Multi-source domain distributed migration target identification method and system based on evidence fusion

Abstract

The invention provides a multi-source-domain distributed migration target identification method and system based on evidence fusion, and the method comprises the steps: inputting image data through an initialization module, and setting the maximum number of iterations and an importance coefficient; the feature learning module performs domain invariant feature learning and migration classification on the image data according to the setting to obtain a fusion soft classification result of the target domain image; a weight calculation module estimates the relative weight of the source domain based on the accuracy and the relative weight of the source domain based on the distribution distance, and calculates the final weight of the source domain; and the evidence fusion module discounts the fusion soft classification result according to the final weight of the source domain, weighted evidence fusion is carried out, and a category decision is carried out to obtain the category of the target image to be identified. According to the method, the problem that the classification performance is limited due to insufficient multi-domain information mining in multi-source migration classification is solved, an accurate recognition result is still obtained under the condition that the inter-domain difference is large, the classification accuracy of target domain samples is improved, and the negative influence of information source quality on fusion is effectively reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing. Specifically, it relates to a multi-source domain distributed migration target recognition method and system based on evidence fusion. Background Art

[0002] In target recognition, it is often possible to use multiple sensors to detect a target to obtain attribute information, and then use this attribute information to construct a classification model to process the target to be recognized. In transfer classification, data from multiple domains can often be collected. Due to sensor differences and complex target backgrounds, the data collected by different sensors often vary greatly, that is, the data collected by multiple sensors can be regarded as data from multiple domains (source domains or target domains). The data distributions of different domains are different, and they can all provide useful information for the target domain to assist in the construction of the target domain recognition model. That is to say, the data of different source domains have a certain complementarity in the classification of target domain samples, and integrating the useful information of different source domains can achieve more accurate recognition.

[0003] The current multi-source domain transfer target recognition methods usually establish a unified objective function and simultaneously learn the data representations of multiple domains to obtain a global domain-invariant feature representation. These methods all mine and integrate multi-source domain information at the data level and are a multi-source domain centralized transfer fusion target recognition framework. However, in the process of learning data representations, the distributions between different source domains are also different, and the unified objective function may lead to internal consumption and loss of useful information, making it difficult to fully exploit the complementary information of different source domains, resulting in serious limitations in the classification performance of the transfer target recognition model. For the transfer target recognition task of image data, this problem needs to be particularly emphasized and solved in order to obtain more accurate target recognition results.

[0004] The patent document "A Crop Disease Image Recognition Method Based on Feature Transfer Learning" (CN110457982A) selects an appropriate auxiliary domain with a large data scale, obtains its feature expression through a deep learning framework, and introduces the theory and method of transfer learning to improve the recognition accuracy. However, it requires large public database resources and a large amount of crop disease text information on the Internet, and it is to use the transfer learning method to transform the problem-solving domain rather than improve the recognition accuracy of the transfer method itself.

[0005] The patent document "Image Classification Method Based on Adversarial Fusion of Multi-Source Transfer Learning" (CN111738315A) unifies the data of multiple source domains and the target domain into the same domain-invariant feature learning framework to obtain global domain-invariant features, which can effectively improve the classification accuracy of various images and can be used for image classification under the condition of missing training data labels. However, as a centralized multi-source transfer framework, when the differences between domains are relatively large, the obtained features cannot guarantee strong discriminability for the data of all domains, resulting in highly conflicting prediction results and reducing the recognition rate.

[0006] Therefore, there is a need for a multi-source domain distributed transfer target recognition method that can construct a domain-invariant feature learning framework for different source domains respectively, ensure the high discriminability of domain-invariant features, and thus improve the classification performance of transfer target recognition. Summary of the Invention

[0007] Aiming at the defects in the prior art, the purpose of the present invention is to provide a multi-source domain distributed transfer target recognition method and system based on evidence fusion.

[0008] A multi-source domain distributed transfer target recognition method based on evidence fusion provided by the present invention includes:

[0009] Step 1: The initialization module inputs the image data of a set number of source domains and a target domain, and sets the maximum number of iterations and the importance coefficient;

[0010] Step 2: The feature learning module performs domain-invariant feature learning and transfer classification on the image data respectively according to the settings to obtain the fused soft classification results of the target domain images;

[0011] Step 3: The weight calculation module estimates the relative weights of the source domains based on accuracy and the relative weights based on distribution distance, and calculates the final weights of the source domains;

[0012] Step 4: The evidence fusion module discounts the fused soft classification results according to the final weights of the source domains, performs weighted evidence fusion, and makes a class decision to obtain the class of the target image to be recognized.

[0013] Repeat Step 4 to traverse the image data of all target domains to obtain the classes of all target images to be recognized.

[0014] Preferably, there are n source data in the source domain, which are composed of images and the labels corresponding to the images.

[0015] The target domain includes multiple target data, and the target data are images.

[0016] The n source domains input by the initialization module in Step 1 and a target domain D T ={χ T ,P(XT )}, and the corresponding source domain dataset is The target domain dataset is and set the class space of the target recognition task as Ω = {ω 1 , ω 2 , ……, ω C};

[0017] Among them, χ T respectively represent the feature spaces of the i-th source domain sample target domain sample ;

[0018] N i , N T respectively represent the number of samples in the i-th source domain dataset and target domain dataset;

[0019] represents the feature of the p-th sample in the i-th source domain;

[0020] represents the label of the p-th sample in the i-th source domain;

[0021] represents the q-th sample in the target domain;

[0022] respectively represent the sets of the i-th source domain sample and target domain sample;

[0023] R represents the real number space;

[0024] k represents the dimension of the feature space;

[0025] P(X T ) respectively represent the data distributions of the i-th source domain sample set and target domain sample set;

[0026] P(·) represents the data distribution;

[0027] ω c represents the c-th category.

[0028] Preferably, the feature learning module includes a construction loss unit, a fusion projection unit, and a weighted fusion unit.

[0029] The second step includes:

[0030] S1. The construction loss unit constructs a loss term based on the maximum mean discrepancy metric criterion based on the image data of the source domain and the target domain

[0031] Quantify the inter-domain marginal distribution difference as Parameter matrix

[0032] The kernel matrix of the domain - to - domain conditional distribution difference between the source - domain image data and the target - domain image data with respect to the class ω c is

[0033] where P i represents the domain - to - domain marginal distribution difference between the source - domain image data and the target - domain image data;

[0034] represents the feature of the p - th sample in the i - th source domain;

[0035] represents the label of the p - th sample in the i - th source domain;

[0036] represents the domain - to - domain conditional distribution difference between the source - domain image data and the target - domain image data with respect to the c - th class ω c ;

[0037] represents the transformation matrix;

[0038] represents the dimension of the domain - invariant feature;

[0039] tr(·) represents the trace of a matrix;

[0040] represents the sample in the i - th source domain;

[0041] X T represents the set of target - domain samples;

[0042] X i represents the set of the i - th source - domain sample and target - domain samples;

[0043] k represents the dimension of the sample feature space;

[0044] respectively represent all - one vectors with N i 、N T elements;

[0045] represents a vector with N i 、N T elements. If the true label and the pseudo - label of the p - th and q - th samples of X Si 、X T are ω c , then the values of the p - th and q - th elements in are 1, otherwise they are 0;

[0046] respectively represent the number of true labels and pseudo - labels of data in the source domain and the target domain as ω c ;

[0047] N i represents the number of the i - th source - domain sample;

[0048] N T represents the number of target - domain samples;

[0049] The superscript T represents the matrix transpose operation.

[0050] S2. Construct a loss unit to minimize the marginal distribution difference P in the i - th source domain and the target domain i , and obtain a feature transformation matrix

[0051] where, represents the transpose matrix of A i ;

[0052] represents the regularization term.

[0053] S3. Construct a loss unit to train a multi - classifier for heterogeneous integration based on the image data in the source domain, and perform integrated fusion on the trained multi - classifiers according to the estimated weights and weighted arithmetic mean to determine the soft classification result corresponding to the target data, thereby determining the pseudo - label of the target data, and updating the loss term based on the MMD metric criterion according to the pseudo - label.

[0054] S4. Construct a loss unit to judge the number of iterations. If the number of iterations is less than the set maximum number of iterations, then the number of iterations is incremented by one, and S2 and S3 are repeatedly executed. If the number of iterations is equal to the set maximum number of iterations, then set this feature transformation matrix as the final feature transformation matrix.

[0055] Preferably, the domain - invariant feature representation of the i - th source - domain and target - domain data obtained in S2 is

[0056] where, represents the transpose matrix of the feature transformation matrix;

[0057] represents the x - axis coordinate of the p - th sample in the i - th source domain;

[0058] represents the number of samples in the i - th source domain;

[0059] represents the q - th sample in the target domain;

[0060] N T represents the number of target - domain samples.

[0061] Preferably, in step S3, a loss unit is constructed for the domain-invariant features of the image data of the i-th source domain and the target domain Establish H 1 types of different classifiers, and the soft classification result of the target domain samples under the domain-invariant feature representation is

[0062] Estimate the weight α using the classification accuracy rate of the classifier on the source domain data h ∈(0,1) is estimated as

[0063] where α h represents the estimated weight of the h-th classifier;

[0064] θ h represents the classification accuracy rate of the h-th classifier on the source domain data ;

[0065] H 1 represents the total number of classifiers;

[0066] ω C represents the c-th category.

[0067] Preferably, step two further includes:

[0068] S5. The loss unit uses the final feature transformation matrix to obtain the image data of the source domain and the target domain with the final domain-invariant feature representation.

[0069] S6. The weighted fusion unit trains a multi-classifier based on the source domain image data, classifies the target image to be recognized under the domain-invariant feature representation, obtains the corresponding soft classification result, and fuses it based on the estimated weight and the weighted arithmetic mean fusion rule to obtain the final fused soft classification result

[0070] where α h represents the estimated weight of the h-th classifier;

[0071] represents the soft classification result obtained by training a multi-classifier based on the source domain data to classify the target to be recognized for the q-th sample under the domain-invariant feature representation.

[0072] The feature learning module repeatedly executes S1 to S6, iteratively obtains a new transformation matrix, and respectively obtains the final fused soft classification results of the target domain image data assisted by each source domain image data.

[0073] Preferably, the relative weight based on the accuracy rate is

[0074] The relative weight based on the distribution distance is

[0075] The final weight of the source domain is

[0076] where η i represents the accuracy rate on the i-th source domain

[0077] n represents that there are n source domains in total;

[0078] represents the maximum value of the accuracy rates of the source domains;

[0079] represents the relative weight based on the accuracy rate on the i-th source domain;

[0080] represents the relative weight based on the distribution distance on the i-th source domain;

[0081] represents the distance between the i-th source domain and the target domain;

[0082] represents the minimum value of the distances between the source domains and the target domain;

[0083] D T respectively represent the distances of the source domain and the target domain;

[0084] β i represents the final weight of the i-th source domain;

[0085] represents the weight of the i-th source domain, that is, the comprehensive relative weight based on the distance and the relative weight based on the accuracy rate; represents the maximum value of the weights among all source domains;

[0086] μ represents the importance coefficient.

[0087] Preferably, when the distributions of the source domain data and the target domain data are inconsistent in step three, d(D S , D T ) represents the inter-domain distribution difference, then with the assistance of the source domain, the generalization error bound of the target domain is

[0088] The relative weight based on the accuracy rate is estimated as

[0089] When the classification accuracy rate of a certain source domain is the highest, the corresponding soft classification result can obtain the maximum relative weight.

[0090] The average classification loss is

[0091] The relative weight based on the distribution distance is estimated to be

[0092] The distance between the i-th source domain and the target domain is

[0093] where

[0094] all represent domain labels;

[0095] represents the optimal joint error;

[0096] ε T (f), respectively represent the generalization error and the empirical error of the target domain and the source domain;

[0097] respectively represent the empirical error of the i-th source domain and the distribution difference between the i-th source domain and the target domain. Then, the generalization error bound of the target domain with the assistance of the i-th source domain is

[0098] η i represents the accuracy of the classification model on the i-th source domain;

[0099] respectively represent the samples with domain labels of the i-th source domain and the target domain;

[0100] C represents a classifier learned from data with domain labels;

[0101] represents the set of samples with domain labels of the i-th source domain and the target domain;

[0102] Ⅱ(·) represents the indicator function, and C(·) ∈ {0, 1}.

[0103] Preferably, in step four, the fused soft classification result is discounted, and the soft classification result under the assistance of the i-th source domain is discounted to The weighted evidence fusion result is Let be converted into the probability value BetP(·), corresponding to the category of the target domain image data

[0104] Where, |X| represents the number of elements in X;

[0105] represents the final fused soft classification result;

[0106] β i represents the final weight of the i-th source domain;

[0107] Ω represents the class space;

[0108] A represents a subset of the class space Ω;

[0109] represents the fused soft classification result of the target to be recognized assisted by the n-th source domain.

[0110] According to a multi-source domain distributed migration target recognition system based on evidence fusion provided by the present invention, the multi-source domain distributed migration target recognition method described above is used for operation.

[0111] Compared with the prior art, the present invention has the following beneficial effects:

[0112] 1. The present invention adopts a distribution alignment strategy with classifier fusion, and uses different types of classifiers to improve the reliability of the pseudo-labels of the target domain image data, and solves the problem that the classification performance is limited due to insufficient mining of multi-domain information in multi-source transfer classification.

[0113] 2. The present invention uses a multi-source weighted evidence fusion model to synthesize the soft classification results assisted by different source domain data, and fuses the complementary information in multiple source domains to improve the classification accuracy of the target domain samples.

[0114] 3. The present invention belongs to a distributed multi-source transfer framework. By establishing objective functions for learning local domain invariant features, local feature representations are obtained, and the best recognition results assisted by different source domains are obtained. Then, these information are synthesized at the decision level, which can effectively utilize the complementary information of multi-source domains and still obtain accurate recognition results even when the differences between domains are relatively large.

[0115] 4. The present invention estimates the weights according to the generalization error bound of the target domain, and obtains the generalization errors of different source domains and the distribution differences between domains, which is more reasonable and reliable, thereby effectively reducing the negative impact of the source quality on the fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0116] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objects and advantages of the present invention will become more obvious:

[0117] Figure 1 is a schematic flow chart of a multi-source domain distributed migration target recognition method based on evidence fusion;

[0118] Figure 2 It is a schematic diagram of the multi-source domain data distribution;

[0119] Figure 3 It is a schematic diagram of the multi-source domain distributed migration target recognition system based on evidence fusion;

[0120] Figure 4 It is a schematic diagram of the SAR target recognition dataset samples;

[0121] Figure 5 It is a schematic diagram of the basic information of the SAR target recognition dataset. Specific implementation manners

[0122] The present invention will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those of ordinary skill in the art can make several changes and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0123] According to a multi-source domain distributed migration target recognition method and system based on evidence fusion provided by the present invention, considering the migration classification problem that data in multiple source domains and a target domain are in the same feature space but have different distributions, by comprehensively using various types of classifiers to improve the reliability of pseudo-labels in the distribution alignment process, and at the same time fusing complementary information of different source domains to obtain accurate migration target recognition results, taking Figure 1 as an example, the method includes the following steps:

[0124] Step 1. Taking Figure 2 as an example, input the image data of multiple source domains and the image data of a target domain, and set the maximum number of iterations and the importance coefficient for balancing the source domain weight estimation;

[0125] The source domain includes multiple source data, and the source data includes images and the labels corresponding to the images; the target domain includes multiple target data, and the target data includes images.

[0126] In more preferred examples, if there are n source domains and a target domain D T ={χ T , P(X T )}, then the corresponding source domain dataset and target domain dataset can be expressed as and

[0127] and χ T respectively represent the i-th source domain sample and the target domain sample Feature space, N i and N T respectively represent the number of samples in the i-th source domain dataset and the target domain dataset.

[0128] represents the feature of the p-th sample in the i-th source domain, represents the label of the p-th sample in the i-th source domain, represents the q-th sample in the target domain.

[0129] and respectively represent the sets of the i-th source domain samples and the target domain samples, where k is the dimension of the feature space, and P(X T ) respectively represent the data distributions of the i-th source domain sample set and the target domain sample set. Here, considering the transfer target recognition problem where the image data of multiple source domains and the target domain are in the same feature space but have different distributions, that is and assuming that the class space of the target recognition task is Ω = {ω 1 , ω 2 , ……, ω C}.

[0130] P(X) represents the distribution of data X. ω c represents the c-th class. R represents the real number space.

[0131] In order to obtain the best classification result with the assistance of the source domain, the most direct strategy is to obtain better domain-invariant features. The accuracy of the fused multiple types of classifiers is often higher than that of a single classifier, that is, the prediction of classes is more accurate. By synthesizing the soft output results of multiple classifiers, the reliability of the obtained pseudo-labels is higher, and the learned domain-invariant features are more robust.

[0132] Step 2: Perform domain-invariant feature learning and transfer classification on the image data of each different source domain and the image data of the target domain respectively to obtain the soft classification results of the target domain images with the assistance of different source domains.

[0133] Embed the multi-classifier fusion technology into the domain-invariant feature learning, and then use the source domain image data represented by the domain-invariant features to classify the target domain image samples to obtain the soft classification results with the assistance of different source domains. Specifically, it includes:

[0134] S1. Construct a loss term based on the maximum mean discrepancy (MMD) metric criterion using the source domain image data and the target domain image data;

[0135] The loss term based on the MMD metric criterion is as follows:

[0136]

[0137] In the formula, P i represents the inter-domain marginal distribution difference between the source domain image data and the target domain image data; represents the inter-domain conditional distribution difference between the source domain image data and the target domain image data with respect to the category ω c .

[0138] Based on the MMD metric criterion, the kernel matrix for calculating the inter-domain conditional distribution difference between the source domain image data and the target domain image data with respect to the category ω c is:

[0139]

[0140] where, and respectively represent all-ones vectors with N i and N T elements, and represent vectors with N i and N T elements (if and X T the true label and the pseudo label of the p-th and q-th samples are ω c , then and the values of the p-th and q-th elements in are 1, otherwise 0), N i represents the number of the i-th source domain samples, N T represents the number of target domain samples, and respectively represent the numbers of the true labels and the pseudo labels of ω c in the source domain and the target domain.

[0141] In more preferred examples, for the i-th source domain and the target domain D T , in order to reduce the inter-domain distribution difference, a feature transformation matrix is to be learned so that the transformed source domain data and the target domain data have similar distributions. Here, the MMD metric criterion can be used to quantify the inter-domain marginal distribution difference after feature transformation as:

[0142]

[0143] where, the parameter matrix

[0144] where, represents the feature transformation matrix, is the dimension of the domain-invariant feature, and tr(·) represents the trace of the matrix. and respectively represent all - one vectors with N i and N T elements,

[0145] X i represents the set of the i - th source - domain samples and target - domain samples.

[0146] k represents the feature dimension of the samples.

[0147] S2. Find the optimal solution for the loss term to obtain the feature transformation matrix;

[0148] Minimize the marginal distribution difference P i in the i - th source domain and target domain, and the transformation matrix can be learned. At this time, the distribution difference between the source domain and the target domain can be effectively reduced, so that the low - dimensional representations learned in the source domain and the target domain have domain invariance. At this time, the domain - invariant feature representations of the image data in the i - th source domain and target domain are and

[0149] where, represents the number of samples in the i - th source domain.

[0150] The transformation matrix obtained by finding the optimal solution for the loss term, that is, the feature transformation matrix, is:

[0151]

[0152] In the formula, represents the transpose matrix of A i ; X i =[X Si , X T , where X Si represents the i - th source - domain sample, and X T represents the set of target - domain samples; represents the kernel matrix for calculating the conditional distribution difference of the source - domain image data and the target - domain image data with respect to the class ω c ; represents the regularization term.

[0153] S3. Train multiple types of classifiers based on the source - domain image data described in S1, and based on the estimated weights and the weighted arithmetic - mean fusion rule, that is, the weighted arithmetic mean, perform an ensemble fusion on the trained multi - classifiers to determine the target data, that is, the soft classification result corresponding to the target - domain image data. Then, determine the pseudo - label of the target data according to the soft classification result corresponding to the target data, and finally update the loss term based on the MMD metric criterion according to the pseudo - label.

[0154] Specifically, for the training data under domain-invariant feature representation Build H 1 different types of classifiers, and then obtain the soft classification results of the target domain samples under domain-invariant feature representation as

[0155] Aiming at the problem that the classification performance is limited due to insufficient mining of multi-domain information in multi-source transfer classification, a distribution alignment strategy with classifier fusion is adopted, and different types of classifiers are used to improve the reliability of the pseudo-labels of the target domain image data, and robust domain-invariant features are learned with the assistance of different source domain data.

[0156] Specifically, since the importance of different types of classifiers is different, when fusing the soft classification results, the weights of the classifiers need to be considered. Classifiers with high recognition accuracy on the training set usually also have good recognition effects on the test set, that is, the higher the accuracy on the training set, the higher the accuracy on the test set. Therefore, the importance of classifiers with good performance on the training set is higher. According to this phenomenon, the weights can be estimated using the classification accuracy of the classifiers on the source domain data Denote the classification accuracy of the h-th classifier on the source domain data h as θ h ∈(0,1), then the weight α

[0157] S4. Repeat S2 and S3 until the maximum number of iterations is reached, and determine the transformation matrix obtained in the last iteration as the final feature transformation matrix;

[0158] S5. Use the final feature transformation matrix described in S4 to obtain the source domain image data and target domain image data with domain-invariant feature representation;

[0159] In more preferred examples, although the multi-classifier fusion is embedded in the learning of domain-invariant features, improving the reliability of the pseudo-labels, they are still not real labels, that is, there is a certain risk of error. Here, a class EM iterative strategy is adopted to enhance the learning of the feature transformation matrix. Set the number of iteration rounds as L, then the final domain-invariant features of the i-th source domain data and target domain data are and The domain-invariant features obtained at this time are more reliable than traditional methods, and better results will be obtained for downstream classification tasks.

[0160] S6. Train multiple types of classifiers based on the source domain image data, classify the target image to be recognized under the domain-invariant feature representation to obtain the corresponding soft classification results, and fuse these soft classification results based on the estimated weights and the weighted arithmetic mean fusion rule to obtain the final fused soft classification result.

[0161] Since these classifiers are trained using the same dataset, there is a high correlation between the classifiers. In this case, the common additive fusion rule, i.e., the weighted arithmetic mean (WAA), can be used to reduce the negative impact of source correlation on fusion. Based on the estimated weights and the arithmetic mean fusion rule, the soft classification results can be fused into

[0162] In more preferred examples, since the information contained in different source domain datasets is different and there is complementarity, fusing the complementary information in multiple source domains is expected to improve the classification accuracy of target domain samples. Therefore, according to the source domain image data under the domain-invariant feature representation, train multiple types of classifiers for heterogeneous integration, and use this model to classify the target domain image samples under the domain-invariant feature representation, and output the soft classification results.

[0163] The fused and trained multi-classifier determines the fused soft classification result corresponding to the target domain image data as shown in the following formula:

[0164]

[0165] In the formula, represents the estimated weight of the h-th classifier; represents the soft classification result obtained by training multiple types of classifiers based on the source domain data to classify the target to be recognized under the domain-invariant feature representation. Among them, θ h represents the classification accuracy of the h-th classifier on the source domain data; H 1 represents the number of classifiers.

[0166] Step 3. Estimate the relative weights based on accuracy and the relative weights based on distribution distance for each source domain, and use the relative weights based on accuracy and the relative weights based on distribution distance to estimate the final weights of each source domain;

[0167] Furthermore, a weighted evidence fusion model is proposed to comprehensively combine the soft classification results assisted by different source domains, and estimate the weights of the soft classification results assisted by different source domains using the upper bound of the generalization error of the source domain and the distribution difference between domains. Since this weight is obtained based on the generalization error bound, the estimated weight is more reasonable and reliable, thus effectively reducing the negative impact of source quality on fusion.

[0168] The relative weight based on accuracy can be estimated as:

[0169]

[0170] where η i represents the accuracy of the classification model on the i-th source domain;

[0171] The relative weight based on distribution distance can be calculated using A-distance:

[0172]

[0173] where, represents the distance (A-distance) between the i-th source domain and the target domain;

[0174] The final weights of each different source domain are calculated by the following formula:

[0175]

[0176] where, n is the number of source domains. Among them, μ represents the importance coefficient used to balance weight estimation; represents the weight of the i-th source domain, that is, the relative weight based on distance and the relative weight based on accuracy are combined, represents the maximum value of the weights among all source domains.

[0177] In more preferred examples, the auxiliary effects of different source domains are different, so the soft classification results usually have different reliabilities, that is, weights. When the distribution difference between a certain source domain and the target domain is very small, the target domain samples can obtain a very high classification accuracy with the assistance of this source domain, and vice versa. For the target domain, the soft classification results obtained by using the source domain with a small distribution difference from the target domain are relatively reliable. In addition, different classification models have different performances on the same source domain data and target domain data, that is, the selection of the classification model will also affect the reliability of the soft classification results. When the classification model has a very high recognition accuracy on the training data, it will also perform well on the test data, that is, the reliability of the soft classification results is relatively high. To sum up, the weights of the soft classification results are related to both factors of the inter-domain distribution difference and the performance of the classification model.

[0178] Let ε T (f) and The generalization error and empirical error of the target domain and the source domain are represented respectively. First, domain-invariant feature learning is performed, and then the construction of the classification model is carried out to find an algorithm. Under the condition of ensuring that the empirical error of the source domain is small, the generalization error ε T (f) of the target domain can be directly obtained and is also very small. When the distributions of the source domain and the target domain are exactly the same, there is no need to learn domain-invariant features. Instead, the classification model is directly constructed using the source domain data. At this time, the generalization error bound of the target domain is

[0179] When the distributions of the source domain data and the target domain data are inconsistent, let d(D S , D T ) represent the inter-domain distribution difference. Then, with the assistance of the source domain, the generalization error bound of the target domain is

[0180] Among them, is the optimal joint error, which can usually be regarded as a relatively small constant. The smaller the generalization error bound of the target domain, the higher the classification accuracy of the classification model f on the target domain, that is, the more reliable the soft classification result of the target domain samples under the assistance of this source domain. Under the assistance of the source domain, the generalization error bound of the target domain is affected by two factors: the empirical error of the source domain and the inter-domain distribution difference d(D S , D T ). That is, for the soft classification result obtained under the assistance of the source domain, its reliability is determined by these two factors.

[0181] Let represent the empirical error of the i-th source domain and the distribution difference between the i-th source domain and the target domain respectively. Then, with the assistance of the i-th source domain, the generalization error bound of the target domain is

[0182] The generalization error bound of the target domain will change with the selection of the source domain. If the distribution difference between the i-th source domain and the target domain is very small, and the constructed classification model f i has very good performance on the source domain data, then using this classification model to classify the target domain samples, its accuracy will be very high.

[0183] The reliability / weight of the soft classification result obtained under the assistance of the i-th source domain is determined by the empirical error of the i-th source domain and the distribution difference d(D Si , D T ) between the i-th source domain and the target domain. To estimate the weight, the classification accuracy of the classification model on the source domain data is used to quantify The A-distance between the source domain data and the target domain data is used to quantify Furthermore, the relative weights based on accuracy and the relative weights based on distribution distance are obtained.

[0184] Let η i represent the accuracy of the classification model on the i-th source domain. For the soft classification results assisted by different source domains, its relative weight based on accuracy can be estimated as:

[0185]

[0186] where When the classification accuracy of a certain source domain is the highest, the corresponding soft classification result can obtain the maximum relative weight.

[0187] The calculation method of A-distance is as follows:

[0188] Let represent the samples with domain labels in the i-th source domain and the target domain respectively, where and are domain labels. Then the average classification loss is:

[0189]

[0190] where C is a classifier learned from these data with domain labels; represents the set of samples with domain labels in the i-th source domain and the target domain; Ⅱ(·) represents the indicator function, C(·) ∈ {0, 1}. Therefore, the A-distance between the i-th source domain and the target domain can be calculated as

[0191] For the soft classification results assisted by different source domains, its relative weight based on distribution distance can be estimated as:

[0192]

[0193] where When the distribution difference between a certain source domain and the target domain is the smallest, the corresponding soft classification result can obtain the maximum relative weight.

[0194] Since the reliability / weight of the soft classification results obtained with source domain assistance depends on the empirical error of the source domain and the distribution difference between the source domain and the target domain, we need to comprehensively consider the relative weights based on accuracy and the relative weights based on distribution distance to estimate the final weight of the soft classification results Using μ ∈ (0, 1] to dynamically adjust the importance of the two, the final weight can be calculated as

[0195] where and The maximum weight (i.e., 1) can be obtained when a certain source domain has a very high classification accuracy and a small distribution difference from the target domain.

[0196] Step 4: Discount the soft classification results of the target domain images using the final weights of each source domain, then perform weighted evidence fusion on the discounted soft classification results, and finally make a class decision based on the fusion results to obtain the class of the target image to be recognized.

[0197] The soft classification of the target sample For discounting, the estimated weight β i can be used for calculation. According to the estimated weight β i , the soft classification result under the assistance of the i-th source domain can be discounted to:

[0198]

[0199] where Ω = {ω 1 , ω 2 , ……, ω C} represents the class space of the target recognition task.

[0200] The evidence fusion result of the soft classification result can be calculated as:

[0201]

[0202] When making the final class decision, usually is converted into a probability value BetP(·):

[0203]

[0204] A represents a subset of the class space Ω.

[0205] represents the fused soft classification result of the target to be recognized under the assistance of the n-th source domain.

[0206] where |X| represents the number of elements in X.

[0207] The class of the corresponding target domain image data obtained according to the fusion result is to divide the target domain sample into the class The calculation formula is:

[0208]

[0209] Step 5: Repeat Step 4 for all the image data of the target domain to obtain the classes of all the target images to be recognized.

[0210] Align multiple source domains with the target domain separately for distribution alignment to learn domain-invariant features, obtain classification results with the assistance of different source domains, and then use decision-level fusion technology to aggregate the complementary information of multiple source domains, that is, construct a multi-source distributed transfer fusion framework, and more accurate target recognition results can be obtained.

[0211] Taking Figure 4 as a preferred example, the synthetic aperture radar image target recognition dataset (MSTAR-Multi-DGREE) is extracted from the MSTAR dataset. Samples with pitch angles of {15°, 30°, 45°} are taken out to construct three different domains, denoted as 15D, 30D, and 45D. It contains three types of targets: rocket launcher (2S1), vehicle (BRDM-2), and armored personnel air defense unit (ZSU-234). Each sample is a sliced image with a size of 88×88 pixels. By directly stretching the pixel matrix into a vector, shallow features with a dimension of 7744 can be obtained. At the same time, variational autoencoders can be used to extract deep features with a dimension of 1024. The difference in pitch angles results in different data distributions in the three domains, and three multi-source transfer target recognition tasks can be constructed: {30D, 45D}→15D, {15D, 45D}→30D, {15D, 30D}→45D. Example samples and basic information of the target recognition dataset are as Figure 4 and Figure 5 shown.

[0212] Select three different types of classification models, namely support vector machine (SVM) with a linear kernel, K-nearest neighbor (KNN), and linear discriminant analysis (LDA) as the base classifiers to obtain the soft classification results and pseudo-labels of the target domain samples.

[0213] The number of iterations is set to L = 10, and the regularization parameter is set to λ = 1. The number of nearest neighbors of KNN is set to 5. In this example, the classification accuracy Acc of the target domain is used as the evaluation index of the algorithm. The test results are shown in Tables 1 and 2.

[0214] Table 1. Classification accuracies (%) of different methods under the shallow feature representation of the MSTAR-Multi-DGREE dataset:

[0215]

[0216] Table 2. Classification accuracies (%) of different methods under the deep feature representation of the MSTAR-Multi-DGREE dataset:

[0217]

[0218] For SAR image data in shallow feature representation and SAR image data in deep feature representation, embedding multi-classifier fusion into the process of domain-invariant feature learning improves the target recognition performance. Compared with other single-source transfer recognition algorithms and multi-source transfer recognition algorithms, the highest accuracy rate can be achieved, and the recognition accuracy rate can reach more than 88%. The results show that using a distributed transfer fusion framework to integrate useful information from multiple source domains is an effective means to achieve accurate target recognition and has important practical application value.

[0219] The present invention also provides a multi-source domain distributed transfer target recognition system based on evidence fusion. The multi-source domain distributed transfer target recognition system based on evidence fusion can be implemented by executing the process steps of the multi-source domain distributed transfer target recognition method based on evidence fusion. That is, those skilled in the art can understand the multi-source domain distributed transfer target recognition method based on evidence fusion as a preferred implementation manner of the multi-source domain distributed transfer target recognition system based on evidence fusion.

[0220] According to a multi-source domain distributed transfer target recognition system based on evidence fusion provided by the present invention, taking Figure 3 as an example, it includes:

[0221] An initialization module, configured to input image data of multiple source domains and image data of a target domain, and set a maximum number of iterations and an importance coefficient for balancing weight estimation.

[0222] A feature learning module, respectively performing domain-invariant feature learning and transfer classification on each source domain and the target domain to obtain a fusion soft classification result of target domain samples assisted by different source domains;

[0223] A weight calculation module, estimating the relative weight based on accuracy rate and the relative weight based on distribution distance of each source domain, and estimating the final weight of each source domain by using the relative weight based on accuracy rate and the relative weight based on distribution distance;

[0224] An evidence fusion module, discounting the fusion soft classification of target samples by using the final weights of each source domain, then performing weighted evidence fusion on the discount results, and then obtaining the category of target domain samples according to the evidence fusion results.

[0225] Wherein, the source domain includes multiple source data, the source data includes an image and a label corresponding to the image; the target domain includes target data, and the target data includes an image;

[0226] The feature learning module includes:

[0227] A loss construction unit, configured to find the optimal solution for a loss term to obtain a feature transformation matrix;

[0228] A fusion projection unit for obtaining source domain and target domain data under domain-invariant feature representations;

[0229] A weighted fusion unit for training multiple types of classifiers based on source domain image data, classifying the target to be recognized under domain-invariant feature representations to obtain soft classification results, and fusing these soft classification results based on the estimated weights and the weighted arithmetic mean fusion rule to obtain a fused soft classification result.

[0230] In more preferred examples, an initialization module inputs image data of a set number of source domains and one target domain, and sets the maximum number of iterations and importance coefficients;

[0231] A feature learning module respectively performs domain-invariant feature learning and transfer classification on the image data according to the settings to obtain a fused soft classification result of the target domain image;

[0232] A weight calculation module estimates the relative weights of the source domains based on accuracy and relative weights based on distribution distances, and calculates the final weights of the source domains;

[0233] An evidence fusion module discounts the fused soft classification results according to the final weights of the source domains, and performs weighted evidence fusion to make a class decision to obtain the class of the target image to be recognized;

[0234] Traverse all the image data of the target domain to obtain the classes of all the target images to be recognized.

[0235] Those skilled in the art know that in addition to implementing the system and its various devices, modules, and units provided by the present invention in the form of pure computer-readable program codes, the method steps can be logically programmed to enable the system and its various devices, modules, and units provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc. to achieve the same functions. Therefore, the system and its various devices, modules, and units provided by the present invention can be regarded as a kind of hardware component, and the devices, modules, and units included therein for implementing various functions can also be regarded as the structures within the hardware component; the devices, modules, and units for implementing various functions can also be regarded as both software modules for implementing the method and the structures within the hardware component.

[0236] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined arbitrarily with each other.

Claims

1. A multi-source domain distributed migration target recognition method based on evidence fusion, characterized in that: include: Step 1: The initialization module inputs a set number of source domain and one target domain image data, and sets the maximum number of iterations and importance coefficient; Step 2: The feature learning module performs domain-invariant feature learning and migration classification on the image data according to the settings to obtain the fusion soft classification result of the target domain image; Step 3: The weight calculation module estimates the relative weight of the source domain based on accuracy and the relative weight based on distribution distance, and calculates the final weight of the source domain; Step 4: The evidence fusion module discounts the fused soft classification results according to the final weight of the source domain, and weights the evidence fusion to make a category decision to obtain the category of the target image to be identified; Repeat step 4 to traverse the image data of all target domains and obtain the categories of all target images to be identified.

2. The multi-source domain distributed migration target identification method based on evidence fusion according to claim 1 is characterized in that: There are n source data in the source domain, which are composed of images and labels corresponding to the images; The target domain includes a plurality of target data, and the target data is an image; The n source domains input by the initialization module in step 1 and a target domain D T ={ T ,P(X T )}, the corresponding source domain dataset is The target domain dataset is And set the category space of the target recognition task to Ω={ω1,ω2,……,ω C }; Among them, χ Si , χ T Represents the i-th source domain sample Target domain sample The feature space of Represents the features of the pth sample in the i-th source domain; represents the label of the pth sample in the i-th source domain; represents the qth sample in the target domain; Respectively represent the set of the i-th source domain sample and the target domain sample; N i 、N T Respectively represent the number of samples in the i-th source domain dataset and target domain dataset; R represents the real number space; k represents the dimension of the feature space; P(X T ) represent the data distribution of the i-th source domain sample set and the target domain sample set respectively; P(·) represents the distribution of data; ω c represents the cth category.

3. The multi-source domain distributed migration target identification method based on evidence fusion according to claim 1 is characterized in that: The feature learning module includes a construction loss unit, a fusion projection unit and a weighted fusion unit; The second step comprises: S1. Construct loss unit to construct loss term based on the image data of source domain and target domain based on the maximum mean difference metric The difference in marginal distribution between domains is quantified as Parameter Matrix Source domain image data and target domain image data about category ω c The kernel matrix of the inter-domain conditional distribution difference is Among them, P i Represents the difference in the marginal distribution between the source domain image data and the target domain image data; Represents the features of the pth sample in the i-th source domain; represents the label of the pth sample in the i-th source domain; Represents the source domain image data and the target domain image data about the cth category ω c Differences in inter-domain conditional distributions; represents the transformation matrix; Dimensions that represent domain-invariant features; tr(·) represents the trace of the matrix; represents the sample of the i-th source domain; X T Represents the set of target domain samples; X i represents the set of the i-th source domain sample and target domain sample; k represents the feature space dimension of the sample; Respectively represent i 、N T A vector of all 1 elements; Indicates with N i 、N T elements of a vector, if X T The true label and pseudo label of the p-th and q-th samples are ω c ,but The value of the pth and qth elements in is 1, otherwise it is 0; Respectively represent the real label and pseudo label of the data in the source domain and target domain as ω c The number of N i represents the number of samples in the i-th source domain; N T Indicates the number of samples in the target domain; The superscript T indicates the matrix transpose operation; S2. Construct a loss unit to minimize the marginal distribution difference P between the i-th source domain and the target domain i , and obtain the feature transformation matrix in, Indicates A i The transposed matrix of represents the regularization term; S3. Construct a loss unit to train multiple classifiers based on the image data of the source domain for heterogeneous integration, integrate and fuse the trained multiple classifiers according to the estimated weights and weighted arithmetic mean, determine the soft classification results corresponding to the target data, thereby determining the pseudo labels of the target data, and update the loss items based on the MMD metric criterion according to the pseudo labels; S4. Construct a loss unit to determine the number of iterations. If the number of iterations is less than the set maximum number of iterations, the number of iterations is increased by one, and S2 and S3 are repeated. If the number of iterations is equal to the set maximum number of iterations, the feature conversion matrix is ​​set as the final feature conversion matrix.

4. The multi-source domain distributed migration target identification method based on evidence fusion according to claim 3 is characterized in that: The domain invariant features of the i-th source domain and target domain data obtained in S2 are expressed as in, represents the transposed matrix of the feature transformation matrix; Represents the x-coordinate axis of the p-th sample in the i-th source domain; represents the number of samples in the i-th source domain; represents the qth sample in the target domain; N T Represents the number of target domain samples.

5. The multi-source domain distributed migration target identification method based on evidence fusion according to claim 3 is characterized in that: The loss unit constructed in S3 has domain-invariant features for image data of the i-th source domain and target domain. Establish H1 different types of classifiers, and obtain the soft classification results of the target domain samples under the domain invariant feature representation: The weight is estimated using the classification accuracy of the classifier on the source domain data, α h ∈(0,1) is estimated to be Among them, α h represents the estimated weight of the h-th classifier; θ h Indicates that the h-th classifier is in the source domain data Classification accuracy on ; H1 represents the total number of classifiers; ω C represents the cth category; represents the transposed matrix of the feature transformation matrix; Represents the x-coordinate axis of the p-th sample in the i-th source domain; represents the number of samples in the i-th source domain; represents the qth sample in the target domain; N T Represents the number of target domain samples.

6. The multi-source domain distributed migration target identification method based on evidence fusion according to claim 3 is characterized in that: The step 2 also includes: S5, constructing a loss unit to obtain the image data of the source domain and the target domain represented by the final domain-invariant feature using the final feature conversion matrix; S6, the weighted fusion unit trains multiple classifiers based on the source domain image data, classifies the target image to be identified under the domain invariant feature representation, obtains the corresponding soft classification result, and obtains the final fused soft classification result based on the estimated weight and weighted arithmetic mean fusion rule Among them, α h represents the estimated weight of the h-th classifier; H1 represents the total number of classifiers; It represents the soft classification result obtained by training multiple classifiers based on source domain data to classify the target to be identified of the qth sample under the domain invariant feature representation; The feature learning module repeatedly executes S1 to S6, iteratively obtains a new transformation matrix, and obtains the final fusion soft classification results of the target domain image data assisted by each source domain image data.

7. The multi-source domain distributed migration target identification method based on evidence fusion according to claim 1 is characterized in that: The relative weight based on accuracy is The relative weight based on the distribution distance is The final weight of the source domain is Among them, η i Represents the accuracy on the i-th source domain n means there are n source domains in total; Indicates the maximum accuracy of the source domain; Represents the relative weight based on accuracy on the i-th source domain; represents the relative weight based on the distribution distance on the i-th source domain; Represents the distance between the i-th source domain and the target domain; Indicates the minimum value of the distance between the source domain and the target domain; D T Represent the distances between the source domain and the target domain respectively; β i represents the final weight of the i-th source domain; represents the weight of the i-th source domain, which is a combination of the relative weight based on distance and the relative weight based on accuracy; Represents the maximum value of weights in all source domains; μ represents the importance coefficient.

8. The multi-source domain distributed migration target identification method based on evidence fusion according to claim 7 is characterized in that: In step 3, when the distribution of source domain data and target domain data is inconsistent, d(D S ,D T ) represents the distribution difference between domains. Then, with the assistance of the source domain, the generalization error bound of the target domain is Relative weight based on accuracy Estimated When the classification accuracy of a source domain is the highest, the corresponding soft classification result can obtain the largest relative weight; The average classification loss is Relative weights based on distribution distance Estimated The distance between the i-th source domain and the target domain is in, Both represent domain labels; represents the optimal joint error; ε T (f) Represent the generalization error and empirical error of the target domain and source domain respectively; denote the empirical error of the i-th source domain and the distribution difference between the i-th source domain and the target domain, respectively. Then, the generalization error bound of the target domain with the assistance of the i-th source domain is η i Represents the accuracy of the classification model on the i-th source domain; Respectively represent samples with domain labels in the i-th source domain and target domain; C represents a classifier learned from data with domain labels; represents the sample set with domain labels of the i-th source domain and target domain; Denotes the indicator function, C(·)∈{0,1}.

9. The multi-source domain distributed migration target identification method based on evidence fusion according to claim 1 is characterized in that: In step 4, the fusion soft classification result is discounted, and the soft classification result assisted by the i-th source domain is Discount for The weighted evidence fusion result is The weighted evidence fusion results Converted into probability value BetP(·), The category of the corresponding target domain image data Among them, |X| represents the number of elements in X; Represents the final fusion soft classification result; β i represents the final weight of the i-th source domain; Ω represents the category space; A represents a subset of the category space Ω; It represents the fused soft classification result of the target to be identified with the assistance of the nth source domain.

10. A multi-source domain distributed migration target recognition system based on evidence fusion, characterized in that: The method for distributed migration target identification in multiple source domains based on evidence fusion is adopted and operated.

Images