A bidirectional domain-adaptive data classification method

Abstract

The application relates to a bidirectional generation field self-adaption data classification method, which comprises the following steps: S1, establishing a bidirectional generation field self-adaption framework; S2, setting a difference loss function of a corresponding branch for data output by a source domain generator and data output by a target domain generator; S3, setting a consistency loss function; and S4, inputting actual data into the trained bidirectional generation field self-adaption model to obtain an actual classification result. Compared with the prior art, the application reduces the distribution difference between source domain data and target domain data, simultaneously trains two classifiers by using a source domain and a target domain double branch, and restrains the two classifiers by using a consistency loss, so that field self-adaption transfer learning meeting complex task requirements is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data classification, and in particular to a bidirectional generative domain adaptation data classification method. BACKGROUND

[0002] With the continuous development and in-depth research of transfer learning, various domain adaptation transfer learning algorithms have emerged to improve the transfer performance of heterogeneous distribution data. These transfer algorithms can be divided into two categories according to their adopted strategies:

[0003] The first is the transfer learning method based on statistical learning. The statistical learning idea is the earliest method applied to solve the transfer learning problem. It is based on the statistical characteristics between the source domain data and the target domain data, and artificially designs a cross-domain transfer transformation mechanism. Most of this method analyzes the difference between the marginal distribution or conditional distribution of the source domain data and the target domain data, and uses the designed domain transfer transformation function to map the instance features of cross-domain data, thereby reducing the distribution difference between the mapped source domain and target domain data. Among them, the representative methods include the distribution adaptation method based on the maximum mean difference (MMD) criterion, the instance weight method based on KL divergence, and the joint distribution adaptation (JDA) algorithm based on joint distribution difference.

[0004] The second is the transfer learning method based on generative adversarial network. With the complexity of the application demand of transfer learning, it is more and more difficult to meet the increasingly wide application scenarios of transfer learning by artificially designing a domain transfer transformation mechanism for a specific transfer task. Therefore, the model adaptive transfer transformation mechanism becomes a more efficient and universal choice. The distribution characteristics between the source domain data and the target domain data are matched by using deep neural networks in the model training iteration process, thereby improving the transfer performance of the domain adaptation transfer learning method between different source domains and target domains. Among them, the representative method is to realize domain adaptation transfer learning based on generative adversarial network (GAN), such as domain adversarial training network (DANN), deep domain confusion (DDC), adversarial domain adaptation (ADDA), conditional adversarial domain adaptation network (CDAN), etc. Because of its high degree of compatibility with deep neural networks in network structure, it has become the mainstream method for realizing domain adaptation transfer learning.

[0005] Both of the two methods have certain defects and deficiencies. The transfer learning method based on statistical learning can steadily reduce the distribution difference between the source domain and the target domain data, but the inter-domain mapping transformation method needs to be designed artificially for different tasks, lacks universality, and cannot meet the transfer requirements in actual application scenarios. The transfer learning method based on the generative adversarial network has better performance for the classification model transfer of heterogeneous distribution data, but needs to consume more computing resources. In the case of some specific field problems and large distribution difference between the source domain and the target domain, the method cannot fully utilize the structure characteristics of the target domain data due to the one-way transfer. SUMMARY

[0006] The purpose of the present application is to overcome the defects of the prior art and provide a bidirectional generative domain adaptive data classification method. The method reduces the distribution difference between the source domain data and the target domain data, simultaneously trains two classifiers by using the source domain and the target domain double branches, and is constrained by consistency loss, thereby realizing the domain adaptive transfer learning meeting the requirements of complex tasks.

[0007] The purpose of the present application can be achieved by the following technical solutions.

[0008] A bidirectional generative domain adaptive data classification method, the method comprising the following steps:

[0009] S1, a bidirectional generative domain adaptive framework is established, the framework comprising two branches, respectively a labeled source domain and an unlabeled target domain, the two branches performing bidirectional generative adversarial and iterative training, in the target domain training process, pseudo labels are used instead of target domain label information,

[0010] Among them, the source domain includes a source domain generator and a source domain classifier, the target domain includes a target domain generator and a target domain classifier, the input of the source domain generator is the source domain data, the input of the source domain classifier is the data output by the source domain generator and the source domain data, the input of the target domain generator is the target domain data, the input of the target domain classifier is the data output by the target domain generator and the data output by the source domain generator, and the classifier outputs a classification result;

[0011] The loss function of the generative adversarial network is set for the source domain and the target domain respectively;

[0012] S2, the difference loss function of the corresponding branch is set for the data output by the source domain generator and the data output by the target domain generator respectively, and the distribution difference between the source domain and the target domain is constrained based on the difference loss function, wherein the expression of the difference loss function of the source domain is:

[0013]

[0014] Among them, k is the number of categories, a gMMD loss of the source domain as a whole, a cMMD loss of the source domain between classes;

[0015] An expression of the difference loss function of the target domain is:

[0016]

[0017] wherein k is the number of classes, a gMMD loss of the target domain as a whole, a cMMD loss of the target domain between classes;

[0018] S3, setting a consistency loss function to constrain the classification results output by the source domain classifier and the target domain classifier;

[0019] S4, training the framework based on the loss function of the generative adversarial network, the difference loss function and the consistency loss function to obtain a trained bidirectional generative domain adaptation model, inputting actual data serving as target domain data of the model into the trained bidirectional generative domain adaptation model to obtain actual classification results.

[0020] Further, the pseudo label is obtained based on a classification model trained based on the source domain data.

[0021] Further, an expression of the pseudo label is:

[0022]

[0023] wherein, is the pseudo label, X t is the target domain data, and C0 is a classification model trained based on the source domain data.

[0024] Further, the difference loss function of the target domain constrains the distribution difference between the data output by the target domain generator and the source domain data.

[0025] Further, the difference loss function of the source domain constrains the distribution difference between the data output by the source domain generator and the target domain data.

[0026] Further, for the source domain, the loss function of the generative adversarial network is:

[0027]

[0028] wherein, is the loss function of the generative adversarial network of the source domain, X s is the source domain data, is the discrimination error of the source domain, is the classification error of the source domain.

[0029] Further, for the target domain, the loss function of the generative adversarial network is:

[0030]

[0031] wherein, is the loss function of the generative adversarial network of the target domain, X t is the target domain data, is the discriminant error of the target domain, is the classification error of the target domain.

[0032] Further, the error in the loss function of the generative adversarial network is cross-entropy loss.

[0033] Further, the bidirectional generative adversarial and iterative training includes training of the source domain classifier and the target domain classifier.

[0034] Further, the mutual iterative training of the source domain classifier and the target domain classifier is realized based on the consistency loss function.

[0035] Compared with the prior art, the present application has the following advantages:

[0036] (1) The present application aims at the problem that the one-way generative adversarial domain adaptation migration method cannot fully utilize the structural characteristics of the target domain data. Through the bidirectional generative adversarial double-branch structure of the source domain and the target domain, the distribution matching in the two directions of the source domain to the target domain and the target domain to the source domain is realized, the distribution difference between the source domain data and the target domain data is efficiently and stably reduced, and the present application is especially suitable for model migration in complex task scenarios with large distribution difference between the source domain and the target domain.

[0037] (2) The difference loss function of the source domain generator sets two losses of the whole and the class as distance constraints, which better reduces the probability distribution difference between the cross-domain data, matches the overall distribution characteristics of the source domain data and the target domain data, further reduces the distribution difference between the same class data of the source domain and the target domain, and thus the class boundary of the target domain data is clear, and the influence of the class overlap of the target domain data on the model adaptive migration performance is avoided.

[0038] (3) The present application can flexibly adjust the model structure for different application scenarios, and has good universality. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is the flowchart of the present application;

[0040] Figure 2 is the bidirectional generative domain adaptation framework diagram of the present application. DETAILED DESCRIPTION

[0041] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0042] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0043] Embodiment 1

[0044] The present application improves a kind of bilateral generative domain adaptation (BGDA) data classification method, the flow chart of method is as shown in Figure 1 .

[0045] The classification method adopted by the present application is given with label source domain D s ={X s ,Y s} With unlabeled target domain D t ={X t}, wherein P (X s ) ≠ P (X t ) or Q s (Y s |X s ) ≠ Q t (Y t |X t ) Condition, through the generator and discriminator of generative adversarial network in the bidirectional generative adversarial process of source domain and target domain, reduce the distribution difference of source domain data and target domain data, simultaneously utilize source domain and target domain double branch respectively training two classifiers, and be constrained by consistency loss, thereby realizing the domain adaptive transfer learning that meets complex task requirements.

[0046] The present application obtains two intermediate domains respectively generated based on another domain by the way of bidirectional generation of source domain and target domain, and utilizes the intermediate domains to bridge and balance the distribution difference between source domain and target domain, so as to more easily achieve the goal of reducing the distribution difference between source domain and target domain data.

[0047] The variable description of the present application is shown in Table 1.

[0048] Table 1 Variable Description Table

[0049]

[0050] The bidirectional generative domain-adaptive data classification method includes the following steps:

[0051] S1. Establish a bidirectional generative domain adaptive framework. The structure of the bidirectional generative domain adaptive framework is as follows: Figure 2 As shown.

[0052] The bidirectional generative domain adaptation framework comprises two branches: a labeled source domain and an unlabeled target domain. These two branches perform bidirectional generative adversarial training and iterative training, including training of both source and target domain classifiers. The source domain includes the source domain generator G. s Source Domain Classifier C s The target domain includes the target domain generator G. t and target domain classifier C t Source domain generator G s The input is source domain data X s Source domain classifier C s The input is the source domain generator G. s Output data F t and source domain data X s Target domain generator G t The input is the target domain data X t Target domain classifier C t The input is the target domain generator G. t Output data F s Source domain generator G s Output data F t The classifier outputs the classification result.

[0053] like Figure 2 As shown, X s and X t These are source domain data and target domain data. To fully utilize the source domain data and target domain data to assist in the migration, a source domain generator G is set up for the source domain and the target domain respectively. s and target domain generator G t These are used to generate source domain generators G that have a similar data distribution to the target domain. s Output data F t And a target domain generator G with a distribution similar to that of the source domain data t Output data F s .

[0054] Set the loss function for the generative adversarial network for both the source and target domains. For the source domain, the source domain classifier C is required to... s Not only must it be able to distinguish source domain data X s and generator Gs The generated data F t , i.e. the source domain data X s is discriminated as real data, F t is discriminated as G s The generated fake data, while also being able to correctly classify it. Therefore, the loss function of the source domain generative adversarial network is:

[0055]

[0056] wherein, is the loss function of the source domain generative adversarial network, X s is the source domain data, is the source domain discriminant error, is the source domain classification error.

[0057] For the target domain, the target domain classifier C t is required to not only be able to distinguish the source domain generator G s The generated data F t and the target domain generator G t The generated data F s , but also to be able to correctly classify it, so the loss function of the target domain generative adversarial network is:

[0058]

[0059] wherein, is the loss function of the target domain generative adversarial network, X t is the target domain data, is the target domain discriminant error, is the target domain classification error. The error in the generative adversarial network is all cross-entropy loss.

[0060] The loss function of the generative adversarial network based on the above setting can optimize the classification performance of the classifier.

[0061] The two branches of the framework are bidirectionally generated and iteratively trained. During the target domain training process, the label information that the target domain data does not have is needed, in order to solve this problem, the pseudo label is used instead of the target domain label information. The pseudo label is obtained based on the classification model C0 trained by the source domain data. Although the pseudo label is not accurate, due to the great similarity between the source domain and the target domain, the pseudo label can also bring most of the target domain class information into the training process of the classifier, thereby more greatly utilizing the target domain data to assist training. The expression of the pseudo label is:

[0062]

[0063] wherein, is the pseudo label, X t is the target domain data, C0 is a classification model trained based on the source domain data.

[0064] S2, the difference loss function of the corresponding branch is set for the data output by the source domain generator and the data output by the target domain generator, and the distribution difference between the source domain and the target domain is constrained based on the difference loss function.

[0065] Since the framework includes two branch structures of the source domain and the target domain and the classifiers are trained respectively, the data F t and F s generated by the source domain generator and the target domain generator are respectively constrained by two loss functions.

[0066] The difference loss function of the source domain is used to constrain the distribution difference between the data F s generated by the source domain generator G t and the target domain data X t , so that the distribution between F t and X t is more similar.

[0067] The expression of the difference loss function of the source domain is:

[0068]

[0069] wherein, k is the number of categories, is the overall gMMD loss of the source domain, is the inter-class cMMD loss of the source domain.

[0070] The difference loss function of the target domain is used to constrain the distribution difference between the data F t generated by the target domain generator G s and the source domain data X s , so that the distribution between F s and X s is more similar.

[0071] The expression of the difference loss function of the target domain is:

[0072]

[0073] wherein, k is the number of categories, is the overall gMMD loss of the target domain, is the inter-class cMMD loss of the target domain.

[0074] The category label of the unlabeled data of the target domain is still the generated pseudo label Carry out instead.

[0075] S3, set a consistency loss function to constrain the classification results output by the source domain classifier and the target domain classifier.

[0076] Through the MMD constraint of the source domain and the target domain double-branch, bidirectional distribution matching between the source domain data and the target domain data is realized, so that the distribution difference between the source domain data and the target domain data is better reduced, and the final effect of the migration model is improved.

[0077] In the framework, the source branch classifier C s is responsible for discriminating and classifying the source domain data X s , and the data F s generated by the source domain generator G t , the target branch classifier C t is responsible for discriminating and classifying the data F s generated by the source domain generator G t and the data F t generated by the target domain generator G s , in the source domain and the target domain two branches, the classifier is trained by distribution matching of the source domain data or the data similar to the source domain and the target domain data or the data similar to the target domain, therefore, in the ideal state, the two classifiers C s and G t should finally obtain the same classification results for the target domain unlabeled data. In order to constrain the classification results of the two classifiers in the source domain and the target domain branch, the consistency loss is set as:

[0078] l c =C t (F s )-C s (F s )1

[0079] Through such bidirectional inter-domain distribution matching, it is beneficial to reduce the distribution difference between the source domain and the target domain to a greater extent, and to jointly approximate the final target domain classification model, at the same time, the consistency loss can also make the classifiers on the two branches iteratively trained, so as to ensure the stability of the whole framework in the training process.

[0080] S4, based on the loss function of the generative adversarial network, the difference loss function and the consistency loss function, the framework is trained to obtain a trained bidirectional generative domain adaptation model, and the actual data as the target domain data of the model is input into the trained bidirectional generative domain adaptation model to obtain an actual classification result.

[0081] The classification method of the application can be applied to the following two scenes:

[0082] 1. Image recognition model migration

[0083] Image recognition classification is a hot field of deep learning application in recent years, and data dependence is one of the most serious problems in deep learning, therefore, in order to solve the problems of missing target domain data labels and high labeling cost, the field adaptive transfer learning method has broad application prospects.

[0084] The standard image dataset Office-31 is used to construct six groups of heterogeneous distribution data distribution matching transfer experiments, and the Office-31 dataset has 4652 images, including Amazon, DSLR and Webcam three different fields, and has 2817, 795 and 498 images respectively: 1) The dataset Amazon is a picture of goods displayed on the Amazon platform, and each kind of goods has different shooting angles; 2) The dataset DSLR is a picture taken by a digital single-lens reflex camera, and the shooting is carried out in a good indoor lighting condition, so the picture clarity is high and the noise is small; 3) The dataset Webcam is a picture taken by a webcam, and the picture clarity is generally low due to the limited pixels of the webcam, and there is color distortion caused by light.

[0085] For the three fields Amazon (referred to as A), DSLR (referred to as D) and Webcam (referred to as W) in the Office-31 dataset, two of the three fields are selected in turn as the source domain and the target domain, and six groups of heterogeneous distribution data transfer tasks are constructed, which are A→D, A→W, D→A, D→W, W→A and W→D respectively. In the experiment, the following advanced transfer algorithms are compared: 1) Source domain CNN; 2) DANN; 3) DDC; 4) ADDA; 5) CDAN, and the results are shown in Table 2.

[0086] Table 2 Accuracy rate comparison of algorithms (%)

[0087]

[0088] It can be seen that the method of the application has obvious improvement in transfer performance compared with the existing method.

[0089] 2. Mail package number recognition model transfer

[0090] In recent years, with the continuous development of the express logistics industry, the number of business of express service companies has increased year by year. In the face of a large number of express mail packages, relying on manual sorting by express delivery personnel requires a large amount of manpower and financial resources, therefore, the entire express logistics industry is constantly exploring a more intelligent and efficient express mail package sorting system to improve the sorting speed and optimize the operating cost.

[0091] With the standardization of express single style, at present, most express service enterprises are sorted by scanning the form barcode, but in the actual application scene, the form barcode has problems such as shielding, pollution, damage and the like, and still needs manual processing sorting. The application of an intelligent sorting model can greatly improve the sorting efficiency and reduce the labor cost, but in the actual scene, the font of the mail package form of different batches and different sources is different, and the deformation of the package caused by extrusion also causes the distortion of the digital number, so that the fixed recognition model is difficult to meet the digital number recognition problem demand of all mail packages. The BGDA algorithm can realize the migration of the mail package number recognition model in multiple scenes, and the automatic sorting is completed by recognizing the digital number of the mail destination information, so as to meet the demand of actual application in different scenes.

[0092] The real image data of express package mail provided by China Post is used to complete the positioning and segmentation of the mail number digits, and finally a 32x32 pixel number digit gray image is obtained. We use it to construct two groups of heterogeneous distribution data sets: one group is the standard express single number digit image data set, denoted as PN, which contains 1000 images; the other group is the deformation number digit image data set caused by the deformation of the number digits due to the extrusion of the package, denoted as PD, which contains 2000 images. Since in the actual mail package sorting scene, there are still some handwritten destination number information of foreign packages, the USPS data set is used to simulate handwritten number digit images, and the data set contains 7291 handwritten number images, all of which are 28x28 pixel black and white images.

[0093] Let the data sets PN and PD be the source domain and the target domain, and perform PN→PD and PD→PN migration experiments; let the data sets PN and USPS be the source domain and the target domain, and perform PN→USPS and USPS→PN migration experiments. In the two groups of experiments, the following leading migration algorithms are compared in performance: 1) source domain CNN; 2) DANN; 3) ADDA; 4) CDAN, and the results are shown in Table 3.

[0094] Table 3 Comparison of classification accuracy of experimental algorithms (%)

[0095]

[0096] It can be seen that the method of the present application has obviously improved the migration performance compared with the existing method.

[0097] The preferred embodiments of the present application have been described above in detail. It should be understood that modifications and variations to the preferred embodiments could be made by those skilled in the art in light of the teachings above. It is therefore contemplated that the application can encompass other variations and modifications that fall within the scope of the claims.

Claims

1. A bidirectional generative domain adaptation data classification method, characterized in that, The method comprises the following steps: S1, a bidirectional generative domain adaptation framework is established, the framework comprises two branches, a labeled source domain and an unlabeled target domain, and the two branches are subjected to bidirectional generative adversarial and iterative training, during the training of the target domain, pseudo labels are used to replace the label information of the target domain, and the bidirectional generative domain adaptation framework is an image recognition classification model; The source domain comprises a source domain generator and a source domain classifier, the target domain comprises a target domain generator and a target domain classifier, the input of the source domain generator is source domain data, the input of the source domain classifier is the data output by the source domain generator and the source domain data, the input of the target domain generator is target domain data, the input of the target domain classifier is the data output by the target domain generator and the data output by the source domain generator, and the classifier outputs a classification result; the source domain and the target domain are obtained in the following process: a standard image data set is obtained, and two of the three domains of the standard image data set are selected as the source domain and the target domain in sequence; Loss functions of generative adversarial networks are set for the source domain and the target domain respectively; S2, difference loss functions corresponding to the data output by the source domain generator and the data output by the target domain generator are set respectively, and the distribution difference between the source domain and the target domain is constrained based on the difference loss functions, wherein the expression of the difference loss function of the source domain is: wherein, k is the number of classes, is the source domain overall gMMD loss, is the source domain inter-class cMMD loss; The expression of the difference loss function of the target domain is: wherein, k is the number of classes, is the overall loss for the target domain gMMD loss, is the inter-class loss for the target domain cMMD loss; S3, a consistency loss function is set to constrain the classification results output by the source domain classifier and the target domain classifier; S4, the framework is trained based on the loss functions of the generative adversarial networks, the difference loss functions and the consistency loss function, a bidirectional generative domain adaptation model after training is obtained, actual data serving as the target domain data of the model is input into the bidirectional generative domain adaptation model after training, and actual classification results are obtained.

2. The method of claim 1, wherein, The pseudo labels are obtained based on a classification model trained by the source domain data.

3. The method of claim 2, wherein, The expression of the pseudo label is: wherein, is a pseudo label, X t is target domain data, is a classification model trained based on the source domain data.

4. The bidirectional domain adaptation data classification method of claim 1, wherein, The difference loss function of the target domain constrains the distribution difference between the data output by the target domain generator and the source domain data.

5. The bidirectional domain adaptation data classification method according to claim 1, wherein, The difference loss function of the source domain constrains the distribution difference between the data output by the source domain generator and the target domain data.

6. The bidirectional domain adaptation data classification method according to claim 1, wherein, For the source domain, the loss function of the generative adversarial network is: wherein, a loss function of the generative adversarial network for the source domain, X s source domain data, a discrimination error for the source domain, a classification error for the source domain.

7. The bidirectional domain adaptation data classification method of claim 1, wherein, For the target domain, the loss function of the generative adversarial network is: wherein, a loss function of the generative adversarial network for the target domain, X t target domain data, a discrimination error for the target domain, a classification error for the target domain.

8. The bidirectional domain adaptation data classification method according to claim 6 or 7, characterized in that, The error in the loss function of the generative adversarial network is a cross-entropy loss.

9. The bidirectional domain adaptation data classification method of claim 1, wherein, The bidirectional generative adversarial and iterative training comprises training of the source domain classifier and the target domain classifier.

10. The bidirectional domain adaptation data classification method of claim 1, wherein, The mutual iterative training of the source domain classifier and the target domain classifier is realized based on the consistency loss function.

Images