Self-correlation domain adaptive image classification method based on dynamic balance

Abstract

The invention is applicable to the technical field of computer vision, and provides an autocorrelation domain adaptive image classification method based on dynamic balance, which comprises the following steps of: acquiring image data sets of two style fields, and processing the image data sets to obtain a source domain data set with a label and a target domain data set without a label; constructing a data weighting balance module, inputting the two image data sets into the data weighting balance module, and weighting according to the number of image samples in each domain; constructing an adversarial training module, and inputting the weighted image data set into an adversarial generative network multiplexed by a discriminator for training; constructing a dynamic balance parameter, and weighting by using the dynamic parameter to obtain a loss function; and based on the back propagation optimization generator and the discriminator, a classification model is obtained through multiple rounds of adversarial training. According to the method, a feature generator and a classifier are optimized by using balance parameters and adversarial loss, and a classification model capable of realizing reliable image classification in multiple domains is finally obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to an autocorrelation domain adaptive image classification method based on dynamic balance. Background Art

[0002] As a research branch of transfer learning, unsupervised domain adaptation has achieved rapid development in recent years. UDA aims to use some labeled source domain data and unlabeled target domain data to build a model with certain generalization ability. Existing adversarial learning-based methods have achieved good performance. These adversarial-based methods aim to obtain feature representations with domain-invariant characteristics to reduce cross-domain distribution bias.

[0003] Adversarial methods are inspired by adversarial generative networks. They minimize the difference between the source domain and the target domain by constructing a game. They usually consist of a generative model G that extracts data distribution and an additional discriminant model D. The discriminator needs to distinguish whether the sample belongs to the source domain or the target domain. Adversarial methods are divided into two types according to whether there is an additional discriminator. The first type of adversarial training method uses an additional discriminator, which causes additional training overhead, and when the data distribution of the source domain and the target domain in the training data is inconsistent, it will cause the discriminator to be biased. Methods without specific discriminators usually only consider intra-class differences and lead to inferior solutions. At the same time, some of the above methods excessively pursue the alignment of the source domain and the target domain, ignoring the distribution differences and sample imbalance problems in the source domain and the target domain, resulting in final domain misalignment. Therefore, it is necessary to provide an autocorrelation domain adaptive image classification method based on dynamic balance to solve the above problems. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an autocorrelation domain adaptive image classification method based on dynamic balance to solve the problems existing in the above-mentioned background technology.

[0005] The present invention is implemented as follows: a method for adaptive image classification based on autocorrelation domain of dynamic balance, the method comprising the following steps:

[0006] Obtain image datasets in two style domains, and process the image datasets to obtain labeled source domain datasets and unlabeled target domain datasets;

[0007] Constructing a data weighted balancing module, inputting the two image data sets into the data weighted balancing module, and weighting them according to the number of image samples in their respective domains;

[0008] Construct an adversarial training module and input the weighted image dataset into the adversarial generative network reused by the discriminator for training;

[0009] Construct a dynamic balance parameter, determine the balance factor, and use the dynamic parameter weighting to obtain the final loss function;

[0010] The generator and discriminator are optimized based on back-propagation, and the final classification model is obtained through multiple rounds of adversarial training.

[0011] As a further solution of the present invention: the step of obtaining the image data set specifically includes:

[0012] Get the source domain containing N S Data samples with labels D S , in X S represents the source domain sample, Y S Represents the label corresponding to the source domain sample, and the label contains k categories;

[0013] Get data samples without labels in the target domain in, X t Represents the target domain sample.

[0014] As a further solution of the present invention: the step of constructing a data weighted balancing module specifically includes:

[0015] Calculate the number of samples n according to the source domain dataset and the target domain dataset respectively s and n t , the source domain data and the target domain data are weighted according to the sample size, and the weighting method is: in and They represent the weighted source domain and target domain data samples respectively, λ∈(0,1], λ is a hyperparameter used to dynamically control the sample weight.

[0016] As a further solution of the present invention: the step of constructing the adversarial training module specifically includes:

[0017] Construct a feature generator and a classifier, and reuse the classifier as a discriminator;

[0018] The weighted image dataset is input into the feature generator to generate features and To deceive the discriminator;

[0019] Adversarial training is performed based on the classifier as a discriminator of different categories.

[0020] As a further solution of the present invention: the step of performing adversarial training based on the classifier as a discriminator of different categories specifically includes:

[0021] When the classifier is used as a domain information alignment discriminator, based on the adversarial loss L adv Conduct adversarial training:

[0022] When the classifier is used as a class-information aligned discriminator, the adversarial loss L based on the nuclear norm Wasserstein difference w Conduct adversarial training:

[0023] When the classifier does not serve as a discriminator, the source domain data is classified and the cross entropy loss is used as the classification loss:

[0024] As a further solution of the present invention: the steps of constructing a dynamic balance parameter, determining a balance factor, and obtaining a final loss function by weighting the dynamic parameter specifically include:

[0025] A balance parameter was constructed based on maximum mean difference (MMD) and linear discriminant analysis (LDA);

[0026] Normalize the evaluation values ​​of MMD and LDA, use minimum-maximum scaling to linearly transform the evaluation values, and map the results to the range of [0, 1];

[0027] Using the normalized and Construct the balance factor α;

[0028] L using the balance factor α to align the information of the reaction domain adv and the response type discrimination information L w After dynamic balance adjustment, the final optimized loss function is:

[0029] As a further solution of the present invention: the construction of the balance factor α, Based on the balance factor α, L adv and the response type discrimination information L w To make dynamic balance adjustments:

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The present invention utilizes balanced parameters and two types of adversarial losses to optimize feature generators and classifiers, and ultimately obtains a classification model that can achieve reliable image classification in multiple domains. The present invention is based on adversarial domain adaptation, does not add additional discriminators, but reuses the classifier as a discriminator, and adds a dynamically balanced learning method to achieve clear domain alignment and category distinction. The present invention is suitable for scenarios where source domain data and annotations are rich, but target domain labels are scarce, and can provide more efficient cross-domain migration image classification functions. Compared with traditional methods, the present invention simultaneously utilizes domain alignment and category distinction information to achieve good performance in classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Flowchart of the autocorrelation domain adaptive image classification method based on dynamic balance.

[0033] Figure 2 Flowchart for building the adversarial training module in the autocorrelation domain adaptive image classification method based on dynamic balance.

[0034] Figure 3 Flowchart for constructing dynamic balance parameters in the autocorrelation domain adaptive image classification method based on dynamic balance.

[0035] Figure 4 This is an application framework diagram of the autocorrelation domain adaptive image classification method based on dynamic balance. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0037] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0038] like Figure 1 and Figure 4 As shown, an embodiment of the present invention provides an autocorrelation domain adaptive image classification method based on dynamic balance, and the method includes the following steps:

[0039] S100, acquiring image datasets in two style domains, and processing the image datasets to obtain a labeled source domain dataset and an unlabeled target domain dataset;

[0040] S200, constructing a data weighted balancing module, inputting the two image data sets into the data weighted balancing module, and weighting them according to the number of image samples in their respective domains;

[0041] S300, constructing an adversarial training module, inputting the weighted image data set into the adversarial generative network reused by the discriminator for training;

[0042] S400, construct a dynamic balance parameter, determine the balance factor, and use the dynamic parameter weighting to obtain the final loss function to ensure that the model pays attention to both the domain information and the class information of the image;

[0043] S500, based on back-propagation optimization of the generator and discriminator, obtains the final classification model through multiple rounds of adversarial training, that is, a classifier that can accurately classify in multiple fields.

[0044] In the embodiment of the present invention, the styles of the image datasets in the two style domains are quite different. The step of obtaining the image datasets specifically includes: obtaining the source domain containing N S Data samples with labels D S , in X S represents the source domain sample, Y S Represents the label corresponding to the source domain sample, the label contains k categories; obtain the data sample without label in the target domain in, X t Represents the target domain sample. After the data of the source domain and the target domain are acquired, step S200 is executed to weight the data.

[0045] In the embodiment of the present invention, the step of constructing a data weighted balancing module specifically includes: calculating the number of samples n according to the source domain dataset and the target domain dataset respectively. s and n t , the source domain data and the target domain data are weighted according to the sample size, and the weighting method is: in and Respectively represent the weighted source domain and target domain data samples, λ∈(0,1], λ is a hyperparameter used to dynamically control the sample weight. After obtaining the weighted data set, execute step S300 to perform the adversarial training module.

[0046] like Figure 2 As shown, in the embodiment of the present invention, the step of constructing the adversarial training module specifically includes:

[0047] S301, construct a feature generator G(·) and a classifier C(·), do not additionally establish a discriminator of the adversarial network, and reuse the classifier as the discriminator D(·);

[0048] S302, input the weighted image data set into the feature generator to generate features and To deceive the discriminator;

[0049] S303, performing adversarial training based on the classifier as a discriminator of different categories.

[0050] In the embodiment of the present invention, the step of performing adversarial training based on the classifier as a discriminator of different categories specifically includes: when the classifier is used as a domain information alignment discriminator, based on the adversarial loss L adv Conduct adversarial training: When the classifier is used as a class-information aligned discriminator, the adversarial loss L based on the nuclear norm Wasserstein difference w Conduct adversarial training: When the classifier does not serve as a discriminator, the source domain data is classified and the cross entropy loss is used as the classification loss:

[0051] It should be noted that the adversarial loss L for calculating the nuclear norm Wasserstein difference is w The method is to form a matrix of true labels, predicted labels and prediction accuracy to reflect the correlation within and between data classes. For an image set with h samples and k categories, the prediction matrix Z = C(f)∈S for the sample by classifier C h×k have:

[0052] Then the autocorrelation matrix is ​​R = Z T Z∈S k×k For the autocorrelation matrix R, the intraclass correlation I is calculated using the main diagonal elements a , the non-main diagonal elements calculate the inter-class correlation I b : From the definition, we can get: I a +I b =h.

[0053] It should be noted that, since it is targeted at the source domain, prediction helps to make I a The value of I b For the target domain, due to the lack of supervised training, I a The value of I b The value of is too large. a -I e To represent the domain difference. At the same time, since L a is equal to the F-norm of the prediction matrix Z, that is , so we have: I a -I b =2||Z|| F -h.

[0054] The prediction matrix Z is calculated by the classifier C, so use ||C|| F to replace ||Z|| F :I a -I b =2||C|| F -h, introduce 1-Wasserstein distance to measure the distribution of two features Distance between: where ||·|| L represents the Lipschitz seminorm [2], and P represents the Lipshitz constant. However, ||C|| F It is now possible to effectively use the critic function for the discriminator D, which can be written as: In order to ensure prediction diversity, the nuclear norm ||·||* is used instead of the F-norm ||·|| F :

[0055] After constructing the adversarial module and calculating the loss of each process, execute step S400.

[0056] like Figure 3 As shown, in the embodiment of the present invention, the steps of constructing a dynamic balance parameter, determining a balance factor, and obtaining a final loss function by weighting the dynamic parameter specifically include:

[0057] S401, constructing a balance parameter based on maximum mean difference MMD and linear discriminant analysis LDA;

[0058] S402, normalizing the evaluation values ​​of MMD and LDA, linearly transforming the evaluation values ​​using minimum-maximum scaling, and mapping the results to a range of [0, 1];

[0059] S403, using the normalized and Construct the balance factor α;

[0060] S404, using the balance factor α to adjust the L of the reaction domain alignment information adv and the response type discrimination information L w After dynamic balance adjustment, the final optimized loss function is:

[0061] In the embodiment of the present invention, the calculation method of MMD and LDA is: Where P b is the between-class scattering matrix, P wis the intra-class scattering matrix. The balance factor is constructed using MMD and LDA. Since the evaluation criteria of the two are not in the same order of magnitude, the estimated values ​​are normalized, the evaluation values ​​are linearly transformed using minimum-maximum scaling, and the results are mapped to the range of [0, 1]. The construction method is:

[0062]

[0063]

[0064] In the embodiment of the present invention, the constructing balance factor α is: Based on the balance factor α, L adv and the response type discrimination information L w To make dynamic balance adjustments: That is, by using the above-mentioned dynamic balance method, the final optimized loss function can be obtained.

[0065] It should be noted that the maximum mean difference (MMD) and linear discriminant analysis (LDA) are used to measure the degree of alignment of the current feature representation across domains. and distinguishability

[0066] Among them, P b is the between-class scattering matrix, P w is the intra-class scattering matrix. Obviously, the larger the L(D) value is, the better the resolution performance of the method is. Repeat steps S300 and S400, and obtain a final classification model through multiple rounds of adversarial training, that is, obtain a classifier that can accurately classify in multiple fields.

[0067] The above only describes in detail the preferred embodiments of the present invention, which is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0068] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0069] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0070] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure in the specification and examples. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A method for adaptive image classification based on autocorrelation domain based on dynamic balance, characterized in that: The method comprises the following steps: Obtain image datasets in two style domains, and process the image datasets to obtain labeled source domain datasets and unlabeled target domain datasets; Constructing a data weighted balancing module, inputting the two image data sets into the data weighted balancing module, and weighting them according to the number of image samples in their respective domains; Construct an adversarial training module and input the weighted image dataset into the adversarial generative network reused by the discriminator for training; Construct a dynamic balance parameter, determine the balance factor, and use the dynamic parameter weighting to obtain the final loss function; The generator and discriminator are optimized based on back-propagation, and the final classification model is obtained through multiple rounds of adversarial training.

2. The method for adaptive image classification based on autocorrelation domain based on dynamic balance according to claim 1, characterized in that: The steps to obtain the image dataset include: Get the source domain containing N S Data samples with labels D S , in X S represents the source domain sample, Y S Represents the label corresponding to the source domain sample, and the label contains k categories; Get data samples without labels in the target domain in, X t Represents the target domain sample.

3. The method for adaptive image classification based on autocorrelation domain based on dynamic balance according to claim 2, characterized in that: The step of constructing a data weighted balancing module specifically includes: Calculate the number of samples n according to the source domain dataset and the target domain dataset respectively s and n t , the source domain data and the target domain data are weighted according to the sample size, and the weighting method is: in and They represent the weighted source domain and target domain data samples respectively, λ∈(0,1], λ is a hyperparameter used to dynamically control the sample weight.

4. The method for adaptive image classification based on autocorrelation domain based on dynamic balance according to claim 3, characterized in that: The step of constructing the adversarial training module specifically includes: Construct a feature generator and a classifier, and reuse the classifier as a discriminator; The weighted image dataset is input into the feature generator to generate features and To deceive the discriminator; Adversarial training is performed based on the classifier as a discriminator of different categories.

5. The method for adaptive image classification based on autocorrelation domain based on dynamic balance according to claim 4, characterized in that: The step of performing adversarial training based on the classifier as a discriminator of different categories specifically includes: When the classifier is used as a domain information alignment discriminator, based on the adversarial loss L adv Conduct adversarial training: When the classifier is used as a class-information aligned discriminator, the adversarial loss L based on the nuclear norm Wasserstein difference w Conduct adversarial training: When the classifier does not serve as a discriminator, the source domain data is classified and the cross entropy loss is used as the classification loss:

6. The method for adaptive image classification based on autocorrelation domain based on dynamic balance according to claim 5, characterized in that: The steps of constructing a dynamic balance parameter, determining a balance factor, and obtaining a final loss function by weighting the dynamic parameter specifically include: A balance parameter was constructed based on maximum mean difference (MMD) and linear discriminant analysis (LDA); Normalize the evaluation values ​​of MMD and LDA, use minimum-maximum scaling to linearly transform the evaluation values, and map the results to the range of [0, 1]; Using the normalized and Construct the balance factor α; L using the balance factor α to align the information of the reaction domain adv and the response type discrimination information L w After dynamic balance adjustment, the final optimized loss function is:

7. The method for adaptive image classification based on autocorrelation domain based on dynamic balance according to claim 6, characterized in that: The construction balance factor α, Based on the balance factor α, L adv and the response type discrimination information L w To make dynamic balance adjustments:

Images