Unsupervised field generalization method and device based on data diversity enhancement

Abstract

The invention provides an unsupervised domain generalization method and device based on data diversity enhancement, and the method comprises the steps: taking an original image and a noise map as the input of a deep learning network model, and training the deep learning network model; the deep learning network model comprises a first generator and a second generator; and performing adversarial training on the first generator and the second generator by maximizing and minimizing the adversarial loss function to obtain a data generation model. According to the invention, the two generators carry out joint optimization through resistance training, and the generators generate diversified visual samples through an adaptive instance normalization layer and noise injection. And through maximization and minimization strategies of mutual information amount, diversity and consistency of generated samples in visual attributes are ensured. While the generator diversifies the images, the regulation and control of mutual information ensure that the generated samples have enough similarity to maintain the stability of the model, and also have enough difference to enhance the generalization ability.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and in particular relates to an unsupervised domain generalization method and device based on data diversity enhancement. Background Art

[0002] With the rapid development of data-driven machine learning technology, deep neural networks have made remarkable progress in many tasks such as image classification, object detection, speech recognition and natural language processing. These achievements are mainly attributed to the powerful representation ability of deep neural networks, which can automatically learn multi-level features from large-scale labeled data. However, the success of deep neural networks depends largely on the independent and identically distributed assumption of training data and test data. That is, the model assumes that the source data (i.e., training data) and the target data (i.e., test data) have the same distribution during training. Under this assumption, deep neural networks can learn accurate representations on the source data and generalize to the target data during testing. However, in the real world, data distribution is usually affected by many factors and may shift significantly due to data acquisition conditions, acquisition equipment, environmental changes, etc. This distribution shift phenomenon makes it difficult for the model to maintain high performance during training during testing.

[0003] In order to deal with the problem of domain shift, two classic methods have been proposed in recent years: domain adaptation and domain generalization. Domain adaptation introduces target domain data to perform adaptive training of the model to reduce the distribution difference between the source domain and the target domain. Although this method can significantly improve the performance of the model in the target domain, its dependence on target domain data makes it difficult and costly to obtain data in practical applications. Single-domain generalization is a more challenging but more general strategy. Its goal is to improve its generalization ability in unknown target domains by training the model in the source domain without relying on target domain data. Domain generalization methods usually rely on multiple source domain data for training. Assuming that there are certain differences between different source domains, the model can improve its robustness to unknown target domains by learning the change patterns between source domains.

[0004] In the existing research on domain transfer, both domain adaptation and domain generalization have the following shortcomings:

[0005] 1. Reliance on supervision from labeled data: Most existing domain transfer methods rely on the labeled information of the source domain data for supervised learning, which means that a large amount of labeled data is required for training. Data labeling is not only costly, but also difficult to obtain labeled data in practical applications, especially cross-domain tasks, which limits the widespread application of these methods.

[0006] 2. Insufficient diversity in generated adversarial samples: Existing adversarial data augmentation methods often fail to fully capture the diversity of the target domain when generating samples. The samples generated by the model may be limited to certain specific visual features or textures and fail to cover all potential changes in the target domain, resulting in limited generalization ability of the model, especially when processing target domain data with complex visual changes, the model performs poorly.

[0007] 3. The impact of false correlations: Deep models in the existing technology are prone to learning false correlations related to the task during training. These false correlations are usually specific deviations or patterns in the data, but they do not have practical generalization significance. These false associations will cause the model to make wrong predictions during testing, especially when there is a significant difference in the distribution of the target domain and the source domain. Summary of the invention

[0008] The purpose of the present invention is to address the deficiencies of the prior art and provide an unsupervised domain generalization method and device based on data diversity enhancement, which enhances the domain generalization ability of the model by generating diverse adversarial samples.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is:

[0010] An unsupervised domain generalization method based on data diversity enhancement includes the following steps:

[0011] S1: Get the original image and noise map;

[0012] S2: using the original image and the noise map as inputs of a deep learning network model to train the deep learning network model;

[0013] The deep learning network model includes a first generator and a second generator;

[0014] The first generator includes a first convolution layer, a second convolution layer, a third convolution layer, and a first adaptive instance normalization layer, the first convolution layer and the second convolution layer are both connected to the first adaptive instance normalization layer, and the first adaptive instance normalization layer is connected to the third convolution layer;

[0015] The second generator includes an encoder, a second adaptive instance normalization layer, and a decoder connected in sequence;

[0016] The training process of the deep learning network model includes:

[0017] The original image is processed by a first convolutional layer to obtain a first feature;

[0018] The noise map is processed by a second convolutional layer to obtain adjusted noise;

[0019] The first feature and the adjusted noise are input into a first adaptive instance normalization layer to obtain a second feature;

[0020] The second feature is processed by a third convolutional layer to obtain a feature map;

[0021] The original image is processed by the encoder to obtain a third feature;

[0022] The third feature is input into the second adaptive instance normalization layer to obtain a fourth feature;

[0023] The fourth feature is processed by the decoder to obtain an auxiliary feature map;

[0024] quantifying the correlation between the feature map and the original image through mutual information, and constructing an adversarial loss function;

[0025] The first generator and the second generator are adversarially trained by maximizing and minimizing the adversarial loss function, and the deep learning network model is optimized and trained to obtain a data generation model.

[0026] The two generators in the present invention are jointly optimized through adversarial training, and the generator generates diverse visual samples through adaptive instance normalization layer and noise injection. In this process, the maximization and minimization strategies of mutual information are used to ensure the diversity and consistency of the generated samples in visual attributes. While the generator diversifies the images, the regulation of mutual information ensures that the generated samples have enough similarities to maintain the stability of the model, and enough differences to enhance the generalization ability.

[0027] The present invention relies on unsupervised training, which not only reduces the cost of data collection and annotation, but also avoids the problem of false correlation introduced by supervision. By increasing the diversity of unsupervised generators, the present invention ensures the high quality, diversity and generalization ability of generated samples.

[0028] Furthermore, the first convolution layer includes a plurality of parallel convolution layers, and the convolution kernels of the plurality of parallel convolution layers are different, so as to make the image more expressive in local details and enhance the image details.

[0029] Preferably, the convolution kernels of the multiple parallel convolutional layers are uniformly randomly sampled from the set K={1,3,5,7}.

[0030] Furthermore, the expression of the first adaptive instance normalization layer is as follows:

[0031]

[0032] Where AdaIN1(·) is the output of the first adaptive instance normalization layer, z is the first feature, and β is the noise intensity parameter used to adjust the noise injection intensity. is the style mean, is the style variance, To adjust the noise, μ(z) is the mean of the first feature and σ(z) is the variance of the first feature.

[0033] The first adaptive instance normalization layer is used to comprehensively adjust the style and noise perturbations.

[0034] Furthermore, the expression of the second adaptive instance normalization layer is as follows:

[0035]

[0036] Where AdaIN2(·) is the output of the second adaptive instance normalization layer, h is the third feature, is the style mean, is the style variance, μ(h) is the mean of the third feature, and σ(h) is the variance of the third feature.

[0037] Furthermore, the noise vector is processed by two fully connected layers to obtain the style mean and style variance, which enables the generator to dynamically adjust the style of each generated image, achieve style diversification by changing the style mean and style variance, and give the output image of the generator diverse visual features.

[0038] Furthermore, the noise map and the noise vector are sampled from a standard normal distribution N(0,1).

[0039] Furthermore, the adversarial loss function L aug The expression is as follows:

[0040]

[0041] Where G(·) is the output of the first generator, Φ(·) is the output of the second generator, and x i 、x j is the original image, N is the number of original images, || ||1 is the L1 norm, is the upper bound estimator of mutual information, x is the original image, x + is the feature graph.

[0042] Based on the same inventive concept, the present invention also provides an electronic device, including:

[0043] one or more processors;

[0044] A memory having one or more programs stored thereon, which, when executed by the one or more processors, enables the one or more processors to implement the steps of an unsupervised domain generalization method based on data diversity enhancement.

[0045] Based on the same inventive concept, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of an unsupervised domain generalization method based on data diversity enhancement.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The two generators in the present invention are jointly optimized through adversarial training, and the generator generates diverse visual samples through adaptive instance normalization layer and noise injection. In this process, the maximization and minimization strategies of mutual information are used to ensure the diversity and consistency of the generated samples in visual attributes. While the generator diversifies the images, the regulation of mutual information ensures that the generated samples have enough similarities to maintain the stability of the model, and enough differences to enhance the generalization ability.

[0048] The present invention relies on unsupervised training, which not only reduces the cost of data collection and annotation, but also avoids the problem of false correlation introduced by supervision. By increasing the diversity of unsupervised generators, the present invention ensures the high quality, diversity and generalization ability of generated samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a schematic structural diagram of a first generator of the present invention;

[0050] Figure 2 It is a visualization diagram of the features extracted by the ERM method;

[0051] Figure 3 It is a visualization diagram of the features extracted by the L2D method;

[0052] Figure 4 This is a visualization of the features extracted by the PDEN method;

[0053] Figure 5 is a visualization diagram of the features extracted by the method of the present invention;

[0054] Figure 6 Diversified sample graph generated for the L2D method;

[0055] Figure 7 Diversified sample graph generated for the ADA method;

[0056] Figure 8 A graph of diverse samples generated by the method of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be described in detail below in conjunction with the embodiments. It should be noted that the embodiments and features of the embodiments of the present invention can be combined with each other without conflict. For the convenience of description, if the words "upper", "lower", "left" and "right" appear in the following, they only indicate that the upper, lower, left and right directions are consistent with the drawings themselves, and do not limit the structure.

[0058] Example 1

[0059] The unsupervised domain generalization method based on data diversity enhancement in this embodiment aims to solve the problem of domain shift in single-domain generalization and multi-domain generalization by generating diverse training data. This method overcomes the problems of strong supervision dependence, insufficient diversity of generated samples, and false correlation in the prior art by constructing a variational lower bound optimization function based on mutual information and combining the adversarial training mechanism of two generators. Through the mutual collaboration of these technical features, the generalization ability of the model is greatly improved.

[0060] The difficulty in designing this method lies in how to effectively generate diverse samples under unsupervised conditions while ensuring that these samples are consistent with the original samples in terms of core semantics. Under unsupervised conditions, it is easy to cause semantic deviation when generating diverse samples. This method effectively solves this problem by introducing mutual information maximization and minimization strategies. The generator must not only ensure image diversity, but also maintain reasonable correlation between samples through mutual information optimization strategies. This process requires precise mathematical processing and model design.

[0061] In addition, directly calculating the mutual information is extremely complex and difficult to operate in high-dimensional space. This method solves this problem by introducing variational approximation technology, so that the optimization of mutual information can be effectively applied in the generation process and high-quality adversarial samples can be generated on different visual attributes. The integration and optimization of these technologies need to overcome many difficulties in computational complexity and control of generated samples.

[0062] This embodiment constructs a deep learning network model, which includes a first generator and a second generator. The structure of the first generator is as follows: Figure 1 As shown, the first generator includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a first adaptive instance normalization layer. The first convolutional layer and the second convolutional layer are connected to the first adaptive instance normalization layer, and the first adaptive instance normalization layer is connected to the third convolutional layer. The second generator includes an encoder, a second adaptive instance normalization layer, and a decoder connected in sequence. The original image and the noise map are used as inputs of the deep learning network model, and the deep learning network model is trained to obtain a data generation model.

[0063] 1. Construction and optimization of the first generator

[0064] Multi-factor disturbance generator (first generator) design: Multi-factor disturbance generator G(·; θ G )'s overall structure is as follows Figure 1 As shown in Figure 1, its core design goal is to introduce rich variations and details when generating images by integrating multiple visual attributes (style, texture, structure, etc.), so that the generated images have diverse visual expressions. The following is an analysis and explanation of the structure of the first generator and its connection relationship.

[0065] 1.1 Input layer (noise vector and image)

[0066] The input of the first generator consists of two parts: the original image x and the noise vector n and noise map I sampled from the standard normal distribution N(0,1) N In the generation process, the original image x serves as the content basis, providing the initial semantics and structure, while the noise vector n provides the generator with controllable style and detail perturbations. In this way, the first generator retains the main features of the original image when generating images, and has the ability to dynamically adjust the style.

[0067] 1.2 Dynamic Generation of Style Mean and Variance

[0068] In the process of generating images, style attributes are often represented by channel mean and standard deviation, which are considered to be the core factors of image style. Therefore, the first generator introduces two independent neural networks: T μ (·;θ Tμ ) and T σ (·;θ Tσ ), which are used to calculate the style mean and style variance respectively. These two network structures are composed of fully connected layers, and the calculation process is as follows: and in represents the style mean and represents the style variance, and c is the number of output channels. This design enables the generator to dynamically adjust the style of each generated image by changing the mean and variance to achieve style diversification.

[0069] 1.3 Diversification of noise disturbances

[0070] In order to enhance the diversity of image details, the first generator designs a noise perturbation processing method. First, the Gaussian noise sampled from the standard normal distribution is adjusted pixel by pixel through a 1×1 convolution kernel (the second convolution layer). Given the height h and width w of the noise image, this process can be expressed as: in, is to adjust the noise, Conv1 represents the convolution operation using a 1×1 convolution kernel, It is the noise sampled from the standard normal distribution N(0,1). This pixel-by-pixel noise can introduce local random perturbations to the generated image, thereby improving the detail expression of the image.

[0071] 1.4 Local texture diversity

[0072] In order to make the image more expressive in local details, the first generator uses multi-scale random convolution to process different texture levels. Specifically, the first generator uses convolution kernels of different sizes (the first convolution layer) to extract multi-level features of the original image. The 1×1 convolution kernel is used to retain the color and shape information of the image and can be regarded as a color map. Larger convolution kernels (such as 3×3, 5×5, 7×7) are used to process local textures to enhance image details. The convolution weights in the generator are obtained through training optimization to ensure the consistency and stability of the image texture during the generation process.

[0073] 1.5 Adaptive Instance Normalization Variants

[0074] The generator adds a variant of adaptive instance normalization after multi-scale random convolution, called the first adaptive instance normalization AdaIN1, which is used to comprehensively adjust the style and noise perturbations. Its mathematical expression is:

[0075]

[0076] Among them, μ(z) is the mean of the first feature, σ(z) is the variance of the first feature, Represents the feature map (first feature) output by the first convolutional layer, is a parameter used to adjust the intensity of noise injection. This operation is done by removing the original style (i.e., standardizing it to a state with a mean of zero and a variance of one), and then using the new style mean and style variance To re-style and inject noise disturbance

[0077] 1.6 Multi-factor generation formula and overall connection relationship

[0078] Multi-factor disturbance generator G(x;θ G ) Through multiple modules (style mean and variance extraction, noise perturbation generation, adaptive instance normalization variants, random multi-scale convolution) working together, each module controls the image through different visual attributes, achieving image generation with diverse styles and rich details. By integrating the above modules, the final output of the first generator can be expressed as:

[0079] G(x;θ G)=Conv1(AdaIN1(RC k (x),n,I N )),

[0080] Among them, RC k It represents a convolutional layer (the first convolutional layer) using a convolution kernel of random size k×k, where k is uniformly randomly sampled from the set K={1,3,5,7}.

[0081] In this process:

[0082] 1) The input original image x first passes through a convolution kernel RC of random size k Extract multi-scale features.

[0083] 2) These multi-scale features are input into the AdaIN1 module, and the combination of style and perturbation is achieved by replacing the style mean and style variance and injecting noise.

[0084] 3) Finally, after 3×3 convolution (the third convolution layer) for local mapping, the final generated image x with rich details is obtained + =G(x;θ G ).

[0085] 2. Auxiliary generator design

[0086] Auxiliary generator (second generator) Φ(·; θ Φ ) adopts a style autoencoder structure, which not only focuses on the diversity of images, but also introduces flexibility in style adjustment to ensure that the generated images maintain the core features of the original images under diverse styles. The overall structural design of the second generator includes the encoder, decoder, second adaptive instance normalization (AdaIN2) module and the connection between them. The following is a detailed design analysis and the logical relationship between the modules.

[0087] 2.1 Input and Encoder

[0088] The second generator Φ(·;θ Φ ) is the original image x, which is passed through the encoder module Φ E After processing, it is converted into a high-dimensional feature vector (the third feature). In the encoder, through layer-by-layer convolution operations, the detailed features in the image are gradually extracted and embedded into the latent space, thereby extracting the core information related to the content. This representation facilitates the operation of the subsequent style adjustment module.

[0089] 2.1 Second Adaptive Instance Normalization (AdaIN2)

[0090] The adaptive instance normalization layer is the core. It receives the feature vector (the third feature) generated by the encoder and combines it with the random noise vector n to dynamically adjust the style attributes of the feature. The random noise n is sampled from the standard normal distribution N(0,1), which provides a variety of factors for generating style. Inside AdaIN2, two fully connected layers are used to generate the style mean T μ (·;θ Tμ ) and variance T σ (·;θ Tσ ), the specific operation process is as follows:

[0091]

[0092] Among them, h is the third characteristic, is the style mean, is the style variance, μ(h) is the mean of the third feature, and σ(h) is the variance of the third feature.

[0093] This makes each generated image not only consistent with the input image in content, but also changes in style, giving the output image of the second generator diverse visual features. This style adjustment is driven by the noise vector, so each generated image will have natural differences in details and style.

[0094] 2.3 Decoder

[0095] The feature AdaIN2(h) processed by AdaIN2 is passed to the decoder Φ D The decoder's task is to map the style-adjusted features back to the image space, and reconstruct images with diverse styles by amplifying the feature maps layer by layer and gradually restoring the details.

[0096] 2.4 Overall structure and process of auxiliary generator

[0097] The auxiliary generator adopts the design of style autoencoder. Under the premise of maintaining the content, the style and visual attributes of the image output by the generator can be adjusted within a diverse range. The generation process of the entire auxiliary generator can be written as the following formula:

[0098] Φ(x;θ Φ )=Φ D (AdaIN2(Φ E (x),n))

[0099] The specific process is as follows:

[0100] 1) The input original image x first passes through the encoder Φ E The feature representation h is obtained to preserve the content and structural information of the image.

[0101] 2) The feature representation h is passed to the AdaIN2 module, where it is combined with random noise n to dynamically generate the style mean and style variance to adjust the style attributes of the image.

[0102] 3) The style-adjusted features are input to the decoder Φ D In , auxiliary images are generated through a series of upsampling operations.

[0103] 3. Mutual Information Optimization Strategy and Adversarial Training

[0104] In order to achieve the goal of generating novel images, this section designs a framework based on information theory to quantify the generated image x through mutual information. + The correlation between the original image x and the original image x. The estimation of mutual information, the approximation of conditional distribution, and the establishment of adversarial objectives jointly support the unsupervised diversity enhancement framework, generating diverse images without labeled data or specific task models. The method consists of two generators G(·; θ G ) and Φ(·;θ Φ ), and the adversarial loss L aug The following is the specific workflow and logical relationship:

[0105] 3.1 Using mutual information I(x + ; x) Quantifying correlation

[0106] First, the design goal is to minimize the number of generated images x + and the original image x to generate diversified images. To this end, the mutual information I(x + ; x) is introduced as a measurement method to evaluate the degree of correlation between the two.

[0107]

[0108] where p(·) is the marginal probability distribution, p(x + ,x) represents x + The joint distribution of and x, represents the expectation of the corresponding distribution. However, since the logarithmic density ratio of the true joint distribution and the marginal distribution is difficult to solve, it is difficult to directly calculate the mutual information in practice. For this reason, an upper bound estimator of the mutual information is introduced That is, the upper bound of mutual information is estimated through inequality.

[0109]

[0110] Among them, p(x + |x) is the conditional probability given x. The form of this estimator is linear, has good numerical stability, and is +When x and x are independent, the estimator can effectively quantify and control the correlation between the generated image and the original image in the process of maximizing the mutual information.

[0111] 3.2 Mutual Information Approximation Using N-Sample Estimator

[0112] Using a dataset containing N pairs of images Small batch samples of x i is the original image, through Perform an unbiased N-sample estimate. However, the conditional distribution p(x + ∣x) is unsolvable, which makes it impossible to minimize directly Therefore, in order to perform approximate calculations, we refer to existing research and assume that the conditional distribution follows an approximate l1-Laplace distribution with a unit covariance matrix, whose mean is parameterized by the generator, thus obtaining a variational approximation q(x + ∣x), that is

[0113]

[0114] Where G(·) is the output of the first generator, Φ(·) is the output of the second generator, || ||1 is the L1 norm, is the Laplace distribution, I represents the identity covariance, and μ is the conditional mean.

[0115] 3.3 Defining the optimization objectives for estimating loss and generating networks

[0116] Under the assumption that the conditional distribution is a Laplace distribution, the mutual information estimation loss can be calculated, namely:

[0117]

[0118] Among them, x j is the original image.

[0119] Generator network G(·;θ G ) aims to minimize To generate a new image. In this process, to avoid generating image x + Semantic distortion, also need to limit x + To do this, we exploit the internal correlation by maximizing q(x + |x) to limit the deviation between the generated image and the original image, and define the corresponding loss L Φ ,Right now:

[0120]

[0121] 3.4 Designing Adversarial Objectives and Overall Loss

[0122] From the above losses, we can see that L Φ This is equivalent to the second term in the approximate mutual information estimation loss, making the generator network Φ(·;θ Φ ) is adversarial. It can be trained by Φ(·;θ Φ )maximize Then increase x + Correlation with x.

[0123] Therefore, the multi-factor disturbance generator G(·; θ G ) and auxiliary generator Φ(·;θ Φ ) to form the adversarial loss function L aug :

[0124]

[0125] The adversarial loss describes the overall framework of unsupervised diversity enhancement, which contains two customizable generator networks, namely G(·; θ G ) and Φ(·;θ Φ ). This framework is highly flexible in practical applications and can select appropriate neural network architectures according to task requirements to achieve different generation tasks. aug The design of is the core of unsupervised data enhancement, which does not rely on task-specific models (such as classifiers) or labeled data.

[0126] 3.5 Iterative Process of Adversarial Training

[0127] During training, the generator network is optimized by alternating the following steps:

[0128] Fixed auxiliary generator Φ(·;θ Φ ), update the multi-factor disturbance generator G(·;θ G ). At this point, the multi-cause disturbance generator minimizes the loss Reduce the correlation between the generated image and the original image and promote the diversity of the generated images.

[0129] Fixed generation network G(·;θ G ), update the auxiliary generator Φ(·;θ Φ ): At this point, the auxiliary generator maximizes the loss Maintain the relevance of the generated image to the original image, ensuring that the generated image does not deviate from the original image in content.

[0130] These two steps are performed alternately so that G(·;θ G ) and Φ(·;θ Φ) continuously approaches the optimal solution in adversarial training and ensures that the generated images are diverse while maintaining semantic consistency under unsupervised conditions.

[0131] The unsupervised diversity enhancement method based on mutual information generation in this embodiment enhances the domain generalization ability of the model by generating diverse adversarial samples.

[0132] The characteristics of this method are:

[0133] 1. Unsupervised learning: This method does not rely on labeled data in the source or target domains. It uses an unsupervised generative model to expand the distribution of source domain data, thereby significantly reducing the cost of data labeling. This makes this method more flexible in application and suitable for a variety of practical scenarios.

[0134] 2. Diversity enhancement: This method generates adversarial samples by introducing mutual information. When generating samples, it fully considers the differences and similarities between samples. The generated adversarial samples have high visual diversity and cover a variety of visual changes that may exist in the target domain, such as style, texture, and noise. This diverse sample generation strategy can significantly improve the generalization performance of the model in the target domain.

[0135] 3. Avoiding false correlations: Through the design of the unsupervised generator, this method avoids the model from learning false correlations, thereby reducing incorrect predictions in the target domain. This method helps the model better learn features that are truly helpful for generalization by generating diverse samples that are unrelated to the task semantics, thereby improving performance in different fields.

[0136] This method solves the problems of supervision dependence, insufficient sample diversity and false correlation in existing domain transfer methods through an unsupervised diversity enhancement method, and provides an effective solution for single-domain and multi-domain generalization tasks.

[0137] Example 2

[0138] In order to evaluate the proposed method, the digit dataset commonly used in domain generalization research is used for experiments. The experiment evaluates the accuracy of the model tested on four other digit sets after training the model on the black-and-white handwritten digit set MNIST. The four test sets are SVHN, MNIST-M, SYNDIGIT and USPS. The SVHN dataset is derived from the digits intercepted from the Google Street View house numbers, the MNIST-M dataset is composed of MNIST digits and patches randomly extracted from BSDS500 color photos as the background, the SYNDIGIT dataset is derived from the house number digits collected on the street, and the USPS dataset is derived from the black-and-white handwritten digits sampled differently from the MNIST. The experimental results and the comparison of classification accuracy with other single-domain generalization methods are shown in Table 1. It can be seen from the experimental results that this method has achieved better results in solving the single-domain generalization problem.

[0139] Four methods are used to train ConvNet on MNIST, and then 1000 images are randomly selected from each target domain, and t-SN is used to visualize the target features extracted by different methods. Figure 2-Figure 5 The visualization of the features extracted by different methods for the target domain image shows that the proposed method forms a clear distribution structure in the feature space. This indicates that the features of different categories are better separated in space, reducing feature confusion, helping to improve classification accuracy and achieve better generalization performance.

[0140] Table 1 Comparison of classification accuracy

[0141]

[0142] Example 3

[0143] 1. Significant improvement in generalization ability

[0144] This method significantly improves the generalization ability of the model in multiple target domains by generating diverse samples in an unsupervised manner. Compared with traditional domain generalization methods (which usually rely on labeled data), the samples generated by this method are highly diverse and can cover a wider range of visual attributes. Table 2 shows the experimental data, comparing the task performance of this method with other domain generalization methods.

[0145] Table 2 Comparison of classification accuracy between this method and other generalization methods in other fields

[0146]

[0147] As can be seen from Table 2, after adopting the unsupervised method of the present invention, the accuracy of the model in all target domains has been significantly improved, especially in Cartoon, with an improvement of more than 8 percentage points. This shows that the diversified samples generated by the method of the present invention effectively cover a wider range of visual changes and improve the adaptability of the model to data in different fields.

[0148] 2. Reduction of data dependencies

[0149] Traditional domain generalization and domain adaptation methods rely on the annotation of multiple source domain or target domain data for training, while the method of the present invention relies on the generated diverse samples to enhance the generalization ability of the model in an unsupervised situation. By comparing the generalization performance under supervised and unsupervised conditions, it is shown that the method of the present invention can still maintain a high level of generalization ability while reducing data dependence. Table 3 shows the comparison of classification accuracy of supervised and unsupervised methods.

[0150] Table 3 Comparison of classification accuracy of supervised and unsupervised methods

[0151]

[0152] Under unsupervised conditions, the accuracy of the method of the present invention still surpasses that of traditional supervised methods, with an average accuracy rate of about 2 percentage points higher, proving that the method of the present invention not only maintains a high level of generalization ability while reducing the demand for labeled data, but also performs better in some tasks.

[0153] 3. The effect of increased diversity

[0154] The generator in the method of the present invention introduces diverse visual attributes, and the generated samples show high diversity in terms of style, texture, noise disturbance, etc. This diversity effectively avoids the limitation of insufficient diversity of generated samples in existing methods. Figure 6-Figure 7 As shown, by comparing with the diversified sample images generated by L2D and ADA, the diversified sample images generated by the method of the present invention have diversity.

[0155] The method of the present invention effectively solves the problems of existing methods' dependence on supervision and insufficient sample diversity through unsupervised model training and diversity enhancement strategies. Traditional methods such as ERM rely on supervised training to improve performance, JiGen and ADA introduce label guidance in self-supervision and adversarial training, and although ME-ADA, MADA and PDEN improve diversity and generalization capabilities, they still rely on partial supervision. L2D increases sample diversity through style games, while RandConv and MetaCNN focus on the generalization of local features, but have limited control over the overall style of the image. The method of the present invention enhances sample diversity in an unsupervised manner, and experiments have verified its superiority in multi-domain tasks. The generated images achieve higher diversity and generalization without the need for labels.

[0156] Example 4

[0157] This embodiment provides an electronic device, including:

[0158] one or more processors;

[0159] A memory having one or more programs stored thereon, which, when executed by one or more processors, enables the one or more processors to implement the steps of an unsupervised domain generalization method based on data diversity enhancement.

[0160] In some implementations, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0161] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.

[0162] This embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of an unsupervised domain generalization method based on data diversity enhancement.

[0163] The contents explained in the above embodiments should be understood as these embodiments are only used to more clearly illustrate the present invention, and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

Claims

1. An unsupervised domain generalization method based on data diversity enhancement, characterized in that: The following steps are involved: S1: Get the original image and noise map; S2: using the original image and the noise map as inputs of a deep learning network model to train the deep learning network model; The deep learning network model includes a first generator and a second generator; The first generator includes a first convolution layer, a second convolution layer, a third convolution layer, and a first adaptive instance normalization layer, the first convolution layer and the second convolution layer are both connected to the first adaptive instance normalization layer, and the first adaptive instance normalization layer is connected to the third convolution layer; The second generator includes an encoder, a second adaptive instance normalization layer, and a decoder connected in sequence; The training process of the deep learning network model includes: The original image is processed by a first convolutional layer to obtain a first feature; The noise map is processed by a second convolutional layer to obtain adjusted noise; The first feature and the adjusted noise are input into a first adaptive instance normalization layer to obtain a second feature; The second feature is processed by a third convolutional layer to obtain a feature map; The original image is processed by the encoder to obtain a third feature; The third feature is input into the second adaptive instance normalization layer to obtain a fourth feature; The fourth feature is processed by the decoder to obtain an auxiliary feature map; quantifying the correlation between the feature map and the original image through mutual information, and constructing an adversarial loss function; The first generator and the second generator are adversarially trained by maximizing and minimizing the adversarial loss function, and the deep learning network model is optimized and trained to obtain a data generation model.

2. The unsupervised domain generalization method based on data diversity enhancement according to claim 1, characterized in that: The first convolution layer includes multiple parallel convolution layers, and the convolution kernels of the multiple parallel convolution layers are different.

3. The unsupervised domain generalization method based on data diversity enhancement according to claim 2, characterized in that: The convolution kernels of the multiple parallel convolutional layers are uniformly randomly sampled from the set K={1,3,5,7}.

4. The unsupervised domain generalization method based on data diversity enhancement according to claim 1, characterized in that: The expression of the first adaptive instance normalization layer is as follows: Where AdaIN1(·) is the output of the first adaptive instance normalization layer, z is the first feature, β is the noise intensity parameter, is the style mean, is the style variance, To adjust the noise, μ(z) is the mean of the first feature and σ(z) is the variance of the first feature.

5. The unsupervised domain generalization method based on data diversity enhancement according to claim 1, characterized in that: The expression of the second adaptive instance normalization layer is as follows: Where AdaIN2(·) is the output of the second adaptive instance normalization layer, h is the third feature, is the style mean, is the style variance, μ(h) is the mean of the third feature, and σ(h) is the variance of the third feature.

6. The unsupervised domain generalization method based on data diversity enhancement according to claim 4 or 5, characterized in that: The noise vector is processed separately through two fully connected layers to obtain the style mean and style variance.

7. The unsupervised domain generalization method based on data diversity enhancement according to claim 6, characterized in that: The noise map and the noise vector are sampled from a standard normal distribution N(0,1).

8. The unsupervised domain generalization method based on data diversity enhancement according to claim 1, characterized in that: The adversarial loss function L aug The expression is as follows: Where G(·) is the output of the first generator, Φ(·) is the output of the second generator, and x i 、x j is the original image, N is the number of original images, || ||1 is the L1 norm, is the upper bound estimator of mutual information, x is the original image, x + is the feature graph.

9. An electronic device, characterized in that: include: one or more processors; A memory having one or more programs stored thereon, which, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer program is stored therein, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images