Cross-domain small sample classification method based on inter-source domain stylization and related device

Abstract

The application provides a cross-domain small sample classification method based on source domain style and related equipment. The method is based on a labeled source domain dataset and multiple unlabeled auxiliary source domain datasets. A transfer dataset with pseudo labels is obtained through a style transfer algorithm. The transfer dataset has the same class as the source domain dataset and the same style as the auxiliary source domain dataset, thereby expanding the distribution of the source domain dataset. The encoding model is fine-tuned through labeled data in the target domain dataset, and the encoding model learns each dataset in the target domain. A corresponding classifier is used to classify unlabeled data in the target domain dataset. The cross-domain small sample classification method improves the performance of cross-domain small sample classification and improves the generalization ability of the encoding model to the domain. At the same time, the performance decline caused by cross-domain is effectively suppressed, thereby improving the effect of cross-domain small sample classification.

Description

Technical Field

[0001] The present application relates to the field of deep learning and target recognition technology, and in particular to a cross-domain small sample classification method based on inter-source domain stylization and related equipment. Background Art

[0002] Over the past decade, deep learning-based neural networks have greatly promoted progress in various computer vision tasks, such as image classification, object detection, image segmentation, and image synthesis. However, their high accuracy usually requires training the network with a large amount of labeled data, and these labels are usually very expensive to collect. To make matters worse, there are many practical areas such as fine-grained image recognition, medical image analysis, and satellite image analysis, where data labeling is very expensive or even impossible to label proprietary or sensitive data in large quantities. To address these problems, the few-shot task came into being, which aims to solve the target task under very limited supervision information. In the past few years, researchers have extensively explored the few-shot classification (FSC) task and made gratifying progress.

[0003] Due to the limited supervision information of the target task, supervised auxiliary datasets are often utilized to solve the FSC problem. Such auxiliary datasets usually have many categories, a large number of labeled samples in each category, and do not cover the prior knowledge of any category in the target task. However, most existing FSC studies assume that the auxiliary dataset and the target dataset are from the same domain. Unfortunately, as mentioned above, such an assumption is a very strict constraint and may not be applicable in many practical applications. This leads to a new problem, namely cross-domain FSC (CDFSC), in which the auxiliary dataset and the target dataset are from two very different domains. Although CDFSC has important practical value, it is more challenging than ordinary FSC due to the domain difference problem between the auxiliary dataset and the target dataset.

[0004] Since CDFSC is an emerging problem in computer vision, researchers have proposed several methods to solve it from different perspectives, such as adversarial training, model integration, fine-tuning with partial data from the target domain, etc. Despite these methods, CDFSC is still in its early stages and there are many unresolved issues. Summary of the Invention

[0005] In view of this, the purpose of this application is to propose a cross-domain small sample classification method and related equipment based on inter-source domain stylization.

[0006] Based on the above objectives, this application provides a cross-domain small sample classification method based on inter-source domain stylization, including:

[0007] Based on a labeled source domain dataset and at least one unlabeled auxiliary source domain dataset, a migration dataset with pseudo labels is obtained through an image style transfer algorithm;

[0008] Performing a first pre-training on an initial encoding model and a first initial classifier based on the source domain dataset and the migration dataset to obtain a first encoding model and a first classifier;

[0009] Performing a second pre-training on the first encoding model and the second initial classifier corresponding to each sub-dataset using the labeled data in each sub-dataset in the target domain dataset to obtain a second encoding model and a plurality of second classifiers;

[0010] Based on the second encoding model and the plurality of second classifiers, the unlabeled data in each sub-dataset in the target domain dataset are classified to obtain a target classification result.

[0011] The number of the auxiliary source domain datasets is the same as the number of the sub-datasets in the target domain dataset.

[0012] Based on the same inventive concept, the present disclosure also provides a cross-domain small sample classification device based on inter-source domain stylization, comprising:

[0013] The style transfer module is configured to obtain a pseudo-labeled transfer dataset through an image style transfer algorithm based on a labeled source domain dataset and at least one unlabeled auxiliary source domain dataset;

[0014] a cross-domain learning module, configured to perform a first pre-training on an initial encoding model and a first initial classifier based on the source domain dataset and the migration dataset to obtain a first encoding model and a first classifier;

[0015] a target domain learning module, configured to perform second pre-training on the first encoding model and the second initial classifier corresponding to each sub-dataset using the labeled data in each sub-dataset in the target domain dataset, so as to obtain a second encoding model and a plurality of second classifiers;

[0016] a classification module configured to classify the unlabeled data in each of the sub-datasets in the target domain dataset based on the second encoding model and the plurality of second classifiers to obtain a target classification result;

[0017] The number of the auxiliary source domain datasets is the same as the number of the sub-datasets in the target domain dataset.

[0018] Based on the same inventive concept, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0019] Based on the same inventive concept, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method as described above.

[0020] From the above, it can be seen that the present application provides a cross-domain small sample classification method and related equipment based on inter-source domain stylization. Based on a labeled source domain dataset and multiple unlabeled auxiliary source domain datasets, a migration dataset with pseudo labels is obtained through a style transfer algorithm. The migration dataset has the same category as the source domain dataset and the same style as the auxiliary source domain dataset, thereby expanding the distribution of the source domain dataset. The encoding model is then fine-tuned using the labeled data in the target domain dataset, so that the encoding model learns each dataset in the target domain, and the classification of the unlabeled data in the target domain dataset is achieved through the corresponding classifier. The cross-domain small sample classification method provided by the present application improves the performance of cross-domain small sample classification and improves the generalization ability of the encoding model to the domain. At the same time, it effectively suppresses the performance degradation caused by cross-domain, thereby improving the effect of cross-domain small sample classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 Schematic diagram of the process of the cross-domain small sample classification method based on inter-source domain stylization according to an embodiment of the present application;

[0023] Figure 2 This is a schematic diagram of the style transfer process in an embodiment of the present application;

[0024] Figure 3 This is a schematic diagram of the first pre-training process of an embodiment of the present application;

[0025] Figure 4 This is a schematic diagram of the second pre-training process of an embodiment of the present application;

[0026] Figure 5 This is a schematic diagram of the structure of a cross-domain small sample classification device based on inter-source domain stylization according to an embodiment of the present application;

[0027] Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of this application more clear, this application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0029] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0030] In the related art, to address CDFSC, numerous domain adaptation (DA) algorithms aiming to minimize the domain gap between the source and target domains are certainly helpful. Recently, many multi-source DA (MDA) methods have achieved great success by leveraging the abundant explicit supervision information from multiple source domains. Inspired by this, unlike existing CDFSC methods that only use a single source domain, we intend to conduct groundbreaking work by exploring multiple source domains for CDFSC. However, it is worth noting that due to the difficulty and high cost of label acquisition, obtaining a large number of labels for multiple source domains is also very challenging. Therefore, assuming that abundant labels are available for multiple source domains is actually too restrictive. To alleviate the need for large-scale labeled source datasets, in this work, we explore a new multi-source CDFSC (MCDFSC) setting, in which the data in only one source domain is fully labeled (denoted as the original source domain), while the data in the remaining source domains is unlabeled (denoted as the auxiliary source domains). Obviously, how to introduce and utilize the unlabeled source domain data to improve the performance of cross-domain few-shot CDFSC becomes a major challenge in this setting.

[0031] Numerous studies have shown that the human visual system has excellent domain generalization capabilities, while CNNs, in contrast, have poor domain generalization capabilities. This discrepancy is primarily due to the fact that humans tend to identify objects based on image content, while CNNs have a strong inductive bias toward style. Changes in image style are more likely to cause domain shifts than changes in content, which explains why CNNs are inherently sensitive to domain differences.

[0032] Based on the above considerations, this paper considers leveraging style information from auxiliary source domains to improve the model's adaptability to style, thereby enhancing its domain generalization capabilities. We propose an Inter-Source Stylization Network (ISSNet) for the new multi-source cross-domain few-shot classification (MCDFSL) setting. This network improves FSC performance on the target domain by transferring style information from the auxiliary source domain to the original source domain. Furthermore, to further improve cross-domain performance, this paper selects auxiliary source domains with a distribution that is relatively close to that of the target domain.

[0033] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0034] This application provides a cross-domain small sample classification method based on inter-source domain stylization. Figure 1 , including the following steps:

[0035] Step S101: Based on a labeled source domain dataset and at least one unlabeled auxiliary source domain dataset, a transfer dataset with pseudo labels is obtained by using an image style transfer algorithm.

[0036] Before explaining this step, we give the setting of a single source domain, which is specifically given a target domain dataset defined by a training dataset containing only a small number of labeled samples. and a source domain dataset that is unrelated to the target task where n s >>n t , n s Indicates the data volume of the source domain dataset, n t represents the amount of data in the target domain dataset, Y s Representation Label The corresponding space, Y t Representation Label Corresponding space. D t and D s From the probability distribution and Medium sampling, According to the commonly used C-way K-shot (C represents the number of sample categories, K represents the number of samples in each category) setting, D tContains C classes, each class has only K samples, K is very small, such as 1 or 5, that is, n t =CK. In the multi-source domain setting, in addition to a labeled source domain dataset D sl , we also introduce N unlabeled auxiliary source domain datasets, represented as Since labeled data plays an indispensable role in the training process, we choose D according to the principle that the source domain data should be widely distributed and diverse in categories. su . And each They are selected from different fields such as medicine, remote sensing, underwater, etc. su They are all selected based on the principle that their distribution is close to the corresponding target domain. su The label is missing, we do not consider D su content information, and only use its style information to expand the migration data. sl Style. D su Style information and D sl The content information is combined into new data D through the AdaIN network al .

[0037] Step S102 : performing a first pre-training on an initial encoding model and a first initial classifier based on the source domain dataset and the migration dataset to obtain a first encoding model and a first classifier.

[0038] This step belongs to the cross-domain consistency learning stage. sl With D al The content is the same, so they share the same label. Source domain dataset D sl With true labels, migration dataset D al With pseudo labels, the source domain dataset D sl and migration dataset D al Input into the initial encoding model and the first initial classifier, perform the first pre-training on the initial encoding model and the first initial classifier, and use the first initial classifier to classify the migration dataset D with pseudo labels. al Classification is performed, and the generalization performance of the initial encoding model for the domain is trained by optimizing the loss function to obtain a first encoding model and a first classifier.

[0039] Step S103 , performing second pre-training on the first encoding model and the second initial classifier corresponding to each sub-dataset using the labeled data in each sub-dataset in the target domain dataset to obtain a second encoding model and multiple second classifiers.

[0040] Specifically, the target domain dataset includes multiple sub-datasets, each from a different domain. Each sub-dataset includes a small number of labeled training sets. The first encoding model is pre-trained using the training sets to optimize the loss function. Simultaneously, a second initial classifier is trained to improve the classification accuracy of the test set in each sub-dataset, thereby obtaining a second encoding model and multiple second classifiers.

[0041] Step S104: classify the unlabeled data in each of the sub-datasets in the target domain dataset based on the second encoding model and the plurality of second classifiers to obtain a target classification result.

[0042] The number of the auxiliary source domain datasets is the same as the number of the sub-datasets in the target domain dataset.

[0043] Through steps S101 to S103, a second encoding model that has been pre-trained twice and a second classifier for each sub-dataset in the target domain dataset are obtained. The remaining unclassified data in the target domain dataset are classified respectively by the second encoding model and multiple second classifiers to obtain the classification results of all the data in the target domain dataset.

[0044] Based on steps S101 to S104 above, a style transfer algorithm is used to obtain a pseudo-labeled migration dataset based on a labeled source domain dataset and multiple unlabeled auxiliary source domain datasets. The encoding model is then fine-tuned using the labeled data in the target domain dataset, allowing the encoding model to learn from each dataset in the target domain and classify the unlabeled data in the target domain dataset using the corresponding classifier. This improves the performance of cross-domain small-sample classification and enhances the encoding model's ability to generalize across domains. At the same time, it effectively suppresses the performance degradation caused by cross-domain learning, thereby improving the effectiveness of cross-domain small-sample classification.

[0045] In some embodiments, the domain distribution of the auxiliary source domain dataset matches the domain distribution of the sub-dataset. When selecting an auxiliary source domain dataset, the selection principle is to be close to the distribution of the target domain, thereby further reducing the domain difference between the source domain and the target domain. Low-quality auxiliary source domain datasets often produce the opposite effect. There are also domain differences between each auxiliary source domain input, which will interfere with each other and may have a counter-effect on cross-domain performance. This embodiment selects an auxiliary source domain dataset with a distribution close to that of the target domain, fundamentally reducing the difficulty of model cross-domain.

[0046] In some embodiments, the migration dataset and the source domain dataset have the same category, and the migration dataset and the auxiliary source domain dataset have the same style.

[0047] Based on the source domain dataset and the auxiliary source domain dataset, a migration dataset is obtained through the style migration algorithm. The migration dataset learns the content of the source domain dataset and the style of the auxiliary source domain dataset to expand the distribution of the source domain dataset.

[0048] In some embodiments, obtaining a pseudo-labeled migration dataset using an image style migration algorithm based on a labeled source domain dataset and at least one unlabeled auxiliary source domain dataset includes:

[0049] Inputting the source domain dataset and all the auxiliary source domain datasets into an encoder, and mapping the source domain dataset into a first feature vector and mapping the auxiliary source domain dataset into a second feature vector by the encoder;

[0050] Obtaining a calculation result based on the first eigenvector and the second eigenvector using an AdaIN algorithm;

[0051] The calculation result is mapped by a decoder to obtain the migration data set.

[0052] This embodiment is implemented through the AdaIN network, aiming to combine D sl 300 content information and D su 400 style information to reconstruct D al 500, the encoder is the VGG model in the AdaIN method. Figure 2 , the AdaIN network consists of an encoder E A 100 and decoder D A 200, of which AdaIN is calculated between E A 100 and D A Between 200 and 500. A 100 respectively D sl and D su Mapped to the corresponding feature f sl and f su , then the two features f sl and f su The AdaIN calculation was performed and the calculation process is as follows:

[0053]

[0054] Where μ(*) and σ(*) represent the mean and standard deviation respectively. A AdaIN(f sl ,f su ) is upsampled to D with the same size as the input data al .

[0055] The AdaIN network is constructed by perceiving the loss function lper And the style loss function l sty Optimize. per The perceptual similarity of two samples is measured by the distance between the activation maps of the pre-trained network. It is defined as:

[0056]

[0057] Among them, φ i is the activation map of the i-th layer of the pre-trained network, and N represents the number of auxiliary source domain datasets. In addition, the style loss l sty It measures the difference in activation map covariance, which can represent the difference in style texture. sty It can be expressed as:

[0058]

[0059] in, is a graph from the activation map φ j The gram matrix of size C j ×C j The total objective function of the AdaIN network is adain By l per and l sty Weighted combination:

[0060] l adain =l per +λ sty l sty

[0061] Among them, λ sty =10.

[0062] In some embodiments, the performing a first pre-training on the initial encoding model and the first initial classifier based on the source domain dataset and the migration dataset to obtain a first encoding model and a first classifier includes:

[0063] Inputting the source domain dataset and the migration dataset into an initial encoding model for encoding to obtain a third feature vector;

[0064] Inputting the third feature vector into the first initial classifier for classification, and outputting a first classification result via the first classifier;

[0065] The initial encoding model and the first initial classifier are first pre-trained based on the pseudo labels of the migration data set and the first classification result, and the first encoding model and the first classifier are obtained by minimizing the loss function.

[0066] In this embodiment, reference Figure 3 , Dsl 300 and D al 500 are input to the initial coding model Md600 and the first initial classifier Clf 700. Md 600 converts D sl and D al Encoded as features f sl =Md(D sl ) and f al =Md(D al ), and classify by Clf 700 to obtain a first classification result, and perform a first pre-training on the initial encoding model Md 600 and the first initial classifier Clf 700 based on the pseudo label of the migration data set and the first classification result, wherein the target loss function in the training process is a cross entropy loss (CE) with batch spectral regularization (BSR), which regularizes the singular values ​​of the feature matrix of the batch, and the target loss function is formulated as follows:

[0067]

[0068] Among them, λ represents the weight parameter, W is the parameter of the classifier, g i (i=1,2,3,…,n) are the singular values ​​of the batch feature matrix. ce Described as:

[0069]

[0070] in,(·) c The first coding model 800 and the first classifier are obtained through the first pre-training. In this embodiment, the initial coding model Md 600 is ResNet10.

[0071] In some embodiments, the second pre-training of the first encoding model and the second initial classifier corresponding to each sub-dataset using the labeled data in each sub-dataset in the target domain dataset to obtain a second encoding model and a plurality of second classifiers includes:

[0072] Inputting the labeled data in each of the sub-datasets into the first encoding model, and outputting a fourth feature vector via the first encoding model;

[0073] Inputting the fourth feature vector into the second initial classifier to obtain a second classification result;

[0074] The first encoding model and the second initial classifier are subjected to the second pre-training based on the true labels of the labeled data in the sub-dataset and the second classification result, and the second encoding model and the second classifier are obtained by minimizing the loss function.

[0075] Specifically, refer to Figure 4 , using D sl 300 and D al After the first pre-training of Md 600, the learned first encoding model 800 is applied to the target domain dataset D t 900. In this embodiment, the first coding model 800 learned by the first pre-training is adapted to the target domain D through the second pre-training. t On each sub-dataset of 900, D t 900 learn the second initial classifier 1000 parameters. D t The training set is used to pre-train the first encoding model 800 for the second time, and at the same time train the parameters of the second initial classifier 1000 to improve D t The classification accuracy of the 900 test set. t The labeled data in each sub-dataset in 900 is input into the first encoding model 800, and the fourth eigenvector is obtained by encoding. The fourth eigenvector is then classified by the second initial classifier 1000 to obtain a second classification result. The first encoding model 800 and the second initial classifier 1000 are trained by narrowing the gap between the second classification result and the true label of the labeled data. The target loss function in the training process is the cross entropy loss (CE) with batch spectral regularization (BSR), which regularizes the singular values ​​of the feature matrix of the batch. The target loss function is:

[0076]

[0077] Among them, λ represents the weight parameter, W is the parameter of the classifier, g i (i=1,2,3,…,n) are the singular values ​​of the batch feature matrix. ce Described as:

[0078]

[0079] in,(·) c The second encoding model and the second classifier are obtained through the second pre-training. In this embodiment, the initial encoding model Md is ResNet10.

[0080] In some embodiments, classifying the unlabeled data in each of the sub-datasets in the target domain dataset based on the second encoding model and the plurality of second classifiers to obtain a target classification result includes:

[0081] Inputting the unlabeled data in each of the sub-datasets into the second encoding model respectively, and outputting a fifth feature vector through the second encoding model;

[0082] The fifth feature vector is input into the second classifier corresponding thereto for classification, and the target classification result is output via the second classifier.

[0083] Specifically, the initial encoding model obtains the final second encoding model through two pre-trainings. The unlabeled data in the target domain dataset is encoded by the second encoding model to obtain the fifth eigenvector. The fifth eigenvector is input into the corresponding second classifier for classification to obtain the target classification result, thereby realizing the classification of all unlabeled data in the target domain, thereby completing cross-domain small sample classification.

[0084] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0085] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0086] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a cross-domain small sample classification device based on inter-source domain stylization.

[0087] refer to Figure 5 The cross-domain small sample classification device based on inter-source domain stylization includes:

[0088] The style transfer module 501 is configured to obtain a transfer dataset with pseudo labels by using an image style transfer algorithm based on a labeled source domain dataset and at least one unlabeled auxiliary source domain dataset;

[0089] A cross-domain learning module 502 is configured to perform a first pre-training on an initial encoding model and a first initial classifier based on the source domain dataset and the migration dataset to obtain a first encoding model and a first classifier;

[0090] a target domain learning module 503 configured to perform second pre-training on the first encoding model and the second initial classifier corresponding to each sub-dataset using the labeled data in each sub-dataset in the target domain dataset, so as to obtain a second encoding model and a plurality of second classifiers;

[0091] The classification module 504 is configured to classify the unlabeled data in each of the sub-datasets in the target domain dataset based on the second encoding model and the plurality of second classifiers to obtain a target classification result.

[0092] The number of the auxiliary source domain datasets is the same as the number of the sub-datasets in the target domain dataset.

[0093] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0094] The device of the above embodiment is used to implement the corresponding cross-domain small sample classification method based on inter-source domain stylization in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0095] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the cross-domain small sample classification method based on inter-source domain stylization described in any of the above embodiments is implemented.

[0096] Figure 6 10 is a schematic diagram showing a more specific hardware structure of an electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0097] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0098] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0099] The input / output interface 1030 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0100] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0101] The bus 1050 comprises a path for transmitting information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0102] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0103] The electronic device of the above embodiment is used to implement the corresponding cross-domain small sample classification method based on inter-source domain stylization in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0104] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the cross-domain small sample classification method based on inter-source domain stylization as described in any of the above embodiments.

[0105] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0106] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the cross-domain small sample classification method based on inter-source domain stylization as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0107] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0108] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0109] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0110] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A cross-domain small sample classification method based on inter-source domain stylization, characterized by: include: Based on a labeled source domain dataset and at least one unlabeled auxiliary source domain dataset, a migration dataset with pseudo labels is obtained through an image style transfer algorithm; Performing a first pre-training on an initial encoding model and a first initial classifier based on the source domain dataset and the migration dataset to obtain a first encoding model and a first classifier, including: inputting the source domain dataset and the migration dataset into the initial encoding model for encoding to obtain a third feature vector; inputting the third feature vector into the first initial classifier for classification, and outputting a first classification result through the first classifier; performing a first pre-training on the initial encoding model and the first initial classifier based on the pseudo labels of the migration dataset and the first classification result, and obtaining the first encoding model and the first classifier by minimizing a loss function; The first encoding model and the second initial classifier corresponding to each sub-dataset are respectively pre-trained with the labeled data in each sub-dataset in the target domain dataset to obtain a second encoding model and multiple second classifiers, including: inputting the labeled data in each sub-dataset into the first encoding model, and outputting a fourth feature vector via the first encoding model; inputting the fourth feature vector into the second initial classifier to obtain a second classification result; performing a second pre-training on the first encoding model and the second initial classifier based on the true labels of the labeled data in the sub-dataset and the second classification result, and obtaining the second encoding model and the second classifier by minimizing the loss function; Based on the second encoding model and the plurality of second classifiers, the unlabeled data in each sub-dataset in the target domain dataset are classified to obtain a target classification result. The number of the auxiliary source domain datasets is the same as the number of the sub-datasets in the target domain dataset.

2. The method according to claim 1, characterized in that The domain distribution of the auxiliary source domain dataset matches the domain distribution of the sub-dataset.

3. The method according to claim 1, characterized in that The migration dataset has the same category as the source domain dataset, and the migration dataset has the same style as the auxiliary source domain dataset.

4. The method according to claim 1, wherein The method of obtaining a pseudo-labeled migration dataset based on a labeled source domain dataset and at least one unlabeled auxiliary source domain dataset through an image style migration algorithm includes: Inputting the source domain dataset and all the auxiliary source domain datasets into an encoder, and mapping the source domain dataset into a first feature vector and mapping the auxiliary source domain dataset into a second feature vector by the encoder; Obtaining a calculation result based on the first eigenvector and the second eigenvector using an AdaIN algorithm; The calculation result is mapped by a decoder to obtain the migration data set.

5. The method according to claim 1, characterized in that The step of classifying the unlabeled data in each of the sub-datasets in the target domain dataset based on the second encoding model and the plurality of second classifiers to obtain a target classification result includes: Inputting the unlabeled data in each of the sub-datasets into the second encoding model respectively, and outputting a fifth feature vector through the second encoding model; The fifth feature vector is input into the second classifier corresponding thereto for classification, and the target classification result is output via the second classifier.

6. A cross-domain small sample classification device based on inter-source domain stylization, characterized by: include: The style transfer module is configured to obtain a pseudo-labeled transfer dataset through an image style transfer algorithm based on a labeled source domain dataset and at least one unlabeled auxiliary source domain dataset; A cross-domain learning module is configured to perform a first pre-training on an initial encoding model and a first initial classifier based on the source domain dataset and the migration dataset to obtain a first encoding model and a first classifier, including: inputting the source domain dataset and the migration dataset into the initial encoding model for encoding to obtain a third feature vector; inputting the third feature vector into the first initial classifier for classification, and outputting a first classification result through the first classifier; performing a first pre-training on the initial encoding model and the first initial classifier based on the pseudo label of the migration dataset and the first classification result, and obtaining the first encoding model and the first classifier by minimizing a loss function; The target domain learning module is configured to perform a second pre-training on the first encoding model and the second initial classifier corresponding to each sub-dataset using the labeled data in each sub-dataset in the target domain dataset to obtain a second encoding model and multiple second classifiers, including: inputting the labeled data in each sub-dataset into the first encoding model, outputting a fourth feature vector via the first encoding model; inputting the fourth feature vector into the second initial classifier to obtain a second classification result; performing a second pre-training on the first encoding model and the second initial classifier based on the true labels of the labeled data in the sub-dataset and the second classification result, and obtaining the second encoding model and the second classifier by minimizing the loss function; a classification module configured to classify the unlabeled data in each of the sub-datasets in the target domain dataset based on the second encoding model and the plurality of second classifiers to obtain a target classification result; The number of the auxiliary source domain datasets is the same as the number of the sub-datasets in the target domain dataset.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 5.

Images