Cross-domain small sample classification method and system based on local target data enhancement

Abstract

The invention relates to the field of image recognition processing, and provides a cross-domain small sample classification method and system based on local target data enhancement, which performs local data enhancement on image data of a source domain and image data of a small number of target domains by designing a shear mixing method. Secondly, two groups of query set images are enhanced through random data, cutting and mixing are carried out to obtain an enhanced query image, the relation between prediction of the enhanced image and prediction of an original image is analyzed, the relation between local features of the enhanced image and the original image is compared through a comparative learning method, and a final small sample classification model is obtained. According to the cross-domain small sample classification method, the generalization ability of cross-domain small sample classification is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition processing, and in particular to a cross-domain small sample classification method and system based on local target data enhancement. Background Art

[0002] With the rapid development of computer vision, small sample image classification methods have gradually become an important technical means in dealing with image classification tasks in new or special fields because they can achieve good results using a small number of labeled samples when faced with limited labeled information.

[0003] In the prior art, when using small sample image classification models for cross-domain learning, multiple source domains are usually introduced to optimize the model in a specific domain. However, when there is a significant domain offset between the target domain and the trained source domain, the performance of the model will be greatly affected. Due to the existence of domain spacing, the cross-domain small sample model in the prior art has poor generalization performance in the target domain.

[0004] Therefore, how to design a small sample classification method to improve the generalization ability of the small sample classification model to the target domain during cross-domain learning. Summary of the invention

[0005] Based on this, the purpose of the present invention is to provide a cross-domain small sample classification method and system based on local target data enhancement. By designing a cutting and mixing method, local data enhancement is performed on the image data of the source domain and a small amount of image data of the target domain, so that the final small sample classification model can better adapt to the target domain, and then the two groups of query set images are cut and mixed by random data enhancement to obtain enhanced query images, and the relationship between the predictions of the enhanced images and the original images is analyzed, and the relationship between the local features of the two is compared using a contrastive learning method to improve the final small sample classification model's ability to discriminate the target domain data. The present invention greatly improves the generalization ability of cross-domain small sample classification.

[0006] The present invention proposes a cross-domain small sample classification method based on local target data enhancement, comprising: Acquire a query image dataset and a support image dataset, wherein the query image dataset includes source domain query image data and target domain query image data, and the support image dataset includes source domain support image data and target domain support image data, and perform random spatial data enhancement processing on the query image dataset to acquire an enhanced query image dataset, wherein the enhanced query image dataset includes source domain enhanced query image data and target domain enhanced query image data; Inputting the query image dataset and the enhanced query image dataset into a shearing and blending module, performing shearing and blending according to the query image dataset to obtain query set original image data, and performing shearing and blending according to the enhanced query image dataset to obtain query set enhanced image data; Inputting the query set original image data, the query set enhanced image data and the supporting image data set into a feature processing module to obtain domain-specific features and domain-independent features, obtaining domain labels according to the domain-specific features, and inputting them into a domain classifier to obtain a domain classification score; The domain-independent features are input into a small sample classification module for classification prediction, and then local feature optimization is performed according to a local feature contrast learning algorithm.

[0007] In summary, according to the above-mentioned cross-domain small sample classification method based on local target data enhancement, by designing a cutting and mixing method, local data enhancement is performed on the image data of the source domain and a small amount of image data in the target domain, so that the final small sample classification model can better adapt to the target domain, and then the two groups of query set images are cut and mixed by random data enhancement to obtain enhanced query images, and the relationship between the predictions of the enhanced images and the original images is analyzed, and the relationship between the local features of the two is compared using a contrastive learning method to improve the final small sample classification model's ability to discriminate the target domain data. The present invention greatly improves the generalization ability of cross-domain small sample classification. Specifically, a query image dataset and a support image dataset are obtained, wherein the query image dataset includes source domain query image data and target domain query image data, and the support image dataset includes source domain support image data and target domain support image data; random spatial data enhancement processing is performed on the query image dataset to obtain an enhanced query image dataset, wherein the enhanced query image dataset includes source domain enhanced query image data and target domain enhanced query image data, which greatly enriches the enhanced image of the same image without causing a large semantic difference between the enhanced image and the original image; the query image dataset and the enhanced query image dataset are input into a shearing and blending module, and according to the query image dataset Cutting and mixing are performed to obtain the original image data of the query set, cutting and mixing are performed according to the enhanced query image data set to obtain the query set enhanced image data, the query set original image data, the query set enhanced image data and the support image data set are input into the feature processing module to obtain domain-specific features and domain-independent features, domain labels are obtained according to the domain-specific features, and domain labels are input into the domain classifier to obtain domain classification scores, the domain-independent features are input into the small sample classification module to perform classification prediction, and then local feature optimization is performed according to the local feature contrast learning algorithm, which greatly improves the discrimination ability of the final small sample classification model for the target domain data. The present invention greatly improves the generalization ability of cross-domain small sample classification.

[0008] Further, the step of performing shearing and mixing according to the query image data set to obtain the query set original image data, and performing shearing and mixing according to the enhanced query image data set to obtain the query set enhanced image data specifically includes: Performing random spatial data enhancement processing on the query image dataset, wherein the random spatial data enhancement processing includes flipping or inverting the image in the query image dataset as a whole or internally to obtain an enhanced query image dataset; A single image region in the source domain query image data in the query image dataset is selected, and the image region is replaced with a corresponding image region in the target domain query image data in the query image dataset to generate the original image data of the query set by cutting and mixing. The formula for generating the original image data of the query set by cutting and mixing is as follows: in, Represents the original image data of the query set, represents the side length ratio of the image, represents the source domain query image data, represents the target domain query image data, Represents the label data of the original image data of the query set, Represents the label data of the source domain query image data, Label data representing query image data in the target domain; The enhanced query image dataset is then cut and mixed to generate query set enhanced image data.

[0009] Furthermore, the step of inputting the query set original image data, the query set enhanced image data and the supporting image data set into a feature processing module to obtain domain-specific features and domain-independent features specifically includes: Inputting the query set original image data, the query set enhanced image data and the support image data set into a feature processing module, wherein the feature processing module includes a twin feature extraction block and a feature decoupling block; The first feature extraction block of the twin feature extraction block extracts features from the original image data of the query set and the source domain support image data and the target domain support image data in the support image data set, and then inputs all the extracted image features into a feature decoupling block to decompose the image features into domain-specific features and domain-independent features, wherein the domain-specific features include source domain support image-specific features, target domain support image-specific features, and query set original image-specific features, and the domain-independent features include source domain support image-independent features, target domain support image-independent features, and query set original image-independent features; The second feature extraction block of the twin feature extraction block extracts features from the query set enhanced image data, and then inputs all extracted image features into the feature decoupling block to decompose and obtain irrelevant features of the query set enhanced image, and performs overfitting elimination according to the feature perturbation algorithm, and the feature perturbation algorithm is as follows: in, represents the irrelevant features of the original image in the query set, represents the query set to enhance the image-independent features, represents a feature disturbance term, wherein the feature disturbance term includes noise obtained by sampling the query set enhanced image-irrelevant features from a normal distribution; A domain label is obtained according to the domain-specific feature, and a domain label is input into a domain classifier to obtain a domain classification score; The domain-independent features are then input into a small sample classification module for classification prediction.

[0010] Furthermore, the step of obtaining a domain label according to the domain-specific feature and inputting the domain label into a domain classifier to obtain a domain classification score specifically includes: Obtain domain labels that support image-specific features in the source domain and the domain label of the target domain supporting image-specific features , to calculate the domain label of the specific features of the original image of the query set ,in Indicates the side length ratio of the image; Input all domain-specific features and corresponding domain labels into the domain classifier and calculate the domain classification score; The loss optimization is performed according to the domain classification loss function, and the domain classification loss function is as follows: in, represents the domain classification loss, represents the cross entropy loss function, indicates that the source domain supports image-specific features, indicates that the target domain supports image-specific features, Represents the specific features of the original image in the query set, Represents a domain classifier.

[0011] Furthermore, the step of inputting the domain-independent features into a small sample classification module to perform classification prediction specifically includes: Input the image-independent features supported by the source domain, the image-independent features supported by the target domain, and the original image-independent features of the query set into the small sample classification module; Performing small sample classification on the original image-irrelevant features of the query set, the image-irrelevant features supported by the source domain, and the image-irrelevant features supported by the target domain; The predicted distribution on the source domain and the predicted distribution on the target domain are calculated respectively, and the loss is optimized according to the small sample classification loss function, which is as follows: in, represents the small sample classification loss, represents the side length ratio of the image, represents the cross entropy loss function, represents the predicted distribution of the original image-independent features of the query set in the source domain, represents the predicted distribution of the query set’s original image-independent features in the target domain, represents the few-shot classification labels of the source domain support images, Represents the few-shot classification labels for the target domain support images.

[0012] Furthermore, the small sample classification module also includes: The small sample classification module also includes a small sample self-supervised learning block, which inputs the query set enhanced image-independent features into the small sample self-supervised learning block. The small sample self-supervised learning block performs small sample classification self-supervised enhancement according to the small sample classification labels of the original image of the query set to make the semantic information of images in different regions of all pictures consistent, and performs loss optimization according to the small sample self-supervised learning loss function. The small sample self-supervised learning loss function is as follows: in, represents the small sample self-supervised learning loss, represents the predicted distribution of query set augmented image-independent features in the source domain, Represents the predicted distribution of query set augmented image-independent features on the target domain.

[0013] Furthermore, the step of optimizing the local features according to the local feature contrast learning algorithm specifically includes: Locally slicing the query set original image data and the query set enhanced image data to obtain the query set original local image and the query set enhanced local image; Extracting local image features of the original local image of the query set and the enhanced local image of the query set according to the twin feature extraction block in the feature feature processing module, and then decomposing the local image features according to the feature decoupling block to obtain corresponding domain-independent features; Randomly select any original local image of the query set as an anchor point, and use all local feature samples of the same type of image in the query set enhanced local image as positive samples, calculate the distance between the anchor point and the local feature corresponding to each position of the positive sample according to the cosine distance, so as to compare the distribution distance between the anchor point and the local features in the positive sample in the feature space, and at the same time select all local feature samples of any heterogeneous image in the query set enhanced local image as negative samples, and compare the distribution distance between the anchor point and the local features in the negative sample in the feature space; The loss optimization is performed according to the local feature contrast learning loss function, and the local feature contrast learning loss function is as follows: in, represents the local feature contrast learning loss, represents the number of local features, represents the local feature ordinal number, represents the cosine distance, represents the local features of the anchor point, represents the local features of the positive sample, represents the local features of negative samples, represents the boundary parameter, represents the temperature parameter, Represents the maximum value function.

[0014] The present invention proposes a cross-domain small sample classification system based on local target data enhancement, comprising: An image acquisition processing module is used to acquire a query image dataset and a support image dataset, wherein the query image dataset includes source domain query image data and target domain query image data, and the support image dataset includes source domain support image data and target domain support image data, and perform random spatial data enhancement processing on the query image dataset to acquire an enhanced query image dataset, wherein the enhanced query image dataset includes source domain enhanced query image data and target domain enhanced query image data; a shearing and blending module, configured to input the query image dataset and the enhanced query image dataset into the shearing and blending module, perform shearing and blending according to the query image dataset to obtain query set original image data, and perform shearing and blending according to the enhanced query image dataset to obtain query set enhanced image data; a feature processing module, configured to input the query set original image data, the query set enhanced image data and the support image data set into the feature processing module to obtain domain-specific features and domain-independent features, obtain domain labels according to the domain-specific features, and input them into a domain classifier to obtain domain classification scores; The classification optimization module is used to input the domain-independent features into the small sample classification module to perform classification prediction, and then perform local feature optimization according to the local feature contrast learning algorithm.

[0015] The present invention also provides a storage medium, which stores one or more programs, and when the programs are executed by a processor, implement the cross-domain small sample classification method based on local target data enhancement as described above.

[0016] The present invention also provides a computer device, the computer device comprising a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the cross-domain small sample classification method based on local target data enhancement as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a cross-domain small sample classification method based on local target data enhancement proposed in the first embodiment of the present invention; Figure 2 This is a flow chart of a cross-domain small sample classification method based on local target data enhancement proposed in the second embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a cross-domain small sample classification system based on local target data enhancement proposed in the third embodiment of the present invention.

[0018] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0019] In order to facilitate understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0020] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0022] See also Figure 1 , which is a flow chart of a cross-domain small sample classification method based on local target data enhancement proposed in the first embodiment of the present invention, the cross-domain small sample classification method based on local target data enhancement includes steps S01 to S04, wherein: Step S01: obtaining a query image dataset and a support image dataset, wherein the query image dataset includes source domain query image data and target domain query image data, and the support image dataset includes source domain support image data and target domain support image data, performing random spatial data enhancement processing on the query image dataset to obtain an enhanced query image dataset, wherein the enhanced query image dataset includes source domain enhanced query image data and target domain enhanced query image data; It should be noted that in this embodiment, random spatial data enhancement processing is performed according to the query image dataset, and the random spatial data enhancement processing includes flipping or inverting the image in the query image dataset as a whole or internally to obtain an enhanced query image dataset.

[0023] Step S02: inputting the query image dataset and the enhanced query image dataset into a shearing and blending module, performing shearing and blending according to the query image dataset to obtain the query set original image data, and performing shearing and blending according to the enhanced query image dataset to obtain the query set enhanced image data; It should be noted that in this embodiment, a single image region in the source domain query image data in the query image data set is selected, and the image region is replaced with a corresponding image region in the target domain query image data in the query image data set to generate the original image data of the query set by cutting and mixing. The formula for generating the original image data of the query set by cutting and mixing is as follows: in, Represents the original image data of the query set, represents the side length ratio of the image, represents the source domain query image data, represents the target domain query image data, Represents the label data of the original image data of the query set, Represents the label data of the source domain query image data, Label data representing query image data in the target domain; The enhanced query image dataset is then cut and mixed to generate query set enhanced image data.

[0024] Step S03: input the query set original image data, the query set enhanced image data and the supporting image data set into a feature processing module to obtain domain-specific features and domain-independent features, obtain domain labels according to the domain-specific features, and input them into a domain classifier to obtain domain classification scores; It should be noted that in this embodiment, the query set original image data, the query set enhanced image data and the support image data set are input into a feature processing module, and the feature processing module includes a twin feature extraction block and a feature decoupling block; The first feature extraction block of the twin feature extraction block extracts features from the original image data of the query set and the source domain support image data and the target domain support image data in the support image data set, and then inputs all the extracted image features into a feature decoupling block to decompose the image features into domain-specific features and domain-independent features, wherein the domain-specific features include source domain support image-specific features, target domain support image-specific features, and query set original image-specific features, and the domain-independent features include source domain support image-independent features, target domain support image-independent features, and query set original image-independent features; The second feature extraction block of the twin feature extraction block extracts features from the query set enhanced image data, and then inputs all extracted image features into the feature decoupling block to decompose and obtain irrelevant features of the query set enhanced image, and performs overfitting elimination according to the feature perturbation algorithm, and the feature perturbation algorithm is as follows: in, represents the irrelevant features of the original image in the query set, represents the query set to enhance the image-independent features, represents a feature disturbance term, wherein the feature disturbance term includes noise obtained by sampling the query set enhanced image-irrelevant features from a normal distribution; A domain label is obtained according to the domain-specific feature, and a domain label is input into a domain classifier to obtain a domain classification score; Then, the domain-independent features are input into a small sample classification module to perform classification prediction; In this embodiment, the domain label of the source domain supporting the specific features of the image is obtained. and the domain label of the target domain supporting image-specific features , to calculate the domain label of the specific features of the original image of the query set ,in Indicates the side length ratio of the image; Input all domain-specific features and corresponding domain labels into the domain classifier and calculate the domain classification score; The loss optimization is performed according to the domain classification loss function, and the domain classification loss function is as follows: in, represents the domain classification loss, represents the cross entropy loss function, indicates that the source domain supports image-specific features, indicates that the target domain supports image-specific features, Represents the specific features of the original image in the query set, represents the domain classifier; Input the image-independent features supported by the source domain, the image-independent features supported by the target domain, and the original image-independent features of the query set into the small sample classification module; Performing small sample classification on the original image-irrelevant features of the query set, the image-irrelevant features supported by the source domain, and the image-irrelevant features supported by the target domain; The predicted distribution on the source domain and the predicted distribution on the target domain are calculated respectively, and the loss is optimized according to the small sample classification loss function, which is as follows: in, represents the small sample classification loss, represents the side length ratio of the image, represents the cross entropy loss function, represents the predicted distribution of the original image-independent features of the query set in the source domain, represents the predicted distribution of the query set’s original image-independent features in the target domain, represents the few-shot classification labels of the source domain support images, Represents the few-shot classification labels of the target domain support images; In this embodiment, the small sample classification module also includes: The small sample classification module also includes a small sample self-supervised learning block, which inputs the query set enhanced image-independent features into the small sample self-supervised learning block. The small sample self-supervised learning block performs small sample classification self-supervised enhancement according to the small sample classification labels of the original image of the query set to make the semantic information of images in different regions of all pictures consistent, and performs loss optimization according to the small sample self-supervised learning loss function. The small sample self-supervised learning loss function is as follows: in, represents the small sample self-supervised learning loss, represents the predicted distribution of query set augmented image-independent features in the source domain, Represents the predicted distribution of query set augmented image-independent features on the target domain.

[0025] Step S04: input the domain-independent features into the small sample classification module to perform classification prediction, and then perform local feature optimization according to the local feature contrast learning algorithm; It should be noted that in this embodiment, the query set original image data and the query set enhanced image data are locally sliced ​​to obtain the query set original local image and the query set enhanced local image; Extracting local image features of the original local image of the query set and the enhanced local image of the query set according to the twin feature extraction block in the feature feature processing module, and then decomposing the local image features according to the feature decoupling block to obtain corresponding domain-independent features; Randomly select any original local image of the query set as an anchor point, and use all local feature samples of the same type of image in the query set enhanced local image as positive samples, calculate the distance between the anchor point and the local feature corresponding to each position of the positive sample according to the cosine distance, so as to compare the distribution distance between the anchor point and the local features in the positive sample in the feature space, and at the same time select all local feature samples of any heterogeneous image in the query set enhanced local image as negative samples, and compare the distribution distance between the anchor point and the local features in the negative sample in the feature space; The loss optimization is performed according to the local feature contrast learning loss function, and the local feature contrast learning loss function is as follows: in, represents the local feature contrast learning loss, represents the number of local features, represents the local feature ordinal number, represents the cosine distance, represents the local features of the anchor point, represents the local features of the positive sample, represents the local features of negative samples, represents the boundary parameter, represents the temperature parameter, Represents the maximum value function.

[0026] In summary, according to the above-mentioned cross-domain small sample classification method based on local target data enhancement, by designing a cutting and mixing method, local data enhancement is performed on the image data of the source domain and a small amount of image data in the target domain, so that the final small sample classification model can better adapt to the target domain, and then the two groups of query set images are cut and mixed by random data enhancement to obtain enhanced query images, and the relationship between the predictions of the enhanced images and the original images is analyzed, and the relationship between the local features of the two is compared using a contrastive learning method to improve the final small sample classification model's ability to discriminate the target domain data. The present invention greatly improves the generalization ability of cross-domain small sample classification. Specifically, a query image dataset and a support image dataset are obtained, wherein the query image dataset includes source domain query image data and target domain query image data, and the support image dataset includes source domain support image data and target domain support image data; random spatial data enhancement processing is performed on the query image dataset to obtain an enhanced query image dataset, wherein the enhanced query image dataset includes source domain enhanced query image data and target domain enhanced query image data, which greatly enriches the enhanced image of the same image without causing a large semantic difference between the enhanced image and the original image; the query image dataset and the enhanced query image dataset are input into a shearing and blending module, and according to the query image dataset Cutting and mixing are performed to obtain the original image data of the query set, cutting and mixing are performed according to the enhanced query image data set to obtain the query set enhanced image data, the query set original image data, the query set enhanced image data and the support image data set are input into the feature processing module to obtain domain-specific features and domain-independent features, domain labels are obtained according to the domain-specific features, and domain labels are input into the domain classifier to obtain domain classification scores, the domain-independent features are input into the small sample classification module to perform classification prediction, and then local feature optimization is performed according to the local feature contrast learning algorithm, which greatly improves the discrimination ability of the final small sample classification model for the target domain data. The present invention greatly improves the generalization ability of cross-domain small sample classification.

[0027] See also Figure 2 , which is a flow chart of a cross-domain small sample classification method based on local target data enhancement proposed in the second embodiment of the present invention, and the cross-domain small sample classification method based on local target data enhancement includes steps S11 to S16, wherein: Step S11: performing random spatial data enhancement processing according to the query image data set to obtain an enhanced query image data set, selecting a single image region in the source domain query image data in the query image data set, replacing the image region with a corresponding image region in the target domain query image data in the query image data set, cutting and mixing to generate the query set original image data, and then cutting and mixing the enhanced query image data set to generate the query set enhanced image data; Step S12: inputting the original image data of the query set, the enhanced image data of the query set and the supporting image data set into a feature processing module, the feature processing module comprising a twin feature extraction block and a feature decoupling block, the first feature extraction block of the twin feature extraction block extracts features from the original image data of the query set and the source domain support image data and the target domain support image data in the supporting image data set, and then inputs all the extracted image features into the feature decoupling block to decompose the image features into domain-specific features and domain-independent features, the second feature extraction block of the twin feature extraction block extracts features from the enhanced image data of the query set, and then inputs all the extracted image features into the feature decoupling block to decompose and obtain features that are irrelevant to the enhanced image of the query set, perform overfitting elimination according to the feature perturbation algorithm, obtain domain labels according to the domain-specific features, input the domain classifier to obtain domain classification scores, and then input the domain-independent features into the small sample classification module for classification prediction; Step S13: obtaining domain labels of specific features of the source domain supporting images and domain labels of specific features of the target domain supporting images, so as to calculate domain labels of specific features of original images of the query set, inputting all domain specific features and corresponding domain labels into a domain classifier, and calculating domain classification scores, so as to perform loss optimization according to a domain classification loss function; Step S14: inputting the image-independent features supported by the source domain, the image-independent features supported by the target domain, and the original image-independent features of the query set into the small sample classification module, respectively calculating the predicted distribution on the source domain and the predicted distribution on the target domain, and performing loss optimization according to the small sample classification loss function; Step S15: inputting the query set enhanced image-independent features into the small sample self-supervised learning block, the small sample self-supervised learning block performs small sample classification self-supervised enhancement according to the small sample classification labels of the original image of the query set, so as to make the semantic information of images in different regions of all pictures consistent, and performs loss optimization according to the small sample self-supervised learning loss function; Step S16: Locally slice the original image data of the query set and the enhanced image data of the query set to obtain the original local image of the query set and the enhanced local image of the query set, respectively extract the local image features of the original local image of the query set and the enhanced local image of the query set according to the twin feature extraction block in the feature feature processing module, and then decompose the local image features according to the feature decoupling block to obtain the corresponding domain-independent features, randomly select any original local image of the query set as an anchor point, and use all local feature samples of the same type of image in the enhanced local image of the query set as positive samples, calculate the distance between the anchor point and the local feature corresponding to each position of the positive sample according to the cosine distance, so as to compare the distribution distance between the anchor point and the local features in the positive sample in the feature space, and at the same time select all local feature samples of any heterogeneous image in the enhanced local image of the query set as negative samples, and compare the distribution distance between the local features in the anchor point and the negative sample in the feature space, so as to perform loss optimization according to the local feature comparison learning loss function; It should be noted that the cross-domain small sample classification model of the present invention is compared with the small sample classification model in the prior art, and the results are shown in Tables 1 and 2 below: Table 1 Table 2 Among them, Dataset 1, Dataset 2, Dataset 3, and Dataset 4 are four data sets, namely CUB, Cars, Places, and Places. Table 1 is set to 5-way 1-shot, and Table 2 is set to 5-way 5-shot. It can be seen from Tables 1 and 2 that the cross-domain small sample classification model of the present invention has greatly improved generalization ability compared with the existing small sample classification model, and has significant progress.

[0028] In summary, according to the above-mentioned cross-domain small sample classification method based on local target data enhancement, by designing a cutting and mixing method, local data enhancement is performed on the image data of the source domain and a small amount of image data in the target domain, so that the final small sample classification model can better adapt to the target domain, and then the two groups of query set images are cut and mixed by random data enhancement to obtain enhanced query images, and the relationship between the predictions of the enhanced images and the original images is analyzed, and the relationship between the local features of the two is compared using a contrastive learning method to improve the final small sample classification model's ability to discriminate the target domain data. The present invention greatly improves the generalization ability of cross-domain small sample classification. Specifically, a query image dataset and a support image dataset are obtained, wherein the query image dataset includes source domain query image data and target domain query image data, and the support image dataset includes source domain support image data and target domain support image data; random spatial data enhancement processing is performed on the query image dataset to obtain an enhanced query image dataset, wherein the enhanced query image dataset includes source domain enhanced query image data and target domain enhanced query image data, which greatly enriches the enhanced image of the same image without causing a large semantic difference between the enhanced image and the original image; the query image dataset and the enhanced query image dataset are input into a shearing and blending module, and according to the query image dataset Cutting and mixing are performed to obtain the original image data of the query set, cutting and mixing are performed according to the enhanced query image data set to obtain the query set enhanced image data, the query set original image data, the query set enhanced image data and the support image data set are input into the feature processing module to obtain domain-specific features and domain-independent features, domain labels are obtained according to the domain-specific features, and domain labels are input into the domain classifier to obtain domain classification scores, the domain-independent features are input into the small sample classification module to perform classification prediction, and then local feature optimization is performed according to the local feature contrast learning algorithm, which greatly improves the discrimination ability of the final small sample classification model for the target domain data. The present invention greatly improves the generalization ability of cross-domain small sample classification.

[0029] See also Figure 3 , which is a schematic diagram of the structure of a cross-domain small sample classification system based on local target data enhancement proposed in the third embodiment of the present invention, and the system includes: The image acquisition processing module 10 is used to acquire a query image dataset and a support image dataset, wherein the query image dataset includes source domain query image data and target domain query image data, and the support image dataset includes source domain support image data and target domain support image data, and perform random spatial data enhancement processing on the query image dataset to acquire an enhanced query image dataset, wherein the enhanced query image dataset includes source domain enhanced query image data and target domain enhanced query image data; A shearing and blending module 20 is used to input the query image dataset and the enhanced query image dataset into a shearing and blending module, perform shearing and blending according to the query image dataset to obtain query set original image data, and perform shearing and blending according to the enhanced query image dataset to obtain query set enhanced image data; A feature processing module 30 is used to input the query set original image data, the query set enhanced image data and the support image data set into a feature processing module to obtain domain-specific features and domain-independent features, obtain domain labels according to the domain-specific features, and input them into a domain classifier to obtain domain classification scores; The classification optimization module 40 is used to input the domain-independent features into the small sample classification module to perform classification prediction, and then perform local feature optimization according to the local feature contrast learning algorithm.

[0030] The present invention also proposes a computer storage medium on which one or more programs are stored, which, when executed by a processor, implement the above-mentioned cross-domain small sample classification method based on local target data enhancement.

[0031] The present invention also proposes a computer device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the above-mentioned cross-domain small sample classification method based on local target data enhancement.

[0032] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain storage, communication, propagation or transmission of a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0033] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0034] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0035] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0036] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A cross-domain small sample classification method based on local target data enhancement, characterized in that: include: Acquire a query image dataset and a support image dataset, wherein the query image dataset includes source domain query image data and target domain query image data, and the support image dataset includes source domain support image data and target domain support image data, and perform random spatial data enhancement processing on the query image dataset to acquire an enhanced query image dataset, wherein the enhanced query image dataset includes source domain enhanced query image data and target domain enhanced query image data; Inputting the query image dataset and the enhanced query image dataset into a shearing and blending module, performing shearing and blending according to the query image dataset to obtain query set original image data, and performing shearing and blending according to the enhanced query image dataset to obtain query set enhanced image data; Inputting the query set original image data, the query set enhanced image data and the supporting image data set into a feature processing module to obtain domain-specific features and domain-independent features, obtaining domain labels according to the domain-specific features, and inputting them into a domain classifier to obtain a domain classification score; The domain-independent features are input into a small sample classification module for classification prediction, and then local feature optimization is performed according to a local feature contrast learning algorithm.

2. The cross-domain small sample classification method based on local target data enhancement according to claim 1 is characterized in that: The step of performing shearing and mixing according to the query image data set to obtain the query set original image data, and performing shearing and mixing according to the enhanced query image data set to obtain the query set enhanced image data specifically includes: Performing random spatial data enhancement processing on the query image dataset, wherein the random spatial data enhancement processing includes flipping or inverting the image in the query image dataset as a whole or internally to obtain an enhanced query image dataset; A single image region in the source domain query image data in the query image dataset is selected, and the image region is replaced with a corresponding image region in the target domain query image data in the query image dataset to generate the original image data of the query set by cutting and mixing. The formula for generating the original image data of the query set by cutting and mixing is as follows: in, Represents the original image data of the query set, represents the side length ratio of the image, represents the source domain query image data, represents the target domain query image data, Represents the label data of the original image data of the query set, Represents the label data of the source domain query image data, Label data representing query image data in the target domain; The enhanced query image dataset is then cut and mixed to generate query set enhanced image data.

3. The cross-domain small sample classification method based on local target data enhancement according to claim 1 is characterized in that: The step of inputting the query set original image data, the query set enhanced image data and the supporting image data set into a feature processing module to obtain domain-specific features and domain-independent features specifically includes: Inputting the query set original image data, the query set enhanced image data and the support image data set into a feature processing module, wherein the feature processing module includes a twin feature extraction block and a feature decoupling block; The first feature extraction block of the twin feature extraction block extracts features from the original image data of the query set and the source domain support image data and the target domain support image data in the support image data set, and then inputs all the extracted image features into a feature decoupling block to decompose the image features into domain-specific features and domain-independent features, wherein the domain-specific features include source domain support image-specific features, target domain support image-specific features, and query set original image-specific features, and the domain-independent features include source domain support image-independent features, target domain support image-independent features, and query set original image-independent features; The second feature extraction block of the twin feature extraction block extracts features from the query set enhanced image data, and then inputs all extracted image features into the feature decoupling block to decompose and obtain irrelevant features of the query set enhanced image, and performs overfitting elimination according to the feature perturbation algorithm, and the feature perturbation algorithm is as follows: in, represents the irrelevant features of the original image in the query set, represents the query set to enhance the image-independent features, represents a feature disturbance term, wherein the feature disturbance term includes noise obtained by sampling the query set enhanced image-irrelevant features from a normal distribution; A domain label is obtained according to the domain-specific feature, and a domain label is input into a domain classifier to obtain a domain classification score; The domain-independent features are then input into a small sample classification module for classification prediction.

4. The cross-domain small sample classification method based on local target data enhancement according to claim 3 is characterized in that: The step of obtaining a domain label according to the domain-specific feature and inputting the domain label into a domain classifier to obtain a domain classification score specifically includes: Obtain domain labels that support image-specific features in the source domain and the domain label of the target domain supporting image-specific features , to calculate the domain label of the specific features of the original image of the query set ,in Indicates the side length ratio of the image; Input all domain-specific features and corresponding domain labels into the domain classifier and calculate the domain classification score; The loss optimization is performed according to the domain classification loss function, and the domain classification loss function is as follows: in, represents the domain classification loss, represents the cross entropy loss function, indicates that the source domain supports image-specific features, indicates that the target domain supports image-specific features, Represents the specific features of the original image in the query set, Represents a domain classifier.

5. The cross-domain small sample classification method based on local target data enhancement according to claim 3 is characterized in that: The step of inputting the domain-independent features into a small sample classification module to perform classification prediction specifically includes: Input the image-independent features supported by the source domain, the image-independent features supported by the target domain, and the original image-independent features of the query set into the small sample classification module; Performing small sample classification on the original image-irrelevant features of the query set, the image-irrelevant features supported by the source domain, and the image-irrelevant features supported by the target domain; The predicted distribution on the source domain and the predicted distribution on the target domain are calculated respectively, and the loss is optimized according to the small sample classification loss function, which is as follows: in, represents the small sample classification loss, represents the side length ratio of the image, represents the cross entropy loss function, represents the predicted distribution of the original image-independent features of the query set in the source domain, represents the predicted distribution of the query set’s original image-independent features in the target domain, represents the few-shot classification labels of the source domain support images, Represents the few-shot classification labels for the target domain support images.

6. The cross-domain small sample classification method based on local target data enhancement according to claim 5 is characterized in that: The small sample classification module also includes: The small sample classification module also includes a small sample self-supervised learning block, which inputs the query set enhanced image-independent features into the small sample self-supervised learning block. The small sample self-supervised learning block performs small sample classification self-supervised enhancement according to the small sample classification labels of the original image of the query set to make the semantic information of images in different regions of all pictures consistent, and performs loss optimization according to the small sample self-supervised learning loss function. The small sample self-supervised learning loss function is as follows: in, represents the small sample self-supervised learning loss, represents the predicted distribution of query set augmented image-independent features in the source domain, Represents the predicted distribution of query set augmented image-independent features on the target domain.

7. The cross-domain small sample classification method based on local target data enhancement according to claim 1 is characterized in that: The step of optimizing the local features according to the local feature contrast learning algorithm specifically includes: Locally slicing the query set original image data and the query set enhanced image data to obtain the query set original local image and the query set enhanced local image; Extracting local image features of the original local image of the query set and the enhanced local image of the query set according to the twin feature extraction block in the feature feature processing module, and then decomposing the local image features according to the feature decoupling block to obtain corresponding domain-independent features; Randomly select any original local image of the query set as an anchor point, and use all local feature samples of the same type of image in the query set enhanced local image as positive samples, calculate the distance between the anchor point and the local feature corresponding to each position of the positive sample according to the cosine distance, so as to compare the distribution distance between the anchor point and the local features in the positive sample in the feature space, and at the same time select all local feature samples of any heterogeneous image in the query set enhanced local image as negative samples, and compare the distribution distance between the anchor point and the local features in the negative sample in the feature space; The loss optimization is performed according to the local feature contrast learning loss function, and the local feature contrast learning loss function is as follows: in, represents the local feature contrast learning loss, represents the number of local features, represents the local feature ordinal number, represents the cosine distance, represents the local features of the anchor point, represents the local features of the positive sample, represents the local features of negative samples, represents the boundary parameter, represents the temperature parameter, Represents the maximum value function.

8. A cross-domain small sample classification system based on local target data enhancement, characterized in that: include: An image acquisition processing module is used to acquire a query image dataset and a support image dataset, wherein the query image dataset includes source domain query image data and target domain query image data, and the support image dataset includes source domain support image data and target domain support image data, and perform random spatial data enhancement processing on the query image dataset to acquire an enhanced query image dataset, wherein the enhanced query image dataset includes source domain enhanced query image data and target domain enhanced query image data; a shearing and blending module, configured to input the query image dataset and the enhanced query image dataset into the shearing and blending module, perform shearing and blending according to the query image dataset to obtain query set original image data, and perform shearing and blending according to the enhanced query image dataset to obtain query set enhanced image data; a feature processing module, configured to input the query set original image data, the query set enhanced image data and the support image data set into the feature processing module to obtain domain-specific features and domain-independent features, obtain domain labels according to the domain-specific features, and input them into a domain classifier to obtain domain classification scores; The classification optimization module is used to input the domain-independent features into the small sample classification module to perform classification prediction, and then perform local feature optimization according to the local feature contrast learning algorithm.

9. A storage medium, characterized in that: The storage medium stores one or more programs, which, when executed by the processor, implement the cross-domain small sample classification method based on local target data enhancement as described in any one of claims 1-7.

10. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the cross-domain small sample classification method based on local target data enhancement as described in any one of claims 1-7.

Images