A small sample recognition method by introducing noise information and constructing similarity space

Abstract

The present invention relates to a small sample recognition method that introduces noise information and constructs a similarity space, and belongs to the field of digital image processing. The method comprises the following steps: a pre-training stage: a noise interference stage; a new support set is generated; a similarity space construction stage: the similarity between the feature center support set of each class in the basic data set and the feature center of each class in the query set is calculated respectively. The similarity is used to replace the features of the original support set and the query set to form a new feature representation in the similarity space; a classification stage: the results of the first two stages are fused through a fully connected layer, and then the cosine similarity is used for classification to obtain the results. The present invention introduces noise information on the basis of traditional image features and constructs a similarity space. When applied to small sample image recognition tasks, it can better increase the differences between image categories. At the same time, compared with other methods, our method has a smaller amount of calculation and a faster calculation speed.

Description

Technical Field

[0001] The invention belongs to the field of digital image processing and relates to a small sample recognition method that introduces noise information and constructs a similarity space. Background Art

[0002] Deep learning is a data-driven learning paradigm that has achieved state-of-the-art performance in visual recognition tasks such as image classification. However, deep learning requires large amounts of data for training. In many real-world applications, large amounts of labeled data are difficult to obtain. This has sparked interest in few-shot learning.

[0003] More and more methods have been proposed for few-shot learning. Most of these methods are based on the three methods proposed by Li Wenbin et al., namely, metric-based, meta-learning-based, and pre-training-based methods. In recent years, meta-learning has become a popular framework for few-shot learning. It involves pre-training a classifier to learn feature representations that are transferred to the support set and the query set. Subsequently, the model is transferred to learn new classes based on a small sample size. Under the framework of meta-learning, the model is directly optimized to ensure that it performs effectively in multiple recognition tasks. However, due to the lack of labeled data, traditional deep learning methods find it difficult to obtain accurate indicators. Regions of the feature space will be disturbed. The less labeled data, the weaker the network's ability to resist interference. Summary of the Invention

[0004] In view of this, an object of the present invention is to provide a small sample recognition method that introduces noise information and constructs a similarity space.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A small sample recognition method for introducing noise information and constructing a similarity space, the method comprising the following steps:

[0007] S1: Pre-training stage: Use the traditional backbone network to train the feature classifier and migrate it to the small sample recognition task;

[0008] S2: Noise interference stage: merge the feature vectors of the support set and the query set to generate a new support set;

[0009] S3: Similarity space construction phase: Calculate the similarity between the feature center support set of each class in the basic dataset and the feature center of each class in the query set. This similarity is used to replace the features of the original support set and query set to form a new feature representation in the similarity space.

[0010] S4: Discrimination result integration stage: The results of the first two stages are integrated through the fully connected layer, and then the cosine similarity is used for classification to obtain the results.

[0011] Optionally, step S1 includes the following steps:

[0012] S11: Use the traditional ResNet12 as the backbone network to train the feature extractor;

[0013] S12: Remove the last layer of the fully connected network and migrate to the small sample task.

[0014] Optionally, step S2 includes the following steps:

[0015] S21: Calculate the feature center point of each category in the support set as the feature of the category;

[0016] S22: Fuse the features of the query set with the feature centers of each category in the support set to form a new support set;

[0017] S23: The cosine similarity between the newly generated support set and the query set image is calculated, and the results are classified using softmax.

[0018] Optionally, step S3 includes the following steps:

[0019] S31: Calculate the similarity between the feature center support set of each class in the basic dataset and the feature center of each class in the query set;

[0020] S32: This similarity is used to replace the features of the original support set and query set, forming a new support set and query set in the similarity space.

[0021] S33: Calculate the cosine similarity between the newly generated support set and the query set image, and use softmax to classify the results;

[0022] Optionally, step S4 includes the following steps:

[0023] S41: Use the normalization function to normalize the results of S23 and S33 to the same dimension.

[0024] S42: Use a fully connected network to fuse the two results obtained in S41 into the final result.

[0025] The beneficial effect of the present invention is that: the present invention utilizes the characteristic of noise interference and increases the inter-class distance between different image categories by artificially introducing noise information, so that the discriminator can more easily distinguish the image category, and establishes a new similarity space to increase the discriminant information.

[0026] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0028] Figure 1 Schematic diagram of the process of the present invention;

[0029] Figure 2 Schematic diagram of the structure of constructing similarity space in the method of the present invention;

[0030] Figure 3 Schematic diagram of the structure of the similarity space in the method of the present invention. DETAILED DESCRIPTION

[0031] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0032] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0033] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0034] See also Figures 1 to 3 , the process of the present invention comprises the following steps:

[0035] Step 1: Pre-training stage: Use the traditional backbone network to train the feature classifier f(θ) and migrate it to the small sample recognition task.

[0036] Step 101: Use the traditional ResNet12 as the backbone network to train the feature extractor f(θ).

[0037] Step 102: Remove the last layer of the fully connected network and migrate to the small sample task to obtain the feature extractor f(μ)

[0038] Step 2: Fuse the feature vectors of the support set and the query set to generate a new support set.

[0039] Step 201: Calculate the feature center point of each category in the support set as the feature of the category. The calculation process is expressed as follows:

[0040]

[0041] where w c is the feature center point of the current category, s c is the number of images in the current category.

[0042] Step 202: Fuse the features of the query set with the feature centers of each category in the support set to form a new support set. The calculation process is expressed as follows:

[0043]

[0044] where w nc is the feature center point of the current category, f μ (x Q ) Characteristics of the query set

[0045] Step 203: Calculate the cosine similarity between the newly generated support set and the query set image, and use softmax to classify the results.

[0046] Step 3: Constructing the Similarity Space: Calculate the similarity between the feature center support set of each class in the base dataset and the feature center of each class in the query set. This similarity is used to replace the features of the original support set and query set to form a new feature representation in the similarity space.

[0047] Step 301: Calculate the similarity between the feature center support set of each class in the basic data set and the feature center of each class in the query set. The calculation process is expressed as follows:

[0048]

[0049]

[0050] in is the center of the current support set category in the similarity space, is the center of the current query set category in the similarity space. cos(,) represents the cosine similarity.

[0051] Step 302: The similarity is used to replace the features of the original support set and query set, forming a new support set and query set in the similarity space.

[0052]

[0053] represents the new support set in the feature space, Represents a new query set in the feature space.

[0054] Step 303: Calculate the cosine similarity between the newly generated support set and the query set image, and classify the results using softmax.

[0055] Step 4: Discrimination result integration stage: The results of the first two stages are integrated through the fully connected layer, and then the cosine similarity is used for classification to obtain the results.

[0056] Step 401: Use a normalization function to normalize the results of S23 and S33 to the same dimension.

[0057] Step 402: Use a fully connected network to fuse the two results obtained in S41 into a final result.

[0058] In order to verify the effect of the present invention, the following experiments were carried out:

[0059] This small-shot recognition method, based on noise perturbation and the construction of a similarity space, was tested on the miniImageNet and tieredImageNet datasets. The two experimental settings used were: a support set with five categories, one image per category, and a support set with five categories, five images per category. The goal was to accurately classify the query set into its corresponding category.

[0060] Table 1 shows the results of the test on the database. It can be seen that when the support set has 5 categories and 1 image in each category, we have the best results. When the support set has 5 categories and 5 images in each category, we have the best effect in the field of meta-learning, and the effect is also very outstanding among all methods.

[0061] Table 1 Database test results (%)

[0062]

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A small sample recognition method that introduces noise information and constructs a similarity space, characterized by: The method comprises the following steps: S1: Pre-training stage: Use the traditional backbone network to train the feature classifier and migrate it to the small sample recognition task; S2: Noise interference stage: Fuse the feature vectors of the support set image feature set with the feature vectors of the query set image feature set to generate a new support set; S3: Constructing the similarity space: Calculate the similarity between the feature centers of the support set image feature set and the feature centers of the query set image feature set for each class in the basic dataset. This similarity is used to replace the features of the original support set image feature set and the query set image feature set to form a new feature representation in the similarity space. S4: Discrimination result integration stage: The results of the first two stages are integrated through the fully connected layer, and then the cosine similarity is used for classification to obtain the results.

2. The small sample recognition method for introducing noise information and constructing a similarity space according to claim 1, characterized in that: The step S1 comprises the following steps: S11: Use the traditional ResNet12 as the backbone network to train the feature extractor; S12: Remove the last layer of the fully connected network and migrate to the small sample task.

3. The small sample recognition method for introducing noise information and constructing a similarity space according to claim 2, characterized in that: The step S2 comprises the following steps: S21: Calculate the feature center point of each category in the support set as the feature of the category; S22: Fuse the features of the query set with the feature centers of each category in the support set to form a new support set; S23: The cosine similarity between the newly generated support set and the query set image is calculated, and the results are classified using softmax.

4. The small sample recognition method for introducing noise information and constructing a similarity space according to claim 3, characterized in that: The step S3 comprises the following steps: S31: Calculate the similarity between the feature center of each class in the base dataset and the feature center of each class in the support set and query set; S32: This similarity is used to replace the features of the original support set and query set, forming a new support set and query set in the similarity space; S33: The cosine similarity between the newly generated support set and the query set image is calculated, and the results are classified using softmax.

5. The small sample recognition method of introducing noise information and constructing a similarity space according to claim 4, characterized in that: The step S4 comprises the following steps: S41: Use the normalization function to normalize the results of S23 and S33 to the same dimension; S42: Use a fully connected network to fuse the two results obtained in S41 into the final result.

Images