Small Sample Classification Method of Cultural Relics Images Based on Automatic Search Metric Function

Abstract

The present invention discloses a small sample classification algorithm for cultural relic images based on automatic search of metric functions. The features of cultural relic images at different scales are extracted by designing a convolution embedding module based on multi-scale feature extraction, and then a metric function search module based on automatic machine learning is further designed to search for the metric function that best suits the current task, and a two-layer optimization method is used to optimize the search process, and the optimization process is optimized using a gradient descent method. Experiments were conducted on the miniImagenet dataset, and the experimental results showed that compared with other small sample classification algorithms for cultural relic images, the network model composed of the metric functions searched by the method of the present invention achieved the best results, effectively improving the classification performance of the network. The method of the present invention searches for the most suitable metric function according to different small sample classification tasks of cultural relic images, thereby better achieving the purpose of closer sample distances within a class and farther sample distances between classes, and improving the classification accuracy of the network model.

Description

Technical Field

[0001] The present invention relates to the field of cultural relic image classification in computer vision, and more specifically, to a cultural relic image small sample classification method based on automatic search metric function. Background Art

[0002] In recent years, deep neural networks have made breakthrough progress in the task of cultural relic image classification. However, due to the large number of parameters, traditional deep learning cultural relic image classification methods usually require a large number of labeled sample images for training and testing. However, due to the limitations of the real environment, many types of cultural relic images can only obtain one or a few image samples, and it is difficult to obtain a large number of labeled cultural relic images. These reasons have greatly weakened the robustness of deep learning cultural relic image classification methods. Therefore, cultural relic image small sample classification came into being.

[0003] Small sample image classification is a learning mechanism that extracts and abstracts new concepts from very few samples. It is specifically designed to deal with problems that traditional deep learning methods cannot handle when the number of samples is insufficient. Small sample learning means achieving truly super learning and generalization capabilities, and being able to draw inferences from one example even when the number of samples is insufficient, opening up a new world for the fields of deep learning and artificial intelligence. The small sample classification algorithm for cultural relic images can effectively solve the problem of poor robustness of traditional deep learning models caused by insufficient cultural relic images.

[0004] Traditional cultural relic image classification algorithms are not fully trained when there are few cultural relic image samples, and their robustness is poor. Although the existing cultural relic image small sample classification algorithms can alleviate this problem to a certain extent, the model uses a fixed metric function, which is difficult to meet the needs of different cultural relic image classification tasks and has poor applicability to different cultural relic image classification tasks. Summary of the invention

[0005] In order to solve the problems of the prior art, the present invention provides a small sample classification algorithm for cultural relic images based on automatic search of metric functions, which solves the problem that the existing small sample classification method for cultural relic images uses a fixed metric function and cannot meet the requirements of different cultural relic image classification tasks.

[0006] The present invention proposes the following technical solutions:

[0007] A small sample classification method for cultural relic images based on automatic search of metric functions, comprising a convolution embedding module based on multi-scale feature extraction and a metric function search module based on automatic machine learning;

[0008] The convolution embedding module based on multi-scale feature extraction is composed of multiple convolution blocks, each of which includes a convolution layer, a batch normalization layer and a leaky ReLU layer; in addition, an additional 2×2 max-pooling layer is added between every two convolution blocks; first, the convolution embedding module is pre-trained to have a certain feature extraction capability; then, the cultural relic image dataset is divided into multiple small sample task datasets, each of which includes a support set and a query set: the support set includes 5 cultural relic categories, each category includes 1 or 5 cultural relic images; the query set includes 5 cultural relic categories in the support set, each category has 15 cultural relic images, and is not repeated with the cultural relic images in the support set; the cultural relic images in the support set and the query set are input into the convolution embedding module, and the features of the corresponding scale of the cultural relic image are extracted from each max-pooling layer; the shallow network outputs larger scale image features, which contain low-level visual information; the deep network outputs smaller scale image features, which contain high-level semantic information;

[0009] The metric function search module based on automatic machine learning searches for the most suitable metric function for the current small sample task to improve the classification performance of the model:

[0010] Firstly, based on the structural characteristics of the metric function, a suitable metric function is constructed by finding two parts: normalization operation and distance measurement operation;

[0011] Then, according to the content to be searched, a metric function search space is designed. For the features of each scale of the cultural relic image, a standardization operation and a distance measurement operation are searched to form the metric function of the scale feature, and the metric functions corresponding to the features of different scales are assigned weights to form the total metric function.

[0012] The normalization operation is to translate and scale the features of the sample and normalize the distribution of the features to eliminate the influence of scale differences between the features, thereby adapting to different distance measurement operations; the formula for the normalization operation is as follows:

[0013]

[0014] in, represents the features that need to be normalized, μ, σ represent the shift parameters and scaling parameters; the normalization search layer consists of four candidate operations: no normalization, channel normalization, global normalization, and individual normalization.

[0015] The distance measurement operation calculates the distance between the support set samples and the query set samples according to their features; four excellent measurement functions are selected and put into the distance measurement search layer: inner product, Euclidean distance, Manhattan distance and Mahalanobis distance; the inner product is the result of flattening two feature matrices, multiplying their corresponding elements, and then adding them. The specific formula is as follows:

[0016]

[0017] Among them, x and y are both n-dimensional vectors; the Euclidean distance can be interpreted as the length of the line segment connecting two points. The specific formula is as follows:

[0018]

[0019] Among them, x and y are both n-dimensional vectors; when the data dimension is low, the distance measurement effect of Euclidean distance is better; Manhattan distance represents the sum of the absolute axis distances of two vectors in the standard coordinate system, and the specific formula is as follows:

[0020]

[0021] Among them, x and y are both n-dimensional vectors; Manhattan distance is very sensitive to outliers in the data, and the numerical differences in different dimensions have the same impact on the results; Mahalanobis distance represents the covariance distance of the data, and the specific formula is as follows:

[0022]

[0023] Where x is the feature matrix and S is the covariance matrix with the same distribution as y.

[0024] In the metric function search stage, a two-layer optimization strategy is used to train the network, and the optimization process is optimized using the gradient descent method.

[0025] Beneficial effects:

[0026] The small sample classification algorithm of cultural relic images based on automatic search of metric functions proposed in the present invention extracts features of cultural relic images at different scales by designing a convolution embedding module based on multi-scale feature extraction, and then further designs a metric function search module based on automatic machine learning to search for the metric function that best suits the current task, and optimizes the search process using a two-layer optimization method, which is optimized using the gradient descent method. Experiments were conducted on the miniImagenet dataset, and the experimental results show that compared with other small sample classification algorithms for cultural relic images, the network model composed of the metric functions searched by the method of the present invention achieved the best results, effectively improving the classification performance of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1The network structure diagram of the small sample classification algorithm of cultural relics images based on automatic search metric function; DETAILED DESCRIPTION

[0028] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0029] The method proposed in the present invention mainly includes two parts: a convolution embedding module based on multi-scale feature extraction and a metric function search module based on automatic machine learning. The convolution embedding module based on multi-scale feature extraction can extract features of cultural relic images at different scales, so that the network can learn more information about cultural relic images. The metric function search module based on automatic machine learning can search for the metric function that best suits the current task, so that the network can better measure the distance between different cultural relic images. For a specific network structure diagram, see Figure 1 , the details are as follows:

[0030] 1. Convolutional embedding module based on multi-scale feature extraction

[0031] In the existing small sample classification algorithms for cultural relic images, the feature extraction module usually only outputs image features of a single scale, which will lose the effective information in features of other scales, resulting in poor model classification results. Therefore, we designed a convolutional embedding module that outputs features of different scales to extract features of cultural relic images at different scales, so that the model can make full use of the effective information in features of each scale to achieve better classification results.

[0032] The convolutional embedding module based on multi-scale feature extraction consists of multiple convolutional blocks, each of which includes a convolutional layer (with 64 filters of size 3×3), a batch normalization layer, and a Leaky ReLU layer. In addition, an additional 2×2 max-pooling layer is added between every two convolutional blocks.

[0033] First, the convolutional embedding module is pre-trained to enable it to have a certain feature extraction capability. Then, the cultural relic image dataset is divided into multiple small sample task datasets, each of which includes a support set and a query set: the support set includes 5 cultural relic categories, each category includes 1 or 5 cultural relic images; the query set includes 5 cultural relic categories in the support set, 15 cultural relic images in each category, and there is no duplication with the cultural relic images in the support set. The cultural relic images in the support set and the query set are input into the convolutional embedding module, and the features of the corresponding scale of the cultural relic image are extracted from each max-pooling layer. The shallow network outputs larger-scale image features, which contain low-level visual information; the deep network outputs smaller-scale image features, which contain high-level semantic information. Features of different scales enable the entire model to learn more useful information.

[0034] 2. Metric function search module based on automatic machine learning

[0035] Through the convolution embedding module based on multi-scale feature extraction, the network model can extract features of cultural relic images at different scales, so as to better learn the effective information in the cultural relic images. This algorithm takes into account the different data distributions in small sample tasks of different cultural relic images, and the required distance metric functions are also different. Therefore, this paper designs a metric function search module based on automatic machine learning to search for the most suitable metric function for the current small sample task to improve the classification performance of the model.

[0036] First, based on the structural characteristics of the metric function, we construct a suitable metric function by finding two parts: normalization operation and distance measurement operation.

[0037] The normalization operation is to translate and scale the features of the sample and normalize the distribution of the features to eliminate the impact of scale differences between features, so as to adapt to different distance measurement operations. The formula for the normalization operation is as follows:

[0038]

[0039] in, Represents the features that need to be normalized, where d×M represents M one-dimensional vectors of length d, and μ,σ represent shift parameters and scaling parameters. The normalization search layer consists of four candidate operations: no normalization, channel normalization, global normalization, and individual normalization.

[0040] The distance metric operation can calculate the distance between the support set samples and the query set samples based on their features. This paper selects four excellent metric functions to be put into the distance metric search layer: inner product, Euclidean distance, Manhattan distance, and Mahalanobis distance. The inner product is the result of flattening two feature matrices, multiplying their corresponding elements, and then adding them. The specific formula is as follows:

[0041]

[0042] Where x and y are both n-dimensional vectors. The Euclidean distance can be interpreted as the length of the line segment connecting two points. The specific formula is as follows:

[0043]

[0044] Where x and y are both n-dimensional vectors. When the data dimension is low, the distance measurement effect of Euclidean distance is better. Manhattan distance represents the sum of the absolute axis distances of two vectors in the standard coordinate system. The specific formula is as follows:

[0045]

[0046] Where x and y are both n-dimensional vectors. Manhattan distance is very sensitive to outliers in the data, and numerical differences in different dimensions have the same impact on the results. Mahalanobis distance represents the covariance distance of the data, and the specific formula is as follows:

[0047]

[0048] Where x is the feature matrix and S is the covariance matrix with the same distribution as y.

[0049] According to the above contents to be searched, the present invention designs a metric function search space, searches for a standardization operation and a distance measurement operation for the features of each scale of the cultural relic image to form a metric function of the scale feature, and assigns appropriate weights to the metric functions corresponding to the features of different scales, and weights them to form a total metric function. According to different cultural relic image small sample classification tasks, the most suitable metric function is searched, so as to better achieve the purpose of closer distance between samples within a class and farther distance between samples between classes, and improve the classification accuracy of the network model.

[0050] The present invention uses the miniImagenet dataset as a dataset for testing algorithm performance. The miniImagenet dataset is used to search for the best metric function, and the classification accuracy of the model composed of the searched metric function on the miniImagenet dataset is tested on the small sample task. The whole experiment is divided into three stages: the metric function search stage, the model training stage, and the model testing stage. For the 5-way 1-shot classification task, each small sample task dataset includes 5 categories, each category has 15 query set images and 1 support set image, that is, each small sample task dataset has 5×15+5×1=80 images. For the 5-way 5-shot classification task, each small sample task dataset has 5×15+5×5=100 images.

[0051] In the metric function search phase, a two-layer optimization strategy is used to train the network. In the experiment, the metric function search phase includes 30 rounds, each with 2500 iterations. Each iteration is divided into two processes, namely, optimizing the operation parameters and optimizing the network weights. The initial learning rate used in the search process is 1×10 -3 The model is trained using the Adam optimizer with the learning rate halved after every 10 epochs.

[0052] In the model training phase, the searched model is trained using the small sample task training mechanism. In the experiment, the model training phase consists of 40 rounds, each with 10,000 iterations, and the learning rate is the same as the setting in the metric function search phase. In the model testing phase, the top-1 average classification accuracy and corresponding confidence intervals of 3,000 small sample classification tasks are calculated, which are randomly constructed from the test set.

[0053] The performance of the proposed algorithm in the classification of small samples of cultural relics images is verified on the miniImagenet dataset. The specific experimental results can be found in Table 1. From the experimental results, it can be seen that compared with the existing small sample classification algorithms based on deep learning, the proposed algorithm effectively improves the small sample classification effect of the network.

[0054] Table 1

[0055]

[0056] The technical solutions disclosed and proposed by the present invention can be realized by those skilled in the art by referring to the contents of this article and appropriately changing the conditions, routes and other links. Although the methods and preparation techniques of the present invention have been described through preferred embodiments, relevant technicians can obviously modify or re-combine the methods and technical routes described herein without departing from the content, spirit and scope of the present invention to achieve the final preparation technology. It is particularly important to point out that all similar substitutions and modifications are obvious to those skilled in the art, and they are all considered to be included in the spirit, scope and content of the present invention. Matters not covered in the present invention belong to the known technology.

Claims

1. A small sample classification method for cultural relics images based on automatic search metric function, characterized in that: It includes a convolutional embedding module based on multi-scale feature extraction and a metric function search module based on automatic machine learning; The convolutional embedding module based on multi-scale feature extraction is composed of multiple convolutional blocks, each of which includes a convolutional layer, a batch normalization layer and a Leaky ReLU layer; in addition, an additional 2×2 max-pooling layer is added between every two convolutional blocks; first, the convolutional embedding module is pre-trained to enable it to have a certain feature extraction capability; Then, the cultural relic image dataset is divided into multiple small sample task datasets, each of which includes a support set and a query set: the support set includes 5 cultural relic categories, and each category includes 1 or 5 cultural relic images; The query set includes 5 cultural relics categories in the support set, 15 cultural relics images in each category, and there is no duplication with the cultural relics images in the support set; the cultural relics images in the support set and the query set are input into the convolution embedding module, and the features of the corresponding scale of the cultural relics images are extracted from each max-pooling layer; the shallow network outputs larger-scale image features, which contain low-level visual information; the deep network outputs smaller-scale image features, which contain high-level semantic information; The metric function search module based on automatic machine learning searches for the most suitable metric function for the current small sample task to improve the classification performance of the model: Firstly, based on the structural characteristics of the metric function, a suitable metric function is constructed by finding two parts: normalization operation and distance measurement operation; Then, according to the content to be searched, a metric function search space is designed. A standardization operation and a distance measurement operation are searched for the features of each scale of the cultural relic image to form a metric function of the scale feature. The metric functions corresponding to the features of different scales are given weights to form a total metric function. The normalization operation is to translate and scale the features of the sample and normalize the distribution of the features to eliminate the influence of scale differences between the features, thereby adapting to different distance measurement operations; the formula for the normalization operation is as follows: in, represents the features that need to be normalized, μ,σ represent the shift parameters and scaling parameters; the normalization search layer consists of four candidate operations: no normalization, channel normalization, global normalization, and individual normalization; The distance measurement operation calculates the distance between the support set samples and the query set samples according to their features; four excellent measurement functions are selected and put into the distance measurement search layer: inner product, Euclidean distance, Manhattan distance and Mahalanobis distance; the inner product is the result of flattening two feature matrices, multiplying their corresponding elements, and then adding them. The specific formula is as follows: Among them, x and y are both n-dimensional vectors; the Euclidean distance can be interpreted as the length of the line segment connecting two points. The specific formula is as follows: Among them, x and y are both n-dimensional vectors; when the data dimension is low, the distance measurement effect of Euclidean distance is better; Manhattan distance represents the sum of the absolute axis distances of two vectors in the standard coordinate system, and the specific formula is as follows: Among them, x and y are both n-dimensional vectors; Manhattan distance is very sensitive to outliers in the data, and the numerical differences in different dimensions have the same impact on the results; Mahalanobis distance represents the covariance distance of the data, and the specific formula is as follows: Where x is the feature matrix and S is the covariance matrix with the same distribution as y.

2. The method for classifying small samples of cultural relics images based on automatic search metric function according to claim 1, characterized in that: In the metric function search stage, a two-layer optimization strategy is used to train the network, and the optimization process is optimized using the gradient descent method.

Images