Image feature similarity calculation method and system based on similar feature attention

Abstract

The invention provides an image feature similarity calculation method and system based on similar feature attention, and belongs to the field of feature classification. The method aims at solving the problems that existing similarity calculation is not suitable for the condition of few samples such as bearing surface defect classification, background information interference exists, and a calculation result is affected. According to the method, the correlation between a support set and a query set is utilized, a similar feature attention module is designed, a common similar feature region between a class prototype center of the support set and the query set is obtained, similarity scores between the support set and the query set are compared, and interference of background information is avoided; designing an adaptive measurement network, performing category prediction on similar features of a query set, performing category prediction on similarity scores of the similar features of the query set and similar features of each category prototype, finally calculating category loss of the two, and determining a similar feature region between the category prototype and the query set through similarity calculation between the category prototype and the query set. And compared with a fixed measurement distance function, the method has better generalization.

Description

Technical Field

[0001] The present invention relates to the technical field of feature classification, and in particular to an image feature similarity calculation method and system based on similar feature attention. Background Art

[0002] Bearings are a basic component used in large quantities in mechanical equipment. The quality of bearings will seriously affect the stability of equipment operation. In recent years, with the vigorous development of my country's manufacturing industry, the demand for bearing products in various industries has continued to increase, and the requirements for bearing quality have also increased. Although my country's mechanical processing technology has reached a relatively high level, it is inevitable that certain damage will occur in the mass production of bearings. Therefore, bearings must undergo strict testing before leaving the factory to ensure that the bearings can be used normally after leaving the factory. At present, domestic bearing manufacturers have basically realized automated production and assembly line assembly, but the surface quality inspection after bearing assembly still relies on human visual inspection. At present, the research on intelligent detection algorithms for bearing surface defects has gradually received attention, and more and more deep learning algorithms have begun to be applied in the field of bearing surface defect detection.

[0003] With the continuous development of big data and artificial intelligence technology, similarity calculation methods are also constantly innovating and improving. From traditional methods such as Euclidean distance and cosine similarity to similarity calculation methods based on deep learning, the existing similarity calculation is based on a large amount of data and is not suitable for the classification of bearing surface defects with a small number of samples. In addition, when performing similarity calculation, there is also background information interference, which affects the calculation results. Summary of the invention

[0004] The technical problems to be solved by the present invention are:

[0005] In order to solve the problem that the existing similarity calculation is not suitable for the situation of few samples such as bearing surface defect classification, and there is background information interference, which affects the calculation results.

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] The present invention provides an image feature similarity calculation method based on similar feature attention, comprising the following steps:

[0008] S100, similarity calculation, by calculating the cosine distance between the central feature of the support set class prototype and the query set feature, to obtain the correlation score between different feature vectors, wherein the support set and the query set are both bearing surface defect image datasets;

[0009] S200, designing a similar feature attention layer, using the correlation between the support set and the query set, designing a similar feature attention module, obtaining the common similar feature area between the class prototype center of the support set and the query set, and comparing the similarity scores between the two;

[0010] S300, designing an adaptive metric network, including predicting categories for similar features of the query set, and predicting categories for similarity scores between similar features of the query set and similar features of each class prototype, and then calculating the prediction loss of similar features of the class prototype and similar features of the query set and the category prediction loss to update the network, and completing classification according to the similarity metric.

[0011] Furthermore, in step S100, the similarity score is as shown in formula (1):

[0012]

[0013] Among them, Z ij represents the cosine similarity between two feature vectors, i and j represent the local feature vector of the class prototype center and the local feature vector of the query set, and both have H×W local feature vectors; s i Represents the i-th local eigenvector of the class prototype center; q j Represents the jth local feature vector in the query set features;

[0014] Use cosine similarity calculation to find each local vector s of the class prototype i With each local vector q in the query set feature j The cosine similarity between them is used to obtain the similarity matrix Z of the prototype of this class with respect to the query set. s Similarly, the transposed matrix of the query set features is used to perform cosine calculations with the class prototype center feature representation to obtain the similarity matrix Z of the query set with respect to each class prototype center. q ,in, And M=H×W.

[0015] Further, in step S200, it includes:

[0016] S210, similarity matrix Z for class prototypes s A global average pooling operation is performed in each channel to calculate the average similarity Rs between each local region in the class prototype and a single local region in the query set, and the class prototype similarity matrix is ​​converted into a regional similarity of size M×1, as shown in formula (2):

[0017]

[0018] in, Represents the average similarity between the overall class prototype and the i-th local feature in the query set;

[0019] S220, after obtaining the regional similarity through step S210, obtain the similarity fusion coefficient after passing through the similarity feature attention layer, multiply the correlation score in the class prototype similarity matrix with the average similarity of the class prototype with respect to a single region of the query set in the similarity fusion coefficient in turn, and sum them to obtain the average similarity of each region in the class prototype with respect to the global feature of the query set;

[0020] S230, use the classifier to process the average similarity between each region in the class prototype and the global features of the query set, and give a larger weight to the region with higher similarity, so as to highlight the characteristics of the region, and obtain s a ; Based on the fusion features of the class prototype, similar features are noticed to enhance the similar area features, and the final similar features of the class prototype are used for classification; the method for determining the similar features of the query set is the same, and finally the similar features F of the class prototype and the query set are obtained s and F q , as shown in formula (3) and formula (4):

[0021] F s =S+S×s a (3)

[0022] F q =Q+Q×q a (4)

[0023] Among them, S represents the fusion feature obtained by combining the global features and local features of the support set, and Q represents the fusion feature obtained by combining the global features and local features of the query set;

[0024] Through the above operations, the common similar feature areas between the class prototype center and the query set are obtained.

[0025] Further, in step S300, including:

[0026] Adaptive metric network is used to classify similar features F in the query set q For category prediction, the prediction loss is calculated as shown in formula (5):

[0027]

[0028] Among them, U is the number of test query sets, C is the number of categories; y ij The i-th input data is the true label of the j-th category. If the true label of the i-th input is the j-th category, then y ij =1, otherwise y ij =0; p ij is the probability that the model predicts that the i-th input is of the j-th class, and log is the natural logarithm;

[0029] The similarity probability between similar features of the query set and similar features of the class prototype is calculated by cosine similarity, and the one with the highest score is the predicted category; the cosine similarity calculation formula is shown in formula (6), and the similarity probability calculation formula is shown in formula (7):

[0030]

[0031] Where d() represents the cosine similarity calculation function, A and B are two vectors in n-dimensional space, A=(x 11 ,x 12 ,...,x 1n ), B=(x 21 ,x 22 ,...,x 2n );

[0032]

[0033] Among them, a represents the ath query set, k represents the category of the class prototype, and N represents the number of categories in the support set. represents the similar features of the k-th class prototype with respect to the a-th query set, Represents the similarity features of the a-th query set with respect to the k-th class prototype, and the query set category label is consistent with the class prototype; the above formula is used to calculate the similarity matrix of the query set a in multiple class prototypes obtained with respect to the query set a. The prototype-like features that are consistent with the query set a category probability;

[0034] At the same time, the loss of prediction of similar features of the class prototype and similar features of the query set is calculated, as shown in formula (8):

[0035]

[0036] Where p(y=k|Q a ∈k) is the similarity matrix of the prototype obtained by query set a in multiple query set a The prototype-like features that are consistent with the query set a category The probability of; U is the total number of query sets; this loss function ensures that the similar feature attention module strengthens the attention to the similar features shared between the class prototype and the query set; the final loss function is calculated according to formula (9):

[0037] loss=loss1+loss2 (9).

[0038] Furthermore, the class prototype center is P, which is the class prototype center of each support set category obtained by the average value calculation method, as shown in formula (10):

[0039]

[0040] Among them, P Ι represents the class prototype center belonging to the class support set of category Ι, N represents the number of support sets contained in each support set, S Ι Represents the global and local fusion features of the class support set.

[0041] Furthermore, the bearing surface defect image dataset is a public dataset mini-Imagenet.

[0042] Further, the bearing surface defect image dataset includes bearing images with grooves, red rust, abrasions, notches, acceptance, oxide scale and pitting.

[0043] The present invention provides an image feature similarity calculation system based on similar feature attention. The system has a program module corresponding to the above steps and executes the steps in the above image feature similarity calculation method based on similar feature attention when running.

[0044] The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of an image feature similarity calculation method based on similar feature attention when called by a processor.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The present invention discloses an image feature similarity calculation method and system based on similar feature attention. The similar feature attention module is designed by utilizing the correlation between a support set and a query set, obtaining the common similar feature area between the class prototype center of the support set and the query set, and comparing the similarity scores between the two to avoid interference from background information, which is more conducive to classification between similar categories. Then, an adaptive metric network is designed to perform category prediction on similar features of the query set, and to perform category prediction on similarity scores between similar features of the query set and similar features of each class prototype. Finally, the prediction loss of similar features of the class prototype and similar features of the query set and the category prediction loss are calculated to update the network, and classification is completed according to the similarity metric. Similar feature attention determines the similar feature area between the class prototype and the query set by calculating the similarity between the two, and the adaptive metric network has better generalization than the fixed metric distance function. The method is particularly suitable for the challenging task of few-sample bearing surface defect classification, overcomes the limited data availability, and provides an innovative method for classification tasks with scarce or difficult-to-obtain data. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The structure of an image feature similarity calculation method based on similar feature attention in an embodiment of the present invention is Figure 1 ;

[0048] Figure 2 The structure of an image feature similarity calculation method based on similar feature attention in an embodiment of the present invention is Figure 2 ;

[0049] Figure 3 It is a similarity measurement network structure diagram in an embodiment of the present invention;

[0050] Figure 4 This is a physical picture of the self-made bearing surface defect data set in the simulation experiment of the present invention;

[0051] Figure 5 This is a comparison chart of the classification accuracy curves of the small sample image classification method in the simulation experiment of the present invention on the benchmark mini-ImageNet dataset after each round of training;

[0052] Figure 6 This is a bar chart of the classification accuracy of the small sample image classification method in the simulation experiment of the present invention on the benchmark mini-ImageNet dataset, tested after each round of training;

[0053] Figure 7 This is the confusion matrix for the classification of the self-made bearing surface defect data set in the simulation experiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0055] Specific implementation plan 1: Combine Figures 1 to 3 As shown, the present invention provides an image feature similarity calculation method based on similar feature attention, comprising the following steps:

[0056] In the small sample image classification task based on metric learning, the extracted support set and query set features are the key factors affecting the classification effect. In order to mine rich feature information from a small number of image samples and improve the discrimination within and between classes, the present invention uses the correlation between the support set and the query set to design a similar feature attention module, focusing on the similar area between the query set and the class prototype center, and effectively distinguishing them by comparing the similarity scores between the two; combined with Figure 1 The network structure of the similar feature attention module shown in the figure, the prototype center part: 1. The cosine similarity between the prototype center feature and the query set feature is calculated; the prototype similarity matrix Z is obtained s and query similarity matrix Z q ; 2. Similar feature attention layer: Design a convolutional network and use the class prototype similarity map and query similarity map to calculate the class prototype attention map and query set attention map respectively;

[0057] S100, similarity calculation, by calculating the cosine distance between the central feature of the support set class prototype and the query set feature, the correlation score between different feature vectors is obtained, as shown in formula (1):

[0058]

[0059] Among them, Z ij represents the cosine similarity between two feature vectors, i and j represent the local feature vector of the class prototype center and the local feature vector of the query set, and both have H×W local feature vectors; s i Represents the i-th local eigenvector of the class prototype center; q j Represents the jth local feature vector in the query set features;

[0060] The support set and query set are the public dataset mini-Imagenet after unified size, which is used to identify grooves, red rust, abrasions, notches, qualified, oxide scale and pitting in bearing surface defects;

[0061] Combination Figure 1 As shown, the cosine similarity is used to calculate each local vector s of the class prototype i With each local vector q in the query set feature j The cosine similarity between them is used to obtain the similarity matrix Z of the prototype of this class with respect to the query set. s Similarly, the transposed matrix of the query set features is used to perform cosine calculations with the class prototype center feature representation to obtain the similarity matrix Z of the query set with respect to each class prototype center. q ,in, And M = H × W;

[0062] S200, design similar feature attention layer, the similar feature attention layer aims to obtain the similarity matrix Z of the query set from the class prototype s and the query set about the class prototype similarity matrix Z q The similar areas between the two were found in Z s Able to generate prototype-like attention maps for a specific query set a , while Z q Able to generate a query set attention graph q for a specific class prototype a ;Attention maps highlight features that are critical to different classification tasks;

[0063] By the class prototype similarity matrix Z s Get the class prototype attention map s a Methods such as Figure 2 As shown, the query set similarity matrix Z q Get the query set attention graph q aThe method is also calculated according to this structure;

[0064] Specifically include:

[0065] S210, similarity matrix Z for class prototypes s A global average pooling operation is performed in each channel to calculate the average similarity R between each local region in the class prototype and a single local region in the query set. s , convert the class prototype similarity matrix into the regional similarity of size M×1, as shown in formula (2):

[0066]

[0067] in, Represents the average similarity between the overall class prototype and the i-th local feature in the query set;

[0068] S220, after obtaining the regional similarity through step S210, obtain the similarity fusion coefficient after passing through the similarity feature attention layer, multiply the correlation score in the class prototype similarity matrix with the average similarity of the class prototype with respect to a single region of the query set in the similarity fusion coefficient in turn, and sum them to obtain the average similarity of each region in the class prototype with respect to the global feature of the query set;

[0069] S230, use the SoftMax classifier to process the average similarity between each region in the class prototype and the global features of the query set, and give a larger weight to the region with higher similarity to highlight the characteristics of the region, and obtain s a ; Based on the fusion features of the class prototype, similar features are noticed to enhance the similar area features, and the final similar features of the class prototype are used for classification; the method for determining the similar features of the query set is the same, and finally the similar features F of the class prototype and the query set are obtained s and F q , as shown in formula (3) and formula (4):

[0070] F s =S+S×s a (3)

[0071] F q =Q+Q×q a (4)

[0072] Among them, S represents the fusion feature obtained by combining the global features and local features of the support set, and Q represents the fusion feature obtained by combining the global features and local features of the query set;

[0073] Through the above operations, we can obtain the common similar feature areas between the prototype center and the query set, focus on the similar features of the two for classification, and avoid the interference of background information of global features on classification accuracy; at the same time, the similar feature areas and similarity scores between the support set and the query set of the same category are higher than the areas and similarity scores between the support set and the query set of different categories;

[0074] S300, similarity measurement calculation,

[0075] Combination Figure 3 As shown, referring to the relational network concept, the convolutional neural network predicts the query set features, wherein the input is the predicted category feature, the output is the probability value predicted for each category, and the maximum probability is the predicted category type; different from the fixed metric function, the present invention designs a nonlinear metric function, namely, an adaptive metric network; the function adaptively learns a metric network that is more conducive to classification based on the information provided by the input data, including,

[0076] Adaptive metric network is used to classify similar features F in the query set q For category prediction, the prediction loss is calculated as shown in formula (5):

[0077]

[0078] Where: U is the number of test query sets, C is the number of categories; y ij The i-th input data is the true label of the j-th category. If the true label of the i-th input is the j-th category, then y ij =1, otherwise y ij =0; p ij is the probability that the model predicts that the i-th input is of the j-th class, and log is the natural logarithm;

[0079] To ensure the effectiveness of the similarity feature attention module, the attention maps obtained between the class prototype and the query set are made more similar. The category prediction is performed by calculating the similarity score between the similar features of the query set and the similar features of each class prototype to ensure the effectiveness of the similarity feature attention layer;

[0080] The similarity probability between similar features of the query set and similar features of the class prototype is calculated by cosine similarity, and the one with the highest score is the predicted category; the cosine similarity calculation formula is shown in formula (6), and the similarity probability calculation formula (7) is as follows:

[0081]

[0082] Where d() represents the cosine similarity calculation function, A and B are two vectors in n-dimensional space, A=(x 11 ,x 12 ,...,x1n ), B=(x 21 ,x 22 ,...,x 2n );

[0083]

[0084] Among them, a represents the ath query set, k represents the category of the class prototype, and V represents the number of categories in the support set. represents the similar features of the k-th class prototype with respect to the a-th query set, Represents the similarity features of the a-th query set with respect to the k-th class prototype, and the query set category label is consistent with the class prototype; the above formula is used to calculate the similarity matrix of the query set a in multiple class prototypes obtained with respect to the query set a. The prototype-like features that are consistent with the query set a category probability;

[0085] At the same time, the loss of similar features of the class prototype and the prediction of similar features of the query set is calculated, as shown in formula (8):

[0086]

[0087] Where p(y=k|Q a ∈k) is the similarity matrix of the prototype obtained by query set a in multiple query set a The prototype-like features that are consistent with the query set a category The probability of; U is the total number of query sets; this loss function ensures that the similar feature attention module strengthens the attention to the similar features shared between the class prototype and the query set; the final loss function is calculated according to formula (9):

[0088] loss=loss1+loss2 (9).

[0089] Preferably, the class prototype center is P, which is obtained by using the average value calculation method to obtain the class prototype center of each support set category, as shown in formula (10):

[0090]

[0091] Among them, P Ι represents the class prototype center belonging to the class support set of category Ι, N represents the number of support sets contained in each support set, S Ι Represents the global and local fusion features of the class support set.

[0092] Specific implementation scheme 2: The present invention provides an image feature similarity calculation system based on similar feature attention. The system has a program module corresponding to the above steps, and executes the steps in the above image feature similarity calculation method based on similar feature attention during operation.

[0093] The other combinations and connection relationships of this embodiment are the same as those of the first embodiment.

[0094] Specific implementation scheme three: The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of an image feature similarity calculation method based on similar feature attention when called by a processor.

[0095] The other combinations and connection relationships of this embodiment are the same as those of the first embodiment.

[0096] Simulation experiment

[0097] Experimental dataset:

[0098] The data sets used in the experiment of the present invention include the commonly used small sample image classification public data set mini-ImageNet and the self-made bearing surface defect data set under actual factory production conditions. Mini-ImageNet is the most widely used benchmark data set in the field of small sample learning. The data set contains a total of 100 categories, of which 64 categories are used for training, 16 categories are used for verification, and 20 categories are used for testing. Each category contains 600 images. All input image data are adjusted to 84*84 size during the training and testing stages. The effectiveness of the experimental method of the present invention is evaluated by comparing the classification effect of the existing small sample image classification method on mini-ImageNet.

[0099] Combination Figure 4 As shown, in order to verify the performance of the method of the present invention under real factory conditions, we constructed a bearing surface defect data set. All defect data were manually discovered on the actual production line and data collection was performed. The surface defect data set mainly includes seven common defects in actual industrial scenarios, namely: grooves, red rust, abrasions, notches, qualified, oxide scale and pitting, and each category contains 50 images. The collected bearing surface defect data does not participate in training, but is used as a test set to verify the model effect. The defect data is also uniformly adjusted to 84*84 size for testing.

[0100] Experimental setup:

[0101] The experiment of the present invention is implemented on the Ubuntu 18.04 system based on the Pytorch open source deep learning framework, where the Torch version is 1.8.0, the corresponding Torchvison version is 0.8.0, the processor is GPU NVIDIA GeForce RTX3090, and the running memory is 48GB.

[0102] The training and testing phases of the model of the present invention are both carried out under the setting of the small sample learning experimental method based on N-wayK-shot. The model is trained using a cross-task mechanism, and a 5-way 1-shot or 5-way 5-shot setting is adopted, that is, each meta-task contains a total of 5 categories, and each category contains 1 or 5 support set images and 15 query set images. In the model training phase, each epoch includes a total of 2000 tasks, and 100 epochs are performed, including a total of 200,000 meta-tasks. In the model testing phase, the 5-way1-shot and 5-way 5-shot experimental settings are also used for verification. Among them, 15 query set images are selected for verification in each meta-task, and the average classification accuracy obtained from 3000 test tasks within a 95% confidence interval is used as an evaluation indicator.

[0103] Parameter settings: The same parameter settings are used for the training set, validation set, and test set. All experiments use the SGD optimization algorithm to update the model parameters. The initial learning rate is set to 0.1. After the 50th epoch, the learning rate is halved every 10 epochs. The momentum is 0.9, the weight decay is 0.0005, and the rest of the parameters use the default values.

[0104] Experimental results:

[0105] To verify the effectiveness of the algorithm proposed in this paper, this paper uses a small sample public dataset mini-ImageNet to compare with other small sample image classification methods, where the feature embedding network uses Conv4 (four-layer convolution) to ensure the fairness of the experimental results. The classification effects of different methods on the public dataset are shown in Table 1.

[0106] Table 1 Comparison of min i-ImageNet experimental results

[0107]

[0108] First, from the analysis of the experimental results in Table 1, under the condition of the same feature embedding layer, the classification accuracy of the method proposed in the present invention is higher than that of the classic meta-learning method (MAML) and the classic metric learning method (MatchingNetwork, ProtoNetwork, Relation Network), whether in the 5-way 1-shot or 5-way 5-shot experimental setting. Secondly, compared with the metric methods (MATANet, MSLPN, TRNA, FMAF) that perform multi-scale feature fusion or design attention mechanism networks in the feature embedding stage, the method proposed in the present invention is also better than them. Thirdly, compared with the improved meta-learning methods MELR and BaseTransformers in the past two years, the method proposed in the present invention has also achieved significant improvement in classification accuracy. Finally, compared with the RENet and CAN methods that focus on the common similar features between the support set and the query set for classification, the classification accuracy of the method proposed in the present invention is 1.04% and 2.23% higher than that of RENet under 5-way 1-shot or 5-way 5-shot conditions, respectively. Compared with the CAN method, the classification accuracy of the method proposed in the present invention is improved by 4.01% and 6.23% respectively.

[0109] At the same time, during the training process of different small sample image classification methods, after each round of training, in order to test the effect of the network model training in that round, the updated weights of the current network model are used to perform classification tests on the categories that did not participate in the training. A total of 600 test tasks were performed, and the average classification accuracy obtained from the 600 test tasks was taken as the average test accuracy after the end of that round. Figure 5 The average accuracy curve of the classification test after each round of training task during the model training process for different small sample image classification methods shown in the figure. Figure 5 It can be clearly seen from the curve trend that with the increase of training rounds, the accuracy of model classification gradually improves and eventually stabilizes. It proves that the test accuracy of the small sample image classification method proposed in the present invention on the benchmark mini-ImageNet dataset is significantly higher than that of other small sample image classification methods compared.

[0110] The method proposed in the present invention achieves good classification results on public data sets. Since the data categories used in the training and testing stages have no intersection, it shows strong generalization ability. Since the amount of homemade bearing surface defect data is small, each category contains only 50 image data, and the direct use of convolutional neural networks for training and testing results in poor model generalization ability. Therefore, the present invention migrates the network model trained using the mini-ImageNet public data set to the homemade bearing surface defect data set for classification. The classification method learned in the public data set is applied to the classification of bearing surface defect data with a small amount of data. No training is required, and the differences between the features of different categories are directly used for classification, which solves the problem of poor model generalization caused by the small number of bearing defect data sets.

[0111] From the classification results on the public dataset mini-ImageNet in Table 1, it can be seen that the classification effect is better under the experimental method setting of 5way 5shot. Analysis of the reasons: Compared with using only one support set feature as the prototype center feature of the category, using multiple support set features to jointly determine the prototype center feature can better show the common characteristics of the category and help distinguish between different categories. Therefore, in order to improve the accuracy of bearing surface defect classification, the present invention selects the 5way5shot experimental method setting for meta-testing.

[0112] Table 2 Experimental results of different small sample image classification methods on bearing surface defect data

[0113]

[0114]

[0115] According to the analysis in Table 2, when the model trained with the mini-ImageNet public dataset is transferred to the meta-testing stage for bearing surface defect data testing, compared with other classic small sample image classification methods based on metric learning (Proto Network, Relation Network, DN4), the method proposed in the present invention achieves better classification results. Compared with the RENet and CAN methods that focus on the common similar features between the support set and the query set for classification, the classification accuracy of the method proposed in the present invention is 1.07% and 1.69% higher than that of RENet and CAN under the 5-way 5-shot condition, respectively. It also proves the effectiveness of the method proposed in the present invention for the classification of bearing surface defects.

[0116] Combination Figure 6The accuracy bar graph of the classification test of self-made bearing surface defect data by different small sample image classification methods is shown in the figure. The test phase is carried out under the 5-way 5shot experimental setting, and the different small sample image classification network models are trained using the public dataset mini-ImageNet. 15 query sets are selected for each category for testing. Each epoch contains 400 test tasks. A total of 5 epochs are carried out. The average classification accuracy of the 400 test tasks in each epoch is calculated. The purpose of using more test tasks for testing is to ensure the accuracy of the classification results. Figure 6 It can be seen that by comparing the classification test accuracy of different small sample image classification methods in each epoch, the method proposed in the present invention is superior to other solutions.

[0117] In order to further explore the classification effect of the model, the present invention uses a confusion matrix to display the prediction results of different defects, such as Figure 7 At the same time, common image classification indicators, precision (P), recall (R) and F1 score (F1), are used to compare the classification effects of different small sample models.

[0118] Through the confusion matrix, it is found that the defect features distinguish widely-used oxide scale, pitting and qualified data, which are easy to classify, but the defects such as grooves, red rust, scratches and notches with small defect features and large background information are easy to misjudge. The reason for the analysis is that the bearing background is relatively consistent and the defect features are small. When similar features are extracted, it is very likely to focus on the consistent background information between the two, and use the background information as the main feature of the category for classification, which leads to misjudgment.

[0119] It can be seen from Tables 3 and 4 that the model proposed in the present invention has the best prediction effect on groove and scratch defects compared with the other five small sample classification methods.

[0120] Table 3 Evaluation results of the model of the present invention

[0121]

[0122] Table 4 Comparison of evaluation results of small sample image classification models

[0123]

[0124] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A method for calculating image feature similarity based on similar feature attention, characterized in that: The following steps are involved: S100, similarity calculation, by calculating the cosine distance between the central feature of the support set class prototype and the query set feature, to obtain the correlation score between different feature vectors, wherein the support set and the query set are both bearing surface defect image datasets; S200, designing a similar feature attention layer, using the correlation between the support set and the query set, designing a similar feature attention module, obtaining the common similar feature area between the class prototype center of the support set and the query set, and comparing the similarity scores between the two; S300, designing an adaptive metric network, including predicting categories for similar features of the query set, and predicting categories for similarity scores between similar features of the query set and similar features of each class prototype, and then calculating the prediction loss of similar features of the class prototype and similar features of the query set and the category prediction loss to update the network, and completing classification according to the similarity metric.

2. The image feature similarity calculation method based on similar feature attention according to claim 1, characterized in that: In step S100, the similarity score is as shown in formula (1): Among them, Z ij represents the cosine similarity between two feature vectors, i, j represent the local feature vector of the class prototype center and the local feature vector of the query set, and both have H×W local feature vectors; s i Represents the i-th local eigenvector of the class prototype center; q j Represents the jth local feature vector in the query set features; Use cosine similarity calculation to find each local vector s of the class prototype i With each local vector q in the query set feature j The cosine similarity between them is used to obtain the similarity matrix Z of the prototype of this class with respect to the query set. s Similarly, the transposed matrix of the query set features is used to perform cosine calculations with the class prototype center feature representation to obtain the similarity matrix Z of the query set with respect to each class prototype center. q ,in, And M=H×W.

3. The image feature similarity calculation method based on similar feature attention according to claim 2, characterized in that: In step S200, it includes: S210, similarity matrix Z for class prototypes s A global average pooling operation is performed in each channel to calculate the average similarity Rs between each local region in the class prototype and a single local region in the query set, and the class prototype similarity matrix is ​​converted into a regional similarity of size M×1, as shown in formula (2): in, Represents the average similarity between the overall class prototype and the i-th local feature in the query set; S220, after obtaining the regional similarity through step S210, obtain the similarity fusion coefficient after passing through the similarity feature attention layer, multiply the correlation score in the class prototype similarity matrix with the average similarity of the class prototype with respect to a single region of the query set in the similarity fusion coefficient in turn, and sum them to obtain the average similarity of each region in the class prototype with respect to the global feature of the query set; S230, use the classifier to process the average similarity between each region in the class prototype and the global features of the query set, and give a larger weight to the region with higher similarity, so as to highlight the characteristics of the region, and obtain s a ; Based on the fusion features of the class prototype, similar features are noticed to enhance the similar area features, and the final similar features of the class prototype are used for classification; the method of determining the similar features of the query set is the same, and finally the similar features F of the class prototype and the query set are obtained s and F q , as shown in formula (3) and formula (4): F s =S+S×s a (3) F q =Q+Q×q a (4) Among them, S represents the fusion feature obtained by combining the global features and local features of the support set, and Q represents the fusion feature obtained by combining the global features and local features of the query set; Through the above operations, the common similar feature areas between the class prototype center and the query set are obtained.

4. The method for calculating image feature similarity based on similar feature attention according to claim 3, characterized in that: In step S300, it includes: Adaptive metric network is used to classify similar features F in the query set q For category prediction, the prediction loss is calculated as shown in formula (5): Among them, U is the number of test query sets, C is the number of categories; y ij The i-th input data is the true label of the j-th category. If the true label of the i-th input is the j-th category, then y ij =1, otherwise y ij =0; p ij is the probability that the model predicts that the i-th input is of the j-th class, and log is the natural logarithm; The similarity probability between similar features of the query set and similar features of the class prototype is calculated by cosine similarity, and the one with the highest score is the predicted category; the cosine similarity calculation formula is shown in formula (6), and the similarity probability calculation formula is shown in formula (7): Where d() represents the cosine similarity calculation function, A and B are two vectors in n-dimensional space, A=(x 11 ,x 12 ,...,x 1n ), B=(x 21 ,x 22 ,...,x 2n ); Among them, a represents the ath query set, k represents the category of the class prototype, and N represents the number of categories in the support set. represents the similar features of the k-th class prototype with respect to the a-th query set, Represents the similarity features of the a-th query set with respect to the k-th class prototype, and the query set category label is consistent with the class prototype; the above formula is used to calculate the similarity matrix of the query set a in multiple class prototypes obtained with respect to the query set a. The prototype-like features that are consistent with the query set a category probability; At the same time, the loss of prediction of similar features of the class prototype and similar features of the query set is calculated, as shown in formula (8): Where p(y=k|Q a ∈k) is the similarity matrix of the prototype obtained by query set a in multiple query set a The prototype-like features that are consistent with the query set a category The probability of; U is the total number of query sets; this loss function ensures that the similar feature attention module strengthens the attention to the similar features shared between the class prototype and the query set; the final loss function is calculated according to formula (9): loss=loss1+loss2(9).

5. The method for calculating image feature similarity based on similar feature attention according to claim 4, characterized in that: The class prototype center is P, which is the class prototype center of each support set category obtained by the average value calculation method, as shown in formula (10): Among them, P Ι represents the class prototype center belonging to the class support set of category Ι, N represents the number of support sets contained in each support set, S Ι Represents the global and local fusion features of the class support set.

6. The method for calculating image feature similarity based on similar feature attention according to claim 1, characterized in that: The bearing surface defect image dataset is a public dataset mini-Imagenet.

7. The method for calculating image feature similarity based on similar feature attention according to claim 6, characterized in that: The bearing surface defect image dataset includes bearing images having grooves, red rust, scratches, notches, passes, oxide scales, and pitting.

8. An image feature similarity calculation system based on similar feature attention, characterized in that: The system has a program module corresponding to the steps of any one of claims 1 to 7, and executes the steps in the above-mentioned image feature similarity calculation method based on similar feature attention when running.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the image feature similarity calculation method based on similar feature attention according to any one of claims 1 to 7 when called by a processor.

Images