An Image Feature Similarity Calculation Method and System Based on Similar Feature Attention
The similarity feature attention method addresses the challenges of background interference and small-sample scenarios in bearing defect classification by identifying shared features and using an adaptive metric network, enhancing classification accuracy.
Patent Information
- Application Number
- CN202510023729.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-07
- Publication Date
- 2025-07-15
- Estimated Expiration
- 2045-01-07
AI Technical Summary
The existing similarity calculation method is not suitable for the case of small samples such as bearing surface defect classification, and there is interference from background information, which affects the calculation results.
The image feature similarity calculation method based on similar feature attention is adopted. By calculating the cosine distance between the support set and the query set, the similar feature attention module is designed, the common similar feature area between the two is obtained, and the adaptive metric network is used for category prediction and loss calculation to avoid background information interference.
Effectively distinguishing similar categories improves the accuracy of the classification of surface defects of bearings with few samples, overcomes the problem of data scarcity, and has better generalization.
Smart Images
Figure CN119832278B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of feature classification. Specifically, it relates to an image feature similarity calculation method and system based on similar feature attention. Background Art
[0002] As a basic component widely used in mechanical equipment, the quality of bearings will seriously affect the stability of equipment operation. In recent years, with the vigorous development of China's manufacturing industry, the demand for bearing products in various industries has been increasing continuously, and at the same time, the requirements for bearing quality have also been rising. Although China's machining technology has reached a relatively high level, certain damages are inevitable in the mass production of bearings. Therefore, before leaving the factory, bearings must undergo strict inspection work to ensure that they can be used normally after leaving the factory. At present, domestic bearing manufacturers have basically achieved automated production and assembly line assembly, but the surface quality inspection after bearing assembly still depends on manual visual inspection. At present, the research on intelligent detection algorithms for bearing surface defects has gradually received attention, and more and more deep learning algorithms have been applied to the field of bearing surface defect detection.
[0003] With the continuous development of big data and artificial intelligence technologies, the methods of similarity calculation are also constantly innovating and improving. From traditional methods such as Euclidean distance and cosine similarity to similarity calculation methods based on deep learning, existing similarity calculations are not applicable to the case of few samples such as bearing surface defect classification under the condition of a large amount of data; moreover, when performing similarity calculation, there is also interference from background information, which affects the calculation results. Summary of the Invention
[0004] The technical problems to be solved by the present invention are:
[0005] To solve the problems that existing similarity calculations are not applicable to the case of few samples such as bearing surface defect classification and there is interference from background information, which affects the calculation results.
[0006] The technical solutions adopted by the present invention to solve the above technical problems:
[0007] The present invention provides an image feature similarity calculation method based on similar feature attention, including the following steps:
[0008] S100. Similarity calculation, by calculating the cosine distance between the support set class prototype center feature and the query set feature, obtaining the correlation score between different feature vectors, where the support set and the query set are both bearing surface defect image data sets;
[0009] S200. Design a similar feature attention layer. Using the correlation between the support set and the query set, design a similar feature attention module to obtain the common similar feature region between the class prototype center of the support set and the query set, and compare the similarity scores between the two.
[0010] S300. Design an adaptive metric network, including predicting the category of the query set's similar features and predicting the similarity scores between the query set's similar features and the similar features of each class prototype. Then calculate the loss of the similar features of the class prototype and the query set prediction and the category prediction loss to update the network, and complete the classification according to the similarity metric.
[0011] Further, in step S100, it includes that the similarity score is as shown in formula (1):
[0012]
[0013] Among them, Z ij represents the cosine similarity between two feature vectors. i and j respectively represent the local feature vectors of the class prototype center and the local feature vectors of the query set, and both have H×W local feature vectors; s i represents the i-th local feature vector of the class prototype center; q j represents the j-th local feature vector in the query set features;
[0014] Use the cosine similarity calculation to find the cosine similarity between each local vector s i of the class prototype and each local vector q j in the query set features, and obtain the similarity matrix Z s of this class prototype with respect to this query set; Similarly, use the transposed matrix of the query set features and the class prototype center feature representation for cosine calculation to obtain the similarity matrix Z q of the query set with respect to each class prototype center, where and M = H×W.
[0015] Further, in step S200, it includes,
[0016] S210. Perform global average pooling operation on the class prototype similarity matrix Z s in each channel, calculate the average similarity Rs between each local region in the class prototype and a single local region in the query set, and convert the class prototype similarity matrix into a region similarity of size M×1, as shown in formula (2):
[0017]
[0018] Among them, represents the average similarity between the overall class prototype and the i-th local feature in the query set;
[0019] S220. After obtaining the regional similarity through step S210, the similarity fusion coefficient is obtained through the similar feature attention layer. The correlation scores in the class prototype similarity matrix are successively dot-multiplied by the average similarity of the class prototype to a single region in the query set in the similarity fusion coefficient, and the sum is calculated to obtain the average similarity of each region in the class prototype to the global feature of the query set;
[0020] S230. Use the classifier to process the average similarity between each region in the class prototype and the global feature of the query set, assign a larger weight to the region with higher similarity, so as to highlight the region feature and obtain s a ; Based on the class prototype fusion feature, through similar feature attention, enhance the similar region feature, and use the final class prototype similar feature for classification; The method for determining the query set similar feature is the same, and finally the similar features F s and F q are obtained, as shown in formulas (3) and (4):
[0021] F s = S + S×s a (3)
[0022] F q = Q + Q×q a (4)
[0023] Among them, S represents the fusion feature obtained by combining the global feature and the local feature of the support set, and Q represents the fusion feature obtained by combining the global feature and the local feature of the query set;
[0024] Through the above operations, the common similar feature region between the class prototype center and the query set is obtained.
[0025] Furthermore, in step S300, it includes
[0026] Use the adaptive metric network to perform class prediction on the query set similar feature F q , and its prediction loss calculation is as shown in formula (5):
[0027]
[0028] Among them, U is the number of test query sets, C is the number of classes; y ij is the true label that the i-th input data belongs to the j-th class. If the true label of the i-th input is the j-th class, then y ij = 1, otherwise y ij = 0; p ij is the probability that the model predicts the i-th input as the j-th class, and log is the natural logarithm;
[0029] Calculate the similarity probability between the similar features of the query set and the similar features of the class prototype through cosine similarity. The category with the highest score is the predicted category. The cosine similarity calculation formula is shown in formula (6), and the similarity probability calculation formula is shown in formula (7):
[0030]
[0031] Where d() represents the cosine similarity calculation function, and A and B are two vectors in the n-dimensional space, A = (x 11 , x 12 ,..., x 1n ), B = (x 21 , x 22 ,..., x 2n );
[0032]
[0033] Where a represents the a-th query set, k represents the category of the class prototype, N represents the number of categories in the support set, represents the similar features of the k-th class prototype with respect to the a-th query set, represents the similar features of the a-th query set with respect to the k-th class prototype, and at the same time the category label of this query set is consistent with the class prototype; through the above formula, calculate the similar matrix of the class prototype obtained by the query set a for multiple query sets a The probability of the similar features of the class prototype that is consistent with the category of the query set a;
[0034] At the same time, calculate the loss predicted by the similar features of the class prototype and the similar features of the query set, as shown in formula (8):
[0035]
[0036] Where p(y = k|Q a ∈k) is the probability of the similar features of the class prototype that is consistent with the category of the query set a in the similar matrix of the class prototype obtained by the query set a for multiple query sets a ; U is the total number of query sets; this loss function can ensure that the similar feature attention module strengthens the attention to the similar features shared between the class prototype and the query set; the final loss function is calculated according to formula (9):
[0037] loss = loss1 + loss2 (9).
[0038] Furthermore, the class prototype center is P, which is obtained by using the average value calculation method to calculate the class prototype center of each support set category, as shown in formula (10):
[0039]
[0040] Among them, P Ι represents the class prototype center belonging to the support set of category Ι, N represents the number contained in each support set, and S Ι represents the global and local fusion features of the support set of category Ι.
[0041] Furthermore, the bearing surface defect image dataset is the publicly available dataset mini-Imagenet.
[0042] Furthermore, the bearing surface defect image dataset includes bearing images with grooves, red rust, abrasions, scratches, qualified, mill scale, and pitting.
[0043] An image feature similarity calculation system based on similar feature attention according to the present invention has program modules corresponding to the above steps, and executes the steps in the above image feature similarity calculation method based on similar feature attention when running.
[0044] A computer-readable storage medium according to the present invention stores a computer program, and the computer program is configured to implement the steps of the image feature similarity calculation method based on similar feature attention when called by a processor.
[0045] Compared with the prior art, the beneficial effects of the present invention are:
[0046] An image feature similarity calculation method and system based on similar feature attention according to the present invention utilize the correlation between the support set and the query set, design a similar feature attention module, obtain the common similar feature region between the class prototype center of the support set and the query set, compare the similarity scores between the two, avoid the interference of background information, and are more helpful for classification between similar categories; then design an adaptive metric network to perform class prediction on the similar features of the query set, perform class prediction on the similarity scores of the similar features of the query set and the similar features of each class prototype, and finally calculate the loss of the similar features of the class prototype and the class prediction loss of the similar features of the query set for updating the network, and complete classification according to similarity measurement; similar feature attention determines the similar feature region between the two by calculating the similarity between the class prototype and the query set, and the adaptive metric network has better generalization than the fixed metric distance function; this method is particularly suitable for the challenging task of few-shot bearing surface defect classification, overcomes the limited data availability, and provides an innovative method for classification tasks with scarce or difficult-to-obtain data. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 is the structure of an image feature similarity calculation method based on similar feature attention in an embodiment of the present invention Figure 1 ;
[0048] Figure 2 The structure of a method for calculating image feature similarity based on similar feature attention in an embodiment of the present invention Figure 2 ;
[0049] Figure 3 The structure diagram of the similarity metric network in an embodiment of the present invention;
[0050] Figure 4 The physical diagram of the self-made bearing surface defect dataset in the simulation experiment of the present invention;
[0051] Figure 5 The comparison chart of the classification accuracy curves obtained by testing after each round of training of the few-shot image classification method in the simulation experiment of the present invention on the benchmark mini-ImageNet dataset;
[0052] Figure 6 The bar chart of the classification accuracy obtained by testing after each round of training of the few-shot image classification method in the simulation experiment of the present invention on the benchmark mini-ImageNet dataset;
[0053] Figure 7 The confusion matrix of the classification of the self-made bearing surface defect dataset in the simulation experiment of the present invention. Detailed implementation manners
[0054] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the specific embodiments of the present invention in detail with reference to the accompanying drawings.
[0055] Specific implementation manner 1: As shown in combination with Figures 1 to 3 , the present invention provides a method for calculating image feature similarity based on similar feature attention, including the following steps:
[0056] In the few-shot image classification task based on metric learning, the features of the support set and the query set extracted are key factors affecting the classification effect. To mine rich feature information from a small number of image samples and improve the discriminability within and between classes, the present invention utilizes the correlation between the support set and the query set, designs a similar feature attention module, focuses on the similar regions between the query set and the class prototype center, and makes effective distinctions by comparing the similarity scores between the two; As shown in combination with Figure 1 the network structure of the similar feature attention module shown, for the class prototype center part: 1. Calculate the cosine similarity between the class prototype center feature and the query set feature; obtain the class prototype similarity matrix Z s and the query similarity matrix Z q ; 2. Similar feature attention layer: Design a convolutional network, and use the class prototype similarity map and the query similarity map to calculate the class prototype attention map and the query set attention map respectively;
[0057] S100. Similarity calculation: By calculating the cosine distance between the prototype center features of the support set classes and the features of the query set, the correlation scores between different feature vectors are obtained, as shown in formula (1):
[0058]
[0059] where Z ij represents the cosine similarity between two feature vectors. i and j respectively represent the local feature vectors of the prototype center of the class and the local feature vectors of the query set, and both have H×W local feature vectors; s i represents the i-th local feature vector of the prototype center of the class; q j represents the j-th local feature vector in the query set features;
[0060] Among them, the support set and the query set are the publicly available dataset mini-Imagenet after being unified in size, which is used to identify grooves, red rust, abrasions, scratches, qualified, mill scale, and pitting in bearing surface defects;
[0061] Combined with Figure 1 as shown, the cosine similarity between each local vector s i of the class prototype and each local vector q j in the query set features is calculated using cosine similarity to obtain the similarity matrix Z s of this class prototype with respect to this query set; Similarly, the cosine calculation is performed using the transposed matrix of the query set features and the class prototype center feature representation to obtain the similarity matrix Z q of the query set with respect to each class prototype center, where and M = H×W;
[0062] S200. Design similar feature attention layer: The similar feature attention layer aims to find the similar regions between the similarity matrix Z s of the class prototype with respect to the query set and the similarity matrix Z q of the query set with respect to the class prototype; Z s can generate the class prototype attention map s a for a specific query set, while Z q can generate the query set attention map q a for a specific class prototype; The attention map highlights the features crucial for different category classification tasks;
[0063] The method to obtain the class prototype attention map s s from the class prototype similarity matrix Z a is as shown in Figure 2 , and the method to obtain the query set attention map q q from the query set similarity matrix Z aThe method is also calculated according to this structure;
[0064] Specifically,
[0065] S210. For the class prototype similarity matrix Z s Perform global average pooling operation on each channel to calculate the average similarity R between each local area in the class prototype and a single local area in the query set s , and convert the class prototype similarity matrix into a region similarity of size M×1, as shown in formula (2):
[0066]
[0067] Among them, represents the average similarity between the overall class prototype and the i-th local feature in the query set;
[0068] S220. After obtaining the region similarity through step S210, obtain the similarity fusion coefficient through the similar feature attention layer, multiply the correlation scores in the class prototype similarity matrix by the average similarity of the class prototype with respect to a single region in the query set in the similarity fusion coefficient in turn, and sum to obtain the average similarity of each region in the class prototype with respect to the global feature of the query set;
[0069] S230. Use the SoftMax classifier to process the average similarity between each region in the class prototype and the global feature of the query set, assign a larger weight to the region with higher similarity to highlight the region feature, and obtain s a ; Based on the class prototype fusion feature, enhance the similar region feature through similar feature attention, and use the final class prototype similar feature for classification; The method for determining the similar feature of the query set is the same, and finally obtain the similar features F s and F q , as shown in formula (3) and formula (4):
[0070] F s =S + S×s a (3)
[0071] F q =Q + Q×q a (4)
[0072] Among them, S represents the fusion feature obtained by combining the global feature and local feature of the support set, and Q represents the fusion feature obtained by combining the global feature and local feature of the query set;
[0073] Through the above operations, the common similar feature regions between the class prototype center and the query set can be obtained. Focus on the similar features of the two for classification to avoid the interference of the background information existing in the global features on the classification accuracy. At the same time, the similar feature regions and similarity scores between the support set and the query set of the same category should be higher than those between the support set and the query set of different categories.
[0074] S300, similarity metric calculation,
[0075] Combined with Figure 3 As shown, referring to the idea of the relational network, the convolutional neural network predicts the features of the query set. Among them, the input end is the predicted category features, and the output is the probability value predicted for each category. The category type with the highest probability is the predicted category. Different from using a fixed metric function, the present invention designs a non-linear metric function, that is, an adaptive metric network. This function adaptively learns a metric network that is more conducive to classification according to the information provided by the input data, including,
[0076] Use the adaptive metric network to perform category prediction on the similar features F of the query set q The prediction loss calculation is as shown in formula (5):
[0077]
[0078] Among them: U is the number of test query sets, and C is the number of categories; y ij is the true label that the i-th input data belongs to the j-th class. If the true label of the i-th input is the j-th class, then y ij = 1, otherwise y ij = 0; p ij is the probability that the model predicts the i-th input as the j-th class, and log is the natural logarithm;
[0079] To ensure the effectiveness of the similar feature attention module and make the attention maps obtained between the class prototype and the query set more similar, perform category prediction by calculating the similarity scores between the similar features of the query set and the similar features of each class prototype to ensure the effectiveness of the similar feature attention layer;
[0080] Calculate the similarity probability between the similar features of the query set and the similar features of the class prototype through cosine similarity. The category with the highest score is the predicted category. The cosine similarity calculation formula is as shown in formula (6), and the similarity probability calculation formula (7) is as follows:
[0081]
[0082] Among them, d() represents the cosine similarity calculation function, A and B are two vectors in the n-dimensional space, A = (x 11 , x 12 ,..., x1n ), B = (x 21 , x 22 ,..., x 2n );
[0083]
[0084] Among them, a represents the a-th query set, k represents the category of the class prototype, V represents the number of categories of the support set, represents the similar features of the k-th class prototype with respect to the a-th query set, represents the similar features of the a-th query set with respect to the k-th class prototype, and at the same time the category label of this query set is consistent with the class prototype; through the above formula, the similar matrix of the class prototypes obtained for the query set a from multiple query sets a is calculated Among them, the probability of the similar features of the class prototype consistent with the category of the query set a ;
[0085] At the same time, calculate the loss predicted by the similar features of the class prototype and the query set, as shown in Equation (8):
[0086]
[0087] Among them, p(y = k|Q a ∈ k) is the probability of the similar features of the class prototype consistent with the category of the query set a in the similar matrix of the class prototypes obtained for the query set a from multiple query sets a Among them, the probability of the similar features of the class prototype consistent with the category of the query set a ; U is the total number of query sets; this loss function can ensure that the similar feature attention module strengthens the attention to the similar features shared between the class prototype and the query set; the final loss function is calculated according to Equation (9):
[0088] loss = loss1 + loss2 (9).
[0089] Preferably, the class prototype center is P, which is obtained by using the average value calculation method for the class prototype center of each support set category, as shown in Equation (10):
[0090]
[0091] Among them, P Ι represents the class prototype center of the support set belonging to the Ι category, N represents the number contained in each support set, and S Ι represents the global and local fusion features of the support set of the Ι category.
[0092] Specific Embodiment 2: A system for calculating the similarity of image features based on similar feature attention according to the present invention, the system has program modules corresponding to the above steps, and executes the steps in the method for calculating the similarity of image features based on similar feature attention when running.
[0093] The other combinations and connection relationships in this embodiment are the same as those in Specific Embodiment 1.
[0094] Specific Embodiment 3: A computer-readable storage medium according to the present invention, the computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the method for calculating the similarity of image features based on similar feature attention when called by a processor.
[0095] The other combinations and connection relationships in this embodiment are the same as those in Specific Embodiment 1.
[0096] Simulation Experiment
[0097] Experimental Datasets:
[0098] The datasets used in the experiments of the present invention include the commonly used small-sample image classification public dataset mini-ImageNet and the self-made bearing surface defect dataset under actual factory production conditions. Mini-ImageNet is the most widely used benchmark dataset in the field of small-sample learning. This dataset contains a total of 100 categories, of which 64 categories are used for training, 16 categories are used for validation, and 20 categories are used for testing. Each category contains 600 images. During the training and testing phases, all input image data is adjusted to a size of 84*84. By comparing the classification effects of existing small-sample image classification methods on mini-ImageNet, the effectiveness of the experimental method of the present invention is evaluated.
[0099] Combined with Figure 4 As shown, to verify the performance of the method of the present invention under actual factory conditions, we constructed a bearing surface defect dataset. All defect data was discovered manually on the actual production line and data was collected. The surface defect dataset mainly includes seven common defects in actual industrial scenarios, namely: grooves, red rust, scratches, nicks, qualified, mill scale, and pitting. Each category contains 50 images. The collected bearing surface defect data is not involved in training and is used as the test set to verify the model effect. The defect data is also uniformly adjusted to a size of 84*84 for testing.
[0100] Experimental Settings:
[0101] The experiments of the present invention were implemented based on the open-source deep learning framework Pytorch on the Ubuntu 18.04 system, where the Torch version was 1.8.0, the corresponding Torchvison version was 0.8.0, the processor was the GPU NVIDIA GeForce RTX3090, and the running memory was 48GB.
[0102] Both the training and testing phases of the model of the present invention were carried out under the settings of the few-shot learning experimental method based on N-way K-shot. The model was trained using the cross-task mechanism, with settings of 5-way 1-shot or 5-way 5-shot, that is, each meta-task included a total of 5 categories, and each category included 1 or 5 support set images and 15 query set images. Each epoch in the model training phase included 2,000 tasks, and 100 epochs were carried out, including a total of 200,000 meta-tasks. In the model testing phase, the experimental settings of 5-way 1-shot and 5-way 5-shot were also used for verification. Among them, 15 query set images were selected for verification in each meta-task, and within the 95% confidence interval, the average classification accuracy obtained from 3,000 test tasks was used as the evaluation index.
[0103] Parameter settings: The training set, validation set, and test set all used the same parameter settings. All experiments used the SGD optimization algorithm to update the model parameters. The initial learning rate was set to 0.1. After the 50th epoch, every 10 epochs, the learning rate was halved, the momentum was 0.9, and the weight decay was 0.0005. The remaining parameters used the default values.
[0104] Experimental results:
[0105] To verify the effectiveness of the algorithm proposed in the present invention, the present invention used the few-shot public dataset mini-ImageNet to compare with other few-shot image classification methods, where the feature embedding network used Conv4 (four-layer convolution) to ensure the fairness of the experimental results. The classification effects of different methods on the public dataset are shown in Table 1.
[0106] Table 1 Comparison of mini-ImageNet experimental results
[0107]
[0108] First, from the analysis of the experimental results in Table 1, under the condition of the same feature embedding layer, compared with the classical meta-learning method (MAML) and the classical metric learning methods (MatchingNetwork, ProtoNetwork, Relation Network), the classification accuracy of the method proposed in the present invention is relatively high in both the 5-way 1-shot and 5-way 5-shot experimental settings. Secondly, compared with the metric methods (MATANet, MSLPN, TRNA, FMAF) that perform multi-scale feature fusion in the feature embedding stage or design attention mechanism networks, the method proposed in the present invention is also superior to them. Thirdly, compared with the improved meta-learning methods MELR and BaseTransformers in the past two years, the method proposed in the present invention has also achieved a significant improvement in classification accuracy. Finally, compared with the RENet and CAN methods that classify by focusing on the common similar features between the support set and the query set, the classification accuracy of the method proposed in the present invention is increased by 1.04% and 2.23% respectively compared with RENet under the conditions of 5-way 1-shot or 5-way 5-shot. Compared with the CAN method, the classification accuracy of the method proposed in the present invention is increased by 4.01% and 6.23% respectively.
[0109] Meanwhile, during the training process of different few-shot image classification methods, after each round of task training is completed, in order to test the training effect of the network model in this round, the updated weights of the current network model are used to classify and test the categories that have not participated in the training. A total of 600 test tasks are carried out, and the average of the classification accuracies obtained from the 600 test tasks is used as the average test accuracy after the end of this round. Combining Figure 5 with the average accuracy curve of classification tests carried out after each round of training tasks during the model training process of the different few-shot image classification methods shown. Observing Figure 5 the curve trend in it, it can be clearly seen that as the number of training rounds increases, the classification accuracy of the model gradually improves and finally tends to be stable. It is proved that the test accuracy of the few-shot image classification method proposed in the present invention on the benchmark mini-ImageNet dataset is significantly higher than that of other compared few-shot image classification methods.
[0110] The method proposed in the present invention achieves good classification results on the public dataset. Since the data categories used in the training and testing phases have no intersection, it demonstrates strong generalization ability. Due to the small amount of self-made bearing surface defect data, each category only contains 50 image data. Directly using a convolutional neural network for training and testing results in poor model generalization ability. Therefore, the present invention transfers the network model trained on the mini-ImageNet public dataset to the self-made bearing surface defect dataset for classification. Applying the classification method learned on the public dataset to the classification of bearing surface defect data with a small amount of data, without going through training, directly classifying using the differences between the features of different categories, solves the problem of poor model generalization caused by the small number of bearing defect datasets.
[0111] As can be seen from the classification results of Table 1 on the public dataset mini-ImageNet, the classification effect is better under the experimental method setting of 5way 5shot. Analyzing the reasons: Compared with using only one support set feature as the class prototype center feature of this category, using multiple support set features to jointly determine the class prototype center feature can better show the common features of this category and contribute to the distinction between different categories. Therefore, to improve the accuracy of bearing surface defect classification, the present invention selects the experimental method setting of 5way5shot for meta-testing.
[0112] Table 2 Experimental results of different few-shot image classification methods on bearing surface defect data
[0113]
[0114]
[0115] As analyzed in Table 2, when the model trained using the mini-ImageNet public dataset for meta-training is transferred to the meta-testing phase for testing bearing surface defect data, compared with other classic few-shot image classification methods based on metric learning (Proto Network, Relation Network, DN4), the method proposed in the present invention achieves good classification results. Compared with the RENet and CAN methods that focus on classifying by the common similar features between the support set and the query set, the classification accuracy of the method proposed in the present invention is increased by 1.07% and 1.69% respectively compared with RENet and CAN under the 5-way 5-shot condition. This also proves the effectiveness of the method proposed in the present invention for bearing surface defect classification.
[0116] Combined with Figure 6The bar chart of the accuracy of classifying the self-made bearing surface defect data by different few-shot image classification methods shown below. The test phase was carried out under the experimental setting of 5-way 5-shot, and different few-shot image classification network models were all trained using the publicly available dataset mini-ImageNet. 15 query sets were selected for testing in each category. Each epoch contained 400 test tasks, and a total of 5 epochs were carried out. The average classification accuracy of the 400 test tasks in each epoch was calculated. The purpose of using more test tasks for testing was to ensure the accuracy of the classification results. As Figure 6 can be seen, when comparing the classification test accuracies of different few-shot image classification methods in each epoch, the method proposed by the present invention is better than other solutions.
[0117] To further explore the classification effect of the model, the present invention uses a confusion matrix to show the prediction results of different defects, as Figure 7 shown. At the same time, common image classification metrics, precision (P), recall (R), and F1 score (F1), are used to compare the classification effects of different few-shot models.
[0118] It is found through the confusion matrix that it is easy to classify the mill scale, pitting, and qualified data with widely distinguishable defect features, but it is easy to misjudge the defects of grooves, red rust, scratches, and indentations with relatively small defect features and large background information. The reason for the analysis is that the bearing background is relatively consistent and the defect features are small. When extracting similar features, it is very likely to pay attention to the consistent background information between the two, and classify using the background information as the main feature of this category, thus resulting in misjudgment.
[0119] As can be seen from Tables 3 and 4, the model proposed by the present invention has the best prediction effect on groove and scratch defects compared with the other 5 few-shot classification methods for comparison.
[0120] Table 3 Evaluation results of the model of the present invention
[0121]
[0122] Table 4 Comparison of evaluation results of few-shot image classification models
[0123]
[0124] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art of the present invention can make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will all fall within the protection scope of the present invention.
Claims
1. A method for calculating the similarity of image features based on similar feature attention, characterized in that Including the following steps: S100. Similarity calculation: By calculating the cosine distance between the class prototype center features of the support set and the query set features, the correlation scores between different feature vectors are obtained. Both the support set and the query set are bearing surface defect image datasets; S200. Designing a similar feature attention layer: Using the correlation between the support set and the query set, a similar feature attention module is designed to obtain the common similar feature region between the class prototype center of the support set and the query set, and compare the similarity scores between them; Including: S210. For the class prototype similarity matrix Z s In each channel, perform global average pooling operation to calculate the average similarity Rs between each local area in the class prototype and a single local area in the query set, and convert the class prototype similarity matrix into a region similarity of size M×1, as shown in formula (2): Among them, represents the average similarity between the overall class prototype and the i-th local feature vector in the query set; S220. After obtaining the regional similarity through step S210, a similarity fusion coefficient is obtained through the similar feature attention layer. The correlation scores in the class prototype similarity matrix are successively dot-multiplied by the average similarity of the class prototype with respect to a single region of the query set in the similarity fusion coefficient, and summed to obtain the average similarity of each region in the class prototype with respect to the global features of the query set; S230. Use a classifier to process the average similarity between each region in the class prototype and the global features of the query set, assign a larger weight to the region with higher similarity to highlight the region features, and obtain s a ; Based on the class prototype fusion features, enhance the similar region features through similar feature attention, and use the final class prototype similar features for classification; The method for determining the query set similar features is the same, and finally obtain the similar features F s and F q , as shown in formulas (3) and (4): F s = S + S×s a (3) F q = Q + Q×q a (4) Wherein, S represents the fusion feature obtained by combining the global features and local features of the support set, and Q represents the fusion feature obtained by combining the global features and local features of the query set; Through the above operations, the common similar feature region between the class prototype center and the query set is obtained; S300. Designing an adaptive metric network, including predicting the category of the query set similar features, and predicting the similarity scores between the query set similar features and the similar features of each class prototype, and then calculating the loss of the similar features of the class prototype and the query set similar features prediction and the category prediction loss to update the network, and completing classification according to the similarity metric.
2. The method for calculating the similarity of image features based on similar feature attention according to claim 1, wherein: In step S100, including: The similarity score is as shown in formula (1): Among them, Z ij represents the cosine similarity between two feature vectors. i and j respectively represent the local feature vectors of the class prototype center and the query set, and both have H×W local feature vectors; s i represents the i-th local feature vector of the class prototype center; q j represents the j-th local feature vector in the query set features. Calculate the cosine similarity between each local feature vector s of the class prototype i and each local feature vector q in the query set feature j to obtain the similarity matrix Z of the class prototype with respect to the query set s ; Similarly, use the transpose matrix of the query set feature to perform cosine calculation with the class prototype center feature representation to obtain the similarity matrix Z of the query set with respect to each class prototype center q , where and M = H × W.
3. The image feature similarity calculation method based on similar feature attention according to claim 2, wherein: In step S300, including: Use an adaptive metric network to calculate the similarity feature F of the query set q for class prediction. The calculation of its prediction loss is shown in Equation (5): where U is the total number of the test query set, and C is the number of categories; y uc is the true label that the u-th input data belongs to the c-th category. If the true label of the u-th input is the c-th category, then y uc = 1, otherwise y uc = 0; p uc is the probability that the model predicts the u-th input as the c-th category, and log is the natural logarithm; Calculating the similarity probability between the query set similar features and the class prototype similar features through cosine similarity, and the category with the highest score is the predicted category; The cosine similarity calculation formula is as shown in formula (6), and the similarity probability calculation formula is as shown in formula (7): Among them, d() represents the cosine similarity calculation function, and A and B are two vectors in the n-dimensional space. A = (x 11 , x 12 ,..., x 1n ). B = (x 21 , x 22 ,..., x 2n ); Among them, a represents the a-th query set, k represents the category of the class prototype, and N represents the number of categories included in each support set. represents the similar features of the k-th class prototype with respect to the a-th query set. represents the similar features of the a-th query set with respect to the k-th class prototype, and at the same time, the category label of this query set is consistent with the class prototype; the class prototype similarity matrix of the query set a obtained from multiple query sets a is calculated by the above formula. In the above, the probability of the similar features of the class prototype consistent with the category of the query set a. of. At the same time, calculating the loss of the similar features of the class prototype and the query set similar features prediction, as shown in formula (8): where p(y = k|Q a ∈ k) is the class prototype similarity feature in the similarity matrix of class prototypes obtained for the query set a with respect to multiple query sets a that is consistent with the class of the query set a; U is the total number of test query sets; this loss function can ensure that the similarity feature attention module strengthens the attention to the shared similarity features between the class prototype and the query set; the final loss function is calculated according to formula (9): loss = loss1 + loss2 (9).
4. The method for calculating the similarity of image features based on similar feature attention according to claim 3, wherein: The class prototype center is P, which is obtained by using the average value calculation method to obtain the class prototype center of each support set category, as shown in formula (10): Among them, P Ι represents the class prototype center belonging to the support set of category Ι, N represents the number of categories included in each support set, and S Ι represents the global and local fusion features of the support set of category Ι.
5. A method for calculating image feature similarity based on similar feature attention according to claim 1, characterized in that: The bearing surface defect image dataset is the public dataset mini-Imagenet.
6. The method for calculating the similarity of image features based on similar feature attention according to claim 5, characterized in that: The bearing surface defect image dataset includes bearing images with grooves, red rust, abrasions, scratches, qualified, mill scale and pitting.
7. An image feature similarity calculation system based on similar feature attention, characterized in that: The system has program modules corresponding to the steps of any one of the above claims 1-6, and executes the steps in the above image feature similarity calculation method based on similar feature attention when running.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the image feature similarity calculation method based on similar feature attention described in any one of claims 1-6 when called by a processor.