Local feature extraction method and system for small sample bearing surface defects

Abstract

The invention provides a local feature extraction method and system for small sample bearing surface defects, and belongs to the technical field of feature extraction. The objective of the invention is to solve the problems of insufficient feature extraction and poor extraction precision in the surface defect feature extraction process of a bearing with small sample size. According to the design of local feature extraction, local feature information is further extracted on the basis of global features, and more detailed feature representation is carried out on the features. Compared with an existing small sample image classification method, a public data set and a self-made bearing defect data set are improved to a certain degree. Meanwhile, the bearing surface defect classification task is solved by applying a small sample learning method for the first time, the challenging task of few-sample bearing surface defect classification is solved, the limited data availability is overcome, and an innovative method is provided for the classification task of data scarcity or difficult acquisition.

Description

Technical Field

[0001] The present invention relates to the technical field of feature extraction, and in particular to a method and system for extracting local features of surface defects of a small sample bearing. Background Art

[0002] Bearings are a basic component used in large quantities in mechanical equipment. The quality of bearings will seriously affect the stability of equipment operation. In recent years, with the vigorous development of my country's manufacturing industry, the demand for bearing products in various industries has continued to increase, and the requirements for bearing quality have also increased. Although my country's mechanical processing technology has reached a relatively high level, it is inevitable that certain damage will occur in the mass production of bearings. Therefore, bearings must undergo strict testing before leaving the factory to ensure that the bearings can be used normally after leaving the factory. At present, domestic bearing manufacturers have basically realized automated production and assembly line assembly, but the surface quality inspection after bearing assembly still relies on human visual inspection. At present, the research on intelligent detection algorithms for bearing surface defects has gradually received attention, and more and more deep learning algorithms have begun to be applied in the field of bearing surface defect detection.

[0003] However, in the intelligent detection of bearing surface defects, there is a common problem of small sample size, which will cause poor extraction accuracy and insufficient extraction during feature extraction, and have a serious impact on subsequent classification or recognition. In order to fully obtain the category feature information, the present invention uses self-similarity descriptors to calculate the correlation between global features and their adjacent local features, enrich local feature information, increase the amount of category information, and help improve classification accuracy. Summary of the invention

[0004] The technical problems to be solved by the present invention are:

[0005] In order to solve the common problems of insufficient feature extraction and poor extraction accuracy in the process of bearing surface defect feature extraction with small sample size.

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] The present invention provides a method for extracting local features of surface defects of a small sample bearing, comprising the following steps:

[0008] S100, mapping the input image data to the same feature space through a feature embedding network, including passing the preprocessed input image through the feature embedding network to obtain a global feature representation of the input support set and the query set in the feature space; the image data is a bearing surface defect image dataset;

[0009] S200, based on the input data features extracted by the feature embedding module, a local feature extraction module is designed to further extract local features of the sample;

[0010] S300, combining global and local information to obtain fusion features, helps to distinguish between different categories in small sample image classification tasks.

[0011] Furthermore, in step S200, it specifically includes:

[0012] S210, after data preprocessing, the support set and the query set are sent to the feature embedding module to obtain the global feature representation matrix of the support set and the query set of the input image data: Among them, C represents the number of global feature channels, H and W are the height and width of the global feature respectively;

[0013] S220, design a local feature extraction module to process the global features of the class prototype and the query set obtained in step S210 to obtain local features of the support set and the query set for enhancing the feature information of the input data.

[0014] Further, in step S220, including:

[0015] S221, design a local feature extraction module to enhance the feature information of the input data, including processing the global features of the obtained support set and query set by the local feature extraction module, extracting local features through a sliding window on each channel; setting the sliding window size to M × N , sliding on the global features one by one, each pixel in the spatial dimension gets its surrounding M × N Therefore, there are C local blocks on the same channel, each of which is M×N in size. The local feature block obtained for one pixel is The local features extracted from each pixel on all channels together form the neighborhood feature matrix Among them, C represents the number of channels, H and W represent the height and width of the global feature, and M and N represent the width and height of the sliding window, respectively;

[0016] S222, process the global feature G, and the size after dimension upgrade is Make its dimension consistent with the neighborhood feature matrix D;

[0017] S223, perform Hadamard product calculation on each pixel point in the dimension-upgraded global feature G and the neighborhood feature matrix D composed of local feature blocks around each pixel point to obtain the feature correlation matrix R. The calculation process of a certain pixel point is shown in formula (1), and the calculation formula of the global feature G and the domain feature matrix D is shown in formula (2):

[0018] R(x)=G(x)☉D(x) (1)

[0019] R=G☉D (2)

[0020] Among them, G(x)∈G, size is 1*1, x represents a pixel point in the global feature G, x∈[1,H]×[1,W]; D(x)∈D, size is M×N, represents the local feature block around the pixel point x; the dot product represents the Hadamard product, R(x)∈R, size is M×N, represents the correlation between the pixel point and its surrounding features; the Hadamard product of the pixel point G and the neighborhood feature matrix D in all dimensions together constitutes the final correlation matrix R, and

[0021] S224. Design a four-layer convolutional neural network to extract features from the correlation matrix R and obtain local features of the support set and the query set.

[0022] Furthermore, in step S300, it includes combining global and local information to obtain fusion features, as shown in formula (3) and formula (4);

[0023]

[0024]

[0025] Among them, S represents the fusion feature obtained by combining the global features and local features of the support set, Q represents the fusion feature obtained by combining the global features and local features of the query set; G s G represents the global features of the output support set of the feature embedding network; q represents the global features of the query set output by the feature embedding network; L s L represents the local graph features of the feature embedding network output support set; q Represents the local features of the query set output by the feature embedding network.

[0026] Furthermore, the bearing surface defect image dataset is a public dataset mini-Imagenet, and the dataset preprocessing is to resize the input data size to a uniform size of 84*84.

[0027] Further, the bearing surface defect image dataset includes bearing images with grooves, red rust, abrasions, notches, acceptance, oxide scale and pitting.

[0028] Furthermore, the feature embedding network is a Conv64 structure, including 4 convolution blocks; each convolution block includes an activation function layer, a batch normalization layer and a convolution layer.

[0029] The present invention provides a local feature extraction system for small sample bearing surface defects. The system has a program module corresponding to the above steps, and executes the steps in the above method for extracting local features of small sample bearing surface defects when running.

[0030] The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program is configured to implement the steps of a method for extracting local features of surface defects of a small sample bearing when called by a processor.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] The present invention provides a method and system for extracting local features of small sample bearing surface defects. The design of local feature extraction is to further extract local feature information based on global features, and to perform a more detailed feature representation of the feature. Compared with existing small sample image classification methods, a certain degree of improvement has been achieved on both public data sets and self-made bearing defect data sets. At the same time, the present invention applies a small sample learning method for the first time to solve the task of classifying bearing surface defects, solving the challenging task of classifying bearing surface defects with a small number of samples, overcoming limited data availability, and providing an innovative method for classification tasks where data is scarce or difficult to obtain. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A structural diagram of a method for extracting local features of surface defects of a small sample bearing according to an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of a feature embedding network structure in an embodiment of the present invention;

[0035] Figure 3 The thermal comparison diagram of the output features of the mini-ImageNet dataset after the baseline network and the final method in the simulation experiment of the present invention is disclosed;

[0036] Figure 4 The thermal comparison diagram of the output features of the self-made bearing surface defect data set in the simulation experiment of the present invention after the baseline network and the final method. DETAILED DESCRIPTION

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0038] Specific implementation plan 1: Combine Figure 1 and Figure 2 As shown, the present invention provides a method for extracting local features of surface defects of a small sample bearing, comprising the following steps:

[0039] S100, mapping the input image data to the same feature space through a feature embedding network, including passing the preprocessed input image through the feature embedding network to obtain a global feature representation of the input support set and the query set in the feature space; wherein the data set is a public data set mini-Imagenet, and the data set preprocessing is to resize the input data size to a uniform size of 84*84;

[0040] S200, based on the input data features extracted by using the common feature embedding module, a local feature extraction module is designed to further extract the local features of the sample, so as to extract as much feature information as possible under the limitation of few samples and shallow feature embedding network;

[0041] Combination Figure 2 As shown, the feature embedding network in this step can select the commonly used Conv64 structure, including 4 convolution blocks; each convolution block consists of an activation function layer, a batch normalization layer and a convolution layer; at the same time, in order to reduce overfitting, a maximum pooling layer is introduced after the first two convolution blocks;

[0042] S210, after data preprocessing, the support set and the query set are sent to the feature embedding module to obtain the global feature representation matrix of the support set and the query set of the input image data: Among them, C represents the number of global feature channels, H and W are the height and width of the global feature respectively;

[0043] S220. In the meta-training and meta-testing process of small sample image classification, the number of support sets for each category is small, and generally 1 or 5 data are used to represent the characteristics of the category, which is not conducive to distinguishing between different categories. In order to improve the accuracy of small sample image classification, it is necessary to obtain more information that can represent the characteristics of the category under limited data. The local self-similar descriptor can represent the local characteristic information of the image, including the target skeleton, color, edge or other common local feature information. Inspired by the local self-similar descriptor, combined with Figure 1 As shown, the present invention designs a local feature extraction module to enhance the feature information of the input data and increase the discrimination of different categories, including:

[0044] S221, the local feature extraction module processes the global features of the obtained class prototype and query set, uses the unfold function to extract local patches from the multi-dimensional global features through a sliding window, sets the sliding window size to M×N, slides on the global features in sequence, and each pixel point in the spatial dimension obtains a local patch of size M×N around it. Therefore, there are C local patches on the same channel, each of which is M×N in size, and the local feature patch obtained by a pixel point is The local features extracted from each pixel on all channels together form the neighborhood feature matrix Among them, C represents the number of channels, H and W represent the height and width of the global feature, and M and N represent the width and height of the sliding window, respectively;

[0045] S222, process the global feature G, and the size after dimension upgrade is Make its dimension consistent with the neighborhood feature matrix D;

[0046] S223, perform Hadamard product calculation on each pixel point in the dimension-upgraded global feature G and the neighborhood feature matrix D composed of local feature blocks around each pixel point to obtain the feature correlation matrix R. The calculation process of a certain pixel point is shown in formula (1), and the calculation formula of the global feature G and the domain feature matrix D is shown in formula (2):

[0047] R(x)=G(x)☉D(x) (1)

[0048] R=G☉D (2)

[0049] Among them, G(x)∈G, size is 1*1, x represents a pixel point in the global feature G, x∈[1,H]×[1,W]; D(x)∈D, size is M×N, represents the local feature block around the pixel point x; the dot product represents the Hadamard product, R(x)∈R, size is M×N, represents the correlation between the pixel point and its surrounding features; the Hadamard product of the pixel point G and the neighborhood feature matrix D in all dimensions together constitutes the final correlation matrix R, and

[0050] S224. In order to better integrate the channel and space dimensional features, a four-layer convolutional neural network is designed to extract features from the correlation matrix R and obtain local features of the support set and the query set.

[0051] S300, combining global and local information to obtain fusion features, as shown in formula (3) and formula (4); the design of the local feature extraction module calculates the correlation matrix based on the global features, and extracts the local structure, texture information, and color and intensity relationship of the input features; combining the global and local feature information is more helpful to distinguish between different categories in the small sample image classification task;

[0052]

[0053]

[0054] Among them, S represents the fusion feature obtained by combining the global features and local features of the support set, Q represents the fusion feature obtained by combining the global features and local features of the query set; G sG represents the global features of the output support set of the feature embedding network; q represents the global features of the query set output by the feature embedding network; L s L represents the local graph features of the feature embedding network output support set; q Represents the local features of the query set output by the feature embedding network.

[0055] Specific implementation scheme 2: The present invention provides a local feature extraction system for small sample bearing surface defects. The system has a program module corresponding to the above steps, and executes the steps in the above-mentioned local feature extraction method for small sample bearing surface defects when running.

[0056] The other combinations and connection relationships of this embodiment are the same as those of the first embodiment.

[0057] Specific implementation scheme three: The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of a method for extracting local features of surface defects of small sample bearings when called by a processor.

[0058] The other combinations and connection relationships of this embodiment are the same as those of the first embodiment.

[0059] Simulation experiment

[0060] The method of the present invention is improved on the basis of the framework structure of the prototype network, and the prototype network mainly includes two parts: a feature embedding network and a metric function. Among them, the feature embedding network is a commonly used four-layer convolution layer, and the Euclidean distance is selected as the metric function.

[0061] First, training and testing are performed on the baseline network (Base); secondly, the local feature extraction module and the similarity feature attention module are introduced on the basis of the baseline network to form Base+LFE and Base+SFA; finally, the above two modules are added to the baseline network at the same time to form the final method of the present invention, Base+LFE+SFA.

[0062] Table 1 Simulation experiment of the method of the present invention on bearing surface defect data

[0063]

[0064] By observing the simulation experiment results in Table 1, it can be found that on the self-made bearing defect data set, under the experimental setting of 5N 5shot, compared with the classification accuracy of the benchmark network, after the local feature extraction module is introduced, the classification accuracy is improved by 1.82%; after the similar feature attention module is introduced, the classification accuracy is improved by 6.71%; after adding the two modules to the benchmark network at the same time, the classification accuracy is improved by 8.60%. The simulation experiment proves that the method for classifying small sample bearing surface defects designed by the present invention is effective.

[0065] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A method for extracting local features of surface defects of small sample bearings, characterized in that: The following steps are involved: S100, mapping the input image data to the same feature space through a feature embedding network, including passing the preprocessed input image through the feature embedding network to obtain a global feature representation of the input support set and the query set in the feature space; the image data is a bearing surface defect image dataset; S200, based on the input data features extracted by the feature embedding module, designing a local feature extraction module to further extract local features of the sample; S300, combining global and local information to obtain fusion features, helps to distinguish between different categories in small sample image classification tasks.

2. The local feature extraction method of a small sample bearing surface defect according to claim 1 is characterized by: In step S200, specifically including: S210, after data preprocessing, the support set and the query set are sent to the feature embedding module to obtain the global feature representation matrix of the support set and the query set of the input image data: Among them, C represents the number of global feature channels, H and W are the height and width of the global feature respectively; S220, design a local feature extraction module to process the global features of the class prototype and the query set obtained in step S210 to obtain local features of the support set and the query set for enhancing the feature information of the input data.

3. The local feature extraction method of a small sample bearing surface defect according to claim 2 is characterized by: In step S220, it includes: S221, designing a local feature extraction module to enhance feature information of input data, including the local feature extraction module processing the obtained global features of the support set and the query set, and extracting local features through a sliding window on each channel; Set the sliding window size to M×N, slide on the global features one by one, and each pixel point in the spatial dimension obtains a local block of size M×N around it. Therefore, there are C local blocks in the same channel, each of which is M×N in size. The local feature block obtained by a pixel point is The local features extracted from each pixel on all channels together form the neighborhood feature matrix Among them, C represents the number of channels, H and W represent the height and width of the global feature, and M and N represent the width and height of the sliding window, respectively; S222, process the global feature G, and the size after dimension upgrade is Make its dimension consistent with the neighborhood feature matrix D; S223, perform Hadamard product calculation on each pixel point in the dimension-upgraded global feature G and the neighborhood feature matrix D composed of local feature blocks around each pixel point to obtain the feature correlation matrix R. The calculation process of a certain pixel point is shown in formula (1), and the calculation formula of the global feature G and the domain feature matrix D is shown in formula (2): R(x)=G(x)☉D(x) (1) R=G☉D (2) Among them, G(x)∈G, size is 1*1, x represents a pixel point in the global feature G, x∈[1,H]×[1,W]; D(x)∈D, size is M×N, represents the local feature block around the pixel point x; the dot product represents the Hadamard product, R(x)∈R, size is M×N, represents the correlation between the pixel point and its surrounding features; the Hadamard product of the pixel point G and the neighborhood feature matrix D in all dimensions together constitutes the final correlation matrix R, and S224. Design a four-layer convolutional neural network to extract features from the correlation matrix R and obtain local features of the support set and the query set.

4. The local feature extraction method of a small sample bearing surface defect according to claim 3 is characterized by: In step S300, it includes combining global and local information to obtain fusion features, as shown in formula (3) and formula (4); Among them, S represents the fusion feature obtained by combining the global features and local features of the support set, Q represents the fusion feature obtained by combining the global features and local features of the query set; G s G represents the global features of the output support set of the feature embedding network; q represents the global features of the query set output by the feature embedding network; L s L represents the local graph features of the feature embedding network output support set; q Represents the local features of the query set output by the feature embedding network.

5. The local feature extraction method of a small sample bearing surface defect according to claim 1 is characterized by: The bearing surface defect image dataset is a public dataset mini-Imagenet, and the dataset preprocessing is to resize the input data size to a uniform size of 84*84.

6. The local feature extraction method of a small sample bearing surface defect according to claim 5 is characterized by: The bearing surface defect image dataset includes bearing images having grooves, red rust, scratches, notches, passes, oxide scales, and pitting.

7. The local feature extraction method of a small sample bearing surface defect according to claim 1 is characterized by: The feature embedding network is a Conv64 structure, including 4 convolution blocks; each convolution block includes an activation function layer, a batch normalization layer and a convolution layer.

8. A local feature extraction system for small sample bearing surface defects, characterized by: The system has a program module corresponding to the steps of any one of claims 1 to 7, and executes the steps in the above-mentioned method for extracting local features of surface defects of small sample bearings when running.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the method for extracting local features of surface defects of a small sample bearing according to any one of claims 1 to 7 when called by a processor.

Images