Method and system for few-shot industrial defect recognition based on adaptive fusion of multiple salient features

Abstract

A few-shot industrial defect recognition method and system based on multi-salient feature adaptive fusion, the method comprising the following steps: performing wavelet transform and gradient transform on the input images in the support set respectively, and inputting into the feature extractor and the feature adaptive fusion module to obtain the fused features; inputting the fused features into the class-distribution joint measurement model to obtain the feature prototype of each known class; training the network; inputting the test images in the query set into the wavelet transform and gradient transform respectively, and inputting into the trained feature extractor, feature adaptive fusion module and class-distribution joint measurement model to obtain the total open score, and evaluating the test results of the test images in the query set according to the total open score. The present application comprehensively measures from the local and global two levels, and realizes the accurate classification of known class defects and the effective discrimination of unknown class defects.

Description

Technical Field

[0001] This invention relates to the field of industrial defect detection technology, and in particular to a method and system for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features. Background Technology

[0002] In modern industrial production systems, surface defect detection is a core component for ensuring product quality, improving production efficiency, and reducing production costs, directly impacting a company's market competitiveness and industry reputation. As industrial manufacturing moves towards higher precision, automation, and batch production, increasingly stringent requirements are being placed on the speed, accuracy, and robustness of defect detection. Traditional manual inspection methods, due to inherent drawbacks such as low efficiency, strong subjectivity, and susceptibility to fatigue, are no longer sufficient to meet the inspection needs of modern production lines. Automated and intelligent defect detection technologies have become an inevitable trend in the industry.

[0003] In recent years, deep learning technology has made breakthrough progress in the field of industrial defect detection due to its powerful feature learning and pattern recognition capabilities, driving the transformation of defect detection technology from traditional manual inspection to intelligent automated inspection. Currently, mainstream industrial defect detection methods are mainly divided into two categories: industrial defect anomaly detection methods and defect target detection methods. These two types of methods have different focuses in different application scenarios, but both have significant technical bottlenecks.

[0004] Industrial defect and anomaly detection methods primarily identify abnormal regions in products through feature embedding or reconstruction-based approaches. These methods do not require pre-labeling of defect categories and are suitable for scenarios where only a "normal / abnormal" judgment is needed. Feature embedding methods construct feature distribution models using normal samples and detect abnormal samples that deviate from this distribution by setting a threshold. Reconstruction methods (such as autoencoders and generative adversarial networks) learn the reconstruction patterns of normal samples and use the difference between the reconstructed image and the original image as the basis for anomaly judgment. However, these methods have significant limitations: firstly, they typically require a large number of normal samples for training, resulting in high computational costs and poor adaptability in small- to medium-batch production scenarios; secondly, these methods can only determine whether a sample is abnormal, but cannot distinguish specific defect categories, making it difficult to trace the cause of defects during production and hindering timely optimization of production processes; furthermore, in complex industrial environments (such as varying lighting, background noise, and complex component structures), the robustness of these methods decreases significantly, making them prone to missed detections and false detections.

[0005] Defect detection methods utilize supervised learning to train models with a large number of labeled defect samples, enabling defect localization and classification. They exhibit high detection accuracy in scenarios with clearly defined defect categories and sufficient samples. However, these methods face two major challenges: First, they are highly sample-dependent. Many defects in industrial production (such as rare process defects and novel design defects) occur very infrequently, making it extremely difficult to obtain a sufficient number of labeled samples, leading to insufficient model training and limited detection performance. Second, they suffer from the limitations of the closed-world assumption. Most existing methods are designed based on the closed-world assumption that "training samples and test samples belong to the same category set." However, actual industrial production is dynamic and open; new defect categories may appear at any time. Closed-world models cannot identify these unknown defects, easily leading to the release of substandard products and causing quality risks and economic losses for enterprises.

[0006] In addition to the two mainstream methods mentioned above, related research fields have also developed open set recognition and few-shot learning techniques, attempting to solve the problems of unknown category recognition and sample scarcity, respectively. However, existing open set recognition methods typically require a large number of known category samples for training, making them difficult to adapt to the real-world scenario where industrial defect samples are scarce. While few-shot learning methods can classify known categories with a small number of samples, most do not consider the existence of unknown categories in the open world, and therefore cannot effectively distinguish between known and unknown defects. A few methods that combine few-shot learning and open set recognition do not fully consider the sparsity of industrial defect features and the high global similarity of industrial images, resulting in poor application performance in industrial scenarios and failing to meet actual detection needs.

[0007] In summary, existing industrial defect detection methods generally suffer from problems such as reliance on large-scale training samples, insufficient ability to identify unknown defects in open worlds, and lack of robustness in complex industrial environments. In actual industrial production, scenarios with few rare defect samples and the continuous emergence of new defects are extremely common. There is an urgent need for a defect detection method that can simultaneously achieve accurate classification of known defects and effective identification of unknown defects in open-world scenarios where training samples are scarce, in order to overcome existing technological bottlenecks and improve the level of quality control in industrial production. Summary of the Invention

[0008] This invention provides a method and system for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features, in order to solve the technical problems mentioned in the background art.

[0009] To achieve the above objectives, the technical solution of the present invention is implemented as follows: This invention provides a method for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features, comprising the following steps: S1, for input images in the support set. IPerform wavelet transform and gradient transform respectively, then input image I The outputs of the two transformations are input into the feature extractor to obtain the original features, gradient significance features, low-frequency features, and three high-frequency component significance features; S2. Input the original features, gradient saliency features, low-frequency features, and three high-frequency component saliency features into the feature adaptive fusion module for adaptive fusion to obtain the final fused features. ; S3, Final fusion features Inputting the data into the category-distribution joint metric model yields the feature prototypes for each known category; S4. Iteratively train the feature extractor and the adaptive feature fusion module; S5. Obtain the test images from the query set. The input image is processed by performing wavelet transform and gradient transform respectively, resulting in a test image. and test images The output after two transformations is input into the trained feature extractor to obtain the corresponding features. S6. Input the various features corresponding to those in S5 into the trained feature adaptive fusion module to obtain the final fused features. ; S7, Final fusion features The input is fed into the category-distribution joint metric model, and the total open score is calculated based on the feature prototypes of each known category. Then, the total open score is used to test images in the query set. The test results will be evaluated.

[0010] Furthermore, step S1 specifically includes the following steps: S11, For input images in the support set I Perform wavelet transform on the input image I Decomposed into multiple frequency components, including low-frequency components. LL 1. Horizontal high-frequency components LH 1. Vertical high-frequency components HL 1. Diagonal high-frequency components HH 1; S12, For the input image in the support set. I Perform gradient transformation to obtain the gradient features of the defective target. G(I) ; S13, Input image I Gradient features G(I) Multiple frequency components are input into the feature extractor to obtain the original features. Gradient significance characteristics Low-frequency characteristics Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The expressions are as follows: , , ; , , ; The feature extractor uses a six-branch feature extraction network, including branch networks. , , , , , .

[0011] Furthermore, step S2 specifically includes the following steps: S21, Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The input is fed into the feature adaptive fusion module, where the significance features of the horizontal high-frequency components are first calculated. Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The energy of each high-frequency component is used to quantify the importance of its significant features, and the expressions are as follows: ; in, , , Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components Energy; Indicates the variable to be traversed; S22. Based on the energy of the saliency features of the three high-frequency components, calculate the fusion weights of the saliency features of the three high-frequency components, as expressed below: ; in, , , These represent the significance characteristics of the horizontal high-frequency components. Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The fusion weight; S23. Significance characteristics of horizontal high-frequency components based on fusion weights Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components By performing weighted summation, high-frequency fusion features are obtained. The expression is as follows: ; S24. Original features Gradient significance characteristics Low-frequency characteristics High-frequency fusion characteristics The inputs are fed into the MLP network within the feature adaptive fusion module, where attention scores for the four types of features are calculated. The attention scores are then normalized using the softmax function to obtain the fusion weights for the four types of features. , , , The calculation formula is as follows: ; in, Representing four types of features, namely, original features Gradient significance characteristics Low-frequency characteristics High-frequency fusion characteristics ; j Indicates the first j Class features; j =1 represents the original feature. , j =2 represents the significance of the gradient. , j =3 indicates low-frequency characteristics. , j =4 represents high-frequency fusion characteristics. ; Fusion weights representing four types of features , , , ; S25. Fusion weights based on four types of features , , , The four types of features are linearly weighted and fused to obtain the final fused features. The expression is as follows: .

[0012] Furthermore, the category-distribution joint metric model in S3 includes local units, global units, and a total open score calculation unit; the local units and global units are connected in parallel and then connected to the total open score calculation unit. Local units are used to construct feature prototypes for each known category and calculate category-level open scores based on the feature prototypes for each known category. The global unit is used to construct a multivariate Gaussian distribution model, which describes the distribution of defect features for all known categories. Then, the distribution-level open score is calculated based on the distribution of defect features for all known categories. The total open score calculation unit is used to calculate the total open score based on the outputs of the local and global units.

[0013] Furthermore, step S3 specifically includes the following steps: S31, Final fusion features The input is fed into a local unit within the category-distribution joint metric model, and the feature prototype for each known category is calculated as follows: ; in, For the first i The feature prototype of a known category To support the first set i The number of image samples of a known category Indicates the first i A set of image samples of known categories; S32. Construct a multivariate Gaussian distribution model within the global unit. The multivariate Gaussian distribution model utilizes the mean vector. Covariance Matrix The formulas for describing the distribution of defect characteristics for all known categories are as follows: ; ; in, To support the centralized i The final fusion features of the input image. n This represents the total number of image samples in the support set; T This indicates transpose.

[0014] Furthermore, step S4 specifically includes the following steps: S41. Set training parameters, including optimizer name, learning rate, training period, and batch size; S42. Based on the final fusion characteristics Construct a cross-entropy loss function, and then adjust the feature extractor and feature adaptive fusion module based on the cross-entropy loss function; S43. Determine whether the predetermined iteration stopping condition has been met. If so, output the trained feature extractor and the trained feature adaptive fusion module; otherwise, return to S1.

[0015] Furthermore, step S5 specifically includes the following steps: S51. Test images in the query set. Perform wavelet transform on the test image Decomposed into multiple frequency components, including low-frequency components. LL 2. Horizontal high-frequency components LH 2. Vertical high-frequency components HL 2. Diagonal high-frequency components HH 2; S52, Test images in the query set Perform gradient transformation to obtain the gradient transformation characteristics of the defect target. G ( x ); S53. The test images in the query set x Gradient transformation characteristics G ( x ), low-frequency components LL 2. Horizontal high-frequency components LH 2. Vertical high-frequency components HL 2. Diagonal high-frequency components HH 2. The input is fed into the trained feature extractor to obtain the original features. Gradient significance characteristics Low-frequency characteristics Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components .

[0016] Furthermore, step S7 specifically includes the following steps: S71, Final fusion features The input is fed into local units within the category-distribution joint metric model to compute the final fused features. Characteristic prototypes of known categories The Euclidean distance, which reflects the similarity between the test image and the corresponding category, is used to select the negative value of the smallest Euclidean distance as the category-level open score. S72, Final fusion features The input is fed into the global unit to calculate the final fused features. The Mahalanobis distance to the multivariate Gaussian distribution model is used as the open fraction of the distribution level. S73. Input the category-level open scores and distribution-level open scores into the total open score calculation unit to calculate the total open score; S74. Test images of the query set based on the total open score. The test results will be evaluated.

[0017] Furthermore, the final fusion feature in S71 f(x) Characteristic prototypes of known categories The expression for the Euclidean distance is as follows: ; in, Indicates the final fusion feature Characteristic prototypes of known categories The Euclidean distance; Operator for Euclidean distance; The expression for the category-level open score in S71 is as follows: ; in, Indicates category-level open scores; The expression for the open fraction of the distribution level in S72 is as follows: ; in, Indicates the open fraction of the distribution level; The expression for the total open score in S73 is as follows: ; in, This represents the total open score.

[0018] In another aspect, the present invention provides a few-sample industrial defect identification system based on adaptive fusion of multiple saliency features, configured to perform a few-sample industrial defect identification method based on adaptive fusion of multiple saliency features.

[0019] The beneficial effects of this invention are: 1. This invention discloses a few-sample industrial defect identification method based on adaptive fusion of multiple saliency features. It achieves efficient identification for the first time in a scenario where training samples are scarce and unknown defects coexist in the open world, filling the technical gap of existing methods in this application scenario and providing a brand-new solution for industrial defect detection.

[0020] 2. This invention designs a feature adaptive fusion module based on multi-saliency energy guidance. By extracting gradient saliency features and multiple frequency domain saliency features (including low-frequency features and high-frequency component saliency features in three different directions) in a hierarchical manner, the fusion weights are dynamically allocated with energy as the guide. Combined with the attention mechanism, the adaptive fusion of multi-dimensional features is realized, which greatly improves the diversity and robustness of the discriminative features. The feature adaptive fusion module can still accurately capture the key features of defects with limited training samples, and significantly improves the recognition stability of known categories.

[0021] 3. This invention also designs a category-distribution joint measurement model. First, it achieves accurate matching of known categories by using the feature prototypes of known categories. Then, it uses a multivariate Gaussian distribution model to capture the distribution of defect features of all known categories, thus achieving effective division of category discrimination boundaries. Even when category features overlap or are unevenly distributed, it can accurately distinguish between known and unknown categories, greatly improving the recognition accuracy of unknown categories. Attached Figure Description

[0022] Figure 1 This is a flowchart of the few-sample industrial defect identification method in this invention. Detailed Implementation

[0023] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0024] To address the dual challenges of limited training samples and the identification of unknown defects in open-world environments, existing industrial defect detection methods propose a few-sample industrial defect identification method based on adaptive fusion of multi-saliency features. This invention innovatively introduces a feature adaptive fusion module guided by multi-saliency energy. By hierarchically extracting and adaptively fusing the original features, gradient saliency features, low-frequency features, and three high-frequency saliency features of the defect target, the method significantly enhances the discriminative ability of defect features in industrial scenarios with limited training data. Simultaneously, a category-distribution joint metric model is proposed to comprehensively measure defects from both local and global perspectives, achieving accurate classification of known defect categories and effective identification of unknown defect categories. Ultimately, this achieves the goal of efficient and reliable industrial defect identification in few-sample open-set scenarios, as detailed below: Reference Figure 1 This application provides a method for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features, comprising the following steps: S1. First, acquire multiple industrial sample images, then construct a support set and a query set according to a set ratio; for the input images in the support set... I Perform wavelet transform and gradient transform respectively, then input image I The outputs of the two transformations are input into the feature extractor to obtain the original features, gradient significance features, low-frequency features, and three high-frequency component significance features; S2. Input the original features, gradient saliency features, low-frequency features, and three high-frequency component saliency features into the feature adaptive fusion module for adaptive fusion to obtain the final fused features. ; S3, Final fusion features Inputting the data into the category-distribution joint metric model yields the feature prototypes for each known category; S4. Iteratively train the feature extractor and the adaptive feature fusion module; S5. Obtain the test images from the query set. The input image is processed by performing wavelet transform and gradient transform respectively, resulting in a test image. and test images The output after two transformations is input into the trained feature extractor to obtain the corresponding features. S6. Input the various features corresponding to those in S5 into the trained feature adaptive fusion module. Following the same fusion logic as in S2, first complete the fusion of high-frequency component saliency features, and then achieve adaptive fusion of multiple types of features through the channel attention mechanism to obtain the final fused features. ; final fusion features The characteristics of S2 have consistent distribution properties, ensuring the effectiveness of subsequent measurements; S7, Final fusion features The input is fed into the category-distribution joint metric model, and the total open score is calculated based on the feature prototypes of each known category. Then, the total open score is used to test images in the query set. The test results will be evaluated.

[0025] In some embodiments, S1 specifically includes the following steps: S11, For input images in the support set I Perform wavelet transform on the input image I Decomposed into multiple frequency components, encompassing low-frequency structural information and high-frequency texture details, including low-frequency components. LL 1. Horizontal high-frequency components LH 1. Vertical high-frequency components HL 1. Diagonal high-frequency components HH 1; Among them, low-frequency componentsLL 1. Primarily reflects the input image I The global structural information; the other three high-frequency components focus on the input image respectively. I The edges, textures and other detailed features are crucial for defect identification; S12, For the input image in the support set. I Perform gradient transformation by calculating the input image. I The gradient information is used to extract the gradient features of the defect target. G(I) Gradient features G(I) It can effectively highlight the difference in grayscale changes between the defect area and the background area, helping the model to quickly locate potential defect locations; S13, Input image I Gradient features G(I) Multiple frequency components are input into the feature extractor to obtain the original features. Gradient significance characteristics Low-frequency characteristics Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components Original features The input image was preserved. I Complete information; gradient significance features Enhanced edge information of defect areas; low-frequency features Focus on input image I The global structure; the high-frequency component saliency features in the other three different directions focus on capturing the texture details and local features of the defects, and their expressions are as follows: , , ; , , ; The feature extractor uses a six-branch feature extraction network, including branch networks. , , , , , Each branch in the six-branch feature extraction network is a lightweight convolutional neural network. Each lightweight convolutional neural network includes a convolutional layer, a batch normalization layer, and an activation function connected in sequence. While ensuring feature extraction capabilities, the computational complexity of the feature extractor is controlled to meet the real-time requirements of industrial scenarios.

[0026] In some embodiments, the feature adaptive fusion module in S2 includes a high-frequency component feature fusion branch, a fusion weight calculation branch, and a multi-type feature adaptive fusion branch. The high-frequency component feature fusion branch and the fusion weight calculation branch are connected in parallel and then connected to the multi-type feature adaptive fusion branch. The fusion weight calculation branch introduces a frequency domain channel attention mechanism and learns the importance weights of various features through an MLP (Multilayer Perceptron, MLP) network.

[0027] In some embodiments, S2 specifically includes the following steps: S21. In industrial defect images, defect areas often exhibit strong high-frequency characteristics, and the distribution of high-frequency features differs among different types of defects. To fully utilize the discriminative value of high-frequency features, the saliency features of horizontal high-frequency components are... Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The input is fed into the high-frequency component feature fusion branch within the feature adaptive fusion module to calculate the saliency features of the horizontal high-frequency components. Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The energy of each high-frequency component is used to quantify the importance of its significant features, and the expressions are as follows: ; in, , , Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The energy; the higher the energy, the richer the effective discriminative information contained in the feature. Indicates the variable to be traversed; S22. Based on the energy of the saliency features of the three high-frequency components, the fusion weights of the three high-frequency component saliency features are calculated. Through the weight allocation method, the high-energy high-frequency components can obtain greater fusion weights, thereby strengthening their contribution to defect identification. The expressions for the fusion weights are as follows: ; in, , , These represent the significance characteristics of the horizontal high-frequency components. Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The fusion weight; S23. Significance characteristics of horizontal high-frequency components based on fusion weights Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components By performing weighted summation, high-frequency fusion features are obtained. The expression is as follows: ; S24. Original features Gradient significance characteristics Low-frequency characteristics High-frequency fusion characteristics The inputs are fed into the fusion weight calculation branch within the feature adaptive fusion module, and then pass through the MLP network to calculate the attention scores of the four types of input features. The attention scores are then normalized using the softmax function to obtain the fusion weights for the four types of features. , , , The calculation formula is as follows: ; in, Representing four types of features, namely, original features Gradient significance characteristics Low-frequency characteristics High-frequency fusion characteristics ; j Indicates the first j Class features; j =1 represents the original feature. , j =2 represents the significance of the gradient. , j =3 indicates low-frequency characteristics. , j =4 represents high-frequency fusion characteristics. ; Fusion weights representing four types of features , , , The MLP network consists of two layers of perceptrons. S25. Fusion weights based on four types of features , , , The four types of features are linearly weighted and fused to obtain the final fused features. The final fusion features It contains both global structural information and integrates local texture details and edge features, exhibiting strong discriminative power and providing a solid feature foundation for subsequent category classification and unknown class identification; the final fused features The expression is as follows: .

[0028] The core idea of ​​the feature adaptive fusion module in this invention is to dynamically adjust the weights of different features through frequency domain energy analysis, thereby achieving adaptive fusion of multi-dimensional features and enhancing the discriminative power of defect features.

[0029] In some embodiments, the category-distribution joint metric model in S3 includes local units, global units, and a total open score calculation unit; the local units and global units are connected in parallel and then connected to the total open score calculation unit. Local units are used to construct feature prototypes for each known category and calculate category-level open scores based on the feature prototypes for each known category. The global unit is used to construct a multivariate Gaussian distribution model, which describes the distribution of defect features for all known categories. Then, the distribution-level open score is calculated based on the distribution of defect features for all known categories. The multivariate Gaussian distribution model can accurately fit the distribution space of known category features, providing an important basis for determining whether a test sample belongs to the distribution range of a known category. The total open score calculation unit is used to calculate the total open score based on the outputs of the local and global units.

[0030] In some embodiments, S3 specifically includes the following steps: S31, Final fusion features The input is fed into a local unit within the category-distribution joint metric model, and the feature prototype for each known category is calculated as follows: ; in, For the first i The feature prototypes of a known category can effectively represent the core features of that category, providing a standard template for subsequent category matching. To support the first set i The number of image samples of a known category Indicates the first i A set of image samples of known categories; S32. Construct a multivariate Gaussian distribution model within the global unit. The multivariate Gaussian distribution model utilizes the mean vector. Covariance Matrix This describes the distribution of defect features for all known categories, including the overall distribution pattern and correlation between features, providing a global basis for distinguishing between known and unknown categories. The calculation formulas are as follows: ; ; in, To support the centralized i The final fusion features of the input image, where the input image I The corresponding final fusion feature is the final fusion feature calculated in S25. , n This represents the total number of image samples in the support set; T Indicates transpose; It is the mean vector of the feature distribution, reflecting the central location of the known category features; The covariance matrix characterizes the correlation between features and the degree of dispersion of their distributions. In some embodiments, S4 specifically includes the following steps: S41. Set training parameters, including optimizer name, learning rate, training period, batch size, etc. Specifically, the Adam optimizer is used for training, the learning rate is set to 0.001, the training period is 200 epochs (batch), and each batch contains 16 images. S42. Based on the final fusion characteristics Construct a cross-entropy loss function, and then adjust the feature extractor and feature adaptive fusion module based on the cross-entropy loss function; S43. Determine whether the predetermined iteration stopping condition has been met. If so, output the trained feature extractor and the trained feature adaptive fusion module; otherwise, return to S1.

[0031] In some embodiments, S5 specifically includes the following steps: S51. Test images in the query set. Perform wavelet transform on the test image Decomposed into multiple frequency components, including low-frequency components. LL 2. Horizontal high-frequency components LH 2. Vertical high-frequency components HL 2. Diagonal high-frequency components HH 2; S52. Test images in the query set. x Perform gradient transformation to obtain the gradient transformation characteristics of the defect target. G ( x ); S53. The test images in the query set xGradient transformation characteristics G ( x ), low-frequency components LL 2. Horizontal high-frequency components LH 2. Vertical high-frequency components HL 2. Diagonal high-frequency components HH 2. The input is fed into the trained feature extractor to obtain the original features. Gradient significance characteristics Low-frequency characteristics Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components .

[0032] In some embodiments, S7 specifically includes the following steps: S71, Final fusion features The input is fed into local units within the category-distribution joint metric model to compute the final fused features. Characteristic prototypes of known categories The Euclidean distance is used to determine the similarity between the test image and the corresponding category. The smaller the distance, the higher the similarity. The negative value of the smallest Euclidean distance is then selected as the category-level open score. S72, Final fusion features The input is fed into the global unit to calculate the final fused features. Compared to the Mahalanobis distance of a multivariate Gaussian distribution model, the Mahalanobis distance not only considers the mean and variance of the feature distribution but also takes into account the correlation between features, thus providing a more accurate measurement of the test image. The degree of fit with the known category feature distribution; the smaller the Mahalanobis distance, the better the test image. The more likely the sample belongs to the known category feature distribution range, the more likely it is to be an unknown category sample; conversely, the more likely it is to be an unknown category sample; then the Mahalanobis distance is used as the distribution level open score; S73. Input the category-level open scores and distribution-level open scores into the total open score calculation unit to calculate the total open score; S74. A discrimination threshold θ=14.5 is preset. The discrimination threshold θ is determined through a large number of experiments and can achieve the best balance between the classification accuracy of known categories and the recognition rate of unknown categories. Test images in the query set based on the total open score and the discrimination threshold θ The test results will be evaluated.

[0033] In some embodiments, the final fusion feature in S71 Characteristic prototypes of known categories The expression for the Euclidean distance is as follows: ; in, Indicates the final fusion feature Characteristic prototypes of known categories The Euclidean distance; Operator for Euclidean distance; The expression for the category-level open score in S71 is as follows: ; in, Indicates category-level open scores; The expression for the open fraction of the distribution level in S72 is as follows: ; in, Indicates the open fraction of the distribution level; The expression for the total open score in S73 is as follows: ; in, This represents the total open score.

[0034] Extensive experimental verification has shown that this invention demonstrates superior performance on two major industrial defect datasets: ID-OSD and MVTec. Compared with existing advanced open-set recognition and few-shot learning methods, it improves ACC (accuracy) by at least 26% and F1 score by at least 35%. F1 score is a comprehensive indicator used in machine learning to evaluate the performance of classification models; it is the harmonic mean of precision and recall. AUROC (area under the ROC curve) is improved by at least 24%, while maintaining a high detection speed (47 frames per second). This significantly enhances the deployability and reliability of industrial surface defect detection technology in actual production, effectively reducing enterprise production costs and improving production efficiency.

[0035] In another aspect, the present invention provides a few-sample industrial defect identification system based on adaptive fusion of multiple saliency features, configured to perform a few-sample industrial defect identification method based on adaptive fusion of multiple saliency features.

[0036] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features, characterized in that, Includes the following steps: S1, for input images in the support set. I Perform wavelet transform and gradient transform respectively, then input image I The outputs of the two transformations are input into the feature extractor to obtain the original features, gradient significance features, low-frequency features, and three high-frequency component significance features; S2. Input the original features, gradient saliency features, low-frequency features, and three high-frequency component saliency features into the feature adaptive fusion module for adaptive fusion to obtain the final fused features. ; S3, Final fusion features Inputting the data into the category-distribution joint metric model yields the feature prototypes for each known category; S4. Iteratively train the feature extractor and the adaptive feature fusion module; S5. Obtain the test images from the query set. The input image is processed by performing wavelet transform and gradient transform respectively, resulting in a test image. and test images The output after two transformations is input into the trained feature extractor to obtain the corresponding features. S6. Input the various features corresponding to those in S5 into the trained feature adaptive fusion module to obtain the final fused features. ; S7, Final fusion features The input is fed into the category-distribution joint metric model, and the total open score is calculated based on the feature prototype of each known category. Then, the test results of the test image x in the query set are evaluated using the total open score. S2 specifically includes the following steps: S21, Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The input is fed into the feature adaptive fusion module, where the significance features of the horizontal high-frequency components are first calculated. Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The energy of each high-frequency component is used to quantify the importance of its significant features, and the expressions are as follows: ； in, , , Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components Energy; i Indicates the variable to be traversed; S22. Based on the energy of the saliency features of the three high-frequency components, calculate the fusion weights of the saliency features of the three high-frequency components, as expressed below: ； in, , , These represent the significance characteristics of the horizontal high-frequency components. Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The fusion weight; S23. Significance characteristics of horizontal high-frequency components based on fusion weights Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components By performing weighted summation, high-frequency fusion features are obtained. The expression is as follows: ； S24. Original features Gradient significance characteristics Low-frequency characteristics High-frequency fusion characteristics The inputs are fed into the MLP network within the feature adaptive fusion module, where attention scores for the four types of features are calculated. The attention scores are then normalized using the softmax function to obtain the fusion weights for the four types of features. , , , The calculation formula is as follows: ； in, Representing four types of features, namely, original features Gradient significance characteristics Low-frequency characteristics High-frequency fusion characteristics ; j Indicates the first j Class features; j =1 represents the original feature. , j =2 represents the significance of the gradient. , j =3 indicates low-frequency characteristics. , j =4 represents high-frequency fusion characteristics. ; Fusion weights representing four types of features , , , ; S25. Fusion weights based on four types of features , , , The four types of features are linearly weighted and fused to obtain the final fused features. The expression is as follows: ； The category-distribution joint metric model in S3 includes local units, global units, and a total open score calculation unit; the local units and global units are connected in parallel and then connected to the total open score calculation unit. Local units are used to construct feature prototypes for each known category and calculate category-level open scores based on the feature prototypes for each known category. The global unit is used to construct a multivariate Gaussian distribution model, which describes the distribution of defect features for all known categories. Then, the distribution-level open score is calculated based on the distribution of defect features for all known categories. The total open score calculation unit is used to calculate the total open score based on the outputs of the local and global units.

2. The method for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features according to claim 1, characterized in that, S1 specifically includes the following steps: S11, For input images in the support set I Perform wavelet transform on the input image I Decomposed into multiple frequency components, including low-frequency components. LL 1. Horizontal high-frequency components LH 1. Vertical high-frequency components HL 1. Diagonal high-frequency components HH 1; S12, For the input image in the support set. I Perform gradient transformation to obtain the gradient features of the defective target. G(I) ; S13, Input image I Gradient features G(I) The multiple frequency components are input into the feature extractor to obtain the original features. Gradient significance characteristics Low-frequency characteristics Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components The expressions are as follows: ， ， ； ， ， ； The feature extractor uses a six-branch feature extraction network, including branch networks. , , , , , .

3. The method for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features according to claim 2, characterized in that, S3 specifically includes the following steps: S31, Final fusion features The input is fed into a local unit within the category-distribution joint metric model, and the feature prototype for each known category is calculated as follows: ； in, For the first i The feature prototype of a known category To support the first set i The number of image samples of a known category Indicates the first i A set of image samples of known categories; S32. Construct a multivariate Gaussian distribution model within the global unit. The multivariate Gaussian distribution model utilizes the mean vector. Covariance Matrix The formulas for describing the distribution of defect characteristics for all known categories are as follows: ； ； in, To support the centralized i The final fusion features of the input image. n This represents the total number of image samples in the support set; T This indicates transpose.

4. The method for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features according to claim 3, characterized in that, S4 specifically includes the following steps: S41. Set training parameters, including optimizer name, learning rate, training period, and batch size; S42. Based on the final fusion characteristics Construct a cross-entropy loss function, and then adjust the feature extractor and feature adaptive fusion module based on the cross-entropy loss function; S43. Determine whether the predetermined iteration stopping condition has been met. If so, output the trained feature extractor and the trained feature adaptive fusion module; otherwise, return to S1.

5. The method for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features according to claim 4, characterized in that, S5 specifically includes the following steps: S51. Test images in the query set. Perform wavelet transform on the test image Decomposed into multiple frequency components, including low-frequency components. LL 2. Horizontal high-frequency components LH 2. Vertical high-frequency components HL 2. Diagonal high-frequency components HH 2; S52, Test images in the query set Perform gradient transformation to obtain the gradient transformation characteristics of the defect target. G ( x ); S53. The test images in the query set x Gradient transformation characteristics G ( x ), low-frequency components LL 2. Horizontal high-frequency components LH 2. Vertical high-frequency components HL 2. Diagonal high-frequency components HH 2. The input is fed into the trained feature extractor to obtain the original features. Gradient significance characteristics Low-frequency characteristics Significance characteristics of horizontal high-frequency components Significance characteristics of vertical high-frequency components Significance characteristics of diagonal high-frequency components .

6. The method for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features according to claim 5, characterized in that, S7 specifically includes the following steps: S71, Final fusion features The input is fed into local units within the category-distribution joint metric model to compute the final fused features. Characteristic prototypes of known categories The Euclidean distance, which reflects the similarity between the test image and the corresponding category, is used to select the negative value of the smallest Euclidean distance as the category-level open score. S72, Final fusion features The input is fed into the global unit to calculate the final fused features. The Mahalanobis distance to the multivariate Gaussian distribution model is used as the open fraction of the distribution level. S73. Input the category-level open scores and distribution-level open scores into the total open score calculation unit to calculate the total open score; S74. Test images of the query set based on the total open score. The test results will be evaluated.

7. The method for identifying industrial defects with few samples based on adaptive fusion of multiple saliency features according to claim 6, characterized in that, The final fusion feature in S71 Characteristic prototypes of known categories The expression for the Euclidean distance is as follows: ； in, Indicates the final fusion feature Characteristic prototypes of known categories The Euclidean distance; Operator for Euclidean distance; The expression for the category-level open score in S71 is as follows: ； in, Indicates category-level open scores; The expression for the open fraction of the distribution level in S72 is as follows: ； in, Indicates the open fraction of the distribution level; The expression for the total open score in S73 is as follows: ； in, This represents the total open score.

8. A few-sample industrial defect identification system based on adaptive fusion of multiple saliency features, characterized in that, It is configured to perform a few-sample industrial defect identification method based on adaptive fusion of multiple saliency features as described in any one of claims 1 to 7.

Images