Unsupervised learning defect detection method based on few sample data

Abstract

The invention belongs to the technical field of few-sample data, and discloses an unsupervised learning defect detection method based on few-sample data, comprising the following steps: step 1, acquiring a few-sample normal sample and corresponding physical attribute data, the normal sample being a defect-free target object image, and constructing a heterogeneous feature extraction network to obtain a feature extraction result; a visual mode and a physical attribute mode are deeply fused, a visual branch extracts visual features such as textures and edges through MobileNet, a physical branch captures physical features such as surface roughness and temperature distribution through Transformer, and interactive fusion and unified mapping of the two features are achieved by means of a cross attention mechanism. By means of the design, multi-dimensional characteristics of hidden defects such as metal hidden cracks and uneven density in glass can be accurately captured, the defects have no obvious difference visually, but can cause physical attribute abnormity, and characteristic information of the defects can be completely reserved through heterogeneous characteristic vectors.

Description

Technical Field

[0001] This invention belongs to the field of few-sample data technology, specifically a defect detection method based on unsupervised learning using few-sample data. Background Technology

[0002] In fields such as industrial manufacturing, electronic component production, and precision instrument processing, product defect detection is a key link in ensuring production quality, reducing resource consumption, and enhancing market competitiveness. With the rapid iteration of intelligent manufacturing technology, traditional defect detection methods that rely on manual visual inspection are no longer able to meet the needs of modern production lines for high-precision, high-speed inspection due to problems such as low detection efficiency, large subjective errors, and weak ability to identify small or non-visual defects. Automated defect detection technology based on machine learning is gradually becoming the mainstream development direction in the industry.

[0003] However, in actual industrial scenarios, especially in the production of high-end precision products such as aero-engine blades, semiconductor wafers, and customized medical devices, there is often a shortage of sample data. On the one hand, the production batches of such products are small, and the normal number of samples is often only 10-50. Moreover, due to strict production process control, defective samples are extremely rare, and there are even no defective samples available at all during the trial production or small-batch production stages. On the other hand, defect annotation requires professional technicians to combine domain knowledge to complete. For minute defects such as micron-level scratches on metal surfaces, microbubbles inside glass, or non-visual defects such as uneven material density or abnormal local mechanical properties, the annotation is extremely difficult and costly, which further limits the applicability of supervised learning methods. Summary of the Invention

[0004] The purpose of this invention is to provide an unsupervised learning defect detection method based on few sample data, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: An unsupervised learning defect detection method based on few sample data includes the following steps: Step 1: Obtain a small number of normal samples and their corresponding physical property data. The normal samples are images of target objects without defects. The physical property data includes at least one of the following: surface roughness data, local light reflectance data, surface height map data, and temperature distribution data of the target object collected by sensors. Step 2: Construct a heterogeneous feature extraction network to extract and fuse features from normal samples and physical attribute data to obtain heterogeneous feature vectors; Step 3: Generate pseudo-defect features based on the dynamic self-distillation mechanism and optimize the model; Step 4: Calculate the anomaly score and determine the defect through a cross-scale comparison mechanism.

[0006] Preferably, the specific steps for constructing the heterogeneous feature extraction network according to step 2 are as follows: Step 201: The visual branch uses a lightweight CNN network to process normal samples and extract texture and edge features; Step 202: The physical branch uses a Transformer network to process the physical attribute data and extract local physical attribute features; Step 203: The fusion module maps the texture features and edge features output by the visual branch and the local physical attribute features output by the physical branch to a unified feature space through a cross-attention mechanism, generating heterogeneous feature vectors.

[0007] Preferably, the detailed steps for generating pseudo-defect features and optimizing the model based on the dynamic self-distillation mechanism described in step 3 are as follows: 301. In the initial stage, the heterogeneous feature extraction network is used as the teacher network and the parameters are frozen. The heterogeneous feature vector output in step 2 is used as the anchor point to train the student network to learn and reconstruct the heterogeneous feature vector. 302, Mutation stage: The heterogeneous feature vector output in step 2 is subjected to controllable perturbation. Controllable perturbation includes adding local height anomaly perturbation to physical property features and adding texture continuity disruption perturbation to visual features to generate pseudo-defect features. 303, Role Reversal Phase: The trained student network is used as the new teacher network. The new student network is trained using pseudo-defect features as training samples. Steps 3.1 to 3.3 are executed iteratively to optimize the normal and abnormal feature discrimination boundaries.

[0008] Preferably, the detailed steps for calculating the anomaly score and determining the defect based on the cross-scale comparison mechanism described in step 4 are as follows: 401. Features are extracted at different levels of the heterogeneous feature extraction network, with shallow layers extracting detailed features and deep layers extracting semantic features. 402. Calculate the deviation of features at each level. The deviation of shallow features is obtained by comparing the consistency between local texture and physical properties, while the deviation of deep features is obtained by comparing the rationality of the overall structure. 403. Weighted fusion of the deviation of features at each level is performed. The weights are adaptively adjusted through the dynamic self-distillation process in step 3 to obtain the anomaly score. When the anomaly score exceeds the preset threshold, the corresponding area is determined to be a defect.

[0009] Preferably, the number of normal samples in step 1 is 10-50.

[0010] Preferably, the lightweight CNN network in step 201 is the MobileNet network.

[0011] Preferably, the intensity of the controllable perturbation in step 302 is adaptively adjusted by the standard deviation of the normal sample characteristic distribution, and the perturbation intensity is positively correlated with the standard deviation.

[0012] Preferably, the consistency between local texture and physical properties in step 402 is determined by calculating the cosine similarity of feature vectors of adjacent regions. If the similarity is lower than a preset threshold, it is determined to be inconsistent. The rationality of the overall structure in step 402 is determined by comparing the feature vector with the Euclidean distance of the cluster center of normal sample features. If the distance exceeds a preset threshold, it is determined to be unreasonable. The preset threshold in step 403 is determined by multiplying the maximum value of the deviation of normal sample features with a preset safety factor, wherein the preset safety factor is 1.2-1.5.

[0013] Preferably, the number of times the pseudo-defect features are generated and the model is optimized based on the dynamic self-distillation mechanism in step 3 is 5-10.

[0014] Preferably, the cross-attention mechanism in step 203 achieves interactive fusion of the two features by calculating the attention weight matrix of visual features and physical features. The attention weight matrix is ​​obtained by normalizing the feature similarity matrix using the softmax function.

[0015] The beneficial effects of this invention are as follows: 1. This invention generates pseudo-defect features based on a dynamic self-distillation mechanism using only 10-50 normal samples. It optimizes the model without relying on scarce real defect samples. In the initial stage, the heterogeneous features of normal samples are used to train the student network's reconstruction ability. Subsequently, pseudo-samples that closely resemble real defect patterns are generated through controlled perturbation. The discrimination boundary is iteratively optimized by switching the roles of teachers and students in the network. This solves the problem of insufficient model training data in the production of high-precision products, where there are few normal samples and no defect samples. It significantly reduces the dependence on labeled data and can be directly adapted to industrial scenarios such as trial production and small-batch production, significantly expanding the application scope of unsupervised defect detection technology.

[0016] 2. This invention constructs a heterogeneous feature extraction network to deeply integrate visual and physical property modalities. The visual branch extracts visual features such as texture and edges through MobileNet, while the physical branch captures physical features such as surface roughness and temperature distribution through Transformer. Then, a cross-attention mechanism is used to achieve interactive fusion and unified mapping of the two types of features. This design can accurately capture the multi-dimensional features of latent defects such as latent cracks in metals and uneven density inside glass. These defects are not visually obvious, but they will cause abnormal physical properties. The heterogeneous feature vector can completely preserve the feature information of the defects, avoiding the problems of missed detection and false detection caused by the lack of information in a single modality, and significantly improving the recognition accuracy of non-visual and latent defects.

[0017] 3. This invention calculates anomaly scores through a cross-scale comparison mechanism. At the shallow level of the heterogeneous feature extraction network, it focuses on detailed features such as local texture and physical property consistency, using cosine similarity to determine minute defects such as micron-level scratches. At the deep level, it focuses on semantic features such as the overall structural rationality, using Euclidean distance comparison to identify structural defects such as part contour deformation. Simultaneously, the fusion weights of feature deviations at each level are adaptively adjusted through a dynamic self-distillation process, ensuring that the feature weights of different types of defects match their detection requirements. Furthermore, a preset threshold is determined by combining the feature deviation of normal samples with a safety factor of 1.2-1.5, balancing detection sensitivity and false positive control. Ultimately, this achieves comprehensive and accurate detection of both minute and structural defects, meeting the core requirements of industrial quality inspection: high precision and low false negatives. Attached Figure Description

[0018] Figure 1 This is a flowchart of the overall method of the present invention; Figure 2 This is a schematic diagram of step 2 of the heterogeneous feature extraction network of the present invention; Figure 3 This is a schematic diagram illustrating the working principle of step 3 of the dynamic self-distillation mechanism of the present invention. Figure 4 This is a schematic diagram of step 4 of the cross-scale comparison and defect determination method of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figures 1 to 4 As shown, this embodiment of the invention provides an unsupervised learning defect detection method based on few sample data, including the following steps: Step 1: Obtain a small number of normal samples and their corresponding physical property data. The normal samples are images of target objects without defects. The physical property data includes at least one of the following: surface roughness data, local light reflectance data, surface height map data, and temperature distribution data of the target object collected by sensors. Step 2: Construct a heterogeneous feature extraction network to extract and fuse features from normal samples and physical attribute data to obtain heterogeneous feature vectors; Step 3: Generate pseudo-defect features based on the dynamic self-distillation mechanism and optimize the model; Step 4: Calculate the anomaly score and determine the defect through a cross-scale comparison mechanism.

[0021] By acquiring images of defect-free target objects from a small number of normal samples and corresponding physical property data, including at least one of the following data collected by sensors: surface roughness, local light reflectivity, surface height map, and temperature distribution, the basic data preparation for defect detection is completed. This provides complete data support covering both visual and physical dimensions for subsequent feature extraction, avoiding the problem that single visual data cannot capture non-visual defect information.

[0022] The specific steps for constructing the heterogeneous feature extraction network according to step 2 are as follows: Step 201: The visual branch uses a lightweight CNN network to process normal samples and extract texture and edge features; Step 202: The physical branch uses a Transformer network to process the physical attribute data and extract local physical attribute features; Step 203: The fusion module maps the texture features and edge features output by the visual branch and the local physical attribute features output by the physical branch to a unified feature space through a cross-attention mechanism, generating heterogeneous feature vectors.

[0023] By constructing a heterogeneous feature extraction network, the visual branch processes normal samples to extract texture and edge features, the physical branch processes physical attribute data to extract local physical attribute features, and the fusion module maps to a unified feature space through a cross-attention mechanism, a deep fusion of visual and physical features is achieved, generating a heterogeneous feature vector that can fully represent the attributes of the target object. This breaks through the limitations of single-modal features and lays the feature foundation for accurately identifying latent and non-visual defects.

[0024] The detailed steps for generating pseudo-defect features and optimizing the model based on the dynamic self-distillation mechanism in step 3 are as follows: 301. In the initial stage, the heterogeneous feature extraction network is used as the teacher network and the parameters are frozen. The heterogeneous feature vector output in step 2 is used as the anchor point to train the student network to learn and reconstruct the heterogeneous feature vector. 302, Mutation stage: The heterogeneous feature vector output in step 2 is subjected to controllable perturbation. Controllable perturbation includes adding local height anomaly perturbation to physical property features and adding texture continuity disruption perturbation to visual features to generate pseudo-defect features. 303, Role Reversal Phase: The trained student network is used as the new teacher network. The new student network is trained using pseudo-defect features as training samples. Steps 3.1 to 3.3 are executed iteratively to optimize the normal and abnormal feature discrimination boundaries.

[0025] The three-stage operation of the dynamic self-distillation mechanism initially uses a heterogeneous feature extraction network as the teacher network to train the student network's reconstruction ability; the mutation stage controls the perturbation of heterogeneous feature vectors to generate pseudo-defect features; and the role-switching stage iteratively trains and optimizes the discrimination boundary. In the absence of real defect samples, it expands the abnormal feature pattern space, optimizes the discrimination boundary between normal and abnormal features, solves the problem of insufficient model training data in scenarios with few samples, and improves the model's ability to generalize and identify defects.

[0026] The detailed steps for calculating the anomaly score and determining the defect based on the cross-scale comparison mechanism in step 4 are as follows: 401. Features are extracted at different levels of the heterogeneous feature extraction network, with shallow layers extracting detailed features and deep layers extracting semantic features. 402. Calculate the deviation of features at each level. The deviation of shallow features is obtained by comparing the consistency between local texture and physical properties, while the deviation of deep features is obtained by comparing the rationality of the overall structure. 403. Weighted fusion of the deviation of features at each level is performed. The weights are adaptively adjusted through the dynamic self-distillation process in step 3 to obtain the anomaly score. When the anomaly score exceeds the preset threshold, the corresponding area is determined to be a defect.

[0027] By employing a cross-scale comparison mechanism to extract shallow detail features and deep semantic features, deviations are calculated and weighted fusion is performed to obtain an anomaly score. Combined with a preset threshold to determine defects, this achieves full coverage detection of minor defects that rely on shallow detail features and structural defects that rely on deep semantic features. At the same time, through adaptive weight adjustment and reasonable threshold setting, both detection sensitivity and false detection control are taken into account to ensure the accuracy of defect determination.

[0028] In step 1, the number of normal samples is 10-50.

[0029] By limiting the number of normal samples to 10-50, the applicability of the method to scenarios with few samples is clarified, ensuring its effective application in actual industrial scenarios where normal samples are scarce, such as small-batch production and trial production of high-end precision products, thus further enhancing the practicality of the method.

[0030] In step 201, the lightweight CNN network is the MobileNet network.

[0031] By employing the MobileNet network, a lightweight CNN network, in the vision branch, the computational complexity and parameter size of the network are reduced while ensuring the effectiveness of texture and edge feature extraction, thereby improving feature extraction efficiency and adapting to the speed requirements of industrial production lines.

[0032] In step 302, the intensity of the controllable perturbation is adaptively adjusted by the standard deviation of the normal sample characteristic distribution, and the perturbation intensity is positively correlated with the standard deviation.

[0033] By adaptively adjusting the controllable perturbation intensity based on the standard deviation of the normal sample feature distribution, the perturbation intensity is positively correlated with the standard deviation, ensuring that the generated pseudo-defect features deviate from the normal feature distribution while conforming to the feature variation law of real defects. This avoids the problem that excessive perturbation leads to large differences between pseudo-defects and real defects, or that insufficient perturbation fails to effectively expand abnormal patterns, thereby improving the effectiveness of pseudo-defect features.

[0034] In step 402, the consistency between local texture and physical properties is determined by calculating the cosine similarity of feature vectors of adjacent regions. If the similarity is lower than a preset threshold, it is determined to be inconsistent. In step 402, the rationality of the overall structure is determined by comparing the feature vector with the Euclidean distance of the cluster center of the normal sample features. If the distance exceeds a preset threshold, it is determined to be unreasonable. In step 403, the preset threshold is determined by multiplying the maximum value of the deviation of the normal sample features with a preset safety factor of 1.2-1.5.

[0035] By employing cosine similarity to determine the consistency between local texture and physical attributes, Euclidean distance to determine the rationality of the overall structure, and setting a preset threshold as the product of the maximum deviation of normal sample features and a safety coefficient of 1.2-1.5, specific and quantifiable judgment criteria are provided for feature deviation calculation and defect judgment, reducing the interference of subjective factors and improving the objectivity and accuracy of defect judgment.

[0036] In step 3, the number of times the pseudo-defect features are generated and the model is optimized based on the dynamic self-distillation mechanism is executed is 5-10 times.

[0037] By setting the number of iterations of the dynamic self-distillation mechanism to 5-10, the model can fully optimize the normal-abnormal feature discrimination boundary while avoiding the problems of low training efficiency and overfitting caused by too many iterations, or insufficient discrimination boundary optimization caused by too few iterations, thus balancing model performance and training efficiency.

[0038] In step 203, the cross-attention mechanism calculates the attention weight matrix of visual features and physical features to achieve interactive fusion of the two features. The attention weight matrix is ​​obtained by normalizing the feature similarity matrix using the softmax function.

[0039] By clarifying the implementation method of the cross-attention mechanism, the similarity matrix of visual and physical features is calculated. After softmax normalization, the attention weight matrix is ​​obtained to achieve feature fusion. The technical path of feature fusion is clearly defined, ensuring that visual and physical features can achieve effective interaction and fusion based on their mutual correlation, improving the representation ability of heterogeneous feature vectors, and further ensuring the accuracy of defect identification.

[0040] Working principle and usage process: First, 10-50 images of a defect-free target object are acquired using sensors as a small-sample normal sample. Simultaneously, at least one of the following physical attribute data is collected: surface roughness data, local illumination reflectance data, surface height map data, and temperature distribution data. Next, a heterogeneous feature extraction network is constructed. The visual branch uses a MobileNet network to process the normal samples, extracting texture and edge features. The physical branch uses a Transformer network to process the physical attribute data, extracting local physical attribute features. The fusion module calculates the feature similarity matrix between visual and physical features, which is then normalized using a softmax function to obtain the attention weight matrix. This matrix maps the two types of features to a unified feature space, generating a heterogeneous feature vector. Then, the model optimization stage based on the dynamic self-distillation mechanism is entered. In the initial stage, the heterogeneous feature extraction network is used as the teacher network and the parameters are frozen. The student network is trained to learn and reconstruct the heterogeneous feature vector using the heterogeneous feature vector as the anchor point. In the mutation stage, the controllable perturbation intensity is determined according to the standard deviation of the normal sample feature distribution. Local height anomaly perturbation is added to the physical attribute features and texture continuity disruption perturbation is added to the visual features to generate pseudo-defect features. In the role-swapping stage, the trained student network is used as the new teacher network. The new student network is trained using the pseudo-defect features as training samples. The initial, mutation, and role-swapping steps are repeated 5-10 times to optimize the normal-abnormal feature discrimination boundary. Finally, defects are determined through a cross-scale comparison mechanism. The heterogeneous feature extraction network extracts detailed features at the shallow level and semantic features at the deep level. When calculating the shallow feature deviation, the cosine similarity of the feature vectors of adjacent regions is compared to ensure consistency between local texture and physical properties. If the similarity is below a preset threshold, it is considered inconsistent. When calculating the deep feature deviation, the rationality of the overall structure is judged by comparing the Euclidean distance between the feature vector and the cluster center of normal sample features. If the distance exceeds a preset threshold, it is considered unreasonable. The fusion weights of the feature deviations at each level are adaptively adjusted according to the dynamic self-distillation process, and the weighted fusion is used to obtain an anomaly score. The preset threshold is the product of the maximum value of the normal sample feature deviation and a preset safety coefficient between 1.2 and 1.5. When the anomaly score exceeds the preset threshold, the corresponding region is determined to be defective.

[0041] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0042] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A defect detection method based on unsupervised learning using few sample data, characterized in that, Includes the following steps: Step 1: Obtain a small number of normal samples and their corresponding physical property data. The normal samples are images of target objects without defects. The physical property data includes at least one of the following: surface roughness data, local light reflectance data, surface height map data, and temperature distribution data of the target object collected by sensors. Step 2: Construct a heterogeneous feature extraction network to extract and fuse features from normal samples and physical attribute data to obtain heterogeneous feature vectors; Step 3: Generate pseudo-defect features based on the dynamic self-distillation mechanism and optimize the model; Step 4: Calculate the anomaly score and determine the defect through a cross-scale comparison mechanism.

2. The unsupervised learning defect detection method based on few sample data according to claim 1, characterized in that: The specific steps for constructing the heterogeneous feature extraction network as described in step 2 are as follows: Step 201: The visual branch uses a lightweight CNN network to process normal samples and extract texture and edge features; Step 202: The physical branch uses a Transformer network to process the physical attribute data and extract local physical attribute features; Step 203: The fusion module maps the texture features and edge features output by the visual branch and the local physical attribute features output by the physical branch to a unified feature space through a cross-attention mechanism, generating heterogeneous feature vectors.

3. The unsupervised learning defect detection method based on few sample data according to claim 1, characterized in that: The detailed steps for generating pseudo-defect features and optimizing the model based on the dynamic self-distillation mechanism described in step 3 are as follows:

301. In the initial stage, the heterogeneous feature extraction network is used as the teacher network and the parameters are frozen. The heterogeneous feature vector output in step 2 is used as the anchor point to train the student network to learn and reconstruct the heterogeneous feature vector. 302, Mutation stage: The heterogeneous feature vector output in step 2 is subjected to controllable perturbation. Controllable perturbation includes adding local height anomaly perturbation to physical property features and adding texture continuity disruption perturbation to visual features to generate pseudo-defect features. 303, Role Reversal Phase: The trained student network is used as the new teacher network. The new student network is trained using pseudo-defect features as training samples. Steps 3.1 to 3.3 are executed iteratively to optimize the normal and abnormal feature discrimination boundaries.

4. The unsupervised learning defect detection method based on few sample data according to claim 1, characterized in that: The detailed steps for calculating anomaly scores and determining defects based on the cross-scale comparison mechanism described in step 4 are as follows:

401. Features are extracted at different levels of the heterogeneous feature extraction network, with shallow layers extracting detailed features and deep layers extracting semantic features.

402. Calculate the deviation of features at each level. The deviation of shallow features is obtained by comparing the consistency between local texture and physical properties, while the deviation of deep features is obtained by comparing the rationality of the overall structure.

403. Weighted fusion of the deviation of features at each level is performed. The weights are adaptively adjusted through the dynamic self-distillation process in step 3 to obtain the anomaly score. When the anomaly score exceeds the preset threshold, the corresponding area is determined to be a defect.

5. The unsupervised learning defect detection method based on few sample data according to claim 1, characterized in that: The number of normal samples mentioned in step 1 is 10-50.

6. The unsupervised learning defect detection method based on few sample data according to claim 2, characterized in that: The lightweight CNN network mentioned in step 201 is the MobileNet network.

7. The unsupervised learning defect detection method based on few sample data according to claim 3, characterized in that: The intensity of the controllable perturbation in step 302 is adaptively adjusted by the standard deviation of the normal sample characteristic distribution, and the perturbation intensity is positively correlated with the standard deviation.

8. The unsupervised learning defect detection method based on few sample data according to claim 4, characterized in that: The consistency between local texture and physical properties in step 402 is determined by calculating the cosine similarity of feature vectors of adjacent regions. If the similarity is lower than a preset threshold, it is determined to be inconsistent. The rationality of the overall structure in step 402 is determined by comparing the feature vector with the Euclidean distance of the cluster center of normal sample features. If the distance exceeds a preset threshold, it is determined to be unreasonable. The preset threshold in step 403 is determined by multiplying the maximum value of the deviation of normal sample features with a preset safety factor, which is 1.2-1.

5.

9. The unsupervised learning defect detection method based on few sample data according to claim 1, characterized in that: The number of iterations for generating pseudo-defect features based on the dynamic self-distillation mechanism and optimizing the model in step 3 is 5-10.

10. The unsupervised learning defect detection method based on few sample data according to claim 2, characterized in that: The cross-attention mechanism described in step 203 achieves interactive fusion of the two features by calculating the attention weight matrix of visual features and physical features. The attention weight matrix is ​​obtained by normalizing the feature similarity matrix using the softmax function.

Images