Automatic detection and classification method of intelligent manufacturing defects based on machine vision

Abstract

The present invention discloses a method for automatic detection and classification of intelligent manufacturing defects based on machine vision, which specifically relates to the technical field of automatic defect detection and classification. By constructing a high-resolution multi-source sample data set and introducing image preprocessing operations, it covers the defect manifestations under different manufacturing batches, surface conditions and lighting conditions. Through the image segmentation network, a defect probability heat map is generated and primary defect candidate areas are extracted. The pseudo-defect high-frequency interference coefficient is calculated in combination with the high-frequency pseudo-defect feature tensor, and the multi-class defect overlapping coupling coefficient is calculated based on the multi-classification confidence distribution and semantic adjacency. The pseudo-defect interference intensity and the fuzziness of the multi-class defect boundaries in the defect candidate area are accurately characterized. A sample label contamination risk assessment model is constructed to realize automatic identification and screening of high-contamination risk samples in the training data, significantly reducing the interference of mislabeled samples on the deep model training, and effectively preventing the erosion of the model generalization ability by the erroneous feature-label mapping.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic defect detection and classification, and more specifically, to a method for automatic defect detection and classification in intelligent manufacturing based on machine vision. Background Art

[0002] With the continuous advancement of intelligent manufacturing, automatic defect detection and classification technologies based on machine vision have been widely adopted in various industrial quality inspection processes. These methods rely on deep learning models to analyze product surface images in real time to identify and classify multiple types of defects, including cracks, scratches, pores, and foreign matter. However, in practical applications, visual inspection systems face numerous technical challenges related to invisibility and coupling, particularly in complex scenarios with interference from pseudo-defects and the overlapping coupling of multiple defect types. On the one hand, pseudo-defect high-frequency interference scenarios often contain image feature anomalies caused by non-real defects such as reflections, water stains, and dust. These anomalies visually resemble real defects, easily leading to false detections and misclassifications. On the other hand, in scenarios with overlapping coupling of multiple defect types, such as scratches and corrosion, or cracks and contamination, where defects coexist or have overlapping boundaries, deep learning models struggle to accurately establish the discriminative boundaries between multiple classes. These two issues can easily lead to sample label contamination and mislabeling during training. During the learning process, the model struggles to effectively distinguish between the principal components of features and background interference, resulting in feature learning bias and distorted classification boundaries. More seriously, when the system learns from mislabeled pseudo-defects as real defects, a transfer effect of "incorrect feature-label mapping" occurs, causing the model's generalization ability to decline in new production lines, new materials, and other scenarios, manifesting as misclassification or missed detections at high confidence levels. This problem not only limits the robustness and stability of the inspection system but also increases the cost of subsequent manual re-inspections, becoming a key bottleneck hindering the further popularization and implementation of intelligent manufacturing visual quality inspection systems. Summary of the Invention

[0003] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an automatic detection and classification method for intelligent manufacturing defects based on machine vision to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] The automatic detection and classification method of intelligent manufacturing defects based on machine vision includes the following steps:

[0006] Step S1, capturing high-resolution images of the workpiece to be inspected, including multi-source samples of different manufacturing batches, surface conditions, and illumination angles, and preprocessing the captured images;

[0007] Step S2: Use the image segmentation network to output the defect probability heat map of the inspected workpiece and mark the primary defect candidate areas;

[0008] Step S3, extracting a high-frequency pseudo-defect feature tensor from the primary defect candidate area, and calculating a pseudo-defect high-frequency interference coefficient based on the high-frequency pseudo-defect feature tensor, and analyzing the pseudo-defect high-frequency interference degree of the primary defect candidate area;

[0009] Step S4, cross-judging the defect type of each primary defect candidate area, obtaining the overlapping coupling information of multiple defect categories between defect categories, calculating the overlapping coupling coefficient of multiple defects, and analyzing the overlapping coupling degree of multiple defects in the primary defect candidate area;

[0010] Step S5: construct a sample label contamination risk assessment model based on the pseudo-defect high-frequency interference coefficient and the multi-class defect overlapping coupling coefficient, output a sample label contamination risk assessment index, evaluate the contamination risk level of the sample label during the training phase, and identify high-contamination risk samples.

[0011] In a preferred embodiment, in step S2, the standardized image is input into the pre-trained image segmentation network to output the defect probability heat map of the inspected workpiece. ,in , representing pixels The probability of being a defect;

[0012] Heat map of defect probability of workpieces to be inspected The defect probability at each location and the preset defect probability threshold Compare and generate a binary mask image : ;

[0013] Connected regions are extracted from the binary mask image, and each connected region is a primary defect candidate region.

[0014] In a preferred embodiment, a high-frequency pseudo-defect feature tensor is extracted from the primary defect candidate region, and a pseudo-defect high-frequency interference coefficient is calculated based on the high-frequency pseudo-defect feature tensor, as follows:

[0015] The high-frequency pseudo-defect feature tensor includes a texture response tensor , brightness anomaly tensor , structured residual tensor ; Fuse the above tensors by channel to obtain the high-frequency pseudo-defect feature tensor : ;

[0016] For each high-frequency pseudo-defect feature tensor Perform principal component analysis to obtain the principal component vector of high-frequency pseudo defects: ,in is the principal component vector of high-frequency pseudo-defects, Principal component analysis method is used to extract the high-frequency pseudo-defect feature tensor Extract the principal component vector of high-frequency pseudo-defects from ;

[0017] Calculate the principal component vector of high-frequency pseudo-defects The mean : ,in is the high-frequency pseudo-defect principal component vector of the i-th primary defect candidate area, N is the total number of primary defect candidate areas; calculate the high-frequency pseudo-defect principal component vector The covariance matrix of : ,in express The transpose of

[0018] Calculate the deviation rate of pseudo defect distribution: ,in is the pseudo defect distribution deviation rate of the i-th primary defect candidate area, is the covariance matrix The inverse matrix of

[0019] The structural residual tensor of the primary defect candidate region The structural residual tensor with the true defect set Combine and calculate the structural alignment residual: ,in is the structural alignment residual, is the Sobel operator, It is used to perform the two-norm square processing;

[0020] Assign confidence weights to each primary defect candidate area : ,in is the mean defect probability when it is determined to be a primary defect candidate area, is the defect probability threshold;

[0021] Calculate the pseudo defect high frequency interference coefficient: ,in is the high-frequency interference coefficient of pseudo-defect.

[0022] In a preferred embodiment, in step S4, the image blocks of the primary defect candidate areas are fed into a pre-trained multi-class defect discrimination model, and the defect type of each primary defect candidate area is cross-judged to obtain a multi-class defect confidence distribution vector: in is the total number of defect types, ∈[0,1], which indicates the probability that the primary defect candidate area is judged as the kth type of defect.

[0023] In a preferred embodiment, the logic for obtaining the overlapping coupling coefficients of multiple types of defects is as follows:

[0024] Calculate the uncertainty distribution coefficient of the defect category based on the confidence distribution vector of multiple defects: ,in is the uncertainty distribution coefficient, is the probability that the primary defect candidate area is judged as the kth type of defect, is the total number of defect types;

[0025] Calculate the semantic adjacency coefficients of different defect categories based on the semantic feature vector of the defect type: ,in is the semantic adjacency coefficient, is the probability that the primary defect candidate area is judged as the h-th type defect, , is the semantic feature vector of the k-th defect type, is the semantic feature vector of the h-th defect type, and Represent the modulus of the semantic feature vector of the k-th defect type and the h-th defect type respectively;

[0026] Calculate the overlapping coupling coefficients of multiple types of defects: ,in is the overlapping coupling coefficient of multiple defect types, N is the total number of primary defect candidate regions, represent the preset proportional coefficients of the uncertainty distribution coefficient and the semantic adjacency coefficient, respectively, and Both are greater than 0.

[0027] In a preferred embodiment, a sample label contamination risk assessment model is constructed based on the pseudo-defect high-frequency interference coefficient and the multi-class defect overlapping coupling coefficient, and a sample label contamination risk assessment index is output. The sample label contamination risk assessment model is based on the following formula: , where is the sample label contamination risk assessment index, is the pseudo defect high frequency interference coefficient, is the overlapping coupling coefficient of multiple types of defects, They represent the preset proportional coefficients of the pseudo-defect high-frequency interference coefficient and the multi-type defect overlapping coupling coefficient, respectively, and Both are greater than 0.

[0028] In a preferred embodiment, the sample label contamination risk assessment index is compared with a preset sample label contamination risk assessment index threshold to identify high contamination risk samples, as follows:

[0029] If the sample label contamination risk assessment index is greater than the sample label contamination risk assessment index threshold, the current sample is marked as a high contamination risk sample;

[0030] If the sample label contamination risk assessment index is less than or equal to the sample label contamination risk assessment index threshold, the current sample is marked as a usable sample.

[0031] Technical effects and advantages of the present invention:

[0032] 1. The machine vision-based intelligent manufacturing defect automatic detection and classification method proposed in this invention can effectively overcome the key problems of existing technologies such as detection error, training bias and reduced model generalization ability caused by pseudo-defect interference and overlapping coupling of multiple types of defects, and achieve high-precision and high-robustness defect detection and classification. By constructing a high-resolution multi-source sample data set and introducing image preprocessing operations, covering defect manifestations under different manufacturing batches, surface conditions and lighting conditions, the adaptability of the visual inspection system is significantly improved. The image segmentation network is used to generate a defect probability heat map and extract primary defect candidate areas, effectively ensuring the sensitivity and integrity of the defect detection stage. The high-frequency pseudo-defect feature tensor is further combined to calculate the pseudo-defect high-frequency interference coefficient, and the multi-class defect overlapping coupling coefficient is calculated based on the multi-classification confidence distribution and semantic adjacency. The pseudo-defect interference intensity and the degree of fuzziness of the multi-class defect boundaries in the defect candidate area are accurately characterized, providing highly reliable feature support for subsequent sample label contamination risk assessment. The sample label contamination risk assessment model constructed based on the above-mentioned characteristic indicators can realize the automatic identification and screening of high-contamination risk samples in the training data, significantly reduce the interference of mislabeled samples on deep model training, and effectively prevent the erosion of model generalization ability caused by incorrect feature-label mapping. It can improve the accuracy, robustness and generalization ability of defect detection and classification in complex detection scenarios, while reducing the cost of manual review and enhancing the application adaptability and engineering implementation capabilities of intelligent manufacturing visual quality inspection systems in multi-production line and multi-material scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings;

[0034] Figure 1 Flowchart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0036] Example: Figure 1 The present invention provides an intelligent manufacturing defect automatic detection and classification method based on machine vision, which includes the following steps:

[0037] Step S1, capturing high-resolution images of the workpiece to be inspected, including multi-source samples of different manufacturing batches, surface conditions, and illumination angles, and preprocessing the captured images;

[0038] Step S2: Use the image segmentation network to output the defect probability heat map of the inspected workpiece and mark the primary defect candidate areas;

[0039] Step S3, extracting a high-frequency pseudo-defect feature tensor from the primary defect candidate area, and calculating a pseudo-defect high-frequency interference coefficient based on the high-frequency pseudo-defect feature tensor, and analyzing the pseudo-defect high-frequency interference degree of the primary defect candidate area;

[0040] Step S4, cross-judging the defect type of each primary defect candidate area, obtaining the overlapping coupling information of multiple defect categories between defect categories, calculating the overlapping coupling coefficient of multiple defects, and analyzing the overlapping coupling degree of multiple defects in the primary defect candidate area;

[0041] Step S5: construct a sample label contamination risk assessment model based on the pseudo-defect high-frequency interference coefficient and the multi-class defect overlapping coupling coefficient, output a sample label contamination risk assessment index, evaluate the contamination risk level of the sample label during the training phase, and identify high-contamination risk samples;

[0042] In step S1, a high-resolution CMOS camera (the camera specifications must meet the resolution 5000 4000 pixels, frame rate 30fps, using GigE or USB3.0 (supporting high-speed data transmission) as the interface to capture high-resolution images of the inspected workpiece. The captured images include multi-source samples from different manufacturing batches, surface conditions, and lighting angles. The captured images are pre-processed, including grayscale normalization: ,in is the pixel value of the sample image at (x, y) after grayscale normalization, is the pixel value of the kth original sample image at coordinate (x, y), is the minimum value of all pixels in the original image, The maximum value of all pixels in the original image; use Laplace enhancement or Canny edge filter to enhance defect edge contrast; multi-channel conversion (RGB + grayscale + gradient); output standardized image matrix set ;

[0043] In step S2, the standardized image is input into a pre-trained image segmentation network (the image segmentation network is built based on CNN to identify defects of the inspected workpiece, such as U-Net++, DeepLabV3+, SegFormer, etc.) to output a defect probability heat map of the inspected workpiece. ,in , representing pixels The probability of being a defect;

[0044] Heat map of defect probability of workpieces to be inspected The defect probability at each location and the preset defect probability threshold Compare and generate a binary mask image : ;

[0045] Connected regions are extracted from the binary mask image. Each connected region is a primary defect candidate region. Each primary defect candidate region contains the following attributes: ,in is the label of the i-th primary defect candidate area, is the location box, is the heat map response intensity, is the image scale level, is the original eigenvector;

[0046] Step S3, extracting a high-frequency pseudo-defect feature tensor from the primary defect candidate area, and calculating a pseudo-defect high-frequency interference coefficient based on the high-frequency pseudo-defect feature tensor, and analyzing the pseudo-defect high-frequency interference degree of the primary defect candidate area;

[0047] The pseudo-defect high-frequency interference coefficient (Pseudo-Defect High-Frequency Interference Coefficient) in this paper is a quantitative indicator used to measure the degree of interference between pseudo-defect features in sample images. It aims to effectively identify and assess the label contamination risk caused by pseudo-defects during the training phase. The Pseudo-Defect High-Frequency Interference Coefficient not only reflects the frequency and spatial density of pseudo-defects in the image but also comprehensively considers the confusability of their texture features in the semantic space—that is, the degree of visual similarity between them and real defects at the depth model's perception level. A large Pseudo-Defect High-Frequency Interference Coefficient generally indicates the presence of significant pseudo-defect interference in the sample image, whose texture, shape, or edge features are easily misclassified as real defects by the convolutional neural network. In this case, the model is prone to learning these highly visually similar interference information as weak defect labels such as "minor scratches" and "fine cracks" during training, resulting in a shift in the classification boundaries in the feature vector space and ultimately overfitting the pseudo-defects. This is particularly true for samples captured from multiple batches of manufacturing data or under different lighting angles. These pseudo-defects are highly uncertain and non-repeatable, making it difficult for the model to accurately identify them during generalization, leading to large-scale false positives and false detections. On the contrary, a smaller pseudo-defect high-frequency interference coefficient indicates that the pseudo-defect interference features in the sample image are weak or the expression form is clearly distinguishable from the real defect features. The model can learn the main component features of the defects more clearly during training and form a stable perception pattern in the convolution layer, reducing the interference risk of mislabeled labels caused by pseudo-defects, which is conducive to building a clearer and more stable discrimination boundary. In the present invention, the pseudo-defect high-frequency interference coefficient is used as a key input feature of the sample label contamination risk assessment model in the training stage, which can provide a reference dimension of "label credibility" for the training samples. By analyzing the pseudo-defect high-frequency interference coefficient of each sample image in the training set, a spatial distribution map of the label contamination risk can be constructed, and further combined with the overlapping coupling coefficients of multiple types of defects to achieve the identification and filtering of high-contamination risk samples. This process not only improves the quality of training data, but also significantly improves the generalization performance of the model in multi-source data and complex backgrounds. In summary, by introducing the pseudo-defect high-frequency interference coefficient, the present invention realizes the quantitative assessment of the label contamination risk in the training samples, breaking the previous limitations of relying on manual experience screening or rule template elimination, improving the controllability of training data, the accuracy of defect feature learning and the generalization stability of the model, and providing key support for building a robust and high-precision visual inspection system.

[0048] The high-frequency pseudo-defect feature tensor is extracted from the primary defect candidate area, and the pseudo-defect high-frequency interference coefficient is calculated based on the high-frequency pseudo-defect feature tensor, as follows:

[0049] The high-frequency pseudo-defect feature tensor includes a texture response tensor (such as based on Gabor filter banks or Laws texture energy templates), brightness anomaly tensors (such as local mean difference), structural residual tensor (such as Laplacian edge anomaly map); fuse the above tensors by channel to obtain the high-frequency pseudo-defect feature tensor : ;

[0050] For each high-frequency pseudo-defect feature tensor Perform principal component analysis to obtain the principal component vector of high-frequency pseudo defects: ,in is the principal component vector of high-frequency pseudo-defects, Principal component analysis method is used to extract the high-frequency pseudo-defect feature tensor Extract the principal component vector of high-frequency pseudo-defects from ;

[0051] It's important to note that principal component analysis (PCA) is a statistical dimensionality reduction method that maps the original high-dimensional feature space to a low-dimensional subspace through an orthogonal transformation. Its core idea is to find the projection direction with the maximum variance (i.e., the principal component) through eigenvalue decomposition or singular value decomposition of the covariance matrix, thereby achieving dimensionality reduction while preserving the original data information as much as possible. This will not be discussed in detail here.

[0052] Calculate the principal component vector of high-frequency pseudo-defects The mean : ,in is the high-frequency pseudo-defect principal component vector of the i-th primary defect candidate area, N is the total number of primary defect candidate areas; calculate the high-frequency pseudo-defect principal component vector The covariance matrix of : ,in express The transpose of

[0053] Calculate the deviation rate of pseudo defect distribution: ,in is the pseudo defect distribution deviation rate of the i-th primary defect candidate area, is the covariance matrix The inverse matrix of

[0054] The structural residual tensor of the primary defect candidate region The structural residual tensor with the true defect set Combine and calculate the structural alignment residual: ,in is the structural alignment residual, which is used to measure the structural residual vector of the primary defect candidate area The structural residual vector with the true defect set A real scalar value expressing the overall degree of difference between is the Sobel operator, It is used to perform the second norm square processing and is used to measure the degree of residual between the edges of the structure;

[0055] It should be noted that the Sobel operator refers to an image edge detection method based on first-order gradient, which is used to perform gradient edge extraction processing on the structural residual tensor in the primary defect candidate area to capture its local structural change characteristics. It will not be described in detail here.

[0056] Assign confidence weights to each primary defect candidate area : ,in is the mean defect probability when it is determined to be a primary defect candidate area, is the defect probability threshold;

[0057] Calculate the pseudo defect high frequency interference coefficient: ,in is the pseudo defect high frequency interference coefficient;

[0058] It should be noted that the above formulas are all dimensionless and numerical calculations. Common dimensionless methods include Min-Max normalization and Z-Score normalization, which will not be described here.

[0059] In step S4, the image blocks of the primary defect candidate area are fed into the pre-trained multi-class defect discrimination model, and the defect type of each primary defect candidate area is cross-judged to obtain the multi-class defect confidence distribution vector: in is the total number of defect types, ∈[0,1], represents the probability that the primary defect candidate area is judged as the kth type of defect;

[0060] It should be noted that the multi-classification defect discrimination model refers to an image classification model built on a deep neural network structure. Its input is the image region to be classified (i.e., the primary defect candidate image block), and its output is the discrimination probability distribution vector of the image region for various defect categories. The model achieves feature extraction and category discrimination of different types of defects by training and learning a large number of labeled defect samples. The following structure and training method are often used: Model structure: ResNet, EfficientNet, DenseNet, Vision Transformer (ViT) and other typical deep convolutional neural networks or visual Transformer architectures can be used, with a Softmax layer as the output; Training objective: The cross entropy loss function is used as the objective function, and the parameters are optimized through back propagation; Feature expression ability: The model has the ability to extract multi-level representations such as morphology, texture, and edges from the original image, which is suitable for distinguishing various surface defects; Confidence output: The softmax activation function of the last layer outputs the discrimination probability of each type of defect to form a confidence distribution vector;

[0061] Obtain the overlapping coupling information of multiple defect categories between defect categories, calculate the overlapping coupling coefficient of multiple defect categories, and analyze the overlapping coupling degree of multiple defect categories in the primary defect candidate area;

[0062] The logic for obtaining the overlapping coupling coefficients of multiple types of defects is as follows:

[0063] Calculate the uncertainty distribution coefficient of the defect category based on the confidence distribution vector of multiple defects: ,in is the uncertainty distribution coefficient, is the probability that the primary defect candidate area is judged as the kth type of defect, is the total number of defect types;

[0064] Calculate the semantic adjacency coefficients of different defect categories based on the semantic feature vector of the defect type: ,in is the semantic adjacency coefficient, is the probability that the primary defect candidate area is judged as the h-th type defect, , is the semantic feature vector of the k-th defect type, is the semantic feature vector of the h-th defect type, and Represent the modulus of the semantic feature vector of the k-th defect type and the h-th defect type respectively;

[0065] It should be noted that the semantic feature vector of the defect type can be obtained by encoding the semantic description text of the defect category (such as category name, attribute label, process description, etc.) and extracting its semantic vector representation using a natural language processing model (such as pre-trained language models such as Word2Vec, GloVe, BERT, and RoBERTa). This will not be discussed in detail here.

[0066] Calculate the overlapping coupling coefficients of multiple types of defects: ,in is the overlapping coupling coefficient of multiple defect types, N is the total number of primary defect candidate regions, represent the preset proportional coefficients of the uncertainty distribution coefficient and the semantic adjacency coefficient, respectively, and All greater than 0;

[0067] It should be noted that the above formulas are all dimensionless and numerical calculations. Common dimensionless methods include Min-Max normalization and Z-Score normalization, which will not be described here. Set it according to the actual situation. For example, adopt the expert empowerment method, that is, invite experts in related fields to determine the preset proportion coefficients of various indicators through professional opinion surveys and comprehensive evaluations, for example, It can be 0.5, 0.5;

[0068] The multi-class defect overlap coupling coefficient in the present invention is an indicator used to measure the degree of overlap, interference, and aliasing of multiple different types of defects in spatial distribution and feature expression. Its core function is to measure whether coupling occurs between different defect types in the same candidate area in the image, that is, there is significant overlap and mutual interference between structural features, texture features, or semantic labels. In industrial scenarios, there are many types of real defects on the surface of workpieces, such as cracks, peeling, foreign matter, pores, etc., which may physically overlap, optically interfere, or semantically cross during the actual manufacturing or acquisition process, thereby causing "contamination" of training sample labels, directly affecting the classification accuracy and generalization ability of the detection model. When the multi-class defect overlap coupling coefficient is large, it indicates that there is significant spatial overlap or statistical similarity between the edge structures, texture details, or brightness features of multiple defect types in the candidate area, that is, two or more defect features may exist simultaneously in an area, or a certain defect feature may be semantically misclassified as another type. At this time, the sample labels in the training phase are likely to be mislabeled or fuzzy, causing the model to learn a mixed feature distribution, increasing the risk of false detection and missed detection in subsequent detection tasks. For example, crack edges hidden beneath a peeling surface, or pore structures superimposed on foreign object reflections, can cause cross-distribution in the feature space, leading to error propagation during supervised learning. Therefore, a large multi-class defect overlap coupling coefficient is a key signal for identifying samples with high contamination risk, necessitating careful attention during sample set cleaning and annotation recalibration. Conversely, a small multi-class defect overlap coupling coefficient indicates that different defect types are well separable within the image region, with clear boundary outlines, non-overlapping textures, and well-defined semantic labels. In this scenario, the training sample labels are purer, and the supervised learning signal is more reliable, enabling the model to more accurately learn the discriminative features between different defect types and improving the accuracy and robustness of the defect detection system in complex scenarios. Applying this coefficient to sample set evaluation during the training phase can effectively identify sample regions with significant label contamination risk, assisting annotators in re-annotation, removal, or local feature suppression, thereby improving the generalization, interference resistance, and stability of the overall defect detection model's high-confidence output. Therefore, the overlapping coupling coefficient of multiple types of defects is not only a key indicator for data quality control, but also a basic component for building a highly robust defect identification system, with significant practical value and engineering significance.

[0069] Step S5: construct a sample label contamination risk assessment model based on the pseudo-defect high-frequency interference coefficient and the multi-class defect overlapping coupling coefficient, output a sample label contamination risk assessment index, evaluate the contamination risk level of the sample label during the training phase, and identify high-contamination risk samples;

[0070] A sample label contamination risk assessment model is constructed based on the pseudo-defect high-frequency interference coefficient and the multi-class defect overlapping coupling coefficient, and the sample label contamination risk assessment index is output. The sample label contamination risk assessment model is based on the following formula: , where is the sample label contamination risk assessment index, is the pseudo defect high frequency interference coefficient, is the overlapping coupling coefficient of multiple types of defects, They represent the preset proportional coefficients of the pseudo-defect high-frequency interference coefficient and the multi-type defect overlapping coupling coefficient, respectively, and All greater than 0;

[0071] It should be noted that the above formulas are all dimensionless and numerical calculations. Common dimensionless methods include Min-Max normalization and Z-Score normalization, which will not be described here. Set it according to the actual situation. For example, adopt the expert empowerment method, that is, invite experts in related fields to determine the preset proportion coefficients of various indicators through professional opinion surveys and comprehensive evaluations, for example, It can be 0.5, 0.5;

[0072] From the above calculation expression, it can be seen that the larger the pseudo-defect high-frequency interference coefficient and the larger the multi-class defect overlap coupling coefficient, the larger the sample label contamination risk assessment index, indicating that the sample has a higher label contamination risk during the training phase. Its label may be inaccurate due to the high-frequency interference characteristics or the ambiguity of the overlap of multiple classes of defects. Conversely, the smaller the pseudo-defect high-frequency interference coefficient and the smaller the multi-class defect overlap coupling coefficient, the smaller the sample label contamination risk assessment index, indicating that the sample has a lower label contamination risk and is more suitable for inclusion in the model training set as a high-quality training sample.

[0073] Compare the sample label contamination risk assessment index with the preset sample label contamination risk assessment index threshold to identify high contamination risk samples, as follows:

[0074] If the sample label contamination risk assessment index is greater than the sample label contamination risk assessment index threshold, it means that the sample has a high degree of pseudo-defect high-frequency interference and a high degree of overlapping coupling of multiple defect types. The risk of label contamination is significantly greater than normal, and the sample is a potential high-risk sample. Such samples may be inaccurately labeled or have labeling errors due to factors such as fuzzy defect characteristics, significant pseudo-defect interference, or unclear boundaries between multiple defect types, marking the current sample as a high-contamination risk sample.

[0075] If the sample label contamination risk assessment index is less than or equal to the sample label contamination risk assessment index threshold, it means that the high-frequency interference degree of pseudo defects and the overlapping coupling degree of multiple types of defects in the sample are both within an acceptable range. Its label reliability is high and it is not easily affected by interference factors or overlapping ambiguity. The current sample is marked as a usable sample and can be directly included in the training set as a high-quality sample.

[0076] It should be noted that the threshold for the sample label contamination risk assessment index can be determined through data analysis or expert evaluation based on specific application requirements and data characteristics. For example, the threshold can be set based on the label contamination ratio distribution of historically annotated data, training set error analysis, or cross-validation experimental results. It can also be set based on empirical values ​​in conjunction with the opinions of field experts. The reasonable setting of the threshold has a significant impact on the accuracy and actual effectiveness of sample risk identification, and can be flexibly adjusted according to different defect detection scenarios (such as high-precision processes, coarse screening processes, etc.);

[0077] The automatic detection and classification method for intelligent manufacturing defects based on machine vision proposed in the present invention can effectively overcome the key problems of detection error, training bias and reduced model generalization ability caused by pseudo-defect interference and overlapping coupling of multiple types of defects in the existing technology, and achieve high-precision and high-robustness defect detection and classification. By constructing a high-resolution multi-source sample data set and introducing image preprocessing operations, covering defect manifestations under different manufacturing batches, surface conditions and lighting conditions, the adaptability of the visual inspection system is significantly improved. The defect probability heat map is generated through the image segmentation network and the primary defect candidate area is extracted, effectively ensuring the sensitivity and integrity of the defect detection stage. The high-frequency pseudo-defect feature tensor is further combined to calculate the pseudo-defect high-frequency interference coefficient, and the multi-class defect overlapping coupling coefficient is calculated based on the multi-classification confidence distribution and semantic adjacency. The pseudo-defect interference intensity and the degree of fuzziness of the multi-class defect boundaries in the defect candidate area are accurately characterized, providing highly reliable feature support for the subsequent sample label contamination risk assessment. The sample label contamination risk assessment model constructed based on the above-mentioned characteristic indicators can realize the automatic identification and screening of high-contamination risk samples in the training data, significantly reduce the interference of mislabeled samples on deep model training, and effectively prevent the erosion of model generalization ability caused by incorrect feature-label mapping. It can improve the accuracy, robustness and generalization ability of defect detection and classification in complex detection scenarios, while reducing the cost of manual review and enhancing the application adaptability and engineering implementation capabilities of intelligent manufacturing visual quality inspection systems in multi-production line and multi-material scenarios.

[0078] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0079] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0080] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An intelligent manufacturing defect automatic detection and classification method based on machine vision, characterized by: The steps include: Step S1, capturing high-resolution images of the workpiece to be inspected, including multi-source samples of different manufacturing batches, surface conditions, and illumination angles, and preprocessing the captured images; Step S2: Use the image segmentation network to output the defect probability heat map of the inspected workpiece and mark the primary defect candidate areas; Step S3, extracting a high-frequency pseudo-defect feature tensor from the primary defect candidate area, and calculating a pseudo-defect high-frequency interference coefficient based on the high-frequency pseudo-defect feature tensor, and analyzing the pseudo-defect high-frequency interference degree of the primary defect candidate area; Step S4, cross-judging the defect type of each primary defect candidate area, obtaining the overlapping coupling information of multiple defect categories between defect categories, calculating the overlapping coupling coefficient of multiple defects, and analyzing the overlapping coupling degree of multiple defects in the primary defect candidate area; Step S5: construct a sample label contamination risk assessment model based on the pseudo-defect high-frequency interference coefficient and the multi-class defect overlapping coupling coefficient, output a sample label contamination risk assessment index, evaluate the contamination risk level of the sample label during the training phase, and identify high-contamination risk samples; The high-frequency pseudo-defect feature tensor is extracted from the primary defect candidate area, and the pseudo-defect high-frequency interference coefficient is calculated based on the high-frequency pseudo-defect feature tensor, as follows: The high-frequency pseudo-defect feature tensor includes a texture response tensor , brightness anomaly tensor , structured residual tensor ; Fuse the above tensors by channel to obtain the high-frequency pseudo-defect feature tensor : ; For each high-frequency pseudo-defect feature tensor Perform principal component analysis to obtain the principal component vector of high-frequency pseudo defects: ,in is the principal component vector of high-frequency pseudo-defects, Principal component analysis method is used to extract the high-frequency pseudo-defect feature tensor Extract the principal component vector of high-frequency pseudo-defects from ; Calculate the principal component vector of high-frequency pseudo-defects The mean : ,in is the high-frequency pseudo-defect principal component vector of the i-th primary defect candidate area, N is the total number of primary defect candidate areas; calculate the high-frequency pseudo-defect principal component vector The covariance matrix of : ,in express The transpose of Calculate the deviation rate of pseudo defect distribution: ,in is the pseudo defect distribution deviation rate of the i-th primary defect candidate area, is the covariance matrix The inverse matrix of The structural residual tensor of the primary defect candidate region The structural residual tensor with the true defect set Combine and calculate the structural alignment residual: ,in is the structural alignment residual, is the Sobel operator, It is used to perform the two-norm square processing; Assign confidence weights to each primary defect candidate area : ,in is the mean defect probability when it is determined to be a primary defect candidate area, is the defect probability threshold; Calculate the pseudo defect high frequency interference coefficient: ,in is the pseudo defect high frequency interference coefficient; In step S4, the image blocks of the primary defect candidate area are fed into the pre-trained multi-class defect discrimination model, and the defect type of each primary defect candidate area is cross-judged to obtain the multi-class defect confidence distribution vector: in is the total number of defect types, ∈[0,1], represents the probability that the primary defect candidate area is judged as the kth type of defect; The logic for obtaining the overlapping coupling coefficients of multiple types of defects is as follows: Calculate the uncertainty distribution coefficient of the defect category based on the confidence distribution vector of multiple defects: ,in is the uncertainty distribution coefficient, is the probability that the primary defect candidate area is judged as the kth type of defect, is the total number of defect types; Calculate the semantic adjacency coefficients of different defect categories based on the semantic feature vector of the defect type: ,in is the semantic adjacency coefficient, is the probability that the primary defect candidate area is judged as the h-th type defect, , is the semantic feature vector of the k-th defect type, is the semantic feature vector of the h-th defect type, and Represent the modulus of the semantic feature vector of the k-th defect type and the h-th defect type respectively; Calculate the overlapping coupling coefficients of multiple types of defects: ,in is the overlapping coupling coefficient of multiple defect types, N is the total number of primary defect candidate regions, represent the preset proportional coefficients of the uncertainty distribution coefficient and the semantic adjacency coefficient, respectively, and Both are greater than 0.

2. The method for automatic detection and classification of intelligent manufacturing defects based on machine vision according to claim 1, characterized in that: In step S2, the standardized image is input into the pre-trained image segmentation network to output the defect probability heat map of the inspected workpiece. ,in , representing pixels The probability of being a defect; Heat map of defect probability of workpieces to be inspected The defect probability at each location and the preset defect probability threshold Compare and generate a binary mask image : ; Connected regions are extracted from the binary mask image, and each connected region is a primary defect candidate region.

3. The method for automatic detection and classification of intelligent manufacturing defects based on machine vision according to claim 1, characterized in that: A sample label contamination risk assessment model is constructed based on the pseudo-defect high-frequency interference coefficient and the multi-class defect overlapping coupling coefficient, and the sample label contamination risk assessment index is output. The sample label contamination risk assessment model is based on the following formula: , where is the sample label contamination risk assessment index, is the pseudo defect high frequency interference coefficient, is the overlapping coupling coefficient of multiple types of defects, They represent the preset proportional coefficients of the pseudo-defect high-frequency interference coefficient and the multi-type defect overlapping coupling coefficient, respectively, and Both are greater than 0.

4. The method for automatic detection and classification of intelligent manufacturing defects based on machine vision according to claim 3, characterized in that: Compare the sample label contamination risk assessment index with the preset sample label contamination risk assessment index threshold to identify high contamination risk samples, as follows: If the sample label contamination risk assessment index is greater than the sample label contamination risk assessment index threshold, the current sample is marked as a high contamination risk sample; If the sample label contamination risk assessment index is less than or equal to the sample label contamination risk assessment index threshold, the current sample is marked as a usable sample.

Images