Industrial visual quality inspection and defect recognition method, system, device and medium

Abstract

This invention discloses a method, system, device, and medium for industrial visual quality inspection and defect recognition, belonging to the field of industrial visual inspection technology. The method includes: for each type of defect in a temporally smoothed support feature, calculating the mean of the feature vectors of all support samples of that type as a class prototype vector, and inputting this mean vector into a dynamic prototype modulator. The dynamic prototype modulator generates a learnable offset based on the intra-class variance of the samples in that type, and adds this offset to the class prototype vector to obtain a dynamic class prototype vector. This invention, by using a dynamic prototype modulator to generate a learnable offset based on the intra-class variance and adding it to the class prototype vector to obtain a dynamic class prototype vector, enables the prototype to adapt to intra-class changes in defect morphology, improving the coverage and recognition accuracy of defects with variable morphologies.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, and in particular to methods, systems, equipment and media for industrial visual quality inspection and defect identification. Background Technology

[0002] As industrial manufacturing rapidly evolves towards higher precision, higher efficiency, and automation, product quality inspection plays an increasingly important role in production lines. Traditional manual visual inspection methods are limited by the subjective judgment, fatigue level, and inspection speed of inspectors, making it difficult to meet the high throughput and high consistency requirements of modern industry. Therefore, machine vision-based automated defect detection technology has gradually become a core means of industrial quality control, widely used in online inspection scenarios in fields such as electronic components, automotive parts, metal surfaces, and textiles.

[0003] Currently, industrial visual quality inspection methods are mainly divided into two categories: those based on traditional image processing and those based on deep learning. Traditional methods rely on manually designed features (such as edges, textures, and shapes) and classifiers (such as support vector machines and random forests), which have certain effects in specific scenarios, but are less robust to problems such as changes in lighting, complex product surface textures, and varied defect morphologies. In recent years, the introduction of deep learning technology, especially convolutional neural networks (CNNs), has significantly improved the accuracy and generalization ability of defect detection. Existing methods mostly use large-scale labeled samples for fully supervised training. Models such as Faster R-CNN, YOLO, and U-Net have achieved good results in defect detection and segmentation tasks. In addition, few-shot learning methods are also gradually gaining attention, attempting to achieve rapid identification of new defect types with a small number of labeled samples.

[0004] However, existing technologies still have many shortcomings. First, obtaining defect samples in industrial scenarios is difficult and the annotation cost is high, especially since rare defect types often have only a few samples, making it difficult to train traditional deep learning models sufficiently, leading to overfitting or poor generalization. Second, images collected from different batches and production lines vary significantly in terms of lighting, contrast, and noise levels, causing a sharp drop in model performance when applied across different scenarios and resulting in insufficient domain adaptability. Third, existing methods mostly rely on pixel-level prediction based on single-frame images, ignoring the enhancing effect of temporal information on feature stability, making them susceptible to image noise and local anomalies. In addition, defects have diverse morphologies and large intra-class variance, making it difficult for fixed class prototypes or static classification boundaries to adapt to actual changes, while background interference (such as scratches, dust, and texture interference) is often misdetected as defects, reducing detection accuracy. Finally, existing instance segmentation methods have high computational complexity, making it difficult to achieve a good balance between real-time detection and high accuracy.

[0005] Therefore, it is essential to invent methods, systems, equipment, and media for industrial visual quality inspection and defect identification to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide industrial visual quality inspection and defect identification methods, systems, equipment and media to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: an industrial visual quality inspection and defect identification method, comprising the following steps:

[0008] S1: Obtain the support set image and query set image of the target industrial product. The support set image contains K labeled samples of each type of defect, and the query set image contains the samples to be analyzed. Perform spatial adaptive normalization feature alignment enhancement on the two images to obtain the support enhancement image and query enhancement image.

[0009] S2: Input the first branch of the dual-branch feature extraction network with shared weights to support the enhanced image, and extract multi-scale support feature maps; input the query enhanced image into the second branch, and extract multi-scale query feature maps; the dual-branch feature extraction network updates the intermediate features of each branch by exponential moving average through the temporal integration module, and outputs temporal smoothed support features and temporal smoothed query features;

[0010] S3: For each type of defect in the temporal smoothing support features, calculate the mean of the feature vectors of all support samples of that type as the class prototype vector, and input it into the dynamic prototype modulator. The dynamic prototype modulator generates a learnable offset based on the intra-class variance of the samples of that type, and adds the offset to the class prototype vector to obtain the dynamic class prototype vector.

[0011] S4: Calculate the Euclidean distance between each pixel feature and all dynamic class prototype vectors as the defect distance, scale it with an adaptive scaling factor and take the negative exponent to obtain the initial defect prediction map; at the same time, calculate the distance between the pixel feature and the learnable background prototype as the background distance, scale the background distance with the same adaptive scaling factor and take the negative exponent to obtain the background similarity; perform softmax normalization on the background similarity and the initial defect prediction map, and output the defect confidence map by subtracting the background probability from 1.

[0012] S5: Concatenate the defect confidence map with the query enhancement image channel, input it into the lightweight residual correction module, output the residual offset field and add it to the initial defect prediction map to obtain the corrected prediction map; perform softmax normalization on the corrected prediction map along the category channel to obtain the defect probability map, take the pixel in the defect probability map whose maximum probability of each pixel belonging to any defect category is greater than 0.5 as the seed point, perform four-connected region growing, include the pixels in the adjacent pixels whose maximum category probability is greater than 0.3 and whose category corresponding to the maximum probability is the same as the category of the seed point into the same instance, until convergence, and generate the final defect instance segmentation mask;

[0013] S6: Extract the feature vectors of each instance of the final defect instance segmentation mask from the temporal smooth query features through masked average pooling; determine the corresponding dynamic class prototype vector according to the predicted category of each instance, calculate the contrastive loss, update the parameters of the dual-branch feature extraction network and the dynamic prototype modulator based on the contrastive loss through backpropagation, and output the segmentation result image with category labels.

[0014] Preferably, the spatial adaptive normalization feature alignment enhancement in S1 specifically involves: calculating the channel mean and standard deviation of the support set and query set images; performing instance normalization on the support set image; and then performing an affine transformation on the channel mean and standard deviation of the query set image to obtain the support enhancement image; conversely, performing the same operation on the query set image to obtain the query enhancement image, thereby aligning the statistical distributions of the two images.

[0015] Preferably, the temporal integration module in S2 maintains a cached feature queue for each branch, and the queue stores the feature vectors of each spatial location; during each forward propagation, the currently extracted intermediate feature is exponentially averaged with the historical features in the queue using a momentum coefficient of 0.9, and the temporal smoothing support feature and temporal smoothing query feature are output, and the queue length is fixed to the features of the most recent 10 forward propagations.

[0016] Preferably, the dynamic prototype modulator in S3 consists of two fully connected networks. The input is the concatenation of the in-class variance vector and the current class prototype vector, and the output is an offset vector with the same dimension. The offset vector is restricted to the range [-0.2, 0.2] by the tanh activation function and then added to the class prototype vector.

[0017] Preferably, the adaptive scaling coefficient τ in S4 is dynamically calculated based on the total number of support set samples N: τ=a+b / (1+ln(N+1)). The distance of each defect is divided by τ and then the negative exponent is taken to obtain the initial defect prediction map, where a and b are weight coefficients, a and b are positive real numbers and a+b=1.

[0018] Preferably, the lightweight residual correction module in S5 includes two depthwise separable convolutional layers and one channel attention layer. The input is the channel concatenation tensor of the defect confidence map and the query enhancement image. The local correction features are extracted by the first convolutional layer, the channel weights are recalibrated by the channel attention layer, and the residual offset field is output by the second convolutional layer. Its size is the same as the initial defect prediction map.

[0019] Preferably, the contrast loss in S6 includes intra-class compaction loss and inter-class separation loss: intra-class compaction loss calculates the squared Euclidean distance between each instance feature vector and its corresponding dynamic class prototype vector; inter-class separation loss calculates the sum of the powers of the cosine similarity between all different class dynamic class prototype vectors with the natural constant e as the base; the total contrast loss is intra-class compaction loss plus 0.5 times inter-class separation loss.

[0020] Industrial visual quality inspection and defect identification system, including:

[0021] The image acquisition and enhancement module is used to acquire the support set image and query set image of the target industrial product, and perform spatial adaptive normalization feature alignment enhancement on the two images to obtain the support enhancement image and query enhancement image.

[0022] The dual-branch feature extraction network includes a first branch and a second branch with shared weights, as well as a temporal integration module. The first branch is used to receive the support enhancement image and extract multi-scale support feature maps, and the second branch is used to receive the query enhancement image and extract multi-scale query feature maps. The temporal integration module is used to update the intermediate features of each branch by exponential moving average and output temporal smoothed support features and temporal smoothed query features.

[0023] The dynamic prototype modulator is used to calculate the mean of the feature vectors of all support samples of each type of defect in the temporal smooth support features as the class prototype vector, and generate a learnable offset based on the intra-class variance of the samples of that type. The offset is added to the class prototype vector to obtain the dynamic class prototype vector.

[0024] The distance calculation module calculates the Euclidean distance between each pixel feature and all dynamic class prototype vectors as the defect distance. After scaling with an adaptive scaling factor, the negative exponent is taken to obtain the initial defect prediction map. At the same time, the distance between the pixel feature and the learnable background prototype is calculated as the background distance. The background distance is scaled with the same adaptive scaling factor and the negative exponent is taken to obtain the background similarity. The background similarity and the initial defect prediction map are normalized with softmax, and the defect confidence map is output by subtracting the background probability from 1.

[0025] The correction and segmentation module is used to stitch the defect confidence map with the query enhancement image channel, input it into the lightweight residual correction module, output the residual offset field and add it to the initial defect prediction map to obtain the corrected prediction map; the corrected prediction map is normalized by softmax along the category channel to obtain the defect probability map. Using the pixels in the defect probability map whose maximum probability of belonging to any defect category is greater than 0.5 as seed points, four-connected region growing is performed to include the pixels in the adjacent pixels whose maximum category probability is greater than 0.3 and whose category is the same as the category of the seed point, until convergence is reached to generate the final defect instance segmentation mask;

[0026] The training and output module is used to extract the feature vectors of each instance of the final defect instance segmentation mask from the temporal smooth query features through masked average pooling, determine the corresponding dynamic class prototype vector according to the predicted category of each instance, calculate the contrastive loss, update the parameters of the dual-branch feature extraction network and the dynamic prototype modulator based on the contrastive loss through backpropagation, and output the segmentation result image with class labels.

[0027] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described above.

[0028] A computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the steps of the method described above.

[0029] The technical effects and advantages of this invention are as follows:

[0030] 1. This invention enhances the alignment of spatial adaptive normalization features, aligning the statistical distributions of the support set image and the query set image, effectively reducing the domain shift caused by differences in lighting and acquisition conditions, and improving the generalization robustness of the model in cross-scene applications.

[0031] 2. This invention updates the intermediate features of each branch by exponential moving average through a time-domain integration module, uses historical frame information to suppress single-frame noise and local anomaly interference, and outputs time-domain smoothed support features and time-domain smoothed query features, thereby enhancing the stability of feature representation.

[0032] 3. This invention generates a learnable offset based on the intra-class variance using a dynamic prototype modulator and adds it to the class prototype vector to obtain a dynamic class prototype vector. This enables the prototype to adapt to intra-class changes in defect morphology, thereby improving the coverage and recognition accuracy of defects with variable morphology.

[0033] 4. This invention dynamically adjusts the distance metric scale through an adaptive scaling factor and combines a learnable background prototype to calculate background similarity and defect confidence map, effectively suppressing interference from non-defect backgrounds, reducing false detection rate, and improving the salience of defect foregrounds.

[0034] 5. This invention corrects the residual offset field of the initial defect prediction map through a lightweight residual correction module, and generates a defect instance segmentation mask by growing a four-connected region. This improves the accuracy of the segmentation boundary and the integrity of the instance while maintaining low computational complexity.

[0035] 6. This invention extracts instance feature vectors through masked average pooling and calculates a contrast loss that includes intra-class compactness loss and inter-class separation loss. Backpropagation updates network parameters, which enhances the compactness of features of the same class of defects and the separability of features of different classes of defects, thereby improving the defect classification and segmentation performance under small sample conditions. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the overall method of the present invention;

[0037] Figure 2 This is a structural diagram of the dual-branch feature extraction network and the temporal integration module of the present invention;

[0038] Figure 3 This is a flowchart illustrating the lightweight residual correction module and instance segmentation process of the present invention.

[0039] Figure 4 This is a system module connection diagram of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Example 1:

[0042] This embodiment details the complete implementation process of the industrial visual quality inspection and defect identification method. For example... Figure 1 As shown, the overall process includes six stages: image acquisition and enhancement, bi-branch feature extraction, dynamic prototype generation, distance measurement and background suppression, residual correction and instance segmentation, and contrastive learning training.

[0043] This embodiment pre-defines the core terms and symbols:

[0044] The support set consists of sample images with K pixel-level mask annotations for each type of defect, where K ranges from 1 to 10, and in this embodiment, K is preferably 5.

[0045] The query set consists of images of industrial products to be inspected, with the same dimensions and preprocessing rules as the support set.

[0046] Spatial adaptive normalization is a preprocessing method that aligns the statistical distribution of the support set and query set image channels through bidirectional affine transformation.

[0047] The temporal integration module is a feature smoothing module based on exponential moving average fusion of historical frame features, used to suppress single-frame noise;

[0048] The dynamic class prototype is a defect feature center vector that combines intra-class variance with adaptive offset.

[0049] Key symbols: C represents the number of defect categories, D represents the feature dimension, H / W represents the image / width, M represents the total number of defect instances, and N represents the total number of samples in the support set, where N = C × K.

[0050] I. Image Acquisition and Spatial Adaptive Normalization Feature Alignment Enhancement

[0051] In step S1, support set images and query set images of the target industrial product are acquired. The support set contains K labeled samples for each type of defect, with K preferably being 5, and K=1 for extremely rare defect scenarios. The number of samples for each type of defect can be the same or different, and the number of defect categories is C. The images are single-channel grayscale images or three-channel color images, uniformly preprocessed to 512×512 pixels with a bit depth of 8 bits, and pixel values ​​normalized to the [0, 1] range. The query set is the image of the product to be inspected, with the same size and preprocessing rules as the support set.

[0052] Perform spatial adaptive normalization feature alignment enhancement: Calculate the global mean μs and standard deviation σs (support set image) and μq and standard deviation σq (query set image) for each channel of a single image. The mean and standard deviation are calculated over the entire range of pixels in a single image, not batch statistics. Perform instance normalization on the support set image Is: Is norm =(Is-μs) / σs, after normalization the value is truncated to [-5, 5] to avoid gradient explosion, and then affine transformation is performed using the query set statistics: Is enhanced =Is norm ×σq+μq yields the support-enhanced image. The same operation is performed on the query set image Iq: Iq norm =(Iq-μq) / σq, truncated to [-5, 5], Iq enhanced =Iq norm The query-enhanced image is obtained by multiplying σs by μs. Bidirectional distribution alignment eliminates the domain offset problem caused by differences in illumination and acquisition conditions.

[0053] II. Two-branch feature extraction and temporal ensemble

[0054] like Figure 2 As shown, step S2 employs a dual-branch feature extraction network that shares all weights of convolutional layers, batch normalization (BN) layers, and activation layers, with only the output layer of the branch network being independent. The backbone network uses ResNet-50, removing fully connected layers and global pooling layers while retaining the convolutional layer outputs. After inputting the support enhancement image and the query enhancement image, multi-scale feature maps are extracted from conv2x, conv3x, conv4x, and conv5x, respectively, with sizes of 128×128, 64×64, 32×32, and 16×16, and channel numbers of 256, 512, 1024, and 2048, respectively. The conv3x, conv4x, and conv5x feature maps are then uniformly projected to 256 dimensions using a 1×1 convolution (stride 1, padding 0, output channels 256), and finally concatenated along the channel dimensions to complete multi-scale feature fusion.

[0055] The temporal integration module consistently smooths the intermediate features output from each branch of conv3x. The module maintains a cached feature queue with a storage format of [batch, channel, H, W], identical to the dimensions of the current feature map. The queue length is fixed at the features from the last 10 forward propagations. During the first forward propagation, the queue is empty; no exponential moving average is performed, and the currently extracted intermediate feature is directly output as the temporal smoothed feature and stored in the queue. From the second forward propagation onwards, an exponential moving average is performed using a momentum coefficient α=0.9 on the element-wise average of the current feature and all historical features in the queue: F smooth =α×F queue，avg +(1-α)×F current The queue follows a first-in, first-out (FIFO) rule. After each forward propagation, the current feature is added to the queue. If the queue length exceeds 10, the oldest feature is removed. The final output consists of temporally smoothed support features and temporally smoothed query features, with a size of 32×32×256. Single-frame noise is suppressed using historical frame information, improving feature stability.

[0056] III. Dynamic Prototype Modulator Generates Class Prototype

[0057] In step S3, for each type of defect c in the temporal smoothing support features, only the defect pixel features within the pixel-level mask are extracted, ignoring background pixels. The mean of all pixel features for that class is calculated to obtain the initial class prototype vector p. c ∈R^D, D=256; simultaneously calculate the within-class variance vector v dimension by dimension. c ∈R^D, the variance is calculated and normalized to the interval [0, 1].

[0058] The dynamic prototype modulator is a two-layer fully connected network: the first layer has a 2D input dimension (concatenation of the prototype vector and variance vector) and an output dimension of 128, using ReLU activation without Dropout; the second layer has an output dimension of 256, using tanh activation to limit the offset to [-0.2, 0.2], which avoids excessive prototype offset leading to classification failure. The network weights are initialized uniformly using Xavier, and the bias is initialized to 0. The concatenated features are input into the modulator to obtain the offset Δ. c The final dynamic class prototype vector is: p c，dynamic =p c +Δ c This enables the prototype to adapt to intra-class changes in defect morphology, thereby improving the ability to cover diverse defects.

[0059] IV. Distance Metrics and Background Suppression

[0060] In step S4, for each pixel location i in the query-enhanced image, its temporal smoothing query feature is denoted as f. i ∈R^D, calculate f i With all dynamic class prototype vectors p c，dynamic Euclidean distance d c (i)=||f i -p c，dynamic 2. Let the total number of support set samples N = C × K, and the adaptive scaling factor τ = a + b / (1 + ln(N + 1)), where a and b are positive real numbers and a + b = 1. The specific value selection rules use a piecewise function to define the boundaries: when N ≤ 4, a = 0.6, b = 0.4; when N = 5, a = 0.5, b = 0.5 (i.e., the interval 5 ≤ N ≤ 20 includes the endpoint N = 5); when N ≥ 21, a = 0.7, b = 0.3. Divide each defect distance by τ and take the negative exponent: s c (i)=exp(-d c (i) / τ) yields an initial defect prediction map with dimensions H×W×C.

[0061] At the same time, a learnable background prototype p is introduced. bg ∈R^D, initialized as a zero vector, updated during network training. Calculate the distance d between the pixel feature and the background prototype. bg (i)=||f i -p bg ||2, similarly divided by τ and taking the negative exponent, yields the background similarity s. bg (i)=exp(-d bg (i) / τ). s bg With s c (i) Concatenate along the category channels (C+1 dimensions), perform softmax normalization, and obtain the background probability P. bg (i) and the probability of various defects Pc (i). The defect confidence map is obtained by subtracting the background probability from 1: conf(i) = 1 - P bg (i) The defect confidence map highlights the foreground region and suppresses the non-defect background.

[0062] V. Residual Correction and Instance Segmentation

[0063] like Figure 3 As shown, in step S5, the defect confidence map (size H×W×1) and the query augmentation image (H×W×1, or 3 channels if it is a color image) are concatenated along the channel dimension to obtain a tensor of H×W×(1+number of input channels), which is then input into the lightweight residual correction module. This module includes: a first depthwise separable convolutional layer with 3×3 kernels and 32 output channels; a channel attention layer (SENet structure, compression ratio 4); and a second depthwise separable convolutional layer with 3×3 kernels and C output channels, yielding the residual offset field ΔP, with the same size as the initial defect prediction map (H×W×C). The corrected prediction map is: P corr =s c (i)+ΔP.

[0064] The modified prediction map is normalized using softmax along the category channel to obtain the defect probability map M. prob (H×W×C). Using pixels in the defect probability map whose maximum probability of belonging to any defect category is greater than 0.5 as seed points, a four-connected region growing process is performed: pixels with a maximum category probability greater than 0.3 among adjacent pixels and belonging to the same category as the seed point are included in the same instance. Here, "small defect scenario" is defined as the case where the expected defect pixel area is less than 0.5% of the total image area or the longer side of the defect is less than 5% of the shorter side of the image; "large defect scenario" is defined as the case where the expected defect pixel area is greater than 5% of the total image area or the longer side of the defect is greater than 20% of the shorter side of the image. For general scenarios where the defect scale cannot be known in advance, a default threshold (seed point 0.5, region growing 0.3) is used; for production lines with known product types, the corresponding threshold can be selected based on historical defect statistics. The seed point threshold is 0.4 and the region growing threshold is 0.25 for small defect scenarios, and the seed point threshold is 0.6 and the region growing threshold is 0.35 for large defect scenarios. Overlapping instances are assigned according to the category with the highest probability. Convergence is determined when no new pixels are added. Instance IDs are assigned according to the seed point generation order, and finally, a defect instance segmentation mask is generated.

[0065] VI. Comparative Learning and Training

[0066] In step S6, instance feature vectors are extracted using masked average pooling: for the m-th instance in the segmentation mask, edge pixels are weighted by 0.5 and center pixels by 1.0; empty masks are automatically skipped for error tolerance; and the temporal smooth query features of the pixels covered by the instance are weighted and averaged to obtain the instance feature e. mBased on the instance-predicted category matching dynamic class prototype vector, the contrastive loss is calculated:

[0067] Intraclass compact loss: In the formula: L intra For intra-class compactness loss; M is the total number of defective instances; m is the defective instance number; e m p is the feature vector of the m-th defect instance; c，dynamic This is the dynamic class prototype vector of the defect category c to which the instance belongs.

[0068] Inter-class separation loss:

[0069] In the formula: Linter is the inter-class separation loss; C is the total number of defect categories; c and c′ are different defect categories; p c′，dynamic is the dynamic class prototype vector of category c′; cos_sim(·) is the normalized feature cosine similarity; exp(·) is the natural exponential function.

[0070] Total contrast loss: L total =L intra +0.5*L inter .

[0071] The learnable background prototype p_bg is initialized as a zero vector and updated via gradient descent during network training. Its update does not participate in the contrastive loss, but indirectly affects the loss function through the Euclidean distance with pixel features and the subsequent softmax normalization process. The gradient is backpropagated to the background prototype parameters through the chain rule.

[0072] In each training iteration, the support set and query set are constructed as follows: A subset of categories is randomly selected from all C-class defects (the subset size is usually equal to C, i.e., all categories participate; when the number of categories is greater than 10, random sampling of some categories can be used to reduce computational cost). K support samples with pixel-level annotations are selected for each category. The query set image is a single image to be detected, which may contain some or all of the defects in the above categories, and may also contain background regions. The training of the background prototype depends on the background pixels in the query image and does not require additional annotation. A meta-learning paradigm is used during training. In each iteration, multiple support set-query set pairs are used (each pair contains c′-class defects, K support samples for each class, and 1 query image). The optimizer is Adam, with a learning rate of 0.001, momentum of 0.9, and weight decay of 1e-4, and training continues until the model converges.

[0073] The final output is a segmentation image with category labels, and different defect types are rendered in different colors.

[0074] Example 2:

[0075] The difference between this embodiment and Embodiment 1 is as follows: the support set has K=3 samples per class; the adaptive scaling coefficient parameters are a=0.6 and b=0.4; the momentum coefficient of the temporal integration module is set to 0.95 and the queue length is set to 5; the neighborhood expansion threshold for region growing is set to 0.35. The remaining steps are the same, and high-precision defect instance segmentation is also achieved.

[0076] Example 3:

[0077] In this embodiment, the dual-branch feature extraction network uses a lightweight MobileNetV3 backbone, sharing weights from all depthwise separable convolutional layers, making it suitable for deployment on embedded edge devices. The temporal ensemble module only smooths the output features of conv4x, resulting in an output feature map size of 16×16×160. The first fully connected layer of the dynamic prototype modulator has its output dimension changed to 64. The channel attention layer compression ratio in the lightweight residual correction module is set to 8. The remaining implementation methods are the same as in Embodiment 1.

[0078] Example 4:

[0079] This embodiment details the specific implementation of each module of the system. For example... Figure 4 As shown, the industrial visual quality inspection and defect identification system includes:

[0080] Image acquisition and enhancement module: Images are acquired using a Basler acA2500-14gm industrial camera, a Computar 16mm fixed-focus lens, and a 2000lux ring white LED light source. After preprocessing to a uniform size, spatial adaptive normalization feature alignment enhancement is performed.

[0081] Dual-branch feature extraction network: Deployed on the NVIDIA Jetson AGX Xavier platform, FP16 quantization is accelerated for inference via PyTorch→ONNX→TensorRT process;

[0082] Dynamic Prototype Modulator: A two-layer fully connected network is implemented using PyTorch, and the parameters are updated synchronously during the training process;

[0083] Distance calculation module: Based on CUDA, it calculates pixel Euclidean distance in parallel, improving measurement efficiency;

[0084] The correction and segmentation module: The region growing algorithm is written in C++ and integrated into the system through PyBind11. The average time for growing a four-connected region is 8ms.

[0085] Training and output module: Mixed precision training is used to calculate the contrastive loss, the gradient clipping threshold is 1.0, and the segmentation result image is output every 200 steps for real-time monitoring.

[0086] Example 5:

[0087] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the method described in any of embodiments 1-4. Specifically, the device is an industrial control computer, configured with an Intel Core i7-11700K CPU, 32GB RAM, and an NVIDIA RTX 3060 GPU, storing the trained model weight file. The device connects to an industrial camera via a GigE interface, outputs detection results via an HDMI interface, and sends OK / NG detection signals to a PLC via the Modbus TCP protocol, achieving closed-loop detection in the industrial field.

[0088] Example 6:

[0089] This embodiment provides a computer-readable storage medium storing a computer program. When the program is executed by a processor, it implements the steps of the method described in any of embodiments 1-4. The storage medium is a 512GB solid-state drive, storing the operating system, Python 3.8, PyTorch 1.10, model weights, configuration files, and execution scripts. Field engineers can quickly deploy the technical solution to different industrial production lines by changing the storage medium.

[0090] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An industrial visual quality inspection and defect identification method, characterized in that, Includes the following steps: S1: Obtain the support set image and query set image of the target industrial product. The support set image contains K labeled samples of each type of defect, and the query set image contains the samples to be analyzed. Perform spatial adaptive normalization feature alignment enhancement on the two images to obtain the support enhancement image and query enhancement image. S2: Extract multi-scale support feature maps from the first branch of the dual-branch feature extraction network that shares weights with the input of the enhanced image. The query enhancement image is input into the second branch to extract multi-scale query feature maps; the dual-branch feature extraction network updates the intermediate features of each branch by exponential moving average through the temporal integration module, and outputs temporal smoothed support features and temporal smoothed query features. S3: For each type of defect in the temporal smoothing support features, calculate the mean of the feature vectors of all support samples of that type as the class prototype vector, and input it into the dynamic prototype modulator. The dynamic prototype modulator generates a learnable offset based on the intra-class variance of the samples of that type, and adds the offset to the class prototype vector to obtain the dynamic class prototype vector. S4: Calculate the Euclidean distance between each pixel feature and all dynamic class prototype vectors as the defect distance, scale it with an adaptive scaling factor and take the negative exponent to obtain the initial defect prediction map; at the same time, calculate the distance between the pixel feature and the learnable background prototype as the background distance, scale the background distance with the same adaptive scaling factor and take the negative exponent to obtain the background similarity. The background similarity and the initial defect prediction map are normalized using softmax, and the defect confidence map is output by subtracting the background probability from 1. S5: Concatenate the defect confidence map with the query enhancement image channel, input it into the lightweight residual correction module, output the residual offset field and add it to the initial defect prediction map to obtain the corrected prediction map; perform softmax normalization on the corrected prediction map along the category channel to obtain the defect probability map, take the pixel in the defect probability map whose maximum probability of each pixel belonging to any defect category is greater than 0.5 as the seed point, perform four-connected region growing, include the pixels in the adjacent pixels whose maximum category probability is greater than 0.3 and whose category corresponding to the maximum probability is the same as the category of the seed point into the same instance, until convergence, and generate the final defect instance segmentation mask; S6: Extract the feature vectors of each instance of the final defect instance segmentation mask from the temporal smooth query features through masked average pooling; determine the corresponding dynamic class prototype vector according to the predicted category of each instance, calculate the contrastive loss, update the parameters of the dual-branch feature extraction network and the dynamic prototype modulator based on the contrastive loss through backpropagation, and output the segmentation result image with category labels.

2. The industrial visual quality inspection and defect identification method according to claim 1, characterized in that, The spatial adaptive normalization feature alignment enhancement in S1 specifically involves: calculating the channel mean and standard deviation of the support set and query set images; performing instance normalization on the support set image; and then performing an affine transformation on the channel mean and standard deviation of the query set image to obtain the support enhancement image; conversely, performing the same operation on the query set image to obtain the query enhancement image, thereby aligning the statistical distributions of the two images.

3. The industrial visual quality inspection and defect identification method according to claim 1, characterized in that, The temporal integration module in S2 maintains a cached feature queue for each branch, which stores the feature vectors of each spatial location. During each forward propagation, the currently extracted intermediate features are exponentially averaged with the historical features in the queue using a momentum coefficient of 0.9, and the temporal smoothing support features and temporal smoothing query features are output. The queue length is fixed to the features of the last 10 forward propagations.

4. The industrial visual quality inspection and defect identification method according to claim 1, characterized in that, The dynamic prototype modulator in S3 consists of two fully connected networks. The input is the concatenation of the in-class variance vector and the current class prototype vector, and the output is an offset vector with the same dimension. This offset vector is restricted to the range [-0.2, 0.2] by the tanh activation function and then added to the class prototype vector.

5. The industrial visual quality inspection and defect identification method according to claim 1, characterized in that, The adaptive scaling factor τ in S4 is dynamically calculated based on the total number of support set samples N: τ=a+b / (1+ln(N+1)). The distance of each defect is divided by τ and then the negative exponent is taken to obtain the initial defect prediction map, where a and b are weight coefficients, a and b are positive real numbers and a+b=1.

6. The industrial visual quality inspection and defect identification method according to claim 1, characterized in that, The lightweight residual correction module in S5 includes two depthwise separable convolutional layers and one channel attention layer. The input is the channel concatenation tensor of the defect confidence map and the query enhancement image. The local correction features are extracted by the first convolutional layer, the channel weights are recalibrated by the channel attention layer, and the residual offset field is output by the second convolutional layer. Its size is the same as the initial defect prediction map.

7. The industrial visual quality inspection and defect identification method according to claim 1, characterized in that, The contrast loss in S6 includes intra-class compaction loss and inter-class separation loss: intra-class compaction loss calculates the squared Euclidean distance between the feature vector of each instance and its corresponding dynamic class prototype vector; The inter-class separation loss is calculated as the sum of the powers of the cosine similarities between the dynamic class prototype vectors of all different classes, with the natural constant e as the base; the total contrast loss is the intra-class compaction loss plus 0.5 times the inter-class separation loss.

8. An industrial visual quality inspection and defect identification system, characterized in that, For performing the method according to any one of claims 1 to 7, comprising: The image acquisition and enhancement module is used to acquire the support set image and query set image of the target industrial product, and perform spatial adaptive normalization feature alignment enhancement on the two images to obtain the support enhancement image and query enhancement image. The dual-branch feature extraction network includes a first branch and a second branch with shared weights, as well as a temporal integration module. The first branch is used to receive the support enhancement image and extract multi-scale support feature maps, and the second branch is used to receive the query enhancement image and extract multi-scale query feature maps. The temporal integration module is used to update the intermediate features of each branch by exponential moving average and output temporal smoothed support features and temporal smoothed query features. The dynamic prototype modulator is used to calculate the mean of the feature vectors of all support samples of each type of defect in the temporal smooth support features as the class prototype vector, and generate a learnable offset based on the intra-class variance of the samples of that type. The offset is added to the class prototype vector to obtain the dynamic class prototype vector. The distance calculation module calculates the Euclidean distance between each pixel feature and all dynamic class prototype vectors as the defect distance. After scaling with an adaptive scaling factor, the negative exponent is taken to obtain the initial defect prediction map. At the same time, the distance between the pixel feature and the learnable background prototype is calculated as the background distance. The background distance is scaled with the same adaptive scaling factor and the negative exponent is taken to obtain the background similarity. The background similarity and the initial defect prediction map are normalized with softmax, and the defect confidence map is output by subtracting the background probability from 1. The correction and segmentation module is used to stitch the defect confidence map with the query enhancement image channel, input it into the lightweight residual correction module, output the residual offset field and add it to the initial defect prediction map to obtain the corrected prediction map; the corrected prediction map is normalized by softmax along the category channel to obtain the defect probability map. Using the pixels in the defect probability map whose maximum probability of belonging to any defect category is greater than 0.5 as seed points, four-connected region growing is performed to include the pixels in the adjacent pixels whose maximum category probability is greater than 0.3 and whose category is the same as the category of the seed point, until convergence is reached to generate the final defect instance segmentation mask; The training and output module is used to extract the feature vectors of each instance of the final defect instance segmentation mask from the temporal smooth query features through masked average pooling, determine the corresponding dynamic class prototype vector according to the predicted category of each instance, calculate the contrastive loss, update the parameters of the dual-branch feature extraction network and the dynamic prototype modulator based on the contrastive loss through backpropagation, and output the segmentation result image with class labels.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1 to 7.

Images