A product appearance quality determination method for industrial vision

Abstract

The application provides an industrial visual product appearance quality determination method, comprising: for the pre-processed image, a multi-scale feature pyramid is constructed according to the characteristics of different sizes and various forms of flow mark defects, different scale flow mark features are extracted through different scale convolution kernels, different size and form flow mark regions are captured, and a multi-scale feature map is obtained; based on the latent flow mark region, according to the characteristics of low distinction between the flow mark defect and the background region and lack of unified feature representation, the weight of different regions in the feature map is adjusted through the long-range dependence relationship between the image regions, and the feature representation of the flow mark region is enhanced; based on the lightweight feature map, according to the difficulty of constructing the training data set of the flow mark defect, the training samples are expanded through data enhancement, the original defect samples are randomly rotated, translated, scaled and flipped, and diversified flow mark forms and distribution positions are simulated.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method for judging the appearance quality of products using industrial vision. Background Technology

[0002] In the appearance quality inspection of industrial visual products, flow marks are a common and challenging problem. Flow marks exhibit diverse morphological characteristics on the product surface, varying in size and randomly distributed, posing numerous challenges to the design of defect detection algorithms. First, flow marks have low contrast against the product surface background, with indistinct edge contours and poor differentiation from the background. Traditional edge detection and segmentation algorithms struggle to accurately extract flow mark regions. Second, due to the diverse shapes and sizes of flow marks and the lack of a unified feature representation, feature matching-based detection methods are ineffective. Furthermore, the random distribution of flow marks and the lack of prior location information increase the difficulty of localization. Traditional detection methods based on low-level visual features such as grayscale, texture, and edges have poor universality and robustness when dealing with low-contrast, multi-scale, and randomly distributed defects like flow marks. While deep learning-based defect detection methods can learn more robust and abstract feature representations, designing sophisticated detection network structures, balancing detection accuracy and real-time performance, and constructing high-quality flow mark defect training datasets remain crucial challenges for complex defects like flow marks. Summary of the Invention

[0003] This invention provides a product appearance quality assessment method based on industrial vision, mainly comprising:

[0004] Product appearance images are acquired using high-resolution industrial cameras and telecentric lenses. To address the characteristics of low contrast and indistinct edge contours of flow mark defects against the background of the product surface, the images are preprocessed to enhance the contrast between the flow mark area and the background, and to highlight the visual features of the edge contour information.

[0005] For the preprocessed image, considering the varying sizes and shapes of flow mark defects, a multi-scale feature pyramid is constructed. Flow mark features at different scales are extracted using convolutional kernels of different scales, capturing flow mark regions of different sizes and shapes to obtain a multi-scale feature map.

[0006] Based on the multi-scale feature map, and considering the random distribution of flow mark defects and the lack of prior location information, a target detection framework based on a region proposal network is adopted. By densely sampling candidate regions of different scales and aspect ratios on the convolutional feature map, potential flow mark regions are located.

[0007] Based on the potential flow mark region, and considering the characteristics of low differentiation between flow mark defects and background and lack of unified feature representation, the weights of different regions in the feature map are adjusted by the long-range dependency between image regions to enhance the feature representation of the flow mark region.

[0008] Based on the image with enhanced feature representation of the flow mark region, and considering the requirement to balance detection accuracy and real-time performance in flow mark defect detection, a lightweight backbone network is used as a feature extractor to reduce the number of parameters and computational complexity of the flow mark feature model. This improves detection speed while ensuring detection accuracy, resulting in a lightweight feature map.

[0009] Based on lightweight feature maps, and addressing the difficulty of constructing a training dataset for flow mark defects, data augmentation is employed to expand the training samples. By randomly rotating, translating, scaling, and flipping the original defect samples, diverse flow mark morphologies and distribution locations are simulated.

[0010] During the training process simulating diverse flow mark morphologies and distribution locations, difficult sample examples with significant losses are recorded in detail to improve the model's ability to detect weak contrast and multi-scale flow mark defects. The defect classification and localization tasks are jointly optimized, and lightweight feature maps are classified and regressed to obtain the category and location information of flow mark defects. Combined with preset judgment rules, the product appearance quality judgment of industrial vision is completed.

[0011] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0012] This invention discloses a method for judging the appearance quality of products using industrial vision. Addressing the characteristics of flow mark defects, such as low contrast on product surfaces, indistinct edges, varying sizes, diverse shapes, and random distribution, this invention employs image preprocessing to enhance contrast, constructs a multi-scale feature pyramid to capture features at different scales, uses a region proposal network to locate potential defects, enhances flow mark feature representation through long-range dependencies, uses a lightweight backbone network to balance accuracy and speed, and utilizes data augmentation to expand the sample while focusing on difficult cases. By integrating multiple techniques, this invention effectively improves the detection capability of flow mark defects, achieves accurate judgment of the appearance quality of industrial products, and significantly enhances product quality control. Attached Figure Description

[0013] Figure 1 This is a flowchart of a product appearance quality judgment method based on industrial vision according to the present invention. Detailed Implementation

[0014] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] like Figure 1 This embodiment of a product appearance quality judgment method based on industrial vision may specifically include:

[0016] S101. Acquire product appearance images using a high-resolution industrial camera and telecentric lens. To address the characteristics of low contrast and indistinct edge contours of flow mark defects against the background of the product surface, preprocess the images to enhance the contrast between the flow mark area and the background, and highlight the visual features of the edge contour information.

[0017] The process involves acquiring raw product appearance images using a high-resolution industrial camera and a telecentric lens. These raw images are then converted to grayscale to obtain grayscale images. Contrast-limited adaptive histogram equalization is applied to the grayscale images to obtain contrast-enhanced images. A Gaussian difference filter is then used to locally enhance the contrast of the contrast-enhanced images, resulting in locally enhanced images. Canny edge detection is then performed on the locally enhanced images to obtain binarized edge images. Background modeling and a difference method are used to separate the background from the binarized edge images to obtain foreground images. If flow mark defects are present in the foreground images, a logical AND operation is performed between the foreground images and the binarized edge images to obtain a flow mark defect detection result image. Based on the flow mark defect detection result image, the presence of flow mark defects on the product appearance is determined.

[0018] Specifically, the process involves acquiring raw product appearance images using a high-resolution industrial camera and telecentric lens. The raw images are then converted to grayscale, and noise is removed using a 5x5 median filter to obtain a smoothed grayscale image. Contrast-limited adaptive histogram equalization (CLAHE) is applied to the smoothed grayscale image, with a grid size of 8x8 and a contrast limit parameter of 3.0, improving overall image contrast and enhancing the difference between the flow mark area and the background, resulting in a contrast-enhanced image. A difference Gaussian filter is then used to locally enhance the contrast of the contrast-enhanced image, with standard deviations of 1 and 2, specifically targeting the flow mark area, resulting in a locally enhanced image. Canny edge detection is then performed on the locally enhanced image, with a low threshold of 50 and a high threshold of 150 to highlight the edge contour information of the flow mark defects, resulting in a binarized edge image. Morphological processing is then applied to the binarized edge image, using a 3x3 rectangular structuring element for opening operations to remove small noise points, and a 5x5 rectangular structuring element for closing operations to connect edge breaks, resulting in a continuous and complete flow mark defect contour image. To address the complex backgrounds on different product surfaces, background modeling and a difference method are used for background separation. The background update rate is set to 0.01, and the threshold is 20, highlighting the flow mark area to obtain the separated foreground image. A logical AND operation is performed between the separated foreground image and a continuous, complete flow mark defect contour image to obtain the final flow mark defect detection result image. Original product appearance images are acquired using a high-resolution industrial camera (4096x3072 pixels) and a telecentric lens (0.5x magnification). The original images are then converted to grayscale, transforming the RGB three-channel image into a single-channel grayscale image. Noise is removed by median filtering within a 5x5 window. During the filtering process, the pixel values ​​within the 5x5 region surrounding each pixel are sorted, and the median value is selected as the new value for that pixel, resulting in a smoothed grayscale image. The smoothed grayscale image was processed using Contrast-Limited Adaptive Histogram Equalization (CLAHE), with a grid size of 8x8 and a contrast limit parameter of 3.0. The image was divided into 64 regions, and histograms were calculated and equalized for each region. The magnitude of contrast enhancement was limited to avoid excessive noise enhancement. A Gaussian difference filter was then used to locally enhance the contrast of the enhanced image, with standard deviations of 1 and 2. The results of two Gaussian filters at different scales were subtracted to highlight edges and details in the image. Canny edge detection was then performed on the locally enhanced image, with a low threshold of 50 and a high threshold of 150. First, the image was smoothed using a Gaussian filter, then the image gradient was calculated, and non-maximum suppression was applied. Finally, a binarized edge image was obtained through a double thresholding method and edge concatenation.Morphological processing is performed on the binarized edge image. A 3x3 rectangular structuring element is used for opening operations to remove small noise points, and a 5x5 rectangular structuring element is used for closing operations to connect edge breaks. During the opening operation, erosion is performed before dilation, and during the closing operation, dilation is performed before erosion, resulting in a continuous and complete flow mark defect contour image. For the complex backgrounds of different product surfaces, background modeling and a difference method are used for background separation. The background update rate is set to 0.01, and the threshold is 20. A Gaussian mixture model (GMM) is used to establish a background model, creating 3-5 Gaussian distributions for each pixel. The foreground is extracted based on the difference between the current frame and the background model, resulting in a separated foreground image. A logical AND operation is performed between the separated foreground image and the continuous and complete flow mark defect contour image, retaining the foreground portions in both images to obtain the final flow mark defect detection result image.

[0019] S102. For the preprocessed image, in view of the characteristics of flow mark defects with different sizes and shapes, a multi-scale feature pyramid is constructed. Flow mark features of different scales are extracted by convolution kernels of different scales to capture flow mark regions of different sizes and shapes, and a multi-scale feature map is obtained.

[0020] A Gaussian pyramid is constructed from the preprocessed image, and image layers of different resolutions are generated through downsampling to obtain a multi-scale image pyramid. For each layer of the multi-scale image pyramid, convolution operations are performed using convolution kernels of different sizes to extract flow trace features at different scales, resulting in feature maps. Based on the feature maps, the ReLU function is used to perform non-linear activation processing on each pixel value in the feature maps, resulting in activated feature maps. Upsampling operations are performed on the activated feature maps, and bilinear interpolation is used to enlarge the low-resolution feature maps to the same size as the original image, resulting in uniform-sized feature maps. All uniform-sized feature maps are concatenated along the channel dimension to obtain a fused feature map. Based on the fused feature map, the oriented gradient histogram feature extraction method is used to divide the image into cells of a preset size, calculate the gradient magnitude and direction of pixels in each cell, and generate a histogram of a preset number of directions, resulting in a multi-scale, multi-morphological flow trace feature representation.

[0021] Specifically, a Gaussian pyramid is constructed for the preprocessed image using a 5x5 Gaussian kernel with a standard deviation of 1.0. Downsampling is used to generate image layers of different resolutions, each half the size of the previous layer, resulting in a total of four layers and a multi-scale image pyramid. For each layer in the multi-scale image pyramid, three different convolutional kernel sizes (3x3, 5x5, and 7x7) are designed, with Gaussian distribution used to initialize the kernel weights and a standard deviation of 0.01. Convolution operations are performed on each layer, using zero-padding to maintain the feature map size and extracting flow trace features at different scales, resulting in 12 feature maps. Nonlinear activation is then applied to these 12 feature maps using the ReLU function f(x) = max(0, x) to process each pixel value, enhancing the expressive power of the features and yielding the activated feature maps. The activated feature maps are upsampled to unify the size of each layer. Bilinear interpolation is then used to enlarge the low-resolution feature maps to the same size as the original image. Specifically, the four nearest neighbor pixels of the target pixel in the original image are calculated, interpolation weights are calculated based on distance, and a weighted average is used to obtain the interpolation result. Then, all feature maps are concatenated along the channel dimension to obtain a fused feature map containing multi-scale flow mark features. The fused feature map is processed using the histogram of oriented gradients (HARQ) feature extraction method. The image is divided into 8x8 pixel cells, and the gradient magnitude and direction of each pixel in each cell are calculated to generate a histogram with nine directions, capturing the morphological diversity of flow mark defects and obtaining the final multi-scale, multi-morphological flow mark feature representation. A Gaussian pyramid is constructed on the preprocessed 1024x1024 pixel flow mark defect image using a 5x5 Gaussian kernel with a standard deviation of 1.0. Downsampling is then used to generate image layers of different resolutions: 512x512, 256x256, and 128x128 pixel images, forming a multi-scale image pyramid with four layers. For each layer of the pyramid image, three different convolutional kernel sizes (3x3, 5x5, and 7x7) were designed, and the kernel weights were initialized using a Gaussian distribution with a mean of 0 and a standard deviation of 0.01. Convolution operations were performed on each layer, using zero-padding to maintain the feature map size; for example, for a 3x3 kernel, one pixel of zero was padded at the image edges. Flow trace features at different scales were extracted, resulting in 12 feature maps. These 12 feature maps were then activated using ReLU, f(x) = max(0, x), setting values ​​less than 0 to 0 and leaving values ​​greater than 0 unchanged to enhance the non-linear expressiveness of the features. The activated feature maps were then upsampled using bilinear interpolation to enlarge the low-resolution feature maps to 1024x1024 pixels.In practice, for the target pixel (x, y), the four nearest pixels in the original image are found to be (x1, y1), (x1, y2), (x2, y1), and (x2, y2). The interpolation weights w1 = (x2 - x)(y2 - y), w2 = (x2 - x)(y - y1), w3 = (x - x1)(y2 - y), and w4 = (x - x1)(y - y1). A weighted average is then used to obtain the interpolation result. All feature maps are then concatenated along the channel dimension to obtain a fused feature map containing multi-scale flow trace features. The fused feature map is processed using histogram of oriented gradients. The image is divided into 8x8 pixel cells, and the gradient magnitude and direction of each pixel in each cell are calculated to generate histograms in 9 directions (0°, 20°, 40°, 60°, 80°, 100°, 120°, 140°, 160°). The cumulative gradient value in each direction is used as a bin of the histogram to capture the morphological diversity of flow mark defects and obtain the final multi-scale and multi-morphological flow mark feature representation.

[0022] S103. Based on the multi-scale feature map, and considering the random distribution of flow mark defects and the lack of prior location information, a target detection framework based on a region proposal network is adopted. By densely sampling candidate regions of different scales and aspect ratios on the convolutional feature map, potential flow mark regions are located.

[0023] A convolution operation is performed on the multi-scale feature map. The convolution operation uses a 3x3 convolution kernel to scan the feature map through a sliding window, resulting in a dense set of feature points. Each feature point in the set corresponds to a receptive field of the original image. Multiple anchor boxes with different scales and aspect ratios are generated based on this feature point set. The scale range of the anchor boxes is a preset pixel value, and the aspect ratio range is a preset ratio value. A preset number of anchor boxes are generated for each feature point to obtain a candidate region set. Region of Interest (ROI) pooling is used to calculate the feature representation of each anchor box in the candidate region set. ROI pooling maps anchor boxes of different sizes onto a fixed-size feature map. The feature representations are then subjected to classification and regression processing. If fully connected layers are used for classification and regression processing, the fully connected layers contain a preset number of neurons, resulting in refined target detection boxes and corresponding confidence scores. A non-maximum suppression (NMS) algorithm is used to process overlapping detection boxes. The NMS algorithm sets preset IoU thresholds and confidence thresholds, selects the detection box with the highest score, and suppresses other detection boxes with high overlap.

[0024] Specifically, convolution operations are performed on multi-scale feature maps. A 3x3 convolution kernel is used to scan the feature maps using a sliding window. The kernel weights are initialized using the He method, and the activation function is ReLU, generating dense feature points. Each feature point corresponds to a receptive field of the original image, resulting in a feature point set. Multiple anchor boxes of different scales and aspect ratios are generated at each feature point. By calculating the statistical information of flow mark defects in the image, the scale range of the anchor boxes is dynamically adjusted to {64, 128, 256, 512} pixels, and the aspect ratio range is {1:1, 1:2, 2:1, 1:3, 3:1}. Twenty anchor boxes are generated for each feature point, forming a candidate region set. The feature representation of each anchor box in the candidate region set is calculated. A region of interest pooling operation is used to map anchor boxes of different sizes onto a fixed-size 7x7 feature map. In practice, bilinear interpolation is first used to adjust the anchor box region to 14x14 size, and then 2x2 max pooling is performed to obtain region features of uniform dimension. Region features are classified and regressed using two fully connected layers: the first layer contains 1024 neurons, and the second layer contains 512 neurons, with ReLU activation function. Each region feature is classified to obtain a category probability distribution, and the translation and scaling offsets of anchor boxes are generated, resulting in refined target detection boxes and corresponding confidence scores. Non-maximum suppression (NMS) is used to handle overlapping detection boxes, with an IoU threshold of 0.5 and a confidence threshold of 0.7. The detection box with the highest score is selected, and other detection boxes with high overlap are suppressed. An attention map is generated using the U-Net image segmentation network to highlight areas in the image that may have flow marks or defects. The attention map is multiplied with the feature map to guide the region proposal network to focus on key areas, improving detection accuracy. A 1024x1024 pixel multi-scale feature map is convolved using a 3x3 kernel, with weights initialized using the He method (standard deviation of sqrt(2 / 9)). A sliding window scan is performed on the feature map with a stride of 16 pixels, generating 64x64 feature points. Twenty anchor boxes are generated at each feature point. The size distribution of flow mark defects is determined by analyzing the image histogram, and the anchor box scale is dynamically adjusted to {64, 128, 256, 512} pixels with aspect ratios of {1:1, 1:2, 2:1, 1:3, 3:1}. For each of the 81920 (64x64x20) candidate regions, bilinear interpolation is used to resize them to 14x14, followed by 2x2 max pooling to obtain a 7x7 unified feature map. Two fully connected layers are used to process the feature map: the first layer has 1024 neurons, and the second layer has 512 neurons. The weights are initialized using the Xavier method, and the biases are initialized to 0. The output layer contains class probabilities (background and flow mark, two classes in total) and bounding box regression values ​​(center point coordinates, width, and height adjustments, four values ​​in total).Non-maximum suppression is applied to overlapping detection boxes, and the IoU between each pair is calculated. If the IoU is greater than 0.5, the box with higher confidence is retained. Finally, U-Net is used to generate an attention map. The encoder contains four convolutional blocks, each with two 3x3 convolutional layers and one 2x2 max pooling layer. The decoder uses transposed convolution upsampling, and skip connections are used to connect the corresponding encoding layers. The output is an attention map of the same size as the input image, which is multiplied pixel-by-pixel with the original feature map to highlight potential flow trace regions.

[0025] S104. Based on the potential flow mark region, and considering the characteristics of low differentiation between flow mark defects and background and lack of unified feature representation, the weights of different regions in the feature map are adjusted through the long-range dependency relationship between image regions to enhance the feature representation of the flow mark region.

[0026] Self-attention is calculated on the feature map of the potential flow mark region to obtain the feature map, which is then reshaped into an HW×C matrix. A linear transformation yields three matrices: Q, K, and V. Spatial pyramid pooling is used to extract multi-scale features from the feature map. Average pooling is performed on each sub-region according to a preset pooling size to obtain a multi-scale feature representation. Non-local operations are performed on the multi-scale feature representation. If the dot product similarity between each location in the feature map and all other locations is greater than a preset threshold, a weighted average of the features is calculated based on the generated similarity matrix. The original feature map and the enhanced feature map are then fused. A 1×1 convolution is used to reduce the channel dimensionality of the two feature maps, and the features are merged by element-wise addition. A contrastive learning method is used to enhance the feature discriminativeness between the flow mark region and the background. Features from the flow mark region and the background region are randomly selected from the enhanced feature map as positive and negative sample pairs. It is determined whether the positive and negative sample pairs meet the preset contrastive learning conditions. If they do, the InfoNCE loss function is used to optimize the feature representation.

[0027] Specifically, self-attention is calculated on the feature map of the potential flow trace region. The feature map is reshaped into an HW×C matrix, and three matrices Q, K, and V are generated through linear transformation. The weights are initialized using the Xavier method with a dimension of 512. Eight attention heads are set with a scaling factor of 1 / sqrt(64). The attention weight matrix is ​​obtained through matrix multiplication, and V is weighted and summed to obtain the enhanced feature map. Spatial pyramid pooling is used to extract features from the feature map at multiple scales. The pooling size is set to 1×1, 2×2, and 4×4. Average pooling is performed on each sub-region to obtain feature vectors of 1, 4, and 16 sub-regions. These feature vectors are concatenated to obtain a multi-scale feature representation. Non-local operations are performed on the multi-scale feature representation. Dot product similarity is used to calculate the similarity between each position in the feature map and all other positions, generating a similarity matrix. The features are weighted and averaged according to the similarity matrix to obtain a feature map containing global context information. The original and enhanced feature maps are fused. A 1×1 convolution is used to reduce the channel dimensionality of the two feature maps, with 1024 input channels and 512 output channels, and ReLU activation function. Then, the features are merged element-wise. Finally, a 3×3 convolution is used to adjust the number of feature channels, with 512 input channels and 256 output channels, and ReLU activation function, resulting in the final enhanced feature map of the flow trace region. A contrastive learning method is used to enhance the feature discrimination between the flow trace region and the background. Features from the flow trace region and the background region are randomly selected from the enhanced feature map as positive and negative sample pairs. The InfoNCE loss function is used to optimize the feature representation, with the temperature parameter set to 0.07 and the batch size to 256. The 256×256×512 potential flow trace region feature map is processed by first reshaping it into a 65536×512 matrix. Three independent linear transformation layers are used to generate Q, K, and V matrices, with the weight matrix of each layer being 512×512, initialized using the Xavier method, and the standard deviation is [missing value].

[0028] Set sqrt(2 / (512+512)). Set 8 attention heads, each with a dimension of 64 and a scaling factor of 1 / sqrt(64)≈0.125. Calculate the attention weight matrix A = softmax(QK^T / 0.125), with a size of 65536×65536, and then calculate AV to obtain the enhanced features. Perform spatial pyramid pooling on the enhanced features: 1×1 pooling to obtain a 1×512 vector, 2×2 pooling to obtain a 4×512 matrix, and 4×4 pooling to obtain a 16×512 matrix. Concatenate these features into a 21×512 multi-scale feature representation. Calculate the dot product similarity matrix S, where S_ij = the inner product of the i-th feature and the j-th feature, generating a 21×21 similarity matrix. Perform softmax normalization on S, and then perform matrix multiplication with the original features to obtain features containing global information. The original and enhanced features are fused using a 1×1 convolution with 512 kernels, a stride of 1, no padding, 1024 input channels, and 512 output channels, activated by ReLU. Then, a 3×3 convolution is used to adjust the number of channels, with 256 kernels, a stride of 1, and padding of 1, 512 input channels, and 256 output channels, activated by ReLU. Finally, 128 flow mark region features and 128 background region features are randomly selected from the 256×256×256 feature map to form 256 sample pairs. The feature representation is optimized using the InfoNCE loss function L = -log(exp(p·p+ / 0.07) / ∑exp(p·p_i / 0.07)), where p and p+ are positive sample pairs, and p_i is all samples.

[0029] S105. Based on the image representing the enhanced flow mark region, and considering the requirement to balance detection accuracy and real-time performance in flow mark defect detection, a lightweight backbone network is used as the feature extractor to reduce the number of parameters and computational complexity of the flow mark feature model. This improves detection speed while ensuring detection accuracy, resulting in a lightweight feature map.

[0030] The input feature map is processed by replacing standard convolution with depthwise separable convolution, performing channel-wise convolution. The depthwise separable convolution includes a 3x3 kernel, a convolutional layer with a stride of 1 and padding of 1, and a 1x1 convolutional layer. Based on the feature map after depthwise separable convolution, a channel attention mechanism is used for channel weighting. Channel weighting is achieved by obtaining channel descriptors through global average pooling, and then passing them through two fully connected layers to generate channel weights. The first layer has 1 / 16 the number of channels in the feature map, and the second layer has the same number of neurons as the feature map. Skip connections are introduced into the channel-enhanced feature map. The feature maps from layers 3 and 5 are selected as shallow features and fused with the feature map from the last layer. After adjusting the number of channels using 1x1 convolution, the features are merged using element-wise addition. Feature map pruning is performed on the fused feature map. The L1 norm of each convolutional kernel is calculated as an importance score. Important convolutional kernels are selected based on a preset threshold, and kernels with scores below the threshold are removed. Knowledge distillation is performed on the sparse feature maps using a teacher-student network structure. The original model is used as the teacher network and the lightweight network is used as the student network. Distillation is performed using soft labeling and feature map matching loss to obtain a compressed lightweight model.

[0031] Specifically, depthwise separable convolution replaces standard convolution. Channel-wise convolution is performed on the input feature map with a 3x3 kernel, a stride of 1, and padding of 1. Then, 1x1 convolutions are used to combine the features, resulting in a depthwise separable convolution feature map. Channel attention is used to weight the feature map by channel. Global average pooling is used to obtain channel descriptors, which are then passed through two fully connected layers to generate channel weights. The first layer has 1 / 16 the number of channels in the feature map, and the second layer has the same number of neurons as the feature map. ReLU is used as the activation function, and each channel of the feature map is weighted to obtain a channel-enhanced feature map. A skip connection structure is introduced, selecting the feature maps from layers 3 and 5 as shallow features. These are fused with the feature map from the last layer. After adjusting the number of channels using 1x1 convolutions, the features are merged using element-wise addition to enhance the multi-scale expressive power of the features, resulting in a fused feature map. Feature map pruning is performed on the fused feature maps. The L1 norm of each convolutional kernel is calculated as an importance score. Important convolutional kernels are selected based on a preset threshold, and kernels with scores below the threshold are removed, resulting in sparser feature maps. 8-bit fixed-point quantization is used to compress the feature maps and model parameters, converting floating-point numbers to fixed-point numbers to reduce storage space and computation. Knowledge distillation is performed using a teacher-student network structure, with the original model as the teacher network and a lightweight network as the student network. Distillation is performed using soft labeling and feature map matching loss to obtain a compressed lightweight model. A single-stage detector is used as the lightweight detection head, employing 3x3 convolutions to generate predicted feature maps. Target location and class probability are directly regressed, reducing computation and yielding the final lightweight feature maps and detection results. A depthwise separable convolution is applied to the 256x256x64 input feature map. 64 3x3 convolutional kernels are used for channel-wise convolution with a stride of 1 and padding of 1, maintaining the feature map size. Then, 64 1x1 convolutional kernels are used to combine the features, resulting in a 256x256x64 output feature map. A channel attention mechanism is applied to the output feature map. First, global average pooling is performed to obtain a 1x1x64 channel descriptor. This descriptor is then passed through two fully connected layers: the first layer has 4 neurons (64 / 16), and the second layer has 64 neurons. The ReLU activation function is used to generate 64 channel weights, which are then applied to the feature map for channel weighting. The feature maps of the third (64x64x256) and fifth (32x32x512) layers of the network are selected as shallow features and fused with the feature map of the last layer (256x256x64). A 1x1 convolution is used to adjust the number of shallow feature channels to 64, and then upsampled to a size of 256x256. The three layers of features are merged by adding elements one by one.The fused 256x256x64 feature map is pruned. The L1 norm of the 64 convolutional kernels is calculated as the importance score, and a threshold of 0.01 is set. Convolutional kernels with scores below the threshold are pruned, leaving 48 kernels, resulting in a 256x256x48 sparse feature map. The feature map and model parameters are quantized using 8-bit fixed-point quantization, mapping floating-point numbers in the range [-1, 1] to integers in the range [-128, 127]. Knowledge distillation is performed using the original model (teacher network) and a lightweight model (student network), with a temperature parameter set to 3 and a weight of 0.5. The student network is trained using hard-label loss and soft-label loss. Finally, a 3x3 convolution is used to generate the predicted feature map, with 15 output channels (assuming 3 target classes are detected, each with 5 parameters: 4 positional parameters and 1 confidence score), resulting in a 256x256x15 lightweight detection feature map.

[0032] S106. Based on lightweight feature maps, to address the difficulty in constructing a training dataset for flow mark defects, data augmentation is used to expand the training samples. By randomly rotating, translating, scaling, and flipping the original defect samples, diverse flow mark morphologies and distribution locations are simulated.

[0033] The original defect samples undergo geometric transformation by randomly rotating, translating pixels, scaling, and flipping to obtain transformed sample images, generating a geometrically enhanced sample set. Based on this enhanced sample set, image blending techniques are used to alpha-blend different defect samples according to a preset ratio, resulting in new composite defect samples. For the composite defect samples, image filtering and noise injection methods are applied to Gaussian blur, sharpen, and add Gaussian noise to obtain a noise-enhanced sample set. If the noise-enhanced sample set meets preset conditions, contrast and brightness adjustments are made using linear transformations to change the overall brightness and contrast levels of the sample images, generating an illumination-enhanced sample set. For this illumination-enhanced sample set, data augmentation methods are applied to lightweight feature maps, performing enhancement operations in the feature space. Random channel dropout and eigenvalue perturbation are applied to the feature maps to enhance their robustness and generalization ability.

[0034] Specifically, the original defect samples undergo geometric transformation. Transformed sample images are generated through random rotation (0-360 degrees), pixel translation (±20 pixels), scaling (0.8-1.2), and random flipping, resulting in a geometrically enhanced sample set. Image blending techniques are used to alpha-blend different defect samples at a ratio of 0.3-0.7, generating new composite defect samples to increase sample diversity and complexity. These composite defect samples contain the superposition of multiple flow mark features. Image filtering and noise injection methods are used to process the sample images with Gaussian blur (kernel size 5x5, standard deviation 1.5), sharpening, and the addition of Gaussian noise (mean 0, variance 0.01) to simulate flow mark defect features under different imaging conditions, resulting in a noise-enhanced sample set. Contrast and brightness adjustments are made using linear transformation methods to change the overall brightness (±50) and contrast level (0.5-1.5) of the sample images, simulating the flow mark defect behavior under different lighting conditions, generating a lighting-enhanced sample set. The aforementioned data augmentation methods are applied to lightweight feature maps, performing augmentation operations in the feature space. Random channel dropout (dropout probability 0.1–0.3) and eigenvalue perturbation (Gaussian noise with mean 0 and standard deviation 0.05) are applied to the feature maps to enhance their robustness and generalization ability. Data augmentation is performed on the original 256x256 pixel flow mark defect samples. First, geometric transformations are applied: a rotation angle randomly selected between 0 and 360 degrees, such as 45 degrees; image translation within ±20 pixels, such as +10 pixels horizontally and -15 pixels vertically; a scaling ratio randomly selected between 0.8 and 1.2, such as 1.1; and horizontal flipping with a 50% probability. Then, image blending is performed. Two different defect samples are selected and alpha blended using weights of 0.4 and 0.6 to generate a composite sample containing multiple flow mark features. Next, the image is smoothed using a 5x5 Gaussian blur kernel with a standard deviation of 1.5, then sharpened using the Laplacian operator, and finally Gaussian noise with a mean of 0 and a variance of 0.01 is added. Image contrast and brightness are adjusted, with brightness increased by 30 and contrast coefficient set to 1.2. For the 64x64x256 lightweight feature map, 20% of the channels (51 channels) are randomly selected for dropout, and the values ​​of these channels are set to 0. Then, Gaussian noise with a mean of 0 and a standard deviation of 0.05 is added to the remaining channels.

[0035] S107. During the training process simulating diverse flow mark morphologies and distribution locations, focus on recording difficult samples with significant losses to improve the model's ability to detect weak-contrast, multi-scale flow mark defects. Jointly optimize defect classification and localization tasks, classify and regress lightweight feature maps to obtain the category and location information of flow mark defects, and combine them with preset judgment rules to complete the industrial vision product appearance quality judgment.

[0036] A 3x3 convolution operation is performed on the lightweight feature map with a stride of 1 and padding of 1, resulting in a classification branch and a regression branch. The classification branch outputs the flow mark category probability at each location, while the regression branch outputs the coordinate offset of the flow mark bounding box. A feature pyramid network structure is used to fuse feature maps of different scales to obtain feature representations of multi-scale flow mark defects. Classification and regression losses are calculated based on the outputs of the classification and regression branches. Samples are sorted according to the total loss value, and the top 20% of samples with the highest loss values ​​are selected as hard examples. If a sample is a hard example, it is given twice the weight, and its sampling frequency is increased to 1.5 times in the next training round. Post-processing is performed on the detection results. The detection boxes with the highest confidence are selected using a non-maximum suppression algorithm. The IoU threshold of the non-maximum suppression algorithm is set to 0.5. All detection boxes are traversed, retaining the highest-scoring box and suppressing other boxes with high overlap.

[0037] Specifically, a 3x3 convolution operation is performed on the lightweight feature map with a stride of 1 and padding of 1, generating classification and regression branches. The classification branch outputs the flow mark category probability at each location, and the regression branch outputs the coordinate offset of the flow mark bounding box. A feature pyramid network structure is adopted to fuse feature maps of different scales to handle multi-scale flow mark defects. Classification and regression losses are calculated. Focalloss is used as the classification loss function with a gamma value of 2 and an alpha value of 0.25, and smoothL1 loss is used as the regression loss function with a beta value of 1.0. The two losses are weighted and summed to obtain the total loss. The samples are sorted according to the total loss value, and the top 20% of the samples with the highest loss values ​​are selected as hard examples. These samples are given twice the weight, and their sampling frequency is increased to 1.5 times the original in the next training round. Post-processing is performed on the detection results. The detection boxes with the highest confidence are selected using a non-maximum suppression algorithm. An IoU threshold of 0.5 is set, and all detection boxes are traversed, retaining the highest-scoring box and suppressing other boxes with high overlap. The severity of flow mark defects is determined based on preset area, length, and depth thresholds, and the final quality assessment result is output. Adaptive histogram equalization enhances the visibility of weak-contrast flow marks by locally enhancing image contrast. The block size is set to 8x8 pixels, and the contrast limit parameter is 3.0, improving the detectability of weak-contrast flow marks. A 3x3 convolution with a stride of 1 and padding of 1 is applied to a lightweight 256x256x64 feature map, generating 128 classification channels and 256 regression channels. A feature pyramid network is employed, fusing feature maps at three scales: 32x32x256, 64x64x128, and 128x128x64. The number of channels is adjusted using 1x1 convolutions, and the data is upsampled and then added element-wise. Calculate focal loss with gamma = 2 and alpha = 0.25. For a misclassified sample with a confidence level of 0.3, the loss value is approximately 0.0879. Calculate smoothL1 loss with beta = 1.0. For a sample with a prediction bias of 5 pixels, the loss value is 4.5. For a batch size of 256 samples, select the 51 samples with the highest loss values ​​as hard examples, increase their weights from 1 to 2, and increase the sampling frequency by 50%. Perform non-maximum suppression with an IoU threshold of 0.5. For 10 overlapping detection boxes, assuming the highest score is 0.9, and the others are 0.8 and 0.7 respectively, retain the box with a score of 0.9 and suppress other boxes with an IoU greater than 0.5. Set the flow mark area threshold to 100 square pixels, the length threshold to 20 pixels, and the depth threshold to 30 (estimated by grayscale difference).Adaptive histogram equalization is performed on the original image, dividing the 256x256 image into 1024 8x8 blocks. Histogram equalization is performed on each block, the contrast gain is limited to 3.0, the overflow histogram count is redistributed, and finally bilinear interpolation is performed on the block boundaries to obtain the enhanced image, which improves the visibility of weak contrast flow marks.

[0038] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A product appearance quality judgment method based on industrial vision, characterized in that, The method includes: acquiring product appearance images using a high-resolution industrial camera and a telecentric lens; preprocessing the images to enhance the contrast between the flow mark region and the background, highlighting the visual features of the edge contours, addressing the low contrast and indistinct edge contours of flow mark defects against the product surface background; constructing a multi-scale feature pyramid for the preprocessed images, considering the varying sizes and shapes of flow mark defects; extracting flow mark features at different scales using convolutional kernels of different scales to capture flow mark regions of different sizes and shapes, resulting in a multi-scale feature map; based on the multi-scale feature map, and considering the random distribution and lack of prior location information of flow mark defects, employing a target detection framework based on a region proposal network; locating potential flow mark regions by densely sampling candidate regions of different scales and aspect ratios on the convolutional feature map; and adjusting different regions in the feature map based on the potential flow mark regions, considering the low distinguishability between flow mark defects and the background and the lack of unified feature representation, by leveraging the long-range dependencies between image regions. The weights of the images are adjusted to enhance the feature representation of the flow mark region. Based on the enhanced feature representation of the flow mark region, and considering the need to balance detection accuracy and real-time performance in flow mark defect detection, a lightweight backbone network is used as a feature extractor to reduce the number of parameters and computational complexity of the flow mark feature model. This improves detection speed while maintaining detection accuracy, resulting in a lightweight feature map. Based on the lightweight feature map, and addressing the difficulty of constructing a training dataset for flow mark defects, data augmentation is used to expand the training samples. By randomly rotating, translating, scaling, and flipping the original defect samples, diverse flow mark morphologies and distribution locations are simulated. During the training process simulating diverse flow mark morphologies and distribution locations, the top 20% of difficult samples with the highest loss values ​​are recorded to improve the model's ability to detect weak contrast and multi-scale flow mark defects. The defect classification and localization tasks are jointly optimized, and the lightweight feature map is classified and regressed to obtain the category and location information of the flow mark defects. Combined with preset judgment rules, the product appearance quality judgment in industrial vision is completed.

2. The method according to claim 1, wherein, The process involves acquiring product appearance images using a high-resolution industrial camera and a telecentric lens. Addressing the low contrast and indistinct edge contours of flow mark defects against the product surface background, the images undergo preprocessing to enhance the contrast between the flow mark area and the background, highlighting the visual features of the edge contours. This includes: acquiring the original product appearance image captured by the high-resolution industrial camera and telecentric lens; converting the original image to grayscale to obtain a grayscale image; performing contrast-limited adaptive histogram equalization on the grayscale image to obtain a contrast-enhanced image; applying a Gaussian difference filter to locally enhance the contrast of the contrast-enhanced image to obtain a locally enhanced image; performing Canny edge detection on the locally enhanced image to obtain a binarized edge image; performing background modeling and a difference method on the binarized edge image to obtain a foreground image; if flow mark defects exist in the foreground image, performing a logical AND operation between the foreground image and the binarized edge image to obtain a flow mark defect detection result image; and determining whether flow mark defects exist on the product appearance based on the flow mark defect detection result image.

3. The method according to claim 1, wherein, The preprocessed image, considering the varying sizes and shapes of flow mark defects, is used to construct a multi-scale feature pyramid. Flow mark features at different scales are extracted using convolution kernels of different scales, capturing flow mark regions of different sizes and shapes to obtain multi-scale feature maps. This process includes: constructing a Gaussian pyramid on the preprocessed image; generating image layers of different resolutions through downsampling to obtain a multi-scale image pyramid; performing convolution operations with convolution kernels of different sizes on each layer of the multi-scale image pyramid to extract flow mark features at different scales, obtaining feature maps; and applying the ReLU function based on the feature maps. A nonlinear activation process is performed on each pixel value in the feature map to obtain the activated feature map. The activated feature map is then upsampled using bilinear interpolation to enlarge the low-resolution feature map to the same size as the original image, resulting in a uniform-sized feature map. All uniform-sized feature maps are then concatenated along the channel dimension to obtain a fused feature map. Based on the fused feature map, the oriented gradient histogram feature extraction method is used to divide the image into cells of a preset size. The gradient magnitude and direction of each pixel in each cell are calculated to generate a preset number of histograms in different directions, resulting in a multi-scale, multi-morphological flow trace feature representation.

4. The method according to claim 1, wherein, Based on the multi-scale feature map, and considering the random distribution and lack of prior location information of flow mark defects, a target detection framework based on a region proposal network is adopted. This framework locates potential flow mark regions by densely sampling candidate regions of different scales and aspect ratios on the convolutional feature map. The process includes: performing a convolution operation on the multi-scale feature map using a 3x3 convolution kernel to scan the feature map through a sliding window, obtaining a dense set of feature points, where each feature point corresponds to a receptive field of the original image; generating multiple anchor boxes of different scales and aspect ratios based on the feature point set, where the scale range of the anchor boxes is a preset pixel value and the aspect ratio range is a preset ratio value. Feature points generate a preset number of anchor boxes to obtain a candidate region set; region-of-interest (ROI) pooling is used to calculate the feature representation of each anchor box in the candidate region set, mapping anchor boxes of different sizes onto a feature map of a fixed size; the feature representations are then classified and regressed. If a fully connected layer is used for classification and regression, the fully connected layer contains a preset number of neurons, resulting in refined target detection boxes and corresponding confidence scores; non-maximum suppression (NMS) is used to process overlapping detection boxes. The NMS algorithm sets preset IoU thresholds and confidence thresholds, selects the detection box with the highest score, and suppresses other detection boxes with high overlap.

5. The method according to claim 1, wherein, The method based on potential flow mark regions addresses the issues of low differentiation between flow mark defects and background, and the lack of unified feature representation. It enhances the feature representation of flow mark regions by adjusting the weights of different regions in the feature map through long-range dependencies between image regions. This includes: performing self-attention calculation on the feature map of the potential flow mark region, obtaining the feature map and transforming it into an HW×C matrix, and obtaining three matrices Q, K, and V through linear transformation; performing multi-scale feature extraction on the feature map using spatial pyramid pooling, performing average pooling on each sub-region according to a preset pooling size to obtain a multi-scale feature representation; and performing nonlocality processing on the multi-scale feature representation. In the first operation, if the dot product similarity of each position in the feature map with all other positions is greater than a preset threshold, the features are weighted and averaged according to the generated similarity matrix. The original feature map and the enhanced feature map are then fused, and 1×1 convolution is used to reduce the channel dimensionality of the two feature maps. The features are then merged by adding them element by element. The feature discrimination between the flow mark region and the background is enhanced by a contrastive learning method. Features from the flow mark region and the background region are randomly selected from the enhanced feature map as positive and negative sample pairs. It is determined whether the positive and negative sample pairs meet the preset contrastive learning conditions. If they do, the InfoNCE loss function is used to optimize the feature representation.

6. The method according to claim 1, wherein, The image, based on the enhanced feature representation of the flow mark region, addresses the need for balancing detection accuracy and real-time performance in flow mark defect detection. A lightweight backbone network is used as the feature extractor to reduce the number of parameters and computational complexity of the flow mark feature model, improving detection speed while maintaining accuracy. This results in a lightweight feature map, which includes: replacing standard convolution with depthwise separable convolution on the input feature map, performing channel-wise convolution. The depthwise separable convolution includes a 3x3 kernel, a convolutional layer with a stride of 1 and padding of 1, and a 1x1 convolutional layer. Based on the feature map after depthwise separable convolution, a channel attention mechanism is used for channel weighting. Channel weighting is achieved through global average pooling to obtain channel descriptors, which are then processed through two fully connected layers to generate channel weights. The number of neurons in the first layer is equal to the number of neurons in the feature map. The number of channels is 1 / 16, and the number of neurons in the second layer is the same as the number of channels in the feature map. Skip connections are introduced into the channel-enhanced feature map. The feature maps of the third and fifth layers of the network are selected as shallow features and fused with the feature map of the last layer. After adjusting the number of channels using 1x1 convolution, the features are merged using element-wise addition. Feature map pruning is performed on the fused feature map. The L1 norm of each convolution kernel is calculated as the importance score. Important convolution kernels are selected according to a preset threshold, and convolution kernels with scores below the threshold are removed. Knowledge distillation is performed on the sparse feature map using a teacher-student network structure. The original model is used as the teacher network, and the lightweight network is used as the student network. Distillation is performed using soft labeling and feature map matching loss to obtain the compressed lightweight model.

7. The method according to claim 1, wherein, The lightweight feature map-based approach addresses the difficulty of constructing a training dataset for flow mark defects by employing data augmentation to expand the training samples. This involves randomly rotating, translating, scaling, and flipping the original defect samples to simulate diverse flow mark morphologies and distribution locations. Specifically, this includes: performing geometric transformations on the original defect samples by randomly rotating angles, translating pixels, scaling ratios, and randomly flipping to obtain transformed sample images, generating a geometrically enhanced sample set; and using image blending techniques to alpha-blend different defect samples according to a preset ratio to obtain new composite defect samples. For defective samples, image filtering and noise injection methods are used to perform Gaussian blurring, sharpening, and Gaussian noise addition on the sample images to obtain a sample set enhanced by filtering noise. If the sample set enhanced by filtering noise meets preset conditions, the overall brightness and contrast levels of the sample images are changed by adjusting contrast and brightness using a linear transformation method to generate a sample set enhanced by illumination changes. For the sample set enhanced by illumination changes, data augmentation methods are applied to lightweight feature maps to perform augmentation operations in the feature space. Random channel dropout and eigenvalue perturbation are applied to the feature maps to obtain the robustness and generalization ability of the enhanced feature maps.

8. The method according to claim 1, wherein, During the training process simulating diverse flow mark morphologies and distribution locations, the top 20% of difficult samples with the highest loss values ​​are highlighted and recorded to improve the model's ability to detect weak-contrast, multi-scale flow mark defects. The defect classification and localization tasks are jointly optimized by classifying and regressing lightweight feature maps to obtain the category and location information of flow mark defects. Combined with preset judgment rules, this completes the industrial vision product appearance quality judgment, including: performing a 3x3 convolution operation on the lightweight feature map with a stride of 1 and padding of 1, resulting in a classification branch and a regression branch; whereby the classification branch outputs the flow mark category probability at each location, and the regression branch outputs the coordinate offset of the flow mark bounding box; A feature pyramid network structure is used to fuse feature maps of different scales to obtain feature representations of multi-scale flow mark defects. Classification loss and regression loss are calculated based on the outputs of the classification and regression branches. Samples are sorted according to the total loss value, and the top 20% of samples with the highest loss values ​​are selected as hard examples. If a sample is a hard example, it is given twice the weight, and its sampling frequency is increased to 1.5 times the original in the next round of training. The detection results are post-processed, and the detection boxes with the highest confidence are selected by non-maximum suppression algorithm. The IoU threshold of the non-maximum suppression algorithm is set to 0.

5. All detection boxes are traversed, and the box with the highest score is retained while other boxes with high overlap with it are suppressed.

Images