Method for online defect detection of hardware stamping parts based on industrial vision

Abstract

The present application belongs to the technical field of image recognition, and particularly relates to a hardware stamping part online defect detection method based on industrial vision. When the to-be-detected part enters a detection station, a forward diffuse light source, a coaxial light source and a low-angle side light source are triggered in turn according to a preset timing sequence to collect three frames of original images; local gray scale variances at the same coordinates of the three frames of images are calculated, and a pixel region is marked as a high-reflection region or a low-illumination dark area; a spatial weight mask image is constructed, the high-reflection region is assigned a low-angle side light source image weight, the low-illumination dark area is assigned a coaxial light source image weight, and the remaining region is assigned a forward diffuse light source image weight; and the three frames of images are subjected to pixel-level weighted fusion according to the mask image to generate a fused image. The present application eliminates local overexposure and dark area phenomena on a metal surface, restores edge continuity and texture details of a defect region, and ensures complete extraction of defect features in an online detection process.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, specifically relating to an online defect detection method for metal stamping parts based on industrial vision. Background Technology

[0002] Conventional solutions for defect detection of metal stamping parts based on industrial vision typically employ fixed light source illumination, with an industrial camera capturing single-frame images. These images are then preprocessed or directly input into a convolutional neural network for feature extraction and classification. To accommodate the reflective properties of metal surfaces, conventional solutions adjust the angle of the light source or use a diffuse reflector mask. However, the light source remains constant throughout the detection process, and the image acquisition stage only obtains two-dimensional grayscale information under a single illumination condition.

[0003] Due to the characteristics of the metal material and the complex curvature changes caused by the stamping process, the surface of metal stamping parts inevitably produces directional specular reflection areas and dark areas that are not reached by the light under a fixed light source. When using the above-mentioned single-frame image processing scheme under a single lighting condition, the specular reflection areas are in a state of local overexposure, and the dark areas are in a state of low illumination. These two extreme brightness states will cause the gray values ​​of the corresponding areas in the image to tend to be saturated or missing, resulting in the irreversible loss of physical texture information of the defective areas during the image acquisition stage. Subsequent image processing steps cannot restore the scratches or pits and other defect features from the overexposed or dark areas that have lost texture information, causing the defects to be masked by the background and unable to be identified. Summary of the Invention

[0004] The purpose of this invention is to provide an online defect detection method for metal stamping parts based on industrial vision, which can effectively solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: The online defect detection method for metal stamping parts based on industrial vision includes: when the metal stamping part to be tested enters the vision inspection station, the controller triggers the forward diffuse reflection light source, the coaxial light source and the low-angle side light source in sequence according to the preset timing, and the industrial camera simultaneously acquires the corresponding three frames of original images. The processor calculates the local gray-level variance at the same pixel coordinates in the three original images, and marks the pixel area as a high-reflectivity area or a low-illuminance dark area based on the local gray-level variance. A spatial weighted mask is constructed. For the high reflectivity area, the weight value is assigned to the original image pixel corresponding to the low-angle side light source. For the low illumination dark area, the weight value is assigned to the original image pixel corresponding to the coaxial light source. For the remaining areas, the weight value is assigned to the original image pixel corresponding to the forward diffuse reflection light source. The three original images are pixel-level weighted fusion based on the spatial weight mask to generate a fused image; The fused image is input into a pre-trained convolutional neural network, which outputs a pixel-level segmentation mask containing defect categories.

[0006] Preferably, the processor calculates the local gray-level variance at the same pixel coordinates of the three original images, and marks the pixel region as a high-reflectivity region or a low-illuminance dark region based on the local gray-level variance, including: constructing a local sliding window of a fixed size with the pixel coordinates of any one of the three original images as the center; Extract all pixel grayscale values ​​of the three original images within the coverage area of ​​the local sliding window, calculate the grayscale variance of each identical pixel coordinate position within the local sliding window between the three original images, and generate a variance feature map. In the variance feature map, connected regions with gray-level variance greater than a preset reflectivity threshold are marked as high reflectivity regions, connected regions with gray-level variance less than a preset dark area threshold are marked as low illumination dark areas, and connected regions with gray-level variance between the preset reflectivity threshold and the preset dark area threshold are marked as normal transition regions.

[0007] Preferably, the construction of the spatial weighted mask map includes: obtaining the boundary pixel coordinates of the high reflectivity region and the boundary pixel coordinates of the low illumination dark region; Based on the boundary pixel coordinates of the high reflectivity region, a first Gaussian distance attenuation weight is calculated into the interior of the high reflectivity region; based on the boundary pixel coordinates of the low illumination dark region, a second Gaussian distance attenuation weight is calculated into the interior of the low illumination dark region. The first Gaussian distance attenuation weight is used as the weight value of the original image pixel corresponding to the low-angle side light source in the spatial weight mask. The second Gaussian distance attenuation weight is used as the weight value of the original image pixel corresponding to the coaxial light source in the spatial weight mask. For the remaining areas, a fixed constant weight value is used as the weight value of the original image pixel corresponding to the forward diffuse light source in the spatial weight mask.

[0008] Preferably, the step of performing pixel-level weighted fusion of the three original images based on the spatial weight mask to generate a fused image includes: multiplying the weight value corresponding to the low-angle side light source in the spatial weight mask with the pixel grayscale value of the original image corresponding to the low-angle side light source pixel by pixel to generate a first weighted image. The weight value corresponding to the coaxial light source in the spatial domain weight mask image is multiplied pixel by pixel with the pixel gray value of the original image corresponding to the coaxial light source to generate a second weighted image. The weight value corresponding to the forward diffuse light source in the spatial domain weight mask image is multiplied pixel by pixel with the pixel gray value of the original image corresponding to the forward diffuse light source to generate a third weighted image. The gray values ​​at the same pixel coordinates of the first weighted image, the second weighted image, and the third weighted image are summed to generate the fused image.

[0009] Preferably, the step of inputting the fused image into a pre-trained convolutional neural network and outputting a pixel-level segmentation mask containing defect categories includes: the convolutional neural network comprising a feature extraction backbone network, a dilated spatial pyramid pooling layer, and a decoder; The fused image is input into the feature extraction backbone network to extract multiple initial feature maps at different scales; The initial feature maps of different scales are input into the hollow spatial pyramid pooling layer, and multi-scale contextual features are aggregated through parallel convolution operations with different dilation rates to output aggregated feature maps. The aggregated feature map is input into the decoder, which restores the aggregated feature map to the same resolution size as the fused image through an upsampling operation, and outputs the pixel-level segmentation mask, where each pixel value in the pixel-level segmentation mask corresponds to a defect category.

[0010] Preferably, the controller triggers the forward diffuse reflection light source, the coaxial light source and the low-angle side light source in sequence according to a preset timing sequence, and the industrial camera simultaneously acquires the corresponding three frames of original images, including: after receiving the metal stamping part arrival signal sent by the external photoelectric sensor, the controller generates three pulse trigger signals with fixed time intervals. The three pulse trigger signals are respectively connected to the driving ends of the forward diffuse reflection light source, the coaxial light source and the low-angle side light source, and the duration of each pulse trigger signal corresponds to the stable light emission time of the light source. The industrial camera's hardware trigger pin is connected to the three pulse trigger signals. The industrial camera performs a single exposure when it receives the rising edge of each pulse trigger signal, and completes image signal reading after the falling edge of each pulse trigger signal, outputting the three original images.

[0011] Preferably, the step of constructing a local sliding window of a fixed size centered on the pixel coordinates of any one of the three original images includes: calculating the horizontal gradient magnitude and the vertical gradient magnitude of any one of the three original images at the current pixel coordinates. Based on the root mean square results of the horizontal gradient magnitude and the vertical gradient magnitude, the local texture complexity index at the current pixel coordinates is calculated. The local texture complexity index is input into a preset mapping function, and the dynamic window size at the current pixel coordinates is output. Centered on the current pixel coordinates, an asymmetric local sliding window is constructed according to the dynamic window size. The major axis of the asymmetric local sliding window is consistent with the gradient direction synthesized from the horizontal gradient magnitude and the vertical gradient magnitude.

[0012] Preferably, the step of calculating the first Gaussian distance attenuation weight into the interior of the highly reflective region based on the boundary pixel coordinates of the highly reflective region includes: extracting the gray values ​​of all pixels within the highly reflective region and calculating the spatial distribution standard deviation of the gray values ​​within the highly reflective region; The standard deviation of the spatial distribution is used as the variance parameter of the Gaussian distribution function, and the geometric center pixel coordinates of the highly reflective region are used as the mean parameter of the Gaussian distribution function. For each pixel coordinate within the highly reflective area, calculate the Euclidean distance from each pixel coordinate to the geometric center pixel coordinate; The Euclidean distance and the variance parameter are substituted into the Gaussian distribution function for calculation to generate the first Gaussian distance decay weight corresponding to each pixel coordinate.

[0013] Preferably, the multi-scale contextual feature aggregation through parallel convolution operations with different dilation rates includes: extracting the maximum feature map size from the multiple initial feature maps at different scales; Based on the proportional relationship between the maximum feature map size and the physical resolution size of the fused image, the field of view scaling factor of the current metal stamping part to be detected is calculated; Input the field scaling factor into a preset hole rate lookup table to match multiple target hole rates corresponding to the field scaling factor; Multiple target void ratios are configured within the void space pyramid pooling layer. Convolutional kernels of the multiple target void ratios are used to perform convolution operations on the multiple initial feature maps of different scales to extract spatial context features of different receptive field ranges. The extracted spatial context features of different receptive field ranges are concatenated along the channel dimension to output the aggregated feature map.

[0014] Preferably, the three pulse trigger signals have a fixed time interval, including: acquiring the real-time running speed of the upstream conveyor belt of the workstation where the industrial camera is located; Based on the real-time operating speed and the single exposure time of the industrial camera, the displacement offset of the metal stamping part during a single exposure is calculated. The displacement offset is compared with the preset maximum allowable displacement error. When the displacement offset is less than the maximum allowable displacement error, the fixed time interval between the three pulse trigger signals remains unchanged. When the displacement offset is greater than or equal to the maximum allowable displacement error, the fixed time interval between two adjacent pulse trigger signals in the three pulse trigger signals is shortened based on the difference between the displacement offset and the maximum allowable displacement error, and the single exposure gain of the industrial camera is reduced simultaneously.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention acquires three original images sequentially from a forward diffuse light source, a coaxial light source, and a low-angle side light source according to a preset time sequence. Based on the local grayscale variance at the same pixel coordinates, it divides high-reflectivity areas into low-illuminance dark areas and constructs a spatial weighted mask map to assign weights to the corresponding light source images for different regions, performing pixel-level weighted fusion. This invention utilizes the difference in reflected energy distribution at the same physical location on a metal surface from different angle light sources to reconstruct the low-angle side light source image information in the high-reflectivity area, the coaxial light source image information in the low-illuminance dark area, and the diffuse reflection image information in the normal area. This eliminates local overexposure caused by specular reflection and dark areas caused by insufficient illumination in single-frame images, restoring the edge continuity and texture grayscale details of defective areas in the fused image, enabling subsequent convolutional neural networks to extract complete defect features from the fused image.

[0016] 2. By employing a local sliding window of fixed size or dynamically determined size based on gradient magnitude to calculate grayscale variance, the system adapts to the boundary division of regions with varying texture complexity. By calculating Gaussian distance attenuation weights based on boundary pixel coordinates, a smooth transition in weight distribution between different illumination regions is achieved. By matching the void ratio of the void space pyramid pooling layer based on the field-of-view scaling factor, the system adapts to the extraction of contextual features from stamped parts of different sizes. By dynamically adjusting the trigger pulse time interval and exposure gain based on the displacement offset calculated from the real-time running speed of the conveyor belt, the system compensates for image motion blur and positional offset under high-speed motion conditions, ensuring the spatial alignment accuracy of dynamically acquired images and static fusion processing in online pipeline conditions. Attached Figure Description

[0017] Figure 1 This is a flowchart of the multi-source timing triggering and image synchronous acquisition process of the present invention; Figure 2 This is a flowchart of the local grayscale variance calculation and region classification labeling of the present invention; Figure 3 This is a flowchart of the spatial weight mask map construction process of the present invention; Figure 4This is a flowchart of the pixel-level weighted fusion process for generating a fused image according to the present invention; Figure 5 This is a flowchart illustrating the feature aggregation of the void space pyramid pooling layer of the present invention. Figure 6 This is a flowchart of the convolutional neural network defect segmentation output of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please refer to Figure 1 This embodiment provides an online defect detection method for metal stamping parts based on industrial vision. The metal stamping parts to be tested are transported to the vision inspection station via a conveyor mechanism. The vision inspection station is equipped with a controller, an industrial camera, a forward diffuse reflection light source, a coaxial light source, a low-angle side light source, and a processor. The controller is communicatively connected to the driving ends of the industrial camera and the three sets of light sources, and the processor is communicatively connected to the controller and the industrial camera. When the metal stamping parts to be tested enter the vision inspection station, the controller triggers the forward diffuse reflection light source, the coaxial light source, and the low-angle side light source sequentially according to a preset timing sequence. The industrial camera simultaneously acquires three corresponding original images. In the preset timing sequence, the trigger signals of two adjacent light sources have a fixed time interval, ensuring that when one light source is triggered to emit light, the other two light sources are in an off state, avoiding mutual interference between different light sources. The exposure process of the industrial camera is completely synchronized with the emission process of the corresponding light source, ensuring that each original image corresponds to the illumination conditions of only one light source. The three original images captured by the industrial camera are the original image corresponding to the forward diffuse reflection light source, the original image corresponding to the coaxial light source, and the original image corresponding to the low-angle side light source. The resolution of the three original images is completely consistent, and the same pixel coordinates correspond to the same physical position on the surface of the metal stamping part to be tested. The pixel grayscale value of the three original images ranges from 0 to 255.

[0020] refer to Figure 2After acquiring three original images from the industrial camera, the processor calculates the local gray-level variance at the same pixel coordinates in the three original images. Based on the local gray-level variance, the pixel region is labeled as a high-reflectivity area or a low-light dark area. Specifically, for each pixel coordinate in the three original images, a local sliding window is constructed centered on that pixel coordinate. All pixel gray-level values ​​within the coverage area of ​​the local sliding window in the three original images are extracted. The gray-level variance of each pixel coordinate position in the three original images is calculated, generating a variance feature map with the same resolution as the original images. For each connected component in the variance feature map, the region type is labeled based on the comparison between the gray-level variance value within the connected component and a preset threshold.

[0021] After region labeling, the processor constructs a spatial weight mask, which contains three single-channel weight matrices with the same resolution as the original images, corresponding to the fusion weights of the three original images. For highly reflective areas, weight values ​​are assigned to the original image pixels corresponding to low-angle sidelight sources; for low-illumination dark areas, weight values ​​are assigned to the original image pixels corresponding to coaxial light sources; and for the remaining areas, weight values ​​are assigned to the original image pixels corresponding to forward diffuse light sources. The sum of the three weight values ​​corresponding to each pixel coordinate is 1, ensuring that the dynamic range of grayscale values ​​remains consistent during the fusion process.

[0022] The processor performs pixel-level weighted fusion of the three original images based on the spatial weight mask map to generate a fused image. Specifically, each single-channel weight matrix is ​​multiplied pixel-by-pixel with the original image of the corresponding light source to generate three weighted images. Then, the gray values ​​at the same pixel coordinates of the three weighted images are summed to obtain the gray value of the fused image at that pixel coordinate. After traversing all pixel coordinates, a complete fused image is generated. The fused image eliminates the loss of texture information caused by local overexposure and low-light dark areas in the single-frame image. High-reflectivity areas retain the texture details of the low-angle side light source image, low-light dark areas retain the gray-scale information of the coaxial light source image, and the remaining areas retain the overall gray-scale uniformity of the forward diffuse light source image.

[0023] refer to Figure 6The processor inputs the fused image into a pre-trained convolutional neural network (CNN), outputting a pixel-level segmentation mask containing defect categories. The pre-trained CNN employs an encoder-decoder architecture, taking a single-channel fused image as input and outputting a pixel-level segmentation mask with the same resolution as the fused image. Each pixel value in the segmentation mask corresponds to a defect category, including background, scratches, dents, cracks, and deformation. The CNN is trained using a labeled dataset of defect images from metal stamping parts. This dataset contains fused image samples with different lighting conditions and defect types, along with corresponding pixel-level labels. The training process uses a cross-entropy loss function to optimize network parameters until the network converges. The trained CNN can then perform end-to-end processing on the fused image, extracting defect features and classifying them at the pixel level, outputting the final segmentation mask.

[0024] Table 1 Comparison of pixel grayscale values ​​and variance characteristics in different regions under different light sources. Highly reflective areas 220-255 220-255 20-80 >1200 Weighting of images from low-angle sidelight sources Low-light dark areas 0-30 0-30 0-30 <50 Weights assigned to coaxial light source image Normal transition region 50-200 50-200 50-200 50-1200 Weights assigned to images with positive diffuse light sources The table above shows the grayscale value distribution characteristics, corresponding variance ranges, and weight allocation rules of different regions under three light sources in this embodiment. It clarifies the correspondence between region division and weight allocation, and clearly presents the physical logic of grayscale differences and variance calculation in different regions, ensuring that those skilled in the art can fully reproduce the entire process of region marking and weight mask construction.

[0025] In a preferred embodiment, after receiving a signal indicating the arrival of a metal stamping part from an external photoelectric sensor, the controller generates three pulse trigger signals with fixed time intervals. The photoelectric sensor is deployed upstream of the vision inspection station. When the metal stamping part under test passes through the detection area of ​​the photoelectric sensor, the sensor generates a level-flipping signal and transmits it to the controller. Upon receiving the arrival signal, the controller calculates the delayed trigger duration based on the operating status of the conveyor mechanism. After ensuring the metal stamping part is fully within the field of view of the vision inspection station, the controller begins outputting pulse trigger signals. The three pulse trigger signals are respectively connected to the driving ends of the forward diffuse reflection light source, the coaxial light source, and the low-angle side light source. The duration of each pulse trigger signal corresponds to the stable emission time of the light source. The light source driving end starts constant current output upon receiving the rising edge of the pulse trigger signal, driving the light source into a stable emission state. Upon receiving the falling edge of the pulse trigger signal, the constant current output is turned off, and the light source is extinguished. The industrial camera's hardware trigger pins are connected to three pulse trigger signals. The camera initiates a single exposure upon receiving the rising edge of each pulse trigger signal and completes image signal reading and output after the falling edge of each signal, generating the original image of the corresponding light source. The industrial camera employs a global exposure mode, where all pixels are simultaneously exposed during exposure, preventing motion distortion. Furthermore, the exposure duration perfectly matches the high-level duration of the pulse trigger signals, ensuring the entire exposure process is conducted within a stable light-emitting phase and guaranteeing uniform image brightness.

[0026] The processor acquires the real-time operating speed of the conveyor mechanism. Based on the real-time operating speed and the single exposure time of the industrial camera, it calculates the displacement offset of the metal stamping part during a single exposure. The formula for calculating the displacement offset is: in, This is the displacement offset. For the real-time operating speed of the transmission mechanism, This refers to the single exposure time of the industrial camera. The processor compares the displacement offset with a preset maximum allowable displacement error, which is half the physical size of a single pixel. When the displacement offset is less than the maximum allowable displacement error, the fixed time interval between the three pulse trigger signals remains unchanged. When the displacement offset is greater than or equal to the maximum allowable displacement error, based on the difference between the displacement offset and the maximum allowable displacement error, the fixed time interval between two adjacent pulse trigger signals is shortened, and the single exposure gain of the industrial camera is reduced simultaneously. The formula for adjusting the fixed time interval is: in, The fixed time interval before adjustment The adjusted fixed time interval, This is the preset maximum allowable displacement error. The formula for adjusting the exposure gain is: in, This is the single exposure gain before adjustment. This represents the adjusted single-exposure gain. Adjusting the exposure gain compensates for the exposure duration variation caused by the shortened time interval, maintaining overall brightness consistency across the three original images and preventing errors in subsequent grayscale variance calculations due to brightness differences. By dynamically adjusting the trigger time interval and exposure gain, the positional offset of the metal stamping part under test during high-speed operation of the conveyor mechanism can be compensated, ensuring that the same pixel coordinates in the three original images strictly correspond to the same physical position on the surface of the metal stamping part under test, thus guaranteeing spatial alignment accuracy in subsequent image processing.

[0027] When calculating the local grayscale variance at the same pixel coordinates in three original images, the processor constructs a fixed-size local sliding window centered on the pixel coordinates of any one of the three original images. The window size is an odd number to ensure that the window has a unique center pixel coordinate. For pixel coordinates at image edges, edge mirroring is used to supplement the pixel values ​​within the window coverage area to avoid deviations in the calculation results for edge pixels. All pixel grayscale values ​​within the coverage area of ​​the local sliding window in the three original images are extracted, and the grayscale variance of each pixel coordinate position within the local sliding window across the three original images is calculated to generate a variance feature map. The formula for calculating the mean of the grayscale values ​​of any pixel coordinate across the three images is: in, pixel coordinates The average grayscale values ​​of the three frames. For a positive diffuse light source, the original image corresponds to the pixel coordinates. grayscale value at that location For the original image corresponding to the coaxial light source, in pixel coordinates grayscale value at that location For low-angle side light sources, the original image corresponds to the pixel coordinates The grayscale value at that location. The formula for calculating the grayscale variance corresponding to that pixel coordinate is: in, pixel coordinates The grayscale variance at each location is used to generate a variance feature map with the same resolution as the original image after traversing all pixel coordinates.

[0028] In the variance feature map, the processor uses an 8-neighborhood connected component analysis method to extract all connected components. Simultaneously, a minimum connected component area threshold is set to remove noisy connected components with areas smaller than the threshold, avoiding misclassification. Connected components with gray-level variance greater than a preset reflectivity threshold are marked as high-reflectivity regions, connected components with gray-level variance less than a preset dark area threshold are marked as low-light dark areas, and connected components with gray-level variance between the preset reflectivity threshold and the preset dark area threshold are marked as normal transition regions. The preset reflectivity threshold is greater than the preset dark area threshold. Pixels in the high reflectivity area experience specular overexposure under both forward diffuse and coaxial light sources, with grayscale values ​​approaching the upper limit. Under low-angle side lighting, no specular reflection enters the lens, resulting in low grayscale values. The grayscale values ​​of the three frames differ greatly, thus the grayscale variance is greater than the preset reflectivity threshold. Pixels in the low-light dark area cannot obtain effective illumination under any of the three light sources, and their grayscale values ​​are all close to the lower limit. The grayscale values ​​of the three frames differ very little, thus the grayscale variance is less than the preset dark area threshold. Pixels in the normal transition area can obtain uniform illumination under all three light sources, with small differences in grayscale values. Therefore, the grayscale variance is between the two thresholds.

[0029] In a preferred embodiment, when the processor constructs the local sliding window, it first calculates the horizontal and vertical gradient magnitudes at the current pixel coordinates in any one of the three original images. The Sobel operator is used for gradient calculation, and the formula for calculating the horizontal gradient magnitude is: The formula for calculating the vertical gradient magnitude is: in, The magnitude of the gradient in the horizontal direction. The magnitude of the gradient in the vertical direction. For the horizontal Sobel operator, This is the Sobel operator in the vertical direction. Based on the root mean square results of the horizontal and vertical gradient magnitudes, the local texture complexity index at the current pixel coordinates is calculated using the following formula: in, This is a local texture complexity index. A higher index value indicates richer texture and more pronounced edge features at the current pixel coordinates. The local texture complexity index is input into a predefined mapping function, which outputs the dynamic window size at the current pixel coordinates. The mapping function is linear, and the window size ranges from 3 to 11 (odd numbers) to ensure the window has a unique center pixel coordinate. An asymmetric local sliding window is constructed centered on the current pixel coordinates according to the dynamic window size. The major axis of the asymmetric local sliding window aligns with the gradient direction resulting from the combined horizontal and vertical gradient magnitudes. The gradient direction is calculated using the following formula: in, The angle between the gradient direction and the horizontal direction is denoted as . The major axis of the asymmetric local sliding window is the dynamic window size, and the minor axis is half the major axis length but not less than 3. The major axis of the window extends along the gradient direction, and the minor axis extends perpendicular to the gradient direction. For non-integer pixel coordinates within the window coverage area, bilinear interpolation is used to obtain the corresponding pixel grayscale value. Using a dynamically sized asymmetric local sliding window can adapt to regions with varying texture complexity. For textured edge regions, a larger window size improves the stability of variance calculation; for smooth, flat regions, a smaller window size improves the accuracy of region boundary segmentation. Simultaneously, the window's major axis extending along the gradient direction better fits the defect edges and the contour edges of the stamped part, avoiding misclassification of boundary pixels.

[0030] Table 2 Comparison of Trigger Timing and Exposure Parameter Adjustments under Different Conveyor Belt Speeds 0.2 5 0.01 20 0 0.4 5 0.02 20 0 0.6 5 0.03 20 0 0.8 5 0.04 20 0 1.0 5 0.05 20 0 1.2 5 0.06 16.7 -1.5 The table above shows the calculation results of the displacement offset, the trigger time interval, and the adjustment parameters of the exposure gain for different conveyor belt operating speeds in this embodiment. It clarifies the numerical change law of dynamic adjustment and clearly presents the correspondence between displacement offset and parameter adjustment, ensuring that those skilled in the art can fully reproduce the dynamic adjustment process of trigger timing and exposure parameters.

[0031] In a preferred embodiment, reference Figure 3When constructing the spatial weighted mask map, the processor first obtains the boundary pixel coordinates of the high-reflectivity regions and the low-illuminance dark regions. Specifically, edge detection is performed on the marked high-reflectivity regions. The Canny edge detection operator is used to extract the set of boundary pixel coordinates of the high-reflectivity regions, and similarly, the set of boundary pixel coordinates of the low-illuminance dark regions is extracted. High and low thresholds are set during the edge detection process to ensure the accuracy of the boundary pixel coordinate extraction and avoid boundary breaks or false boundaries. Based on the boundary pixel coordinates of the high-reflectivity regions, a first Gaussian distance attenuation weight is calculated into the interior of the high-reflectivity regions, and based on the boundary pixel coordinates of the low-illuminance dark regions, a second Gaussian distance attenuation weight is calculated into the interior of the low-illuminance dark regions.

[0032] Specifically, the grayscale values ​​of all pixels within the highly reflective area are extracted, and the spatial distribution standard deviation of the grayscale values ​​within the highly reflective area is calculated using the following formula: in, The standard deviation of the spatial distribution of grayscale values ​​in highly reflective areas. This represents the total number of pixels in the highly reflective area. This is the set of pixel coordinates for the highly reflective area. This is the average grayscale value of all pixels within the highly reflective region. The geometric center pixel coordinates of the highly reflective region are calculated using the following formula: in, Let be the pixel coordinates of the geometric center of the highly reflective region. For each pixel coordinate within the highly reflective region, calculate the Euclidean distance from that pixel coordinate to the geometric center pixel coordinates using the following formula: in, Pixel coordinates within the highly reflective area The Euclidean distance to the geometric center. Substituting the Euclidean distance and the standard deviation of the spatial distribution into the Gaussian distribution function, the first Gaussian distance attenuation weight corresponding to the pixel coordinate is calculated using the following formula: in, The first Gaussian distance attenuation weight is defined as (0,1). At the geometric center of the highly reflective region, the Euclidean distance is 0 and the weight is 1. As the pixel coordinates move further away from the geometric center and closer to the region boundary, the weight gradually decreases.

[0033] For low-light dark areas, the same calculation logic is used to generate a second Gaussian distance attenuation weight. All pixel grayscale values ​​within the low-light dark area are extracted, and the spatial distribution standard deviation and geometric center pixel coordinates of the grayscale values ​​within the dark area are calculated. For each pixel coordinate within the dark area, the Euclidean distance to the geometric center is calculated and substituted into the Gaussian distribution function to generate the second Gaussian distance attenuation weight. The weight value also ranges from (0,1), with a weight value of 1 at the geometric center of the dark area, and the weight value gradually decreases as it approaches the area boundary.

[0034] The first Gaussian distance attenuation weight is used as the weight value of the original image pixel corresponding to the low-angle side light source in the spatial weight mask. The second Gaussian distance attenuation weight is used as the weight value of the original image pixel corresponding to the coaxial light source in the spatial weight mask. For other regions, a fixed constant weight value is used as the weight value of the original image pixel corresponding to the forward diffuse light source in the spatial weight mask. To ensure that the sum of the three weight values ​​at each pixel coordinate is 1, the weight values ​​are normalized. For pixel coordinates in highly reflective regions, the weight value corresponding to the low-angle side light source is the first Gaussian distance attenuation weight, the weight value corresponding to the forward diffuse light source is 1 minus the first Gaussian distance attenuation weight, and the weight value corresponding to the coaxial light source is 0. For pixel coordinates in low-illuminance dark regions, the weight value corresponding to the coaxial light source is the second Gaussian distance attenuation weight, the weight value corresponding to the forward diffuse light source is 1 minus the second Gaussian distance attenuation weight, and the weight value corresponding to the low-angle side light source is 0. For pixel coordinates in normal transition regions, the weight value corresponding to the forward diffuse light source is 1, and the weight values ​​corresponding to the coaxial light source and the low-angle side light source are 0. By using Gaussian distance attenuation weights, a smooth transition of weights is achieved between high reflectivity areas, low illumination dark areas, and normal transition areas. This avoids grayscale jumps and artifacts at the boundaries of the fused image caused by abrupt weight changes, thereby improving the grayscale uniformity of the fused image and the accuracy of subsequent feature extraction.

[0035] refer to Figure 4The processor performs pixel-level weighted fusion of the three original images based on the spatial weight mask to generate a fused image. Specifically, the weight values ​​of the low-angle side light source in the spatial weight mask are multiplied pixel-by-pixel by the pixel gray values ​​of the original image corresponding to the low-angle side light source to generate a first weighted image; the weight values ​​of the coaxial light source in the spatial weight mask are multiplied pixel-by-pixel by the pixel gray values ​​of the original image corresponding to the coaxial light source to generate a second weighted image; and the weight values ​​of the forward diffuse light source in the spatial weight mask are multiplied pixel-by-pixel by the pixel gray values ​​of the original image corresponding to the forward diffuse light source to generate a third weighted image. The gray values ​​at the same pixel coordinates of the first, second, and third weighted images are summed to generate the gray value of the fused image at that pixel coordinate. After traversing all pixel coordinates, a complete fused image is generated. The dynamic range of grayscale values ​​in the fused image remains consistent with that of the original image, avoiding information loss caused by grayscale overflow or compression. At the same time, through smooth weight transition, fusion artifacts at the boundaries of different regions are eliminated, and the optimal texture information of different regions is fully preserved.

[0036] Table 3. Comparison of Gaussian distance attenuation weights at different distances within highly reflective areas. The table above shows the first Gaussian distance attenuation weight value corresponding to different distance positions in the highly reflective area under different spatial distribution standard deviations in this embodiment. It clarifies the variation law of weight attenuation and clearly presents the correspondence between spatial distribution standard deviation and weight attenuation rate, ensuring that those skilled in the art can completely reproduce the calculation process of Gaussian distance attenuation weight.

[0037] In a preferred embodiment, the processor fuses the image input into a pre-trained convolutional neural network (CNN) and outputs a pixel-level segmentation mask containing defect categories. The CNN includes a feature extraction backbone, a dilated spatial pyramid pooling layer, and a decoder. The fused image is input into the feature extraction backbone to extract multiple initial feature maps at different scales. The feature extraction backbone uses a residual network structure, removing the final fully connected layers and average pooling layers, and retaining multiple convolutional modules at different stages. Each convolutional module outputs an initial feature map at a corresponding scale. The initial feature maps at different scales correspond to different levels of feature information. Shallow feature maps contain more detailed features such as edges and textures, while deep feature maps contain more semantic features.

[0038] refer to Figure 5The process involves inputting multiple initial feature maps at different scales into a hollow spatial pyramid pooling layer. Multi-scale contextual features are aggregated through parallel convolutional operations with different void ratios, outputting an aggregated feature map. Specifically, the largest feature map size is extracted from the initial feature maps at multiple scales. Based on the ratio between the largest feature map size and the physical resolution of the fused image, the field-of-view scaling factor for the current metal stamping part to be detected is calculated. The physical meaning of the field-of-view scaling factor is the actual physical size of the metal stamping part to be tested corresponding to a single pixel in the feature map. The field-of-view scaling factor is then input into a pre-defined void ratio lookup table to match multiple target void ratios corresponding to the field-of-view scaling factor. The pre-defined void ratio lookup table stores void ratio combinations corresponding to different field-of-view scaling factor ranges. Each combination contains multiple different void ratios to cover feature extraction across different receptive field ranges.

[0039] Multiple target dilation rates are configured within the dilated spatial pyramid pooling layer. Convolutional kernels with these target dilation rates are used to perform convolutional operations on initial feature maps at different scales to extract spatial context features within different receptive fields. The dilated spatial pyramid pooling layer contains multiple parallel convolutional branches, one of which is a 1×1 convolutional layer used to adjust the number of channels in the feature map. The remaining branches are 3×3 dilated convolutional layers with dilation rates corresponding to the multiple target dilation rates. Each convolutional branch is followed by a batch normalization layer and a linear rectified activation function. The initial feature maps at different scales are concatenated along the channel dimension and then input into the parallel branches of the dilated spatial pyramid pooling layer. Each branch outputs a feature map of the same size. Finally, the feature maps output from all branches are concatenated along the channel dimension to generate an aggregated feature map. By dynamically matching the dilation rate based on the field of view scaling factor, the receptive field range of the convolution kernel can be adaptively adjusted. For scenes with a small field of view scaling factor, a smaller dilation rate is used to extract fine small defect features; for scenes with a large field of view scaling factor, a larger dilation rate is used to extract a large range of contextual features, thereby improving the feature extraction accuracy of defects of different sizes.

[0040] The aggregated feature map is input into the decoder, which uses upsampling to restore the aggregated feature map to the same resolution as the fused image, outputting a pixel-level segmentation mask. The decoder employs a multi-level upsampling structure. In each upsampling stage, shallow features from the corresponding stage of the backbone network are extracted and fused to supplement the details lost during upsampling. Specifically, the aggregated feature map is adjusted for channel count using a 1×1 convolutional layer and then upsampled using bilinear interpolation to obtain the first-level upsampled feature map. This first-level upsampled feature map is then concatenated with the shallow features from the corresponding stage of the backbone network by channel dimension, and feature fusion and channel count adjustment are performed using a convolutional layer to obtain the first-level fused feature map. The first-level fused feature map is then upsampled again using bilinear interpolation to obtain the second-level upsampled feature map, which is then fused with the shallow features from the corresponding stage of the backbone network to obtain the second-level fused feature map. This upsampling and feature fusion process is repeated until the resolution of the feature map is restored to the same size as the fused image. Finally, a 1×1 convolutional layer is used to adjust the number of channels in the feature map to match the number of defect categories, outputting a pixel-level segmentation mask. Each pixel value in the pixel-level segmentation mask corresponds to a defect category. Specifically, for each pixel coordinate, the index corresponding to the maximum value in the channel dimension is taken as the defect category label for that pixel, generating a single-channel segmentation mask. Each pixel value corresponds to a unique defect category, realizing pixel-level detection and classification of defects in metal stamping parts.

[0041] Table 4. Comparison of Target Void Ratio Combinations Corresponding to Different Field-of-View Zoom Factors S≤0.05 [1,3,6,9] 3-19 ≤0.95 0.05<S≤0.1 [2,4,8,12] 5-25 0.25-2.5 0.1<S≤0.2 [3,6,12,18] 7-37 0.7-7.4 S>0.2 [4,8,16,24] 9-49 1.8-9.8 The table above shows the target void ratio combinations, corresponding receptive field ranges, and applicable defect size ranges obtained under different field-of-view scaling factors in this embodiment. It clarifies the rules and applicable scenarios of dynamic void ratio matching, and clearly presents the correspondence between the field-of-view scaling factor and the receptive field range, ensuring that those skilled in the art can fully reproduce the entire process of multi-scale context feature aggregation.

Claims

1. An online defect detection method for metal stamping parts based on industrial vision, characterized in that, include: When the metal stamping part to be tested enters the vision inspection station, the controller triggers the forward diffuse reflection light source, the coaxial light source and the low-angle side light source in sequence according to the preset timing, and the industrial camera simultaneously acquires the corresponding three original images. The processor calculates the local gray-level variance at the same pixel coordinates in the three original images, and marks the pixel area as a high-reflectivity area or a low-illuminance dark area based on the local gray-level variance. A spatial weighted mask is constructed. For the high reflectivity area, the weight value is assigned to the original image pixel corresponding to the low-angle side light source. For the low illumination dark area, the weight value is assigned to the original image pixel corresponding to the coaxial light source. For the remaining areas, the weight value is assigned to the original image pixel corresponding to the forward diffuse reflection light source. The three original images are pixel-level weighted fusion based on the spatial weight mask to generate a fused image; The fused image is input into a pre-trained convolutional neural network, which outputs a pixel-level segmentation mask containing defect categories.

2. The online defect detection method for metal stamping parts based on industrial vision according to claim 1, characterized in that, The processor calculates the local gray-level variance at the same pixel coordinates of the three original images, and marks the pixel region as a high-reflectivity region or a low-illuminance dark region based on the local gray-level variance, including: constructing a local sliding window of a fixed size with the pixel coordinates of any one of the three original images as the center; Extract all pixel grayscale values ​​of the three original images within the coverage area of ​​the local sliding window, calculate the grayscale variance of each identical pixel coordinate position within the local sliding window between the three original images, and generate a variance feature map. In the variance feature map, connected regions with gray-level variance greater than a preset reflectivity threshold are marked as high reflectivity regions, connected regions with gray-level variance less than a preset dark area threshold are marked as low illumination dark areas, and connected regions with gray-level variance between the preset reflectivity threshold and the preset dark area threshold are marked as normal transition regions.

3. The online defect detection method for metal stamping parts based on industrial vision according to claim 1, characterized in that, The construction of the spatial weighted mask map includes: obtaining the boundary pixel coordinates of the high reflectivity region and the boundary pixel coordinates of the low illumination dark region; Based on the boundary pixel coordinates of the high reflectivity region, a first Gaussian distance attenuation weight is calculated into the interior of the high reflectivity region; based on the boundary pixel coordinates of the low illumination dark region, a second Gaussian distance attenuation weight is calculated into the interior of the low illumination dark region. The first Gaussian distance attenuation weight is used as the weight value of the original image pixel corresponding to the low-angle side light source in the spatial weight mask. The second Gaussian distance attenuation weight is used as the weight value of the original image pixel corresponding to the coaxial light source in the spatial weight mask. For the remaining areas, a fixed constant weight value is used as the weight value of the original image pixel corresponding to the forward diffuse light source in the spatial weight mask.

4. The online defect detection method for metal stamping parts based on industrial vision according to claim 1, characterized in that, The step of performing pixel-level weighted fusion of the three original images based on the spatial weight mask to generate a fused image includes: multiplying the weight value corresponding to the low-angle side light source in the spatial weight mask with the pixel grayscale value of the original image corresponding to the low-angle side light source pixel by pixel to generate a first weighted image. The weight value corresponding to the coaxial light source in the spatial domain weight mask image is multiplied pixel by pixel with the pixel gray value of the original image corresponding to the coaxial light source to generate a second weighted image. The weight value corresponding to the forward diffuse light source in the spatial domain weight mask image is multiplied pixel by pixel with the pixel gray value of the original image corresponding to the forward diffuse light source to generate a third weighted image. The gray values ​​at the same pixel coordinates of the first weighted image, the second weighted image, and the third weighted image are summed to generate the fused image.

5. The online defect detection method for metal stamping parts based on industrial vision according to claim 1, characterized in that, The step of inputting the fused image into a pre-trained convolutional neural network and outputting a pixel-level segmentation mask containing defect categories includes: the convolutional neural network includes a feature extraction backbone network, a dilated spatial pyramid pooling layer, and a decoder. The fused image is input into the feature extraction backbone network to extract multiple initial feature maps at different scales; The initial feature maps of different scales are input into the hollow spatial pyramid pooling layer, and multi-scale contextual features are aggregated through parallel convolution operations with different dilation rates to output aggregated feature maps. The aggregated feature map is input into the decoder, which restores the aggregated feature map to the same resolution size as the fused image through an upsampling operation, and outputs the pixel-level segmentation mask, where each pixel value in the pixel-level segmentation mask corresponds to a defect category.

6. The online defect detection method for metal stamping parts based on industrial vision according to claim 1, characterized in that, The controller triggers the forward diffuse reflection light source, the coaxial light source and the low-angle side light source in sequence according to a preset timing sequence. The industrial camera simultaneously acquires three corresponding original images, including: after receiving the metal stamping part arrival signal sent by the external photoelectric sensor, the controller generates three pulse trigger signals with fixed time intervals. The three pulse trigger signals are respectively connected to the driving ends of the forward diffuse reflection light source, the coaxial light source and the low-angle side light source, and the duration of each pulse trigger signal corresponds to the stable light emission time of the light source. The industrial camera's hardware trigger pin is connected to the three pulse trigger signals. The industrial camera performs a single exposure when it receives the rising edge of each pulse trigger signal, and completes image signal reading after the falling edge of each pulse trigger signal, outputting the three original images.

7. The online defect detection method for metal stamping parts based on industrial vision according to claim 2, characterized in that, The step of constructing a fixed-size local sliding window centered on the pixel coordinates of any one of the three original images includes: calculating the horizontal gradient magnitude and the vertical gradient magnitude of any one of the three original images at the current pixel coordinates. Based on the root mean square results of the horizontal gradient magnitude and the vertical gradient magnitude, the local texture complexity index at the current pixel coordinates is calculated. The local texture complexity index is input into a preset mapping function, and the dynamic window size at the current pixel coordinates is output. Centered on the current pixel coordinates, an asymmetric local sliding window is constructed according to the dynamic window size. The major axis of the asymmetric local sliding window is consistent with the gradient direction synthesized from the horizontal gradient magnitude and the vertical gradient magnitude.

8. The online defect detection method for metal stamping parts based on industrial vision according to claim 3, characterized in that, The step of calculating the first Gaussian distance attenuation weight into the interior of the highly reflective region based on the boundary pixel coordinates of the highly reflective region includes: extracting the gray values ​​of all pixels within the highly reflective region and calculating the spatial distribution standard deviation of the gray values ​​within the highly reflective region; The standard deviation of the spatial distribution is used as the variance parameter of the Gaussian distribution function, and the geometric center pixel coordinates of the highly reflective region are used as the mean parameter of the Gaussian distribution function. For each pixel coordinate within the highly reflective area, calculate the Euclidean distance from each pixel coordinate to the geometric center pixel coordinate; The Euclidean distance and the variance parameter are substituted into the Gaussian distribution function for calculation to generate the first Gaussian distance decay weight corresponding to each pixel coordinate.

9. The online defect detection method for metal stamping parts based on industrial vision according to claim 5, characterized in that, The multi-scale contextual feature aggregation through parallel convolution operations with different dilation rates includes: extracting the maximum feature map size from the multiple initial feature maps at different scales; Based on the proportional relationship between the maximum feature map size and the physical resolution size of the fused image, the field of view scaling factor of the current metal stamping part to be detected is calculated; Input the field scaling factor into a preset hole rate lookup table to match multiple target hole rates corresponding to the field scaling factor; Multiple target void ratios are configured within the void space pyramid pooling layer. Convolutional kernels of the multiple target void ratios are used to perform convolution operations on the multiple initial feature maps of different scales to extract spatial context features of different receptive field ranges. The extracted spatial context features of different receptive field ranges are concatenated along the channel dimension to output the aggregated feature map.

10. The online defect detection method for metal stamping parts based on industrial vision according to claim 6, characterized in that, The three pulse trigger signals have a fixed time interval, including: acquiring the real-time running speed of the upstream conveyor belt of the workstation where the industrial camera is located; Based on the real-time operating speed and the single exposure time of the industrial camera, the displacement offset of the metal stamping part during a single exposure is calculated. The displacement offset is compared with the preset maximum allowable displacement error. When the displacement offset is less than the maximum allowable displacement error, the fixed time interval between the three pulse trigger signals remains unchanged. When the displacement offset is greater than or equal to the maximum allowable displacement error, the displacement offset is determined based on the difference between the displacement offset and the maximum allowable displacement error.

Images