Automobile sealing gasket appearance defect detection method and system based on deep learning

Abstract

The invention discloses an automobile sealing gasket appearance defect detection method and system based on deep learning, and belongs to the field of computer vision and defect detection. Comprising the steps that firstly, a narrow-band annular shadowless light source and a high-precision industrial camera are adopted to collect an original gray level image of the sealing gasket, and gray level normalization and region-of-interest extraction are carried out; secondly, constructing and pre-training a deep residual network, and extracting a multi-scale feature map; then, introducing a double-branch module containing channel attention and space attention, and performing enhanced training on the feature map to enable the model to focus on defect related features; thirdly, constructing a feature pyramid, and fusing feature maps with different scales; and finally, carrying out defect classification and positioning by utilizing the fusion features, and automatically rejecting unqualified products in linkage with an industrial control assembly. According to the invention, the detection precision and stability of small defects on the surface of the black low-contrast sealing gasket are effectively improved, and a complete industrial closed loop from image acquisition to automatic elimination is realized.

Description

Technical Field

[0001] This application belongs to the field of computer vision and defect detection, specifically relating to a method and system for detecting appearance defects in automotive gaskets based on deep learning. Background Technology

[0002] With the continuous evolution of intelligent manufacturing technology, automated testing technology is being applied more and more widely in the field of automotive parts quality control. As a key sealing component of automotive powertrain and transmission systems, the surface quality of automotive gaskets directly affects the sealing reliability and driving safety of the entire vehicle. High-precision defect detection is not only a core link in ensuring the product's pass rate, but also an important guarantee for reducing vehicle maintenance costs and enhancing brand competitiveness, which is of great significance for achieving precision production in the automotive industry.

[0003] Among them, deep learning detection methods based on computer vision, due to their powerful feature representation capabilities, are gradually replacing traditional manual visual inspection and becoming the mainstream solution for online full-scale inspection of sealing gaskets. This technology automatically extracts the appearance features of the target to be inspected through convolutional neural networks, aiming to achieve high-precision real-time identification and accurate positioning of abnormal conditions such as cracks, pinholes, and burrs that may exist on the surface of the sealing gasket.

[0004] However, existing deep learning detection models have significant limitations when dealing with black rubber or composite material gaskets. Due to the extremely low surface contrast of the target material, conventional convolution operations are prone to severe loss of sub-millimeter-level defect features during downsampling in deep networks. Simultaneously, uneven lighting and dust interference in the production environment often create false features, leading to frequent false detections when processing background noise. Furthermore, traditional operators have weak ability to fuse multi-scale features, making it difficult to balance the global deformation and local features of minute losses in different workpiece models, resulting in insufficient robustness to meet application requirements in complex industrial scenarios. Therefore, a deep learning-based solution for detecting appearance defects in automotive gaskets is desired. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for detecting appearance defects in automotive gaskets based on deep learning, which can effectively solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A deep learning-based method for detecting appearance defects in automotive gaskets includes the following specific steps: Step S1, Image Acquisition and Preprocessing: The automotive sealing gasket to be inspected is placed at the inspection station and illuminated by a ring-shaped shadowless light source with a wavelength set in a narrow band between 450 nm and 650 nm. A charge-coupled device (CCD) industrial camera, with the actual physical size of a single pixel set between 0.01 mm and 0.05 mm, acquires the original grayscale image. Grayscale normalization and region of interest (ROI) extraction are then performed on the original grayscale image to obtain the ROI image. The grayscale normalization process calculates the mean grayscale value μ and standard deviation σ of all pixels within the ROI, and calculates the normalized pixel values ​​according to a preset linear mapping formula. This stabilizes the mean gray level of the normalized image at around 128. Step S2, construct and pre-train a multi-scale feature extraction network: construct a deep residual network and pre-train it using the region of interest image to focus it on the main area of ​​the sealing pad; the deep residual network includes multiple residual stage groups to output feature maps at different scales. Step S3, construct and pre-train the attention enhancement module: introduce attention enhancement modules for feature maps of different scales respectively, the attention enhancement module being a spatial channel attention module containing channel attention branches and spatial attention branches; pre-train the attention enhancement module so that it can assign higher weights to feature channels and spatial regions related to defects; Step S4, Constructing a Feature Pyramid: The feature maps of different scales processed by the attention enhancement module are fused to construct a feature pyramid containing multiple output levels. Step S5, Defect Prediction and Automatic Removal: Defects are classified and located using the output of the feature pyramid, and an industrial control component is controlled to perform removal actions based on the detection results.

[0007] Furthermore, in step S1, the ring-shaped shadowless light source is composed of multiple sets of light-emitting components arranged in a ring at equal intervals on an aluminum alloy substrate, and the light is scattered by a light-diffusing plate made of a high diffuse reflection material. The preset linear mapping formula is: Where I represents the original pixel value, G represents the preset gain coefficient, ranging from 0.5 to 2.0, and B represents the preset brightness offset value, ranging from -50 to 50.

[0008] Further, in step S1, the region of interest extraction includes: smoothing the normalized image using a Gaussian filter; applying the Sobel operator to calculate the gradient magnitude and direction of each pixel in the image, and determining strong and weak edges in the image by setting a dual threshold mechanism; The algorithm combines morphological closing operations to fill small gaps and texture noise, and connects broken edge segments. Finally, it uses a contour tracking algorithm to lock the outer closed contour of the sealing pad, and extends the redundancy by 5 to 10 pixels to crop out a rectangular area containing the main body of the sealing pad as the region of interest image.

[0009] Furthermore, in step S2, the deep residual network includes a second residual stage group and a fourth residual stage group; The second residual stage group is used to output feature map C3, which has a size of 1 / 8 relative to the original input image and 512 channels. The fourth residual stage group is used to output feature map C5, which has a size of 1 / 32 relative to the original input image and 2048 channels. The pre-training process of the deep residual network in step S2 includes: constructing a pre-training dataset containing pixel-level segmentation annotations of the sealing pad region; using the binary classification cross-entropy loss function as the objective function for pre-training; and minimizing the loss function to enable the network to learn to distinguish between the sealing pad region and the background region.

[0010] Further, in step S3, the pre-training process of the attention enhancement module includes: connecting an auxiliary classifier after the attention enhancement module, taking the defect classification task as the auxiliary target, optimizing the parameters of the channel attention branch and the spatial attention branch in the attention enhancement module through the backpropagation algorithm, so that the channel attention branch learns to assign higher weights to the key channels representing the defect texture, and the spatial attention branch learns to assign higher weights to the spatial regions where defects occur; after training is completed, the auxiliary classifier is removed.

[0011] Further, in step S3, the channel attention branch is used to compress the spatial dimension of the input feature map to obtain the channel description vector, and the channel description vector is passed through the compressed fully connected layer and the expanded fully connected layer in sequence to generate the channel attention weight vector; The spatial attention branch is used to perform global max pooling and global average pooling operations on the channel dimension respectively. The results are concatenated and passed through a convolutional layer to generate a spatial attention weight map.

[0012] Furthermore, in step S4, fusing the feature maps of different scales after processing by the attention enhancement module includes: upsampling feature map C5 using a bilinear interpolation algorithm so that its spatial size is consistent with that of feature map C3; The number of channels in the upsampled C5 is reduced from 2048 to 512 using a 1×1 convolutional layer. The dimensionality-reduced C5 is then fused with the feature map C3 element by element, and the fused feature map is input into a 3×3 convolutional layer for smoothing to obtain the fused feature map P3.

[0013] Furthermore, in step S4, constructing a feature pyramid containing multiple output levels also includes: performing 3×3 convolution downsampling on the fused feature map P3 with a stride of 2 to obtain feature map P4, whose scale relative to the original input image is 1 / 16. The feature map P4 is downsampled by a 3×3 convolution with a stride of 2 to obtain the feature map P5, which has a size ratio of 1 / 32 relative to the original input image.

[0014] Furthermore, in step S5, defect classification and localization include: inputting each output level P3, P4, and P5 of the feature pyramid into the corresponding multi-branch detection head; each detection head presets multiple anchor frames with different aspect ratios, including 1:1, 1:3, and 3:1. The detection head learns the offset of the predicted box relative to the anchor box through the bounding box regression branch, and determines the probability distribution of whether the region belongs to a burr, crack, pinhole, deformation or normal surface through the category classification branch. During the inference phase, a classification confidence threshold is set to retain candidate boxes with a confidence greater than 0.5, and a non-maximum suppression algorithm is applied to remove duplicates. The intersection-union ratio threshold is preset to 0.45.

[0015] A deep learning-based automotive gasket appearance defect detection system includes: The image acquisition module includes a ring-shaped shadowless light source with a wavelength set in a narrow band of 450 nm to 650 nm, and a charge-coupled device industrial camera with the actual physical size of a single pixel set between 0.01 mm and 0.05 mm, used to acquire the original grayscale image of the automotive sealing gasket to be inspected. The image preprocessing module is used to perform grayscale normalization and region of interest extraction on the original grayscale image to obtain the region of interest image. The grayscale normalization process calculates the mean grayscale value μ and standard deviation σ of all pixels in the region of interest and calculates the normalized pixel value I' according to the preset linear mapping formula, so that the mean grayscale value of the normalized image is stabilized at around 128. The multi-scale feature extraction network is a pre-trained deep residual network used to extract feature maps of different scales from images of regions of interest. The attention enhancement module is a pre-trained module that includes channel attention branches and spatial attention branches, used to enhance features of feature maps at different scales. The feature pyramid construction module is used to fuse feature maps of different scales after they have been processed by the attention enhancement module and output feature maps of multiple levels. The defect prediction module is used to classify and locate defects using feature maps at multiple levels, and output defect information. The industrial control components, including a programmable logic controller, a solenoid valve, and a pneumatic actuator, are used to receive defect information and track the position of the defective gasket to be rejected based on the pulse data of the production line encoder. When the defective gasket to be rejected moves to the rejection station, the solenoid valve and the pneumatic actuator are driven to perform the rejection action.

[0016] In summary, this application includes at least one of the following beneficial technical effects: 1. By introducing an attention enhancement module, this invention can accurately locate weak signals representing defects in complex channel and spatial information for specific materials. This mechanism actively amplifies the feature differences between defect areas and normal surfaces by reconstructing the feature map, effectively solving the false detection problem that conventional detection methods are prone to under background noise interference, making the identification of minute defects stable and reliable.

[0017] 2. This invention employs a multi-scale feature fusion strategy, organically combining shallow features containing rich details with deep features containing strong semantic information to construct a comprehensive feature pyramid. This structure ensures that even minute pinholes or slender cracks can be preserved through skip-feature stacking after multiple convolutional downsampling processes, fundamentally solving the structural defect of traditional deep learning models that leads to decreased accuracy when handling small object detection.

[0018] 3. The detection framework designed in this invention can simultaneously consider both the global shape deviation and local microscopic loss of the gasket. Through the parallel operation of multi-scale detection heads, it achieves universal detection of gaskets of different models and specifications. Furthermore, combined with the industrial camera and industrial control components, this invention forms a complete industrial closed loop from image acquisition and intelligent analysis to automatic execution. The industrial control components include a programmable logic controller, solenoid valves, and pneumatic actuators, significantly reducing the labor intensity of manual inspection and improving the intelligence level of automotive core component production lines and the reliability of product quality. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall technical solution for a deep learning-based method for detecting appearance defects in automotive gaskets. Figure 2 This is a schematic diagram illustrating the core principle of feature reconstruction based on spatial channel attention mechanism; Figure 3 It is a flowchart of the logic of image acquisition, grayscale normalization and region of interest extraction; Figure 4 It is a flowchart of the logic of multi-level feature upsampling fusion and feature pyramid construction; Figure 5This is a schematic diagram of the multi-level interaction and data flow between defect detection results and industrial control components. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1-5 The present invention will be further described in detail with reference to specific embodiments.

[0021] First, a deep learning-based method for detecting visual defects in automotive gaskets is presented, which is implemented according to the following steps: The first step is S1, the image acquisition and preprocessing step. Step S1 aims to acquire a high-quality original image of the sealing gasket and, through a series of preprocessing operations, eliminate the influence of factors such as uneven illumination and background interference, providing a standardized, high-purity input image containing only the target object for subsequent deep learning models. This stage mainly includes three steps: illumination and imaging settings, grayscale normalization, and region of interest extraction.

[0022] Step S101, Image Acquisition and Illumination Setup: Place the automotive sealing gasket to be inspected at the preset inspection station. A ring-shaped shadowless light source is used to uniformly illuminate the sealing gasket from all directions. This light source consists of multiple sets of light-emitting components arranged in a ring at equal intervals on an aluminum alloy substrate, and the light is scattered by a light-diffusing plate made of a high diffuse reflection material.

[0023] The light source's emission wavelength is set within a narrow band of 450 nm to 650 nm. By adjusting the intensity ratio of different wavelengths within this range, the visual contrast between the inspected surface and micro-cracks, pinholes, and burrs is enhanced. Under this illumination condition, a charge-coupled device (CCD) industrial camera mounted directly above the gasket acquires the gasket's original grayscale image.

[0024] The camera has a resolution of 2592 pixels × 1944 pixels, with the actual physical size of each pixel set between 0.01 mm and 0.05 mm to ensure clear imaging of tiny defects larger than 0.1 mm.

[0025] Step S102, grayscale normalization processing; after acquiring the original grayscale image, grayscale normalization processing is performed. This process first calculates the mean μ and standard deviation σ of the grayscale values ​​of all pixels within the region of interest. The normalized pixel values... The following is calculated using a preset linear mapping formula: in, Represents the original pixel value. Represents the average gray level of the pixels in the original image. The standard deviation of pixel grayscale This represents the preset gain coefficient, with a value range of [value range missing]. to , This represents the preset brightness offset value, with a range of [value missing]. to The specific values ​​are adjusted based on the actual image brightness distribution to ensure that the mean grayscale value of the normalized image is stable at a certain level. This eliminates the uneven lighting caused by fluctuations in brightness between different batches of light sources or differences in camera exposure time.

[0026] Step S103, Region of Interest Extraction; After grayscale normalization, the region of interest is extracted. This process uses an edge detection algorithm based on gradient information, and the specific steps are as follows: First, a Gaussian filter is used to smooth the normalized image to suppress high-frequency noise. The kernel size of the Gaussian filter is set to 5×5 and the standard deviation is set to 1.0.

[0027] Subsequently, the Sobel operator is applied to calculate the gradient magnitude and direction of each pixel in the image, and a dual-threshold mechanism is used to determine strong and weak edges in the image. The high threshold in the dual-threshold mechanism is set to 150, and the low threshold is set to 50. Pixels with gradient magnitudes higher than the high threshold are marked as strong edges, pixels with gradient magnitudes between the low and high thresholds are marked as weak edges, and pixels with gradient magnitudes lower than the low threshold are suppressed.

[0028] After obtaining the initial edge contour, morphological closing operations are used—first dilation, then erosion—to fill the small gaps and texture noise on the surface of the sealing gasket and connect broken edge segments. The morphological closing operation uses a 3×3 square structuring element as the convolution kernel. The dilation operation expands the edge region outward to fill the voids, and the erosion operation restores the dilated edge region to its original size, thereby filling the small gaps and connecting the breaks.

[0029] Finally, the outer closed contour of the sealing gasket is locked by a contour tracking algorithm, and based on this contour, a redundancy of 5 to 10 pixels is added to the outside to crop out a rectangular area containing the main body of the sealing gasket as the region of interest image.

[0030] In summary, step S1 completes the acquisition and preprocessing of the original image. Through step S1, background interference in the input image is effectively suppressed, and the target to be detected is normalized, providing an ideal data foundation for the subsequent training and inference of the multi-scale feature extraction network.

[0031] The next step, S2, involves constructing a multi-scale feature extraction network. Step S2 builds a deep residual network to extract feature maps of different scales from the preprocessed region of interest image, providing a rich feature base for subsequent attention enhancement and defect detection. This stage mainly includes three parts: network structure design, forward feature extraction, and network pre-training, comprising the following steps: Step S201, Network Structure Design: The region of interest image obtained in step S1 is input into a deep residual network model. This deep residual network model adopts a backbone network based on the ResNet architecture, and its structure consists of an initial convolutional layer, 5 residual stage groups, and a global average pooling layer.

[0032] The initial convolutional layer uses a 7×7 kernel with a stride of 2 and 64 output channels to perform preliminary feature extraction on the input image.

[0033] The first residual stage group contains three residual blocks, and the output feature map has 256 channels, with a scale of 1 / 4 relative to the original input image. This stage group is mainly used to extract the coarse contour features of the sealing gasket.

[0034] The second residual stage group contains four residual blocks, and the output feature map has 512 channels, with a scale size of 1 / 8 relative to the original input image. The feature map output by this stage group retains high-resolution texture information.

[0035] The third residual stage group contains 6 residual blocks, and the output feature map has 1024 channels, with a scale size of 1 / 16 relative to the original input image. This stage group further abstracts semantic information.

[0036] The fourth residual stage group contains three residual blocks, and the output feature map has 2048 channels, with a scale of 1 / 32 relative to the original input image. The feature map output by this stage group contains the highest-level abstract semantics.

[0037] Each residual block employs a skip connection mechanism, which directly passes input features across layers to subsequent outputs via identity mapping. The expression for this is: Here, x is the input feature map, and F(x) is the residual mapping learned by the convolutional layer group. This mechanism effectively alleviates the gradient vanishing or gradient exploding problems that occur during the training of deep neural networks, enabling the network to learn deeper abstract features.

[0038] Step S202, forward feature extraction: During the forward propagation of the network, hierarchical feature maps of different scales are extracted for subsequent processing.

[0039] The second feature map C3 is obtained at the output of the second residual stage group. This feature map has 512 channels and is 1 / 8 the size of the original input image. It retains rich high spatial resolution information and texture details, and can effectively capture the tiny burr features of the sealing gasket edge with a width ranging from 0.05 mm to 0.2 mm.

[0040] The fourth layer feature map C5 is obtained at the output of the fourth residual stage group. This feature map has 2048 channels and a scale size of 1 / 32 relative to the original input image. After multiple downsampling and nonlinear activation, it contains strong abstract semantic information and global topological structure, and can focus on capturing large-scale defect information such as the overall geometric deformation and fracture of the sealing gasket.

[0041] The spatial size ratio between C3 and C5 is 4:1, meaning that the width and height of C3 are both 4 times that of C5. This ratio provides the basis for size matching in the subsequent step S4, where C5 is upsampled by 4 times and then fused with C3.

[0042] Step S203, Network Pre-training: To enable the network to accurately extract features related to the sealing pad region, it needs to be pre-trained. The goal of this pre-training is to teach the network to accurately segment the main sealing pad region from complex backgrounds, so that the feature maps output by each layer of the network can focus on the sealing pad itself and suppress background interference. The pre-training process is as follows: First, a pre-training dataset was constructed. A total of 10,000 images of automotive sealing gaskets from different models and under varying lighting conditions were collected, including 2,000 samples of typical defects. Professional quality inspectors used annotation tools to perform pixel-level segmentation and annotation of the sealing gasket region in each image, generating corresponding mask images. The main sealing gasket region was labeled as 1, and the background region as 0. These annotations provided the network with a standard answer for learning the features of the target region.

[0043] Secondly, a pre-training loss function is designed. The binary classification cross-entropy loss function is used as the objective function for pre-training, and its calculation formula is as follows: Where N represents the total number of pixels in a training batch. This represents the actual label of the i-th pixel, and its value can be either 0 or 1. This represents the probability that the network predicts the pixel belongs to the sealing pad region, with a value ranging from 0 to 1. By minimizing this loss function, the network can accurately distinguish the sealing pad region from the background region.

[0044] Finally, the pre-training parameters were set. The network parameters were iteratively optimized using the stochastic gradient descent optimization algorithm, with an initial learning rate of 0.001, momentum parameter of 0.9, and weight decay coefficient of 0.0001. A total of 200 training cycles were performed, with each cycle iterating through all training data once.

[0045] After pre-training, the weight parameters of each layer of the network are retained, and the top segmentation prediction layer is removed. At this point, the network becomes a general feature extractor. The feature map C3 output from the second residual stage group and the feature map C5 output from the fourth residual stage group will serve as the core input for subsequent defect detection tasks, used to capture the detailed features of small defects and the global features of large-scale defects, respectively.

[0046] In summary, step S2 completes the construction and pre-training of the multi-scale feature extraction network. The pre-trained network's output feature maps C3 and C5 will serve as the core inputs for subsequent attention enhancement processing.

[0047] Step S3 aims to selectively enhance the multi-scale feature maps extracted in step S2, enabling the model to more accurately focus on weak features related to defects while suppressing interference from complex backgrounds and noise. Specifically, for the third-layer feature map C3 and the fifth-layer feature map C5 obtained from the deep residual network, an identical attention module is introduced for processing. This attention module includes a channel attention branch and a spatial attention branch, which act sequentially on the input feature maps, as detailed below.

[0048] Step S301, Construction and training of channel attention branches; First, construct the channel attention branches. For the input feature map, its dimension is... ,in and These represent the height and width of the feature map, respectively. This represents the total number of channels. The feature map is spatially compressed using the global average pooling operator, aggregating the spatial information from each channel into a single value, resulting in a value with dimension [value missing]. The channel description vector.

[0049] The vector is then fed into a compression and expansion structure consisting of two fully connected layers. The first fully connected layer compresses the number of channels to... The ReLU activation function is used for nonlinear transformation to learn the nonlinear dependencies between channels. The second fully connected layer then restores the number of channels to the original number. The final channel attention weight vector is generated by applying a sigmoid activation function. Each element in this weight vector takes a value between 0 and 1, indicating the importance of the corresponding feature channel.

[0050] To ensure that the channel attention mechanism can effectively identify feature channels related to defects, it needs to be pre-trained. The training data comes from the pre-training dataset labeled in step S2. This dataset contains images of car gaskets under different lighting conditions and of different models, and the defect areas in the images have been accurately labeled by professionals.

[0051] The training process employs supervised learning, with defect classification as the auxiliary objective. Specifically, a classifier is connected after the channel attention module, with its loss function defined as binary cross-entropy loss, used to distinguish whether the image region corresponding to the input feature map contains defects. The parameters of the two fully connected layers in the channel attention module are optimized using the backpropagation algorithm, enabling the model to automatically learn to assign higher weights to key channels representing defect textures such as cracks, pinholes, or burrs, while assigning lower weights to invalid channels representing background or noise. After training, the auxiliary classifier is removed, retaining only the trained channel attention module for subsequent processing.

[0052] Step S302: Construction and training of the spatial attention branch; after completing the channel domain feature enhancement, spatial domain analysis is performed on the feature map. For the feature map after channel attention weighting, the spatial attention branch first performs global max pooling and global average pooling operations on the channel dimension respectively, resulting in two independent two-dimensional feature maps, both with dimension 1. These two feature maps capture information about salient regions from different perspectives.

[0053] These two two-dimensional feature maps are concatenated along the channel dimension to form a dimension of The feature map. Then, a convolutional kernel with a size of [size missing] is used. The convolutional layer performs operations on the concatenated feature map. This convolutional layer learns during training how to fuse the information provided by max pooling and average pooling, and outputs a dimensionless vector map. The spatial weight map is then generated. Finally, the sigmoid activation function is used to normalize each pixel value in the map to between 0 and 1, forming the final spatial attention weight map.

[0054] The training of this spatial attention mechanism is performed simultaneously with that of the channel attention mechanism. During the training phase, its parameters are optimized using the loss function of the defect classification task, with the goal of assigning high weights (close to 1) to local regions corresponding to defects in the spatial weight map, while assigning low weights (close to 0) to background regions. Through this training method, the model can learn how to accurately locate the spatial positions where defects may occur in the feature map.

[0055] Step S303, Weighted Reconstruction of Feature Map: After obtaining the weight vectors output by the channel attention branch and the weight map output by the spatial attention branch, they are fused with the original input feature map. First, the channel attention weight vector is multiplied element-wise with the original feature map along the channel dimension to achieve weighting for different channels. Then, the channel-weighted feature map is multiplied element-wise with the spatial attention weight map along the spatial dimension. Through this double-weighting method, the weighted reconstruction of the feature map is finally completed.

[0056] After the above processing, the defect-related regions in the feature map are significantly enhanced in both channel and spatial dimensions, while invalid information related to diffuse reflection noise on the black rubber surface is effectively suppressed, providing a clearer and purer feature basis for subsequent feature fusion and defect detection.

[0057] In summary, step S3 constructs a complete attention enhancement module by combining the channel attention branch and the spatial attention branch. This module first trains the model to identify which feature channels are most critical for defect detection and assigns them higher weights. Secondly, it teaches the model to locate the spatial region where defects are most likely to occur and strengthens the response in that region. This combination enables the model to accurately pinpoint weak signals representing defects from complex industrial images, actively amplifying the feature differences between defects and normal surfaces, and solving the false detection problem that conventional detection methods are prone to in the presence of background noise.

[0058] Step S4 aims to effectively fuse the multi-scale feature maps after attention enhancement processing to construct a feature pyramid. This allows the network to simultaneously utilize the strong semantic information of deep features and the high-resolution detail information of shallow features, thereby improving its ability to detect defects at different scales. The specific implementation process is as follows.

[0059] Step S401: Deep feature upsampling and channel alignment; In step S3, the third-layer feature map C3 and the fifth-layer feature map C5 undergo attention enhancement processing respectively. This step first adjusts the spatial size of the feature map C5 after the attention enhancement processing in step S3.

[0060] The aspect ratio of C5 to the original input image is 1 / 32, while the aspect ratio of C3 is 1 / 8, resulting in a spatial size ratio of 1:4. To merge C5 and C3, a bilinear interpolation algorithm is used to upsample C5, increasing its height and width by a factor of 4 in the horizontal and vertical directions, respectively, so that the spatial size of the upsampled C5 is consistent with that of C3.

[0061] Since C5 has 2048 channels while C3 has 512 channels, their different channel counts would lead to a dimension mismatch if directly added together. Therefore, before element-wise addition and fusion of the upsampled C5 and the enhanced C3, a 1×1 convolutional layer is applied to each feature map to align their channel counts.

[0062] Specifically, C3 can be adjusted to have the same number of channels as the upsampled C5 by passing a 1×1 convolutional layer, or the upsampled C5 can be adjusted to have the same number of channels as C3 by passing a 1×1 convolutional layer. This embodiment uses the latter, reducing the number of channels in the upsampled C5 from 2048 to 512, making it consistent with the number of channels in C3.

[0063] Step S402: Feature map addition, fusion, and smoothing; after channel alignment, the upsampled and dimensionality-reduced C5 and the enhanced C3 are added and fused element-wise. This addition operation adds the pixel values ​​at corresponding spatial locations, achieving a combination of deep semantic information and shallow detail information.

[0064] The fused feature maps are then fed into a 3×3 convolutional layer for smoothing. This convolutional layer serves two purposes: first, it eliminates aliasing effects that may be introduced by upsampling, making the fused feature maps spatially smoother; second, it further extracts complementary information from the fused feature maps, enhancing their expressive power. The stride of this 3×3 convolutional layer is set to 1, and the padding method is set to "same," meaning the spatial dimensions of the feature maps remain unchanged, and the number of output channels is maintained at 512.

[0065] Step S403, complete construction of the feature pyramid; based on the above fusion logic, a complete feature pyramid structure is further constructed so that it can output multiple feature maps of different scales, which are used to detect defects of different size levels.

[0066] First, the fused feature map obtained in step S402 is denoted as P3, which has a size ratio of 1 / 8 to the original input image and 512 channels. This layer retains high spatial resolution and is specifically designed for detecting small defects such as pinholes and microcracks.

[0067] Secondly, a downsampling operation is performed on P3 to construct a lower-resolution feature layer. Specifically, a 3×3 convolutional layer with a stride of 2 is used to downsample P3 to obtain feature map P4, which has a scale size of 1 / 16 relative to the original input image and the number of channels is set to 256. This layer is used to detect medium-sized defects, such as large-area wear or deformation.

[0068] Next, the downsampling operation is repeated on P4, using a 3×3 convolutional layer with a stride of 2 to downsample P4, resulting in feature map P5, which has a size ratio of 1 / 32 relative to the original input image and 128 channels. This layer has the largest receptive field and contains rich global semantic information, specifically designed for detecting large-scale defects such as overall geometric deformation, breakage, or missing parts of the sealing gasket.

[0069] Thus, the feature pyramid structure contains three feature output levels: P3, P4, and P5, corresponding to three scales: 1 / 8, 1 / 16, and 1 / 32, respectively.

[0070] Step S404, training of the feature pyramid module; in order for the feature pyramid to effectively extract and fuse multi-scale features, it needs to be trained. The training process is carried out in conjunction with the defect prediction module in step S5, using an end-to-end training method.

[0071] The training data comes from the pre-training dataset constructed in step S2. This dataset contains 10,000 images, each of which has been precisely bounding box annotated by professionals for the defect area. The annotation information includes the defect category and location coordinates.

[0072] The total loss function during the training phase consists of a weighted average of the classification loss and the localization loss. For each output level P3, P4, and P5 of the feature pyramid, an independent detection head is connected, and each detection head outputs the defect prediction result corresponding to that level.

[0073] The total loss is calculated by summing the losses from all layers. The parameters of the entire network, including those of each convolutional layer in the feature pyramid, are optimized using the backpropagation algorithm. This enables the network to learn to effectively extract and fuse features at different scales, thereby improving its comprehensive detection capability for multi-scale defects.

[0074] The training parameters were set as follows: the optimization algorithm used stochastic gradient descent, with an initial learning rate of 0.01, momentum parameter of 0.9, and weight decay coefficient of 0.0001. The training lasted for 150 epochs. During training, based on the loss changes on the validation set, the learning rate was reduced to 0.1 times its current value at the 80th and 120th epochs, respectively.

[0075] In summary, step S4 integrates deep features by upsampling them and then fusing them with shallow features, and further constructs a feature pyramid containing multiple scales, thus achieving the organic integration of multi-scale features. This approach retains the ability of high-resolution layers to capture details of minute defects while utilizing the semantic perception capabilities of low-resolution layers for global deformation and large-scale defects. This ensures that even after multiple convolutional downsampling processes, minute pinholes or slender cracks can still be preserved and enhanced through feature stacking, providing a rich and complementary feature base for subsequent defect prediction.

[0076] Finally, step S5, the defect prediction step, aims to accurately classify and locate defects on the sealing gasket surface using the multi-scale fused feature map output from the feature pyramid constructed in step S4, and to achieve automatic rejection control on the production line based on the detection results. This step includes the design and training of the detection head network, post-processing during the inference stage, and the linkage between the detection results and the industrial control system. The specific implementation process is as follows.

[0077] Step S501, Multi-branch detection head structure design: Input the output levels P3, P4, and P5 of the feature pyramid constructed in step S403 into the corresponding multi-branch detection heads. Each detection head adopts an anchor-based detection mechanism, structurally divided into two parallel branches: a bounding box regression branch and a category classification branch.

[0078] To address the geometric characteristics of sealing gasket defects, three anchor frames with different aspect ratios are preset: 1:1, 1:3, and 3:1. The 1:1 anchor frame is suitable for approximately circular defects such as pinholes and spots; the 1:3 anchor frame is suitable for slender crack defects; and the 3:1 anchor frame is suitable for laterally extending burrs or scratches. The size of each anchor frame is set according to the receptive field of its feature level. For the high-resolution P3 level, the anchor frame reference size is set to 32 pixels; for the medium-resolution P4 level, the anchor frame reference size is set to 64 pixels; and for the low-resolution P5 level, the anchor frame reference size is set to 128 pixels.

[0079] The bounding box regression branch is responsible for learning the offset of the predicted box relative to the anchor box, including the offset of the center point coordinates and the scaling of the width and height, so as to accurately locate the specific spatial coordinates of the defect in the image. The category classification branch is responsible for determining the probability distribution of the region belonging to one of the five categories: burr, crack, pinhole, deformation, or normal surface. It outputs a 5-dimensional vector, with the sum of its components being 1.

[0080] Step S502: Construction of training data and design of loss function for the detection head; in order for the detection head to accurately predict the type and location of defects, it needs to be trained. The training process is carried out in conjunction with the feature pyramid training in step S404, using an end-to-end approach.

[0081] The training data comes from the pre-training dataset constructed in step S2, which contains 10,000 images. Based on this, professional quality inspectors precisely annotated the defect regions in each image with bounding boxes. Each bounding box contains five pieces of information: defect category label, the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, width, and height. The defect category label is represented using one-hot encoding, corresponding to five categories: burrs, cracks, pinholes, deformation, and normal surfaces. During training, the images and their corresponding bounding box annotations are input as supervisory signals into the network.

[0082] The total loss function during the training phase is a weighted average of the classification loss and the localization loss, calculated as follows: in, Represents the total loss value. This represents the preset balancing weight, set to 0.6, used to balance the contributions of the classification task and the localization task to the total loss.

[0083] Classification loss The FocalLoss function is used, and its calculation formula is as follows: in, This represents the total number of anchor boxes in a training batch. The representative model predicts the first The probability that each anchor box corresponds to its true category label. To balance the weighting of positive and negative samples, the weighting coefficient is set to 0.25. The focusing parameter, which adjusts the weights of easy and difficult samples, is set to 2.0. This loss function reduces the weights of easily classified samples, causing the model to focus more on the few defective samples during training.

[0084] Location loss The smoothed L1 loss function is used, and its calculation formula is as follows: in, This represents the total number of positive sample anchor boxes in a training batch. Representing the The true offset vector of each positive sample anchor box contains four components: center point x-coordinate offset, y-coordinate offset, width scaling, and height scaling. This represents the offset vector predicted by the model. The function is defined as follows: This loss function is insensitive to outliers and can improve the robustness of the regression branch to outlier coordinates.

[0085] The training parameters were set in the same manner as in step S404: the optimization algorithm used stochastic gradient descent, with an initial learning rate of 0.01, momentum parameter of 0.9, and weight decay coefficient of 0.0001, for a total of 150 training epochs. During training, based on the loss changes on the validation set, the learning rate was reduced to 0.1 times the current value at the 80th and 120th epochs, respectively.

[0086] Step S503, Post-processing during the inference phase: During the model inference phase, the detection head at each feature level outputs a large number of preliminary candidate boxes, which include many overlapping boxes and low-confidence boxes. To obtain the final detection results, the following post-processing steps are performed.

[0087] First, set a classification confidence threshold, retain candidate boxes with a class confidence greater than 0.5, and remove candidate boxes with a confidence lower than the threshold.

[0088] Secondly, non-maximum suppression (NMS) is applied to the retained candidate boxes for deduplication. Specifically, all candidate boxes are sorted from highest to lowest classification confidence. The candidate box with the highest confidence is selected as the retained box. The remaining candidate boxes are then iterated through, and the intersection-union ratio (IUR) is calculated between the area of ​​the overlapping region of the two boxes and the area of ​​their union. If the IUR exceeds a preset overlap threshold of 0.45, the candidate box is discarded. This process is repeated until all candidate boxes have been processed.

[0089] Non-maximum suppression ensures that each physical defect corresponds to only one unique class label and bounding box in the output, avoiding duplicate detection.

[0090] In step S504, the detection results are linked with the industrial control components. After post-processing, the detection system transmits the final confirmed defect information to the industrial control components in real time via the industrial Ethernet communication protocol. In this application, the industrial control components include a programmable logic controller, a solenoid valve, and a pneumatic actuator. The transmitted defect information includes the horizontal and vertical coordinates of the defect center point in the image coordinate system, the width and height of the defect bounding box, and the category number to which the defect belongs.

[0091] After receiving defect information, the programmable logic controller (PLC) uses a production line encoder, which monitors conveyor belt displacement, to obtain real-time feedback pulse data and track the position of the defective sealing gasket on the production line. The encoder pulse data has a definite correspondence with the physical displacement of the production line, with a preset number of pulses per millimeter.

[0092] When a gasket marked as defective moves along the production line conveyor belt to the preset rejection station, the programmable logic controller (PLC) outputs a control signal to drive the solenoid valve and pneumatic push rod to perform the rejection action. The compressed air pressure is preset to 0.6 MPa to 0.8 MPa. Upon receiving the control signal, the solenoid valve opens, allowing compressed air to enter the pneumatic push rod. The push rod extends and precisely pushes the defective gasket into the defective product collection area. After rejection is complete, the solenoid valve closes, and the pneumatic push rod resets, awaiting the next rejection command.

[0093] In summary, step S5, by designing a multi-branch detection head adapted to the geometric features of gasket defects and combining a joint optimization strategy of focal loss and smoothing L1 loss, achieves accurate classification and location of different types of defects such as burrs, cracks, pinholes, and deformations. During the inference stage, a non-maximum suppression algorithm effectively filters out redundant detection results, ensuring the uniqueness and accuracy of the output. Finally, through the linkage of industrial Ethernet and a programmable logic controller (PLC), the detection results are transformed into physical rejection actions on the production line, forming a complete industrial closed loop from image acquisition and intelligent analysis to automated execution.

[0094] In a real-world industrial application, this system was deployed on an automotive engine gasket production line. The production line operates at a speed of 60 pieces per minute. The system's single-process inspection time is controlled within 80 milliseconds, enabling online real-time full-volume inspection of the production line.

[0095] For microcracks larger than 0.15 mm, the system achieves a detection accuracy of over 99.5% and a false detection rate below 0.2%. The core algorithm of the system runs on the graphics processing unit of the industrial control computer. Through multi-threaded parallel processing technology, it realizes the streamlined operation of image acquisition, algorithm inference, and industrial control, ensuring that the detection process is synchronized with the production line cycle.

[0096] A deep learning-based automotive gasket appearance defect detection system includes: The image acquisition module includes a ring-shaped shadowless light source with a wavelength set in a narrow band of 450 nm to 650 nm, and a charge-coupled device industrial camera with the actual physical size of a single pixel set between 0.01 mm and 0.05 mm, used to acquire the original grayscale image of the automotive sealing gasket to be inspected. The image preprocessing module is used to perform grayscale normalization and region of interest extraction on the original grayscale image to obtain the region of interest image. The grayscale normalization process calculates the mean grayscale value μ and standard deviation σ of all pixels in the region of interest and calculates the normalized pixel value I' according to the preset linear mapping formula, so that the mean grayscale value of the normalized image is stabilized at around 128. The multi-scale feature extraction network is a pre-trained deep residual network used to extract feature maps of different scales from images of regions of interest. The attention enhancement module is a pre-trained module that includes channel attention branches and spatial attention branches, used to enhance features of feature maps at different scales. The feature pyramid construction module is used to fuse feature maps of different scales after they have been processed by the attention enhancement module and output feature maps of multiple levels. The defect prediction module is used to classify and locate defects using feature maps at multiple levels, and output defect information. The industrial control components, including a programmable logic controller, a solenoid valve, and a pneumatic actuator, are used to receive defect information and track the position of the defective gasket to be rejected based on the pulse data of the production line encoder. When the defective gasket to be rejected moves to the rejection station, the solenoid valve and the pneumatic actuator are driven to perform the rejection action.

[0097] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0098] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for detecting appearance defects in automotive gaskets based on deep learning, characterized in that, Includes the following steps: Step S1, Image Acquisition and Preprocessing: The automotive sealing gasket to be inspected is placed at the inspection station and illuminated by a ring-shaped shadowless light source with a wavelength set in a narrow band between 450 nm and 650 nm. A charge-coupled device (CCD) industrial camera, with the actual physical size of a single pixel set between 0.01 mm and 0.05 mm, acquires the original grayscale image. Grayscale normalization and region of interest (ROI) extraction are then performed on the original grayscale image to obtain the ROI image. The grayscale normalization process calculates the mean grayscale value μ and standard deviation σ of all pixels within the ROI, and calculates the normalized pixel values ​​according to a preset linear mapping formula. This stabilizes the mean gray level of the normalized image at around 128. Step S2, construct and pre-train a multi-scale feature extraction network: construct a deep residual network and pre-train it using the region of interest image to focus it on the main area of ​​the sealing pad; the deep residual network includes multiple residual stage groups to output feature maps at different scales. Step S3, construct and pre-train the attention enhancement module: introduce attention enhancement modules for feature maps of different scales respectively, the attention enhancement module being a spatial channel attention module containing channel attention branches and spatial attention branches; pre-train the attention enhancement module so that it can assign higher weights to feature channels and spatial regions related to defects; Step S4, Constructing a Feature Pyramid: The feature maps of different scales processed by the attention enhancement module are fused to construct a feature pyramid containing multiple output levels. Step S5, Defect Prediction and Automatic Removal: Defects are classified and located using the output of the feature pyramid, and an industrial control component is controlled to perform removal actions based on the detection results.

2. The method for detecting appearance defects of automotive gaskets based on deep learning according to claim 1, characterized in that, In step S1, the ring-shaped shadowless light source is composed of multiple light-emitting components arranged in a ring at equal intervals on an aluminum alloy substrate, and the light is scattered by a light-diffusing plate made of a high diffuse reflection material. The preset linear mapping formula is: Where I represents the original pixel value, G represents the preset gain coefficient, ranging from 0.5 to 2.0, and B represents the preset brightness offset value, ranging from -50 to 50.

3. The method for detecting appearance defects of automotive gaskets based on deep learning according to claim 1, characterized in that, In step S1, the region of interest extraction includes: smoothing the normalized image using a Gaussian filter; applying the Sobel operator to calculate the gradient magnitude and direction of each pixel in the image; and determining strong and weak edges in the image by setting a dual threshold mechanism. The algorithm combines morphological closing operations to fill small gaps and texture noise, and connects broken edge segments. Finally, it uses a contour tracking algorithm to lock the outer closed contour of the sealing pad, and extends the redundancy by 5 to 10 pixels to crop out a rectangular area containing the main body of the sealing pad as the region of interest image.

4. The method for detecting appearance defects of automotive gaskets based on deep learning according to claim 1, characterized in that, In step S2, the deep residual network includes a second residual stage group and a fourth residual stage group; The second residual stage group is used to output feature map C3, which has a size of 1 / 8 relative to the original input image and 512 channels. The fourth residual stage group is used to output feature map C5, which has a size of 1 / 32 relative to the original input image and 2048 channels. The pre-training process of the deep residual network in step S2 includes: constructing a pre-training dataset containing pixel-level segmentation annotations of the sealing pad region; using the binary classification cross-entropy loss function as the objective function for pre-training; and minimizing the loss function to enable the network to learn to distinguish between the sealing pad region and the background region.

5. The method for detecting appearance defects of automotive gaskets based on deep learning according to claim 4, characterized in that, In step S3, the pre-training process of the attention enhancement module includes: connecting an auxiliary classifier after the attention enhancement module, taking the defect classification task as the auxiliary target, optimizing the parameters of the channel attention branch and the spatial attention branch in the attention enhancement module through the backpropagation algorithm, so that the channel attention branch learns to assign higher weights to the key channels representing the defect texture, and the spatial attention branch learns to assign higher weights to the spatial regions where defects occur; after training is completed, the auxiliary classifier is removed.

6. The method for detecting appearance defects of automotive gaskets based on deep learning according to claim 5, characterized in that, In step S3, the channel attention branch is used to compress the spatial dimension of the input feature map to obtain the channel description vector, and then the channel description vector is passed through the compressed fully connected layer and the expanded fully connected layer in sequence to generate the channel attention weight vector. The spatial attention branch is used to perform global max pooling and global average pooling operations on the channel dimension respectively. The results are concatenated and passed through a convolutional layer to generate a spatial attention weight map.

7. The method for detecting appearance defects of automotive gaskets based on deep learning according to claim 4, characterized in that, In step S4, fusing the feature maps of different scales after processing by the attention enhancement module includes: upsampling feature map C5 using a bilinear interpolation algorithm so that its spatial size is consistent with that of feature map C3; The number of channels in the upsampled C5 is reduced from 2048 to 512 using a 1×1 convolutional layer. The dimensionality-reduced C5 is then fused with the feature map C3 element by element, and the fused feature map is input into a 3×3 convolutional layer for smoothing to obtain the fused feature map P3.

8. The method for detecting appearance defects of automotive gaskets based on deep learning according to claim 7, characterized in that, In step S4, constructing a feature pyramid containing multiple output levels also includes: performing 3×3 convolution downsampling on the fused feature map P3 with a stride of 2 to obtain feature map P4, whose scale relative to the original input image is 1 / 16. The feature map P4 is downsampled by a 3×3 convolution with a stride of 2 to obtain the feature map P5, which has a size ratio of 1 / 32 relative to the original input image.

9. The method for detecting appearance defects of automotive gaskets based on deep learning according to claim 8, characterized in that, In step S5, defect classification and localization include: inputting each output level P3, P4, and P5 of the feature pyramid into the corresponding multi-branch detection head; each detection head has multiple anchor frames with different aspect ratios, including 1:1, 1:3, and 3:

1. The detection head learns the offset of the predicted box relative to the anchor box through the bounding box regression branch, and determines the probability distribution of whether the region belongs to a burr, crack, pinhole, deformation or normal surface through the category classification branch. During the inference phase, a classification confidence threshold is set to retain candidate boxes with a confidence greater than 0.5, and a non-maximum suppression algorithm is applied to remove duplicates. The intersection-union ratio threshold is preset to 0.

45.

10. A deep learning-based automotive gasket appearance defect detection system, used to execute the method according to any one of claims 1 to 9, characterized in that, include: The image acquisition module includes a ring-shaped shadowless light source with a wavelength set in a narrow band of 450 nm to 650 nm, and a charge-coupled device industrial camera with the actual physical size of a single pixel set between 0.01 mm and 0.05 mm, used to acquire the original grayscale image of the automotive sealing gasket to be inspected. The image preprocessing module is used to perform grayscale normalization and region of interest extraction on the original grayscale image to obtain the region of interest image. The grayscale normalization process calculates the mean grayscale value μ and standard deviation σ of all pixels in the region of interest and calculates the normalized pixel value I' according to the preset linear mapping formula, so that the mean grayscale value of the normalized image is stabilized at around 128. The multi-scale feature extraction network is a pre-trained deep residual network used to extract feature maps of different scales from images of regions of interest. The attention enhancement module is a pre-trained module that includes channel attention branches and spatial attention branches, used to enhance features of feature maps at different scales. The feature pyramid construction module is used to fuse feature maps of different scales after they have been processed by the attention enhancement module and output feature maps of multiple levels. The defect prediction module is used to classify and locate defects using feature maps at multiple levels, and output defect information. The industrial control components, including a programmable logic controller, a solenoid valve, and a pneumatic actuator, are used to receive defect information and track the position of the defective gasket to be rejected based on the pulse data of the production line encoder. When the defective gasket to be rejected moves to the rejection station, the solenoid valve and the pneumatic actuator are driven to perform the rejection action.

Images