Vision-based method for surface defect recognition of automotive plastic parts

Abstract

The present invention relates to the field of industrial intelligent detection technology and discloses a vision-based method for identifying surface defects in automotive plastic parts. This method obtains historical surface images of automotive plastic parts and establishes a data set. After preprocessing, labeling, and segmentation, a deep learning model based on a generative adversarial network is built for training to obtain a surface defect recognition model for automotive plastic parts. The model can capture surface images of automotive plastic parts in real time and identify various defect types, such as scratches, bubbles, missing materials, and weld lines, while adapting to changes in lighting conditions and shooting angles. At the same time, a dynamic threshold judgment mechanism and a model interpretation module are used to promptly detect and alert defect problems, thereby improving production efficiency and product quality. This method is highly scalable and flexible, providing strong support for the intelligent transformation of the automotive plastic parts manufacturing industry.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial intelligent detection, and in particular to a method for identifying surface defects of automobile plastic parts based on vision. Background Art

[0002] During the manufacturing process of automotive plastic parts, surface defects seriously impact product quality and customer satisfaction. Traditional manual inspection methods are not only time-consuming and labor-intensive but also susceptible to subjective factors, making it difficult to guarantee detection efficiency and accuracy. With the rapid development of machine vision technology, the use of deep learning models for defect detection has become a research hotspot. However, existing defect detection methods often target specific defect types, have limited generalization capabilities, and struggle to handle complex and changing lighting conditions and defect morphologies. Therefore, there is an urgent need for intelligent detection technologies that can efficiently identify multiple defect types on the surfaces of automotive plastic parts. Summary of the Invention

[0003] In view of the above-mentioned technical deficiencies, the purpose of the present invention is to provide a brake disc surface quality detection method and system to solve the problems of low accuracy and limited generalization ability of the existing automotive plastic parts defect detection methods.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention provides a method for identifying surface defects of automotive plastic parts based on vision, the method comprising:

[0006] Step S100: Acquire historical automobile plastic part surface images, pre-process the images, and establish an automobile plastic part surface dataset;

[0007] Step S200: annotating the automotive plastic parts surface dataset and dividing it into a training set and a validation set;

[0008] Step S300: Building a deep learning model based on a generative adversarial network;

[0009] Step S400: Inputting the training set into a deep learning model based on a generative adversarial network for training, and using a validation set for validation during the training process to obtain a surface defect recognition model for automotive plastic parts;

[0010] Step S500: collecting the surface image of the automobile plastic part in real time, and inputting it into the automobile plastic part surface defect recognition model to obtain the automobile plastic part surface defect recognition result.

[0011] Preferably, in a possible implementation of the first aspect, the historical automobile plastic part surface images are collected by a high-resolution linear array CCD camera, including plastic part surface images taken under different lighting conditions and at multiple angles, and covering sample data within a production cycle of at least 12 months.

[0012] Preferably, in a possible implementation manner of the first aspect, the preprocessing includes:

[0013] Use bilateral filtering algorithm to perform image noise reduction;

[0014] Enhance image contrast through histogram equalization;

[0015] Scale the image to 512×512 pixel resolution.

[0016] Preferably, in a possible implementation manner of the first aspect, LabelImg is used to annotate the preprocessed automotive plastic part surface dataset, and the annotation types include four types of defects: scratches, bubbles, missing materials, and weld lines.

[0017] Preferably, in a possible implementation of the first aspect, the deep learning model based on the generative adversarial network includes:

[0018] The generator network uses a U-Net architecture with an 8-layer downsampling-upsampling structure to gradually extract and restore surface image features of automotive plastic parts. Each layer integrates a residual block and a self-attention mechanism module. Layers 3-5 embed Transformer encoder units to capture long-range dependencies and learn feature associations of cross-region defects. The output uses a spectral normalization convolutional layer to generate a 512×512 high-resolution feature map.

[0019] The discriminator network constructs a multi-scale pyramidal architecture consisting of three parallel convolutional streams, processing original resolution, 1 / 2 downsampled, and 1 / 4 downsampled images, respectively, to capture surface defects of automotive plastic parts at different scales. Each branch uses depthwise separable convolution and adaptive instance normalization techniques to adapt to different image feature distributions. The final step integrates multi-scale image features through a gated fusion module.

[0020] The classification module consists of a parallel attention mechanism layer and a fully connected layer. The attention mechanism layer adopts a dynamic multi-head self-attention mechanism, sets 12 attention heads and integrates a channel attention module to analyze the surface defect characteristics of automotive plastic parts from different angles. The fully connected layer adopts a hybrid expert system architecture, which includes 32 expert networks and a learnable gated controller to determine the defect type.

[0021] Preferably, in a possible implementation of the first aspect, the generator network specifically includes:

[0022] The input layer uses a 4-channel tensor with position encoding to mark the defect position features;

[0023] The downsampling stage contains 4 residual blocks, each of which consists of two 3×3 spectral normalization convolutional layers, with a windowed self-attention module based on the Swin Transformer inserted in the middle;

[0024] The bottleneck layer integrates three cascaded Transformer encoding units, each of which contains 12 attention heads and a 2048-dimensional feedforward network;

[0025] The upsampling stage uses sub-pixel convolution technology and adaptive instance normalization to perform multi-level image feature fusion;

[0026] The output layer uses the tanh activation function and combines the channel attention gating mechanism to generate a synthetic image with enhanced local details.

[0027] Preferably, in a possible implementation of the first aspect, the discriminator network specifically includes:

[0028] The multi-scale input branch uses adaptive resampling technology to dynamically adjust the resolution of the image of the surface of the automotive plastic parts input by each branch;

[0029] Each convolutional stream consists of five stages, each of which uses a combination of depthwise separable convolution and adaptive instance normalization, and uses the LeakyReLU activation function to extract multi-scale features of defects of different sizes and shapes on the surface of automotive plastic parts;

[0030] The feature fusion module uses a topological structure generated by differentiable neural architecture search and integrates a cross-scale feature interaction mechanism to fuse multi-scale features of defects of different sizes and shapes on the surface of automotive plastic parts;

[0031] The discriminant head includes a region focusing mechanism that applies a 3x weight to key regions through a saliency map generated in real time.

[0032] The training process adopts the Wasserstein loss function, integrates gradient penalty and spectral regularization constraints, and improves the training stability and convergence speed of the discriminator network.

[0033] Preferably, in a possible implementation manner of the first aspect, the classification module specifically includes:

[0034] The attention mechanism layer adopts a spatiotemporal joint attention architecture, consisting of 12 dynamically configured attention heads. Four of these heads are dedicated to spatial dimension interactions, six heads process inter-channel relationships, and two heads analyze cross-layer feature correlations to mine the associations between different channel features and defect types.

[0035] The channel attention module integrates a dynamic channel compression mechanism, where the compression rate is automatically adjusted based on the input features to highlight features related to defects.

[0036] The fully connected hybrid expert system consists of 32 heterogeneous expert networks, 16 of which use a 3-layer MLP structure to process local defect features, 8 use a graph convolutional network to classify defect features, and 8 use a time series analysis module to analyze defect change trends.

[0037] The output layer uses dynamic weight fusion technology to integrate the outputs of the attention branch and the fully connected branch through a learnable gating parameter α;

[0038] Label smoothing regularization and contrastive learning auxiliary loss are introduced during training, and the classification decision threshold is dynamically adjusted according to the performance of the validation set.

[0039] Preferably, in a possible implementation of the first aspect, step S400 specifically includes:

[0040] Set the training parameters, including the Adam optimizer with an initial learning rate of 0.002, a batch size of 32, a maximum number of training epochs of 200, and configure an early stopping mechanism to terminate training when the validation set loss does not decrease for 10 consecutive epochs.

[0041] An alternating training strategy is adopted. In the first round, the generator network parameters are frozen and the discriminator network is trained using the Wasserstein loss function. In the second round, the discriminator network parameters are frozen and the generator network is optimized by combining adversarial loss, L1 reconstruction loss, and perceptual loss.

[0042] After every five rounds of training, the validation set evaluation is started. The FID index, defect classification accuracy, and intersection over union ratio of the generated and real images are calculated simultaneously. When the FID is lower than 15 and the IoU is higher than 0.85, the model snapshot is saved.

[0043] After training is completed, the generative adversarial network is compressed into a lightweight recognition model through knowledge distillation technology, retaining more than 97% of the original model accuracy.

[0044] Preferably, in a possible implementation of the first aspect, step S500 specifically includes:

[0045] A high-resolution linear CCD camera deployed on the production line captures 2048×2048 original resolution images at a frame rate of 1 frame per second and performs pre-processing operations in real time.

[0046] The preprocessed image is fed into the automotive plastic surface defect recognition model. The generator network outputs a 512×512 feature map with a defect heat map. The discriminator network generates multi-scale confidence scores. The classification module combines the two outputs to generate structured data containing the defect type, location coordinates, and confidence level.

[0047] Adopting a dynamic threshold judgment mechanism, when the comprehensive confidence exceeds the dynamic decision threshold, an audible and visual alarm is triggered and the defect image and defect coordinate information are recorded;

[0048] The model interpretation module visualizes decision rationale, uses class activation mapping technology to highlight defective areas, and generates interactive inspection reports with timestamps, production line locations, and defect statistics.

[0049] The beneficial effects of the present invention are: establishing a surface dataset of automotive plastic parts and using a generative adversarial network deep learning model for training, thereby achieving high-precision recognition of surface defects of automotive plastic parts.

[0050] This method not only effectively identifies various defect types, such as scratches, bubbles, missing material, and weld lines, but also adapts to varying lighting conditions and camera angles. Furthermore, by acquiring and processing image data in real time, it can promptly detect and alert on defects, improving production efficiency and product quality. Furthermore, this method is highly scalable and flexible, allowing for model optimization and upgrades tailored to actual needs, providing strong support for the intelligent transformation of the automotive plastic parts manufacturing industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 A flow chart of a vision-based method for identifying surface defects in automotive plastic parts is provided for this application. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] Example 1: Figure 1As shown, the present invention provides a method for identifying surface defects of automobile plastic parts based on vision, comprising:

[0055] Step S100: Acquire historical automobile plastic part surface images, pre-process the images, and establish an automobile plastic part surface dataset.

[0056] In this example, a Basler ace acA5472-5gm linear array CCD camera with a 20-megapixel resolution and a 5 fps frame rate was used to capture historical surface images of automotive plastic parts. The linear array CCD camera was deployed on the automotive plastic parts production line, 50 centimeters from the parts, with the camera angled at a 90-degree angle to the direction of plastic part transport. The captured images were transmitted in real time using the TCP / IP transport protocol and stored in a control center. The historical surface images of automotive plastic parts were captured under various lighting conditions and from multiple angles, covering a production cycle of at least 12 months and encompassing automotive plastic parts from different batches and stages of production.

[0057] After completing image acquisition and dataset creation, preprocessing of the dataset is performed. First, the preprocess_image function is defined to implement the specific preprocessing process. This function first uses the cv2.bilateralFilter function to perform bilateral filtering on the input image to reduce noise. By setting parameters such as the filter kernel diameter, the color space filter standard deviation, and the coordinate space filter standard deviation, noise is removed while preserving image edge information. Next, the image is determined to be grayscale or color. If it is grayscale, histogram equalization is performed directly using the cv2.equalizeHist function. If it is color, the image is first converted from BGR color space to YCrCb color space. Histogram equalization is performed on the Y channel to enhance contrast. The processed Y channel is then merged with the unprocessed Cr and Cb channels and converted back to BGR color space. Finally, the cv2.resize function is used to resize the image to a resolution of 512×512 pixels and return the processed image.

[0058] Step S200: annotating the automotive plastic parts surface dataset and dividing it into a training set and a validation set.

[0059] In this example, after obtaining a pre-processed dataset of automotive plastic parts surfaces from the control center, the data is first organized and inspected to ensure standardized and consistent file naming for easy management and search. Each image is also carefully checked for integrity and quality, and images that are damaged, blurred, or have undergone abnormal pre-processing are removed from the dataset to ensure high-quality, usable data for annotation and training.

[0060] Then formulate the marking specifications. For scratches, draw polygons along the edge during marking to accurately outline the shape and range, and clearly mark them as "scratches" in the marking properties; for bubbles, select them completely with a rectangular frame, set the marking property to "bubble", and record relevant information such as size; for missing materials, mark the area with a polygon, set the marking property to "missing material", and mark features such as area; for weld lines, use broken lines or polygons to mark the direction and range, and mark the marking property as "weld line".

[0061] Next, select LabelImg as the annotation tool. Each image is annotated strictly according to the established annotation specifications. Finally, the dataset is partitioned. After reviewing and correcting the annotated data, the annotated dataset is randomly split into a training set and a validation set in an 8:2 ratio using the train_test_split function in the Python sklearn library. After the split is complete, the training and validation sets are stored in separate folders, and the image file names and corresponding annotation information for each dataset are recorded.

[0062] Step S300: Build a deep learning model based on a generative adversarial network.

[0063] In this embodiment, the deep learning model based on the generative adversarial network includes a generator network, a discriminator network, and a classification module.

[0064] The generator network input layer uses a 4-channel tensor with positional encoding to label defect location features. This helps the model identify potential defect locations in the image during subsequent processing, allowing for more targeted feature extraction and processing at these locations. This provides the location information foundation for accurate identification of four types of defects: scratches, bubbles, missing material, and weld lines.

[0065] The overall architecture uses a U-Net structure, comprising an eight-layer downsampling-upsampling structure, to progressively extract and restore surface image features of automotive plastic parts. Downsampling continuously compresses the image size while extracting high-level semantic features. Upsampling restores these high-level features to the original image size, thus fully capturing the surface image features of automotive plastic parts and providing rich feature information for identifying different types of defects.

[0066] The downsampling stage includes four residual blocks, each consisting of two 3×3 spectrally normalized convolutional layers, interposed with a windowed self-attention module based on the Swin Transformer. Spectral normalization enhances the stability of model training, ensuring smooth parameter updates during training and avoiding issues such as vanishing or exploding gradients. The Swin Transformer's windowed self-attention module enables the model to focus on and process image features more closely within a local window, helping to capture the subtle local features of defects such as scratches and bubbles.

[0067] The bottleneck layer integrates three cascaded Transformer encoding units, each containing 12 attention heads and a 2048-dimensional feedforward network. This structure allows for further in-depth processing and fusion of features extracted during the downsampling phase, capturing more complex and abstract feature relationships. This helps distinguish different types of defects, such as missing material and weld lines, which may have relatively complex feature patterns.

[0068] The upsampling stage utilizes sub-pixel convolution technology, combined with adaptive instance normalization (ANU) to fuse multi-level image features. Sub-pixel convolution effectively improves image resolution and gradually restores spatial detail lost during the downsampling process. ANU applies adaptive normalization based on different feature maps, enabling better fusion of features at different levels. This results in a composite image with richer details, facilitating accurate identification of detailed features of various defects.

[0069] The output layer uses a tanh activation function, combined with a channel-wise attention gating mechanism, to generate a composite image with enhanced local details. The tanh activation function maps the output values ​​to the range [-1, 1], helping the model learn more reasonable feature representations. The channel-wise attention gating mechanism further enhances the model's ability to capture defect features by highlighting features related to surface defects on automotive plastic parts. This allows the generated composite image to more clearly depict details of defects such as scratches and bubbles, providing more favorable image data for subsequent defect identification.

[0070] Specifically, we first define a generator model based on a generative adversarial network. This model inherits from torch.nn.Module and is used to generate synthetic images with enhanced local details. Its input is a 4-channel tensor with position encoding and its output is a 3-channel image.

[0071] In the model initialization section, the __init__ method accepts two parameters, input_channels and output_channels, which are 4 and 3 respectively, representing the number of input and output channels. It then defines the downsampling stage, bottleneck layer, upsampling stage, output layer, and channel attention gating mechanism.

[0072] The downsampling stage consists of four modules, stored in the down_blocks file of type nn.ModuleList . Each module contains a residual block and a Swin Transformer. The residual block consists of two 3×3 spectral normalization convolutional layers, interposed with a windowed self-attention module based on the Swin Transformer. Spectral normalization is used to enhance model training stability. The number of input channels of the Swin Transformer is equal to the number of output channels of the residual block, and the number of channels doubles with each module. The bottleneck layer is also stored in nn.ModuleList and consists of three cascaded Transformer encoding units. Its input channel number is equal to the number of output channels of the last module in the downsampling stage. The upsampling stage consists of four modules, stored in up_blocks . Each module first performs sub-pixel convolution using nn.PixelShuffle to increase image resolution. It then undergoes instance normalization, 3×3 convolution, and instance normalization. The number of channels is halved with each module. The output layer consists of a 3x3 convolutional layer followed by a tanh activation function, converting the output of the upsampling stage into a final 3-channel image. The channel-wise attention gating mechanism, consisting of adaptive average pooling, 1x1 convolution, ReLU activation, 1x1 convolution, and sigmoid activation, is used to highlight features related to surface defects on automotive plastic parts.

[0073] The forward method defines the model's forward propagation. First, during downsampling, the input data passes sequentially through the modules in down_blocks . The output of each module is stored in the skip_connections list, and the feature map is downsampled using average pooling. Next, the input data passes through the three Transformer encoding units in the bottleneck layer. During upsampling, the skip_connections list is reversed, and the input data passes sequentially through the modules in up_blocks . Adaptive instance normalization and feature fusion are performed using the custom _adaptive_instance_norm method. The upsampled feature map is concatenated with the previously stored feature map during downsampling. Next, an attention map is generated using a channel-wise attention gating mechanism and multiplied with the feature map to highlight important features. Finally, the final composite image is obtained by passing it through the output layer. The _adaptive_instance_norm method implements adaptive instance normalization. By calculating the mean and standard deviation of the content and style feature maps, the content feature map is normalized and rescaled to have similar statistical properties to the style feature map.

[0074] The discriminator network first constructs a multi-scale pyramidal architecture. The multi-scale input branches utilize adaptive resampling technology to dynamically adjust the resolution of each branch's input image of the automotive plastic part surface. This enables the model to analyze surface images of automotive plastic parts at different resolutions and capture defects of varying sizes, from tiny scratches and bubbles to larger defects such as missing material and weld lines.

[0075] The parallel convolutional stream consists of three parallel convolutional streams, processing original resolution, 1 / 2 downsampled, and 1 / 4 downsampled images, respectively, to capture defects of varying scales on the surfaces of automotive plastic parts. Each convolutional stream comprises five stages, each employing a combination of depthwise separable convolution, adaptive instance normalization, and the LeakyReLU activation function. Depthwise separable convolution effectively extracts multi-scale features of defects of varying sizes and shapes while reducing computational effort. Adaptive instance normalization normalizes the features according to their respective feature maps, enhancing feature distinguishability. The LeakyReLU activation function introduces nonlinearity while avoiding information loss caused by the ReLU function completely zeroing out the negative axis, thereby helping to better extract and retain defect-related feature information.

[0076] The gated fusion module integrates multi-scale image features at the end. This feature fusion module uses a topological structure generated by a differentiable neural architecture search and integrates a cross-scale feature interaction mechanism to fuse the multi-scale features of defects of varying sizes and shapes on the surface of automotive plastic parts. This fusion method effectively integrates the different-scale features extracted by different convolutional streams, enabling the model to comprehensively consider defect information at different scales and more accurately determine the presence of various defects in the image, such as scratches, bubbles, missing material, and weld lines.

[0077] The discriminator head incorporates a region-focusing mechanism that applies a 3x weight to key regions using a real-time saliency map. It also employs the Wasserstein loss function, integrating gradient penalty and spectral regularization constraints to improve the stability and convergence speed of the discriminator network training. This region-focusing mechanism enables the model to focus on key areas of the image where defects may be present, enhancing its ability to detect them. The Wasserstein loss function, combined with gradient penalty and spectral regularization, helps optimize the discriminator training process, enabling it to converge more stably and quickly to a better discrimination result, thereby accurately identifying various defects on the surfaces of automotive plastic parts.

[0078] Specifically, we first define the DifferentiableNASModule class, which inherits from nn.Module. This class is used to search for the topology generated by differentiable neural architectures. During initialization, it receives the number of input channels (in_channels) as a parameter, creates two fully connected layers (fc1 and fc2), and a ReLU activation function layer (relu). fc1 maps the number of input channels to in_channels / / 2, and fc2 maps it back to in_channels. In the forward method, the input tensor x is first reshaped into a two-dimensional tensor. After being processed by fc1, relu, and fc2, the output is reshaped back into a four-dimensional tensor for subsequent operations with other feature maps.

[0079] Next, the core Discriminator class is defined, also inheriting from nn.Module. During initialization, a multi-scale input branch is created, consisting of three parallel convolutional streams: scale_original, scale_half, and scale_quarter, for processing images at original resolution, downsampled by 1 / 2, and downsampled by 1 / 4. These streams are created using the _create_conv_stream method. Each stream consists of five stages, each consisting of a depthwise separable convolution, a 1x1 convolution, an instance normalization layer, and a LeakyReLU activation function. This effectively extracts features from images at different scales. Furthermore, a gated fusion module, gate_fusion, is created, using the previously defined DifferentiableNASModule. The number of input channels is the sum of the final output channels of the three streams. Finally, the discriminator head, discriminator_head, is created. It is a 3x3 convolutional layer that maps feature maps to a single-channel output, resulting in the final discriminant result. The _create_conv_stream method creates each stream, accepting the number of input channels (in_channels) as a parameter and looping through five steps to build the layer structure for each stage. In each loop, depthwise separable convolution is first used to double the number of channels, followed by 1x1 convolution for channel adjustment, followed by instance normalization and LeakyReLU activation, while also updating the number of input channels. Finally, all layers are combined into a nn.Sequential object and returned.

[0080] In the forward method, the forward propagation process of the discriminator is implemented. First, the input image x is adaptively resampled to obtain the original resolution x_original, 1 / 2 downsampled x_half, and 1 / 4 downsampled x_quarter. These are then input into the corresponding convolutional streams to obtain feature maps of different scales, output_original, output_half, and output_quarter. For subsequent feature fusion, the downsampled feature maps are restored to the same size as the original resolution feature maps through bilinear interpolation. Next, these three feature maps are concatenated in the channel dimension to obtain concatenated_features. The concatenated feature maps are then input into the gated fusion module gate_fusion to obtain gated features, which are then expanded to the same shape as the concatenated feature maps. The two are then element-wise multiplied to obtain the fused feature maps fused_features. For cross-scale feature interactions, the fused feature maps are reshaped and summed over the specified dimensions to obtain final_features. Finally, the final feature map is input into the discriminator head discriminator_head to obtain the discrimination result and return it.

[0081] The attention mechanism layer in the classification module utilizes a spatiotemporal joint attention architecture, comprising 12 dynamically configured attention heads. Four of these heads are dedicated to spatial dimension interactions, six handle inter-channel relationships, and two analyze cross-layer feature associations, mining the associations between different channel features and defect types. This configuration enables analysis and attention to image features from different dimensions. The spatial dimension interaction head helps capture the spatial characteristics of defects, the inter-channel relationship head analyzes the connections between features from different channels, and the cross-layer feature association head integrates feature information from different levels, thereby more comprehensively mining the characteristic patterns associated with four types of defects: scratches, bubbles, missing material, and weld lines, providing a basis for accurate classification.

[0082] The channel attention module integrates a dynamic channel compression mechanism, automatically adjusting the compression ratio based on input features to highlight defect-related features. This mechanism automatically adjusts the channel compression ratio based on the characteristics of the input features, removing some channel information that contributes less to defect classification while highlighting key channel features related to defects. This further improves the relevance and effectiveness of features, enabling the model to more accurately judge based on features closely related to defects during classification.

[0083] Fully connected layer hybrid expert system: contains 32 heterogeneous expert networks, of which 16 experts use a 3-layer MLP structure to process local features of defects. They can conduct in-depth analysis and processing of the characteristics of defects in local areas, capture subtle local feature differences, and help distinguish the differences in local manifestations of different types of defects; 8 experts use graph convolutional networks to classify defect features, leveraging the advantages of graph convolutional networks in processing graph-structured data to explore the potential relationships between defect features and improve classification accuracy; 8 experts use time series analysis modules to analyze defect change trends. For some defects that may change over time or during the production process, such as material shortages, there may be certain change patterns during continuous production. By analyzing this change trend, the defect type can be judged more accurately.

[0084] The output layer uses dynamic weight fusion technology, integrating the outputs of the attention branch and the fully connected branch through a learnable gating parameter α. Label smoothing regularization and contrastive learning auxiliary loss are introduced during training, and the classification decision threshold is dynamically adjusted based on the performance of the validation set. Dynamic weight fusion technology can automatically adjust the weight contribution of the attention branch and the fully connected branch in the final classification result based on different input features, fully leveraging the advantages of both branches. Label smoothing regularization can prevent overfitting of the model during training and improve the model's generalization ability. Contrastive learning auxiliary loss further enhances the model's ability to distinguish different defect types by comparing the feature differences between different samples. The classification decision threshold is dynamically adjusted based on the performance of the validation set, enabling the model to find an optimal classification decision boundary in different data sets and task scenarios, thereby improving classification accuracy.

[0085] Specifically, the TemporalSpatialAttention class implements a joint spatiotemporal attention architecture. During initialization, three different types of attention heads are defined: four for spatial dimension interactions, six for processing inter-channel relationships, and two for analyzing cross-layer feature correlations. 1x1 convolutional layers for query, key, and value are then created to convert the input feature map into corresponding tensors. Finally, a 1x1 output projection convolutional layer is added to convert the attention result back to the same number of channels as the input. In the forward method, the batch size, number of channels, height, and width of the input feature map are obtained. The input feature map is then passed through the query, key, and value convolutional layers, reshaped, and transposed. Attention scores are then calculated, scaled by dividing them by the square root of the number of channels, and then passed through a softmax function to obtain attention probabilities. Finally, the attention probabilities are multiplied by the value tensor, transposed and reshaped, and then passed through an output projection convolutional layer to obtain the final attention output.

[0086] The DynamicChannelCompression class implements a dynamic channel compression mechanism. In the forward method, the mean of the input feature map across all dimensions is calculated to obtain a scalar value. The torch.clamp function is used to clamp this mean to a value between 0.1 and 0.9, which serves as the compression ratio. The new number of channels is calculated based on the compression ratio. If the new number of channels is greater than 0, the input feature map is clipped to the new number of channels, and the clipped feature map is returned.

[0087] The MixedExpertSystem class implements a hybrid expert system. During initialization, three different types of expert networks are created. Sixteen experts employ a three-layer MLP structure to process local defect features; eight experts employ graph convolutional networks (GCNs) with fully connected layers to classify defect features; and eight experts employ LSTMs for time series analysis to analyze defect trends. In the forward method, the input feature map is flattened and then fed into each type of expert network to generate the output of each expert. For the LSTM expert, the input is reshaped, fed into the LSTM, and the output of the last time step is taken. Finally, the outputs of all experts are stacked together to form the final output of the hybrid expert system.

[0088] During initialization, the ClassificationModule class includes the previously defined attention mechanism layer (TemporalSpatialAttention), channel compression module (DynamicChannelCompression), and mixed expert system (MixedExpertSystem). A learnable gating parameter alpha is also defined for dynamic weight fusion. Two fully connected layers are created: one for processing the output of the attention branch and the other for final classification. In the forward method, the input feature map first passes through the attention mechanism layer to obtain the attention output. It then passes through the channel compression module for channel pruning. The feature map is then converted to a one-dimensional vector using global average pooling. Finally, it passes through a fully connected layer to obtain the output of the attention branch. Simultaneously, the input feature map is also input to the mixed expert system to obtain the output of the fully connected branch. The outputs of the two branches are then dynamically weighted fused using the learnable gating parameter alpha. Finally, the fused output passes through the final fully connected layer to obtain the classification result, which is then returned.

[0089] Step S400: Input the training set into a deep learning model based on a generative adversarial network for training. The training process is verified using a validation set to obtain a surface defect recognition model for automotive plastic parts.

[0090] In this example, a training set was fed into a deep learning model based on a generative adversarial network for training. The training process was validated using a validation set, resulting in a model for identifying surface defects in automotive plastic parts. The Adam optimizer was used for training parameters, with an initial learning rate of 0.002, a batch size of 32, and a maximum number of training epochs of 200. To prevent overfitting, an early stopping mechanism was implemented; training was terminated if the validation set loss did not decrease for 10 consecutive epochs.

[0091] The training process uses an alternating training strategy. In the first round of training, the parameters of the generator network are frozen so that they are not updated during this training round. The discriminator network is then trained using the Wasserstein loss function. The Wasserstein loss function effectively measures the difference between generated data and real data and can avoid the vanishing gradient problem.

[0092] During the second round of training, the parameters of the discriminator network are frozen, and the generator network is optimized. This optimization process is performed by combining adversarial loss, L1 reconstruction loss, and perceptual loss. The adversarial loss measures the difference between the generator's generated images and the real images as seen by the discriminator, encouraging the generator to produce more realistic images. The L1 reconstruction loss focuses on the pixel-level differences between the generated and real images, helping to improve image detail and quality. The perceptual loss measures the similarity between the generated and real images at the feature level, making the generated images more similar in semantic features to the real images.

[0093] During training, validation set evaluation is initiated after every five training rounds. Three key metrics are simultaneously calculated: the Frechet Inception Distance (FID) between the generated and real images, defect classification accuracy, and intersection over union (IoU). The FID metric measures the difference in feature distribution between the generated and real images; a lower value indicates a closer match. The defect classification accuracy reflects the model's accuracy in classifying surface defect types on automotive plastic parts. The IoU measures the degree of overlap between the model's predicted defect areas and the actual defect areas. When the FID is below 15 and the IoU is above 0.85, a model snapshot is saved, preserving the current model parameters for later use or further optimization.

[0094] After training, to improve the model's efficiency and ease of deployment in practical applications, the generative adversarial network is compressed into a lightweight recognition model using knowledge distillation. This process strives to retain at least 97% of the original model's accuracy, resulting in a lightweight recognition model with a smaller model size and faster inference speed while maintaining high recognition accuracy.

[0095] Step S500: collecting the surface image of the automobile plastic part in real time, and inputting it into the automobile plastic part surface defect recognition model to obtain the automobile plastic part surface defect recognition result.

[0096] In this embodiment, the surface image of automobile plastic parts is collected in real time and input into the automobile plastic parts surface defect recognition model to obtain the automobile plastic parts surface defect recognition results. First, image acquisition is performed by a high-resolution linear array CCD camera deployed on the production line. The camera acquires images with an original resolution of 2048×2048 at a rate of 1 frame per second. Such an acquisition rate and resolution can ensure that clear and complete image information of the surface of automobile plastic parts is obtained. The collected images are preprocessed in real time. The preprocessing process is consistent with the preprocessing process when the data set was previously established, that is, bilateral filtering and noise reduction are performed using the cv2.bilateralFilter function, and then corresponding histogram equalization processing is performed according to whether the image is grayscale or color. Finally, the image is scaled to a resolution of 512×512 pixels using the cv2.resize function.

[0097] The preprocessed image is input into the automotive plastic surface defect recognition model. The generator network in the model processes the input image and outputs a 512×512 feature map with a defect heat map. The defect heat map visually indicates areas in the image where defects may be present, with darker colors indicating a greater likelihood of a defect. The discriminator network performs a multi-scale analysis of the input image and generates multi-scale confidence scores, which reflect the likelihood of defects at different scales. The classification module combines the outputs of the generator and discriminator networks to generate structured data containing the defect type, location coordinates, and confidence level. Specifically, the classification module analyzes the generated feature map and confidence scores based on feature patterns learned during training. It determines the specific defect type (such as scratches, bubbles, missing material, or weld lines) in the image, calculates the defect's location coordinates in the image, and provides a confidence level for this determination.

[0098] To accurately determine whether an image truly contains a defect, a dynamic threshold decision mechanism is employed. This threshold is dynamically adjusted based on the model's performance in real-world applications and data distribution. When the overall confidence level exceeds the threshold, the image is deemed to contain a defect, triggering an audible and visual alarm to draw the attention of personnel. The system also records the defect image and coordinates for subsequent analysis and processing.

[0099] The decision-making process is visualized through the model interpretation module. Class Activation Mapping (CAM) technology is used to highlight areas in the image that are critical to the model's decision-making, namely defect areas. In addition, the system generates an interactive inspection report that includes a timestamp, production line location, and defect statistical analysis. The timestamp records the specific time of the inspection, the production line location information can help locate the specific location of the problem, and the defect statistical analysis collects and analyzes information such as the type and quantity of defects detected over a period of time, providing strong support for production process optimization and quality control. Staff can view, filter, and analyze inspection reports through an interactive interface so that appropriate measures can be taken in a timely manner.

[0100] Example 2: The present invention provides a method for identifying surface defects of automotive plastic parts based on vision, which also includes constructing a self-evolutionary model update mechanism to achieve continuous learning and dynamic optimization.

[0101] In this embodiment, cameras deployed on the automotive plastic parts production line capture real-time images of the surface of automotive plastic parts in actual production, and construct an incremental dataset update channel. The acquisition terminal has a built-in edge computing unit and uses a multi-level cache queue to manage newly acquired images. First, online quality screening is performed, and the image clarity and lighting uniformity are scored using a pre-trained lightweight ResNet-18 network. Blurred, overexposed, or underexposed images are eliminated, and only qualified images with a quality score higher than 0.85 are retained and entered into the queue for processing. After the qualified images undergo the same preprocessing process as in Example 1, the automatic labeling process is triggered, using a semi-supervised labeling strategy: for defective areas with a model prediction confidence higher than 0.95, the model prediction results are directly used as pseudo-labels; for prediction results with a confidence of 0.7-0.95, an improved active learning algorithm is used to screen uncertainty samples and push them to the manual labeling interface for review and labeling; samples with a confidence lower than 0.7 trigger the expert review mechanism for manual secondary confirmation. The labeled incremental data is used to dynamically update the training set using a sliding window mechanism, retaining valid data from the last 30 days while randomly retaining 20% ​​of historical representative samples to prevent performance degradation of the model due to data distribution shift.

[0102] During the model update phase, a dual-channel validation strategy is implemented, using a physically isolated A / B testing environment. When the incremental dataset reaches a preset threshold (typically 15% of the original training set size), the model fine-tuning process begins. First, the encoder portion of the generator network is frozen, and only the parameters of the decoder and classification modules are updated. A progressive learning rate strategy is employed, with an initial learning rate of 0.0001 and a cosine annealing algorithm applied after each round of training. During fine-tuning, an elastic weight consolidation (EWC) regularization term is introduced to calculate the Fisher information matrix of key parameters on the old task, constraining the direction of parameter updates to preserve the original knowledge. After every five rounds of fine-tuning, cross-validation is initiated, testing the current model (Model B) in parallel with the production model (Model A) on an independent validation set consisting of 2,000 balanced samples covering various defect types. Evaluation metrics include FID, mAP, and false positive rate. When model B achieves a relative improvement in mAP of more than 1.2%, a decrease in FID of more than 0.8%, and a false positive rate increase of no more than 0.5% over three consecutive validation runs, the model hot-swap protocol is triggered. Model B gradually replaces model A through weight smoothing migration technology. The migration process uses a sliding average algorithm to gradually update model parameters within 10 batches to ensure uninterrupted online recognition services. If model B's performance does not meet the upgrade criteria, a rollback mechanism is initiated, retaining model A and analyzing performance bottlenecks. Samples corresponding to the failure mode are added to the enhanced training set, and the update process is retriggered after the next round of incremental data accumulation is completed.

[0103] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for identifying surface defects of automotive plastic parts based on vision, characterized in that: The method comprises: Step S100: Acquire historical automobile plastic part surface images, pre-process the images, and establish an automobile plastic part surface dataset; Step S200: annotating the automotive plastic parts surface dataset and dividing it into a training set and a validation set; Step S300: Building a deep learning model based on a generative adversarial network; The deep learning model based on the generative adversarial network includes: The generator network uses a U-Net architecture with an 8-layer downsampling-upsampling structure to gradually extract and restore surface image features of automotive plastic parts. Each layer integrates a residual block and a self-attention mechanism module. Layers 3-5 embed Transformer encoder units to capture long-range dependencies and learn feature associations of cross-region defects. The output uses a spectral normalization convolutional layer to generate a 512×512 high-resolution feature map. The generator network specifically includes: The input layer uses a 4-channel tensor with position encoding to mark the defect position features; The downsampling stage contains 4 residual blocks, each of which consists of two 3×3 spectral normalization convolutional layers, with a windowed self-attention module based on the Swin Transformer inserted in the middle; The bottleneck layer integrates three cascaded Transformer encoding units, each of which contains 12 attention heads and a 2048-dimensional feedforward network; The upsampling stage uses sub-pixel convolution technology and adaptive instance normalization to perform multi-level image feature fusion; The output layer uses the tanh activation function and combines the channel attention gating mechanism to generate a synthetic image with enhanced local details; The discriminator network constructs a multi-scale pyramidal architecture consisting of three parallel convolutional streams, processing original resolution, 1 / 2 downsampled, and 1 / 4 downsampled images, respectively, to capture surface defects of automotive plastic parts at different scales. Each branch uses depthwise separable convolution and adaptive instance normalization techniques to adapt to different image feature distributions. The final step integrates multi-scale image features through a gated fusion module. The discriminator network specifically includes: The multi-scale input branch uses adaptive resampling technology to dynamically adjust the resolution of the image of the surface of the automotive plastic parts input by each branch; Each convolutional stream consists of five stages, each of which uses a combination of depthwise separable convolution and adaptive instance normalization, and uses the LeakyReLU activation function to extract multi-scale features of defects of different sizes and shapes on the surface of automotive plastic parts; The feature fusion module uses a topological structure generated by differentiable neural architecture search and integrates a cross-scale feature interaction mechanism to fuse multi-scale features of defects of different sizes and shapes on the surface of automotive plastic parts; The discriminant head includes a region focusing mechanism that applies a 3x weight to key regions through a saliency map generated in real time. The training process uses the Wasserstein loss function, which integrates gradient penalty and spectral regularization constraints to improve the training stability and convergence speed of the discriminator network; The classification module consists of a parallel attention mechanism layer and a fully connected layer. The attention mechanism layer uses a dynamic multi-head self-attention mechanism with 12 attention heads and an integrated channel attention module to analyze the surface defect characteristics of automotive plastic parts from different angles. The fully connected layer uses a hybrid expert system architecture, including 32 expert networks and a learnable gated controller, to determine the defect type. Step S400: Inputting the training set into a deep learning model based on a generative adversarial network for training, and using a validation set for verification during the training process to obtain a surface defect recognition model for automotive plastic parts; Step S500: collecting surface images of automobile plastic parts in real time, and inputting them into an automobile plastic part surface defect recognition model to obtain automobile plastic part surface defect recognition results.

2. The method for identifying surface defects of automotive plastic parts according to claim 1, wherein: The historical automobile plastic parts surface images are collected by a high-resolution linear array CCD camera, including plastic parts surface images taken under different lighting conditions and at multiple angles, and covering sample data within a production cycle of at least 12 months.

3. The method for identifying surface defects of automotive plastic parts according to claim 2, wherein: The pretreatment includes: Use bilateral filtering algorithm to perform image noise reduction; Enhance image contrast through histogram equalization; Scale the image to 512×512 pixel resolution.

4. The method for identifying surface defects of automotive plastic parts according to claim 1, wherein: LabelImg is used to annotate the preprocessed automotive plastic parts surface dataset. The annotation types include four types of defects: scratches, bubbles, missing materials, and weld lines.

5. The method for identifying surface defects of automotive plastic parts according to claim 1, wherein: The classification module specifically includes: The attention mechanism layer adopts a spatiotemporal joint attention architecture, consisting of 12 dynamically configured attention heads. Four of these heads are dedicated to spatial dimension interactions, six heads process inter-channel relationships, and two heads analyze cross-layer feature correlations to mine the associations between different channel features and defect types. The channel attention module integrates a dynamic channel compression mechanism, where the compression rate is automatically adjusted based on the input features to highlight features related to defects. The fully connected hybrid expert system consists of 32 heterogeneous expert networks, 16 of which use a 3-layer MLP structure to process local defect features, 8 use a graph convolutional network to classify defect features, and 8 use a time series analysis module to analyze defect change trends. The output layer uses dynamic weight fusion technology to integrate the outputs of the attention branch and the fully connected branch through a learnable gating parameter α; Label smoothing regularization and contrastive learning auxiliary loss are introduced during training, and the classification decision threshold is dynamically adjusted according to the performance of the validation set.

6. The method for identifying surface defects of automotive plastic parts according to claim 1, wherein: The step S400 specifically includes: Set the training parameters, including the Adam optimizer with an initial learning rate of 0.002, a batch size of 32, a maximum number of training epochs of 200, and configure an early stopping mechanism to terminate training when the validation set loss does not decrease for 10 consecutive epochs. An alternating training strategy is adopted. In the first round, the generator network parameters are frozen and the discriminator network is trained using the Wasserstein loss function. In the second round, the discriminator network parameters are frozen and the generator network is optimized by combining adversarial loss, L1 reconstruction loss, and perceptual loss. After every five rounds of training, the validation set evaluation is started. The FID index, defect classification accuracy, and intersection over union ratio of the generated and real images are calculated simultaneously. When the FID is lower than 15 and the IoU is higher than 0.85, the model snapshot is saved. After training is completed, the generative adversarial network is compressed into a lightweight recognition model through knowledge distillation technology, retaining more than 97% of the original model accuracy.

7. The method for identifying surface defects of automobile plastic parts according to claim 5, wherein: The step S500 specifically includes: A high-resolution linear CCD camera deployed on the production line captures 2048×2048 original resolution images at a frame rate of 1 frame per second and performs pre-processing operations in real time. The preprocessed image is fed into the automotive plastic surface defect recognition model. The generator network outputs a 512×512 feature map with a defect heat map. The discriminator network generates multi-scale confidence scores. The classification module combines the two outputs to generate structured data containing the defect type, location coordinates, and confidence level. Adopting a dynamic threshold judgment mechanism, when the comprehensive confidence exceeds the dynamic decision threshold, an audible and visual alarm is triggered and the defect image and defect coordinate information are recorded; The model interpretation module visualizes decision rationale, uses class activation mapping technology to highlight defective areas, and generates interactive inspection reports with timestamps, production line locations, and defect statistics.

Images