Synthetic leather surface defect detection method based on spatial instance-specific self-attention

Abstract

The present application relates to the technical field of surface defect detection, and particularly relates to a synthetic leather surface defect detection method based on spatial instance-specific self-attention, which comprises constructing a spatial instance-specific self-attention bottleneck structure, which comprises a main branch and an auxiliary branch; wherein the output of the main branch and the output of the auxiliary branch are fused together with the input of the spatial instance-specific self-attention bottleneck structure, and the fusion result is taken as the output of the spatial instance-specific self-attention bottleneck structure; the spatial instance-specific self-attention bottleneck structure is used to replace the bottleneck structure in the backbone network of the YOLOv8 algorithm to obtain a surface defect detection model; a surface defect sample dataset is obtained to train the surface defect detection model; and the detection data to be detected is input into the trained surface defect detection model to obtain a detection result; the spatial instance-specific self-attention bottleneck structure introduced in the present application is helpful to reduce the false detection and missed detection problems caused by insufficient mining of single sample specificity, and improve the accuracy of target detection.

Description

Technical Field

[0001] This invention relates to the field of surface defect detection technology, and specifically to a method for detecting surface defects in synthetic leather based on spatial instance-specific self-attention. Background Technology

[0002] Surface defect detection in synthetic leather is a crucial application area in synthetic leather quality control. With increasing automation in manufacturing, the demands for surface defect detection on production lines are becoming increasingly stringent. Traditional surface defect detection methods rely on manual inspection or rule-based machine vision algorithms, resulting in low efficiency and a tendency to miss or false positives when facing complex environments or varying surface textures. In recent years, the successful application of deep learning in computer vision has led to the mainstream adoption of surface defect detection methods based on convolutional neural networks (CNNs). CNNs can automatically learn multi-level feature representations of images, thus maintaining high detection accuracy even in complex environments. Among these, the YOLO algorithm series, due to its fast and efficient detection capabilities, is particularly suitable for real-time monitoring and large-scale production line surface defect detection scenarios.

[0003] The YOLO (You Only Look Once) algorithm family is a crucial technology in object detection, its core concept being the ability to detect objects with a single forward pass across an image. Compared to traditional two-stage detectors (such as Faster R-CNN), YOLO treats object detection as a single regression problem, directly predicting the object's class, location, and confidence score from the input image. With the evolution of the YOLO series (YOLOv1 to YOLOv7), its models have been continuously optimized, resulting in significant improvements in both detection speed and accuracy. While the deep learning-based YOLO algorithm family can solve diverse scene problems that traditional vision and machine learning algorithms cannot, the YOLO algorithm family itself also has several problems and limitations:

[0004] 1) Neglecting the specificity of single-sample features: In typical object detection algorithms, image feature extraction mainly relies on the bottleneck structure in the backbone network, which consists of multiple convolutional layers, batch normalization (BN) layers, and activation functions. The BN layers calculate the mean and variance of batch images and adjust the distribution of batch features using trainable parameters. This design tends to focus on the generalized features of the entire batch of images, ignoring the unique characteristics of each sample. This deficiency may cause the algorithm to fail to fully capture single-sample features, thus affecting the overall detection accuracy.

[0005] 2) Lack of a global weighting mechanism for single-sample features: The bottleneck structure of existing object detection algorithms is mainly implemented through convolution operations, lacking a global feature attention module. This design limits the network's ability to perceive global information in the image, making it difficult to effectively extract global weight features of single samples, thus adversely affecting detection performance and reducing detection accuracy.

[0006] 3) Failure to effectively integrate batch generalization features and single-sample features: Current target detection algorithms typically employ a single convolutional method as their bottleneck structure, resulting in limited feature extraction capabilities and a failure to combine multiple feature fusion strategies. This design is insufficient to achieve effective synergy between batch generalization features and single-sample features, further limiting the network's performance in detection tasks and leading to less than ideal detection accuracy. Summary of the Invention

[0007] To address the above problems, this invention provides a method for detecting surface defects in synthetic leather based on spatial instance-specific self-attention, the specific scheme of which includes the following steps:

[0008] S1. Construct a spatial instance-specific self-attention bottleneck structure, which includes a main branch and an auxiliary branch; wherein, the outputs of the main branch and the auxiliary branch are fused together with the input of the spatial instance-specific self-attention bottleneck structure, and the fusion result is used as the output of the spatial instance-specific self-attention bottleneck structure.

[0009] S2. A surface defect detection model is obtained by replacing the bottleneck structure in the backbone network of the YOLOv8 algorithm with a spatial instance-specific self-attention bottleneck structure.

[0010] S3. Obtain a surface defect sample dataset to train a surface defect detection model;

[0011] S4. Input the data to be detected into the trained surface defect detection model to obtain the detection results.

[0012] Furthermore, the main branch includes two cascaded CBS modules, each of which includes a 3×3 convolutional layer, a BN layer, and a SiLU activation function layer; the number of input channels and the number of output channels of each CBS module are the same.

[0013] Furthermore, the auxiliary branch includes a first CIS module, an open layer, a self-attention mechanism module, a fold layer, and a second CIS module, which are cascaded in sequence.

[0014] Furthermore, the first CIS module includes a 1×1 convolutional layer, an IN layer, and a SiLU activation function layer; the second CIS module includes a 3×3 convolutional layer, an IN layer, and a SiLU activation function layer.

[0015] Furthermore, the first CIS module and the second CIS module have the same stride, denoted as s; the number of input channels of the spatial instance-specific self-attention bottleneck structure is denoted as c, the number of input channels of the first CIS module is c, and the number of output channels is (c / / s)×3; the number of input channels of the second CIS module is c / / s, and the number of output channels is c; where / / represents integer division, for example, 5 / / 2=2.

[0016] Furthermore, the computation process of the self-attention mechanism module is represented as follows:

[0017]

[0018] Q = XW Q

[0019] K = XW K

[0020] V = XW V

[0021] Where X represents the input of the self-attention mechanism module, and Output represents the output of the self-attention mechanism module; W Q W K W V represents the weight matrix; Q represents the query vector matrix, K represents the key vector matrix, V represents the value vector matrix, Softmax() represents the softmax function, and d represents the dimension of the key vector.

[0022] The beneficial effects of this invention are:

[0023] This invention introduces a spatial instance-specific self-attention bottleneck structure, which helps reduce false positives and false negatives caused by insufficient exploitation of single-sample specificity, thereby improving the accuracy of target detection. By modeling the specificity of single samples, this invention better adapts to differences between samples, improving the overall network performance.

[0024] The spatial instance-specific self-attention bottleneck structure introduces a self-attention mechanism, enabling the network to redistribute the weights of feature maps globally, thus improving its ability to perceive important regions in images. Compared to traditional bottleneck structures that rely on convolutional operations for feature extraction, the self-attention mechanism of this invention enhances the network's ability to capture global information, especially by weighting the relationships between space and channels, thereby improving the accuracy and robustness of object detection.

[0025] Spatial instance specialization employs a three-branch fusion approach within the attention bottleneck structure, enabling the network's endpoints to effectively fuse batch generalization features and single-sample specific features. Through multi-level feature fusion, the network can simultaneously utilize features from different sources, fully leveraging the advantages of both to enhance its ability to detect complex targets and maintain high detection accuracy across various detection scenarios. This optimization improves the network's generalization ability, allowing it to perform exceptionally well in diverse environments.

[0026] By employing an improved bottleneck structure, the network can extract and fuse features more efficiently, thereby enhancing the model's expressive power and nonlinear characteristics. This allows the model to better adapt to diverse input data in complex object detection tasks, improving detection performance.

[0027] Despite introducing a self-attention mechanism and a more complex feature fusion strategy, this invention significantly improves performance while maintaining network computational efficiency. By reducing unnecessary floating-point operations and the number of parameters, it increases the model's running speed while ensuring low computational resource consumption, making it suitable for deployment in practical applications. Attached Figure Description

[0028] Figure 1 This is a flowchart of the method of the present invention;

[0029] Figure 2 This is a comparison diagram of the various bottleneck structures of the present invention;

[0030] Figure 3 This is a detailed schematic diagram of the spatial instance-specific self-attention bottleneck structure of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] In the backbone network of the YOLO algorithm series, the most critical component is the bottleneck structure, primarily used for the progressive extraction of image features and the fusion of features from different levels. By reducing network depth, the bottleneck structure avoids the vanishing gradient problem, thus helping the model maintain effective gradient propagation and improving the model's fitting ability and performance. Specifically, the bottleneck structure used in YOLOv4 and its subsequent versions (such as YOLOv5 and YOLOX) is as follows: Figure 2As shown in (a), the CSPNet structure is composed of a combination of 1×1 and 3×3 convolutions. This structure effectively alleviates the gradient vanishing problem, accelerates network convergence, and enhances feature learning ability. However, the 1×1 convolution introduced to reduce network parameters leads to feature loss and a small receptive field.

[0033] To address this issue, the bottleneck architecture used in YOLOv7 is an improved version of the ELAN network. Figure 2 As shown in (b), the bottleneck architecture used in YOLOv8 is a further development based on the ELAN network. Figure 2 As shown in (c), the bottleneck structure improved based on the ELAN network replaces the original 1×1 convolution plus 3×3 convolution combination with two layers of 3×3 convolutions. This effectively improves the information flow of the network, reduces floating-point operations and the number of parameters, and enhances the model's expressive power and nonlinear characteristics, thereby improving the model's generalization ability and optimizing network performance. Although the bottleneck structure in the ELAN structure has achieved very good results, the following problems still exist: 1) The specificity of single-sample features is ignored during feature extraction; 2) There is a lack of a global weighting mechanism for single-sample features; 3) Batch generalization features and single-sample features are not effectively integrated.

[0034] For the reasons mentioned above, this invention proposes an improved bottleneck structure, named the "Spatial Instance-Specific Self-Attention Bottleneck Module"; based on this structure, a method for detecting surface defects in synthetic leather based on spatial instance-specific self-attention is provided, such as... Figure 1 As shown, it includes the following steps:

[0035] S1. Construct a spatial instance-specific self-attention bottleneck structure, which includes a main branch and an auxiliary branch; wherein, the outputs of the main branch and the auxiliary branch are fused together with the input of the spatial instance-specific self-attention bottleneck structure, and the fusion result is used as the output of the spatial instance-specific self-attention bottleneck structure.

[0036] S2. Replace the bottleneck structure in the backbone network of the YOLOv8 algorithm with a spatial instance-specific self-attention bottleneck structure, that is, replace the bottleneck layer Bottleneck in the feature pyramid layer C2f to obtain the surface defect detection model.

[0037] S3. Obtain a surface defect sample dataset to train a surface defect detection model;

[0038] S4. Input the data to be detected into the trained surface defect detection model to obtain the detection results.

[0039] Specifically, spatial instance-specific self-attention bottleneck structures, such as Figure 3As shown, an additional auxiliary branch is added to the existing main branch network with residual connections. This auxiliary branch consists of a first CIS module, an unfold layer, a self-attention mechanism module, a fold layer, and a second CIS module, all cascaded sequentially. The main branch comprises two cascaded CBS modules, each consisting of a 3×3 convolutional layer, a BN layer, and a SiLU activation function layer. Each CBS module has the same number of input and output channels. Figure 3 As shown, the number of input and output channels of the CBS module is c, which is consistent with the number of input image channels.

[0040] Specifically, the first CIS module includes a 1×1 convolutional layer, an instance normalization (IN) layer, and a SiLU activation function layer; the second CIS module includes a 3×3 convolutional layer, an IN layer, and a SiLU activation function layer. This invention introduces an instance normalization layer to independently normalize the features of each sample and learns the feature distribution of each sample through trainable parameters. This design effectively enhances the network's ability to learn single-sample specific differences, thereby improving the overall network performance.

[0041] Specifically, the bottleneck structure in existing object detection algorithms typically uses convolution operations to extract features, but it fails to capture global relationships within the image. This invention introduces a self-attention mechanism into the spatial instance-specific self-attention bottleneck structure to address the lack of a global weighting mechanism during feature extraction. The self-attention mechanism divides the input feature map into spatial blocks and calculates global self-attention based on the relationship between channels and block positions, further capturing the interactions between spatial blocks and enhancing the network's ability to perceive global spatial information, thus effectively improving feature extraction capabilities. Assuming the input feature map is... Where C is the number of channels in the feature map, H is the height of the feature map, and W is the width of the feature map. In this invention, the input feature map X1 is first unfolded into 4×4 image blocks using an unfold layer and then converted into channel dimensions to obtain the feature map. Z indicates that the dimension is 4×4, and L is the number of one-dimensional features that are expanded into image patches. The specific calculation process is as follows:

[0042]

[0043] Q = XW Q

[0044] K = XW K

[0045] V = XW V

[0046] Where Output represents the output of the self-attention mechanism module; W Q W K W V Denotes the weight matrix, and M represents the kernel size; Q represents the query vector matrix; K represents the key vector matrix; V represents the value vector matrix; Softmax() represents the softmax function; d represents the dimension of the key vector; and T represents the matrix transpose operation.

[0047] Specifically, a multi-level feature fusion strategy is adopted at the end of the spatial instance-specific self-attention bottleneck structure. This involves effectively fusing the outputs of the main branch and the auxiliary branch with the input of the spatial instance-specific self-attention bottleneck structure. Through this fusion mechanism, the collaborative work of batch generalization features and single-sample features can be achieved, thereby improving the overall detection accuracy of the network.

[0048] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "setting," "connection," "fixing," "rotation," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components or the interaction between two components. Unless otherwise explicitly limited, those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0049] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A synthetic leather surface defect detection method based on spatial instance-specificized self-attention, characterized in that, The method comprises the following steps: S1. Constructing a space instance-specific self-attention bottleneck structure comprising a main branch and an auxiliary branch; wherein the output of the main branch and the output of the auxiliary branch are fused together with the input of the space instance-specific self-attention bottleneck structure, and the fusion result is taken as the output of the space instance-specific self-attention bottleneck structure; The main branch comprises two CBS modules connected in cascade, each CBS module comprising a 3x3 convolution layer, a BN layer and a SiLU activation function layer; the input channel number of each CBS module is the same as the output channel number; The auxiliary branch comprises a first CIS module, an unfold layer, a self-attention mechanism module, a fold layer and a second CIS module connected in cascade; the first CIS module comprises a 1x1 convolution layer, an IN layer and a SiLU activation function layer; the second CIS module comprises a 3x3 convolution layer, an IN layer and a SiLU activation function layer; S2. Replacing the bottleneck structure in the backbone network of the YOLOv8 algorithm with the space instance-specific self-attention bottleneck structure to obtain a surface defect detection model; S3. Training the surface defect detection model by using a surface defect sample data set; S4. Inputting to-be-detected data into the trained surface defect detection model to obtain a detection result.

2. The synthetic leather surface defect detection method based on spatial instance-specificized self-attention according to claim 1, characterized in that, The stride of the first CIS module and the second CIS module is the same, denoted as s; the input channel number of the space instance-specific self-attention bottleneck structure is denoted as c, the input channel number of the first CIS module is c, and the output channel number is (c / / s) x 3; the input channel number of the second CIS module is c / / s, and the output channel number is c; wherein / / represents an integer division operation.

3. The synthetic leather surface defect detection method based on spatial instance-specificized self-attention according to claim 1, characterized in that, The calculation process of the self-attention mechanism module is represented as Q = XW Q K = XW K V = XW V wherein X represents an input of the self-attention mechanism module, Output represents an output of the self-attention mechanism module; W Q , W K , W V represents a weight matrix; Q represents a query vector matrix, K represents a key vector matrix, V represents a value vector matrix, Softmax() represents a softmax function, and d represents a dimension of the key vector.

Images