Spraying quality monitoring method and system based on convolutional attention neural network, and storage medium
By building a convolutional attention neural network, combining wavelet transformation and multi-head attention layer, the existing spray quality detection problems are solved, and high-precision recognition of spray defects is achieved.
Patent Information
- Application Number
- CN202510466416.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-15
- Publication Date
- 2025-08-08
AI Technical Summary
The existing spray quality detection methods rely on manual or simple optical detection, are inefficient, highly subjective, and are difficult to meet the needs of high-precision and large-scale production. The existing convolutional neural network cannot fully identify spray defects.
The spray quality monitoring method based on the convolutional attention neural network is used to identify spray defects by collecting images in real time, preprocessing, wavelet transformation, and constructing a convolutional attention neural network, including input layer, convolutional layer, multi-head attention layer and fully connected layer.
High-precision identification of spray defects is achieved, obvious and hidden defects can be identified, and the accuracy and efficiency of detection are improved.
Smart Images
Figure CN120451053A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to spraying quality monitoring, and in particular to a spraying quality monitoring method, system and storage medium based on a convolutional attention neural network. Background Art
[0002] Spray coating technology is widely used in the manufacturing industry, particularly in the automotive, aerospace, marine, and electronics sectors. Spray coating quality directly impacts the appearance, performance, and durability of products. Traditional spray coating quality inspection relies primarily on manual visual inspection or simple optical inspection equipment. This method is inefficient, highly subjective, and struggles to meet the demands of high-precision and large-scale production. With the advancement of industrial automation, quality monitoring systems based on machine vision have gradually become a research hotspot. However, existing visual inspection systems lack robustness in complex environments, have limited ability to identify minor defects, and lack the ability to intelligently classify spray coating defects.
[0003] In the existing technology, convolutional neural networks are used for image recognition and then for spray defect identification and classification. However, convolutional neural networks can only extract local features of images and cannot fully identify the overall features of images from multiple angles with a global perspective. Their recognition capabilities are limited. Summary of the Invention
[0004] Purpose of the invention: The purpose of the present invention is to provide a method, system and storage medium for spraying quality monitoring based on a convolutional attention neural network, which can extract local features of an image and long-context long-distance relationship features and integrate the two to more comprehensively and accurately identify spraying defects.
[0005] Technical solution: The spraying quality monitoring method based on convolutional attention neural network described in the present invention includes the following steps:
[0006] Real-time acquisition of surface images of sprayed workpieces;
[0007] Preprocessing the collected image includes enhancing the collected image, performing wavelet transform on the enhanced image to remove noise, and normalizing the denoised image;
[0008] Construct and train a convolutional attention neural network, which consists of an input layer that receives image data, a convolutional layer that extracts local image features, a multi-head attention layer that extracts long-range contextual features, a fully connected layer that maps the extracted image features to a high-dimensional space, and an output layer that outputs defect category probabilities.
[0009] The preprocessed image data is input into the trained neural network to obtain the probability distribution of defect categories.
[0010] By performing wavelet transform on the real-time collected images to remove noise and perform image enhancement processing, the image quality can be optimized and the generalization ability of the model can be improved. The enhanced image is normalized to facilitate subsequent neural network processing. In addition to the convolutional layer, the constructed neural network also includes a multi-head attention layer, which can not only extract local features of the image, but also extract long-distance relationship features of the context in the image. It takes into account details without being limited to detail features, ignoring the overall relationship between various detail features, and can more comprehensively extract image features. This enables this method to not only identify obvious defects, but also identify hidden defects through the connection of global features, thereby achieving more accurate recognition.
[0011] Preferably, the wavelet transform decomposition uses Haar wavelet to perform high-frequency and low-frequency decomposition, retaining the decomposed low-frequency sub-bands and discarding the high-frequency sub-bands.
[0012] By using wavelet filtering and retaining the low-frequency sub-band, the key information of the image is retained and the noise information in the image is removed. In addition, this processing also reduces the amount of data and improves the efficiency of subsequent processing.
[0013] Preferably, the multi-head attention layer extracts the long-range relationship features of the image context, comprising the following steps:
[0014] (1) The multi-head attention module extracts features, which specifically includes the following steps:
[0015] (1.1) Convert the output data of the convolutional layer into a sequence and concatenate the classification head to form an extended sequence. Add a learnable positional encoding to the extended sequence to obtain the final input sequence.
[0016] (1.2), calculate the query matrix, key matrix and value matrix, and calculate the scaled dot product attention from them;
[0017] (1.3) Calculate the output feature value of each head based on the scaled dot product attention, and concatenate and linearly transform the output feature values of multiple heads to obtain the fusion feature;
[0018] (1.4) Perform residual connection and normalization on the fusion features and the final input sequence to obtain the output result;
[0019] (2) The output result of the multi-head attention module is input into the feedforward neural network. After nonlinear transformation by the two fully connected layers of the feedforward neural network, the output result of the multi-head attention module is residually connected and normalized to obtain the final output result of the multi-head attention layer.
[0020] In the above method, the residual connection in step (1.4) and step (2) can prevent the gradient from disappearing during the training process; step (2) inputs the output result of the multi-head attention module into the feedforward neural network for nonlinear transformation, which can enhance the discriminability of local features, such as the sharpening of bubble edges and the contrast enhancement of sagging areas. The combination of the two ensures that the multi-head attention layer has both global perception and local refinement capabilities.
[0021] The spraying quality monitoring system based on convolutional attention neural network of the present invention includes:
[0022] Acquisition module: used to collect surface images of sprayed workpieces in real time;
[0023] Preprocessing module: used to perform wavelet transform on the collected image to remove noise, enhance the image after noise removal, and normalize the enhanced image;
[0024] Neural network construction and training module: used to build and train a convolutional attention neural network, including an input layer for receiving image data, a convolutional layer for extracting local image features, a multi-head attention layer for extracting long-range relationship features of the image context, a fully connected layer for mapping the extracted image features to a high-dimensional space, and an output layer for outputting defect category probabilities.
[0025] Recognition module: used to input the preprocessed image data into the trained neural network to obtain the probability distribution of defect categories.
[0026] The computer-readable storage medium storing one or more programs according to the present invention is characterized in that: the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device executes any one of the above methods.
[0027] Beneficial effects: Compared with the prior art, the present invention has the following significant effects: by simultaneously extracting local features and global features of the image through the convolutional layer and multi-head attention layer in the constructed neural network, the method can not only identify obvious defects, but also identify hidden defects through the connection of global features, thereby improving the accuracy of recognition; and a feedforward neural network is set in the multi-head attention layer, so that the multi-head attention layer itself can strengthen the discrimination of local features, further strengthen the overall recognition ability of the neural network for local features, and further improve the accuracy of the final recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 Schematic diagram of the process of this method;
[0029] Figure 2 Schematic diagram of wavelet transform. DETAILED DESCRIPTION
[0030] As shown in the figure, the spraying quality monitoring method based on convolutional attention neural network described in the present invention includes the following process:
[0031] Real-time acquisition of surface images of sprayed workpieces.
[0032] High-resolution industrial cameras capture surface images of sprayed workpieces, ensuring clear images and coverage of all critical areas. The cameras are equipped with a uniform light source to reduce ambient light interference and ensure consistent image quality.
[0033] Preprocessing the collected image includes enhancing the collected image, performing wavelet transform on the enhanced image to remove noise, and normalizing the denoised image;
[0034] The image is enhanced through geometric transformation, histogram equalization, and contrast stretching to highlight defect areas and facilitate subsequent feature extraction and classification. The specific process includes the following:
[0035] First, the image is geometrically enhanced at the spatial level, primarily by random rotation, translation, scaling, and mirror flipping operations to simulate the various pose changes and camera acquisition angle variations experienced during the actual spraying process. Histogram equalization then enhances local image detail, making the image brightness more uniform and enhancing texture and edge clarity. It can enhance details in previously dark or bright areas, making the edges of defects such as bubbles and sags more distinct. For spray images with uneven illumination, equalization can reduce the effects of overexposure or underexposure, improving detection stability. It also sharpens texture structures and edge contours in the image, facilitating the model's identification of defects with blurred edges. Contrast stretching linearly or nonlinearly stretches the original image's grayscale range to a wider range, enhancing the contrast between light and dark. A piecewise linear transformation allows for targeted enhancement of specific grayscale intervals, avoiding the potential noise amplification caused by global equalization. Flow defects typically manifest as localized areas of high brightness, and stretching further enhances their contrast with the surrounding coating. This is particularly effective for localized defects such as missed sprays (low grayscale) or paint accumulation (high grayscale).
[0036] The above data-enhanced image is input into the wavelet transform for high-frequency and low-frequency decomposition. Specifically, the wavelet transform first applies a cutoff frequency of The low-pass filter and cutoff frequency are This process will generate two sub-bands X L and X H Then, the same low-pass filter and high-pass filter are applied again in the column direction of these two sub-bands to obtain four wavelet sub-bands X LL , X LH, X HL , X HH The wavelet sub-function uses Haar; these sub-bands are downsampled versions of the original feature map, but still retain the key information of the input. LL Contains the main structure and contour information of the image and is a rough approximation of the image. LH Emphasizes horizontal edges and horizontal details in the image and is sensitive to sudden changes in the horizontal direction. HL Emphasizes vertical edges and vertical details in the image and is sensitive to sudden changes in the vertical direction of the image. HH Extract the diagonal details in the image, which usually contains the most subtle changes, textures, noise, etc. Since the high-frequency sub-band usually contains the details and noise of the image, the present invention only selects the low-frequency sub-band X LL Perform subsequent processing and discard the remaining three high-frequency sub-bands X LH , X HL , X HH This strategy retains the main structural information of the image and also plays a denoising role to a certain extent. By separating high-frequency noise and low-frequency signals using the above Haar wavelet, the noise in the image is effectively removed while retaining the edge and detail features of the defect.
[0037] The image is then normalized to unify the size and pixel distribution to meet the input requirements of the neural network.
[0038] Construct and train a convolutional attention neural network, which consists of an input layer that receives image data, a convolutional layer that extracts local image features, a multi-head attention layer that extracts long-range contextual features, a fully connected layer that maps the extracted image features to a high-dimensional space, and an output layer that outputs defect category probabilities.
[0039] The constructed neural network model is trained using the spraying defect dataset, using back propagation and Adam optimizer, and the loss function is cross entropy loss.
[0040] The preprocessed image data is input into the trained neural network to obtain the probability distribution of defect categories. The specific process is as follows:
[0041] The preprocessed image data is input into the convolution layer, and the shape of the input tensor is (B, C, H, W) after convolution transformation.
[0042]
[0043] Where B is the number of images, C is the number of channels, H is the height of the image, and W is the width of the image; is the value of the position of the i-th sample in the output tensor at the j-th output channel, is the bias of the j-th output channel, weight is the convolution kernel weight matrix, Input(N i ,k) input tensor, N i represents the i-th sample in the input data, Indicates the jth channel of the current output, k indicates the index of the input channel, which is used to traverse all input channels; C m Indicates the total number of input channels; * indicates 2D convolution operation.
[0044] The calculation process of two-dimensional convolution is to slide the convolution kernel weight in the H×W area (step size = s, padding = p) for each channel k of the convolution layer input data. Each slide calculates the weighted sum of the local receptive field 3×3 area (the area size is only the case in this embodiment and can be set to other area sizes). For all input channels k∈[0,C in-1 ] and the convolution results are summed to obtain a single output channel C out The summation result is added with the bias term bias. The number of output channels is determined by the number of convolution kernels. After convolution, the final output is generated through the activation function. The ReLu activation function is selected in this invention.
[0045] Convolution kernels can extract information such as local edges, textures, and color changes. In spray-painted images, details such as the regularity of sprayed edges and the smoothness of regions can be extracted. Downsampling through the convolutional layer also gradually reduces the image size, reducing computational effort while preserving key information.
[0046] After the convolutional layer extracts local features, the data is input into the multi-head attention layer. The multi-head attention layer includes a multi-head attention module and a feedforward neural network. The output features of the multi-head attention module are input into the feedforward neural network to obtain the final output features of the multi-head attention layer. The specific process includes the following:
[0047] (1) The multi-head attention module extracts features, which specifically includes the following steps:
[0048] (1.1) Convert the output data of the convolutional layer into a sequence and concatenate the classification head to form an extended sequence. Add a learnable positional encoding to the extended sequence to obtain the final input sequence.
[0049] For the convolutional layer output data X0 with shape (B, C, H, W), we first modify it to a feature sequence X1 with shape (B, L, C) for input, where L is the number of pixels, L = H * W. Then we construct a classification head CLS and splice it with the feature sequence to obtain an extended sequence. The final classification result is determined by the classification head.
[0050] Through the above operation, the data output by the convolution layer, that is, the original feature map, is modified into a sequence and enters the multi-head attention module, breaking the positional relationship between image pixels. Therefore, it is necessary to add position encoding PE. PE is a learnable parameter, so the final input sequence is constructed as X=PE+cat(CLS,X1), where cat represents the splicing operation.
[0051] (1.2), calculate the query matrix, key matrix and value matrix, and calculate the scaled dot product attention from them;
[0052] Construct the query matrix (Query), key matrix (Key), and value matrix (Value) from the final input sequence X
[0053] Q=XW Q ,K=XW K ,V=XW V
[0054] Among them, W Q 、W K and W V is the learnable weight matrix.
[0055] The original Query, Key, and Value matrices are divided into multiple subspaces according to the number of heads (num_heads). In the code, (B, L, C) -> (B, L, h, d) where h is the number of heads and d is the dimension of each head. Each head independently calculates the scaled dot product attention (Q h ,K h ,V h )
[0056]
[0057] Among them, Q h , K h and V h are the query score matrix, key score matrix and value score matrix in the h-th head respectively; It's K h That is, the dimension of the key matrix in the h-th head.
[0058] (1.3) Calculate the output feature value of each head based on the scaled dot product attention, and concatenate and linearly transform the output feature values of multiple heads to obtain the fusion feature;
[0059] The calculation formula for the output eigenvalue of each head is:
[0060] head h =Attention(Q h ,K h ,V h )*K
[0061] Among them, K is the key matrix; Attention(Q h ,K h ,V h ) is the scaled dot product attention of the h-th head.
[0062] The formula for splicing and linear transformation of multi-head output eigenvalues is:
[0063] MHA(X)=Concat(head1,…,head h )W o
[0064] Among them, MHA(X) is the fusion feature, W o Linear transformation matrix of fused features.
[0065] (1.4) Perform residual connection and normalization on the fusion features and the final input sequence to obtain the output result;
[0066] Residual connection and layer normalization are performed using the following formula
[0067] Z = LayerNorm(X + MHA(X))
[0068] Among them, Z is the output result of the multi-head attention module.
[0069] (2) The output result of the multi-head attention module is input into the feedforward neural network. After nonlinear transformation by the two fully connected layers of the feedforward neural network, the output result of the multi-head attention module is residually connected and normalized to obtain the final output result of the multi-head attention layer.
[0070] The formula for nonlinear transformation of two fully connected layers is
[0071] FFN(Z)=ReLU(ZW1+b1)W2+b2
[0072] Among them, FFN(Z) is the result of nonlinear transformation, W1 and W2 are weight matrices, and b1 and b2 are bias terms;
[0073] The formula for residual connection and normalization of feedforward neural network is:
[0074] Output = LayerNorm(Z+FFN(Z))
[0075] Among them, Output is the final output result of the multi-head attention layer.
[0076] The multi-head attention module in the multi-head attention layer establishes long-range relationships across the entire image, integrating contextual information based on attention weights to enhance defect features. The multi-head attention module can learn multiple attention patterns (such as shape, color, and texture) in parallel. Combined with the local features of the convolutional layer, it achieves high-precision classification of defects such as bubbles and sags. Finally, the output of the convolutional and multi-head attention layers is input into the fully connected layer for mapping and then into the output layer, outputting the probabilities of various defect categories.
[0077] This paper uses data augmentation techniques, wavelet transforms, convolutional neural networks, and multi-head attention networks. The wavelet transform effectively removes noise from the dataset, particularly by retaining low-frequency information while effectively compressing redundant information and preserving the core structure. The convolutional layer extracts local texture features and key features, while the multi-head attention layer further integrates long-range contextual information, enabling the model to identify not only obvious defects but also subtle ones such as bubbles, sags, and spray leaks.
[0078] The spraying quality monitoring system based on convolutional attention neural network of the present invention comprises:
[0079] Acquisition module: used to collect surface images of sprayed workpieces in real time;
[0080] Preprocessing module: used to perform wavelet transform on the collected image to remove noise, enhance the image after noise removal, and normalize the enhanced image;
[0081] Neural network construction and training module: used to build and train a convolutional attention neural network. The neural network includes an input layer that receives image data, a convolutional layer that extracts local image features, a multi-head attention layer that extracts long-range relationship features of the image context, a fully connected layer that maps all extracted image features to a high-dimensional space, and an output layer that outputs defect category probabilities.
[0082] Recognition module: used to input the preprocessed image data into the trained neural network to obtain the probability distribution of defect categories.
[0083] The computer-readable storage medium storing one or more programs according to the present invention includes one or more programs including instructions, which, when executed by a computing device, enable the computing device to perform any of the above methods.
Claims
1. A spraying quality monitoring method based on convolutional attention neural network, characterized in that: The following processes are included: Real-time acquisition of surface images of sprayed workpieces; Preprocessing the collected image includes enhancing the collected image, performing wavelet transform on the enhanced image to remove noise, and normalizing the denoised image; Construct and train a convolutional attention neural network, which consists of an input layer that receives image data, a convolutional layer that extracts local image features, a multi-head attention layer that extracts long-range contextual features, a fully connected layer that maps the extracted image features to a high-dimensional space, and an output layer that outputs defect category probabilities. The preprocessed image data is input into the trained neural network to obtain the probability distribution of defect categories.
2. The method according to claim 1, wherein: The wavelet transform decomposition uses Haar wavelet to perform high-frequency and low-frequency decomposition, retaining the decomposed low-frequency sub-band and discarding the high-frequency sub-band.
3. The method according to claim 1, characterized in that The image enhancement process includes geometric transformation, histogram equalization and contrast stretching, and the contrast stretching adopts piecewise linear transformation.
4. The method according to claim 1, wherein: The output calculation formula of the convolutional layer is: in, is the value of the position of the i-th sample in the output tensor at the j-th output channel, is the bias of the j-th output channel, weight is the convolution kernel weight matrix, Input(N i ,k) input tensor, N i represents the i-th sample in the input data, represents the jth channel of the current output, k represents the index of the input channel, C m Indicates the total number of input channels; ★ indicates a 2D convolution operation.
5. The method according to claim 1, wherein: The multi-head attention layer includes a multi-head attention module and a feedforward neural network. The output features of the multi-head attention module are input into the feedforward neural network to obtain the final output features of the multi-head attention layer.
6. The method according to claim 5, characterized in that: The multi-head attention layer extracts the long-range relationship features of the image context, including the following steps: (1) The multi-head attention module extracts features, which specifically includes the following steps: (1.1) Convert the output data of the convolutional layer into a sequence and concatenate the classification head to form an extended sequence. Add a learnable positional encoding to the extended sequence to obtain the final input sequence. (1.2), calculate the query matrix, key matrix and value matrix, and calculate the scaled dot product attention from them; (1.3) Calculate the output feature value of each head based on the scaled dot product attention, and concatenate and linearly transform the output feature values of multiple heads to obtain the fusion feature; (1.4) Perform residual connection and normalization on the fusion features and the final input sequence to obtain the output result; (2) The output result of the multi-head attention module is input into the feedforward neural network. After nonlinear transformation by the two fully connected layers of the feedforward neural network, the output result of the multi-head attention module is residually connected and normalized to obtain the final output result of the multi-head attention layer.
7. The method according to claim 6, characterized in that: In the step (1.4), residual connection and layer normalization are performed by the following formula: Z = LayerNorm(X + MHA(X)) Among them, Z is the output result of the multi-head attention module, and X is the final input sequence of the multi-head attention layer; MHA(X) is the fusion feature obtained in step (1.3), and the calculation formula is MHA(h)=Concat(head1,…,head h )W o Among them, W o Linear transformation matrix of fused features; head h is the output eigenvalue of the h-th head, and the calculation formula is head h =Attention(Q h ,K h ,V h )*K Among them, K is the key matrix; Attention(Q h ,K h ,V h ) is the scaled dot product attention of the h-th head.
8. The method according to claim 7, wherein: The formula for the nonlinear transformation of the two fully connected layers of the feedforward neural network in step (2) is FFN(Z)=ReLU(ZW1+b1)W2+b2 Among them, FFN(Z) is the result of nonlinear transformation, W1 and W2 are weight matrices, and b1 and b2 are bias terms; The formula for residual connection and normalization of feedforward neural network is: Output = LayerNorm(Z+FFN(Z)) Among them, Output is the final output result of the multi-head attention layer.
9. A spraying quality monitoring system based on convolutional attention neural network, characterized in that: The system comprises: Acquisition module: used to collect surface images of sprayed workpieces in real time; Preprocessing module: used to perform wavelet transform on the collected image to remove noise, enhance the image after noise removal, and normalize the enhanced image; Neural network construction and training module: used to build and train a convolutional attention neural network. The neural network includes an input layer that receives image data, a convolution layer that extracts local image features, a multi-head attention layer that extracts long-range relationship features of the image context, a fully connected layer that maps the extracted image features to a high-dimensional space, and an output layer that outputs the probability of defect categories. Recognition module: used to input the preprocessed image data into the trained neural network to obtain the probability distribution of defect categories.
10. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions which, when executed by a computing device, cause the computing device to perform any one of the methods according to claims 1 to 8.