Image-based pipe weld joint type recognition method

Abstract

Provided is an image-based pipe welding joint type identification method, a deep neural network is labeled and trained through prior data, joint features of pipe welding workpiece image data are extracted and learned, a model is calibrated and trained, and finally a pipe welding joint classification model with high accuracy is obtained, the classification of the type of the pipe welding joint is completed, the interface type of the current workpiece can be quickly output in the case of external interference, the accuracy and speed of joint type identification and classification are improved, the overall welding efficiency is improved, and the effect is improved to a certain extent when classifying workpiece joints with similar arc light, smoke and single workpiece appearance features.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a pipe welding joint type identification method based on images. BACKGROUND

[0002] Traditional image classification technology usually adopts image local feature extraction operators (such as sift surf lbp hog) and related machine learning models (such as random forest SVM) to classify images, and the algorithm flow is long and complex, the design is difficult, and the calculation speed, accuracy and anti-interference ability are relatively poor. The real welding scene is complex, there may be more arc light and smoke pollution in the factory building, and the welding workpiece often needs to be polished, which will form serious glare, making it difficult for the algorithm to work normally. Based on deep learning technology, the model can be further customized and optimized to improve the effect in view of the single scene of welding, so as to improve the accuracy and speed of joint type identification and classification and improve the overall welding efficiency. SUMMARY

[0003] The technical problem solved by the application is to provide a pipe welding joint type identification method based on images, which labels and trains a deep neural network through prior data, extracts and learns joint features of pipe welding workpiece image data, calibrates and trains a model, and finally obtains a high-accuracy pipe welding joint classification model to complete the classification of pipe welding joint types, quickly output the interface type of the current workpiece in the case of external large interference, improve the accuracy and speed of joint type identification and classification, and improve the overall welding efficiency.

[0004] The technical scheme adopted by the application is a pipe welding joint type identification method based on images, which collects image data information of pipe welding workpieces, labels and trains a deep neural network through prior data, extracts and learns joint features of pipe welding workpiece image data, calibrates and trains a model, and finally obtains a high-accuracy pipe welding joint classification model to complete the classification of pipe welding joint types, and the specific steps are as follows:

[0005] 1) Data acquisition and preprocessing: a camera is used to collect image data of pipe welding workpieces and smoke arc light, interference data and original data are fused according to weights of 0.3 and 0.7, and the image data is subjected to projective transformation to simulate image data enhancement in real production;

[0006] 2) After the pre-processing is completed, first, spatial attention modules are added after the first convolutional and last convolutional modules of the ResNeXt-50 network, data is input into the ResNeXt-50 network, first, a global maximum pooling and global average pooling based on channel are performed on the feature map to perform a concat operation; then, a convolution is performed to reduce the dimension to 1 channel; then, a sigmoid activation function is used to generate spatial attention feature weights; finally, the feature is used as a weight to multiply the input feature of the module and input into the subsequent network structure for continuous propagation, that is, the feature maps with the channel number of 1 extracted above are combined to obtain a feature map with the channel number of 2, and the specific combination calculation formula is as follows:

[0007]

[0008] In the above formula, sigma is a sigmoid activation function, f is a convolution operation, the convolution kernel size is 7x7, avgpool is an average pooling operation, and maxpool is a maximum pooling operation.

[0009] 3) The processing result in step 2) is input into a ResNeXt-50 network residual block with a group convolution to improve performance, and a CBAM module structure is added at the end of the residual block to improve performance; the CBAM module sequentially infers an attention map according to the channel dimension and the spatial dimension through a channel attention mechanism, and then the attention map is multiplied with the input feature map for adaptive feature optimization, the module is repeated 3, 4, 6 and 3 times as a stage, and then the four stages are stacked to complete the main body construction of the network;

[0010] 4) After the output of the main residual network, spatial attention is introduced again, and then global average pooling is performed, the probability of the workpiece joint belonging to each category is output, the value with the maximum output is the final prediction category, and the cross entropy is used to calculate the loss of multi-classification, and the calculation formula is:

[0011]

[0012] Where J is the classification loss of a certain sample, p i is the predicted value (0, 1) of the sample, y i is a symbol function, which is 1 when the predicted category is correct, and 0 otherwise.

[0013] In step 3) above, the channel attention mechanism is to compress the feature map in the spatial dimension, the results of the average pooling and the maximum pooling are input into an MLP, and the output is mapped to 0-1 through a sigmoid activation function to obtain the channel weight, and the channel attention mechanism can be expressed as:

[0014]

[0015] In the above formula, AvgPool is an average pooling operation, MaxPool is a maximum pooling operation, MLP is a fully connected network, and sigma is a sigmoid activation function.

[0016] Compared with the prior art, the present application has the following advantages:

[0017] 1. The technical solution adopts simulated smoke arc smoke pollution with different proportions to expand the data, and uses a projection transformation matrix to modify the image, so that the generated data is more consistent with the real scene of the actual image, and the matching degree and adaptability of the model to the actual generated scene are improved.

[0018] 2. The technical solution adjusts the model based on the actual scene of the pipe welding joint image data set, tests the algorithm, and finally determines that the Top1_Accuracy is the highest when the spatial attention is added to the convolution module at the beginning and end of the model, and the CBAM module is added to all residual connection blocks, and a network that performs well on the current pipe welding joint type is obtained.

[0019] 3. The technical solution can quickly output the interface type of the current workpiece in the presence of external interference, improve the accuracy and speed of joint type recognition and classification, and improve the overall welding efficiency. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0021] It should be noted that in this paper, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations. The element defined by the statement "includes a..." does not exclude the presence of another identical element in the process, method, article or device including the element.

[0022] The image-based pipe welding joint type identification method collects image data information of the pipe welding workpiece, labels and trains a deep neural network through prior data, extracts and learns the joint features of the pipe welding workpiece image data, calibrates and trains the obtained model, and finally obtains a high-accuracy pipe welding joint classification model to complete the classification of the pipe welding joint type. The specific steps are as follows:

[0023] 1) Data acquisition and preprocessing: camera is used to collect image data of pipe welder and smoke arc light, and interference data and original data are fused according to the weight of 0.3 and 0.7, and the image data is subjected to projective transformation, and the image in the real production is simulated for data enhancement;

[0024] 2) After the pre-processing is completed, first, a spatial attention module is added after the first convolution and the last convolution module of the ResNeXt-50 network, the data is input into the ResNeXt-50 network, first, the feature map is subjected to a concat operation based on the global maximum pooling and the global average pooling of the channel; then it is reduced to 1 channel through a convolution; then the sigmoid activation function is used to generate the spatial attention feature weight; finally, the feature is used as the weight and the input feature of the module is multiplied to input the subsequent network structure for continuous propagation, that is, the feature map with the channel number of 1 extracted above is combined to obtain a feature map with the channel number of 2, and the specific combination calculation formula is as follows:

[0025]

[0026] In the above formula, σ is the sigmoid activation function, f is the convolution operation, the convolution kernel size is 7x7, avgpool is the average pooling operation, and maxpool is the maximum pooling operation;

[0027] 3) The processing result in step 2) is input into the ResNeXt-50 network residual block with group convolution to improve the performance, and a CBAM module structure is added at the end of the residual block to improve the performance; CBAM is a lightweight general module attention module for feedforward convolutional neural networks; given an intermediate feature map, CBAM module sequentially infers an attention map according to the channel dimension and the spatial dimension through the channel attention mechanism, and then multiplies the attention map with the input feature map for adaptive feature optimization, and the module is repeated 3, 4, 6 and 3 times as a stage, and then the four stages are stacked to complete the network main body construction; specifically, the channel attention mechanism is to compress the feature map in the spatial dimension, and the average value pooling has feedback for each pixel point on the feature map, while the maximum value pooling only has gradient feedback in the place with the largest response in the feature map when calculating the gradient back propagation, and the results of the average pooling and the maximum pooling are mapped to 0-1 through the sigmoid activation function after the MLP to obtain the channel weight, and the channel attention mechanism can be expressed as:

[0028]

[0029] In the above formula, AvgPool is the average pooling operation, MaxPool is the maximum pooling operation, MLP is a layer of fully connected network, and σ is the sigmoid activation function.

[0030] 4) After the output of the main body residual network, introduce spatial attention once, then perform global average pooling, output the probability that the workpiece joint belongs to each category, and the output with the maximum value is the final prediction category, and the loss of multi-classification is calculated by cross entropy, and the calculation formula is:

[0031]

[0032] Wherein J is the classification loss of a sample, p i is the predicted value (0, 1) of the sample, y i is a symbol function, which is 1 when the predicted category is correct, otherwise it is 0.

[0033] The technical scheme adopts simulated smoke arc smoke pollution of different proportions to expand the data, and uses a projection transformation matrix to modify the image, so that the generated data is more consistent with the real scene of the actual image, and the matching degree and adaptability of the model to the actual generated scene are improved; based on the actual scene of the pipe welding joint image data set, the model is adjusted, the algorithm is tested, and finally it is determined that only the spatial attention is added to the convolution module at the beginning and end of the model, the Top1_Accuracy is the highest when the CBAM module is added to all residual connection blocks, and a network with good performance for the current pipe welding joint type is obtained; the interface type of the current workpiece can be quickly output in the case of large external interference, the accuracy and speed of joint type recognition and classification are improved, and the overall welding efficiency is improved.

[0034] It is apparent to those skilled in the art that the present application is not limited to the details of the foregoing exemplary embodiments, and that the present application can be implemented in other concrete forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and not restrictive, and the scope of the present application is defined by the appended claims rather than the foregoing description, and it is intended to encompass all changes falling within the meaning and range of equivalents of the elements of the claims. Any reference signs in the claims should not be considered as limiting the claims to which they relate.

[0035] In addition, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description manner of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be properly combined to form other embodiments that those skilled in the art can understand.

Claims

1. An image-based tube weld joint type recognition method, characterized by: Image data information of pipe welding workpieces is collected, a deep neural network is labeled and trained through prior data, joint features of pipe welding workpiece image data are extracted, a model is calibrated and trained, and finally a pipe welding joint classification model with high accuracy is obtained to complete the classification of pipe welding joint types. The specific steps are as follows: 1) Data acquisition and preprocessing: camera is used to collect image data of pipe welding workpieces and smoke arc light, interference data and original data are fused according to the weights of 0.3 and 0.7, and the image data is subjected to projective transformation, and the data is enhanced by simulating the image in actual production; 2) After preprocessing, first add a spatial attention module after the first convolution and the last convolution module of the ResNeXt-50 network, input the data into the ResNeXt-50 network, first input the feature map to do a concat operation based on the global maximum pooling and global average pooling of channel; then pass through a convolution to reduce the dimension to 1 channel; then pass through a sigmoid activation function to generate a spatial attention feature weight; finally, the feature is used as a weight and multiplied with the input feature of the module to input the subsequent network structure for further propagation, that is, the feature map with a channel number of 1 extracted above is combined to obtain a feature map with a channel number of 2, and the specific combination calculation formula is as follows: In the formula, sigma is a sigmoid activation function, f is a convolution operation, the convolution kernel size is 7x7, avgpool is an average pooling operation, and maxpool is a maximum pooling operation; 3) The processing result in step 2) is input into the ResNeXt-50 network residual block with group convolution to improve performance, and a CBAM module structure is added at the end of the residual block to improve performance; the CBAM module sequentially infers an attention map according to the channel dimension and the spatial dimension through the channel attention mechanism, and then multiplies the attention map with the input feature map for adaptive feature optimization, and the module is repeated 3, 4, 6 and 3 times as a stage, and then the four stages are stacked to complete the main body construction of the network; 4) After the output of the main residual network, a spatial attention is introduced again, and then a global average is performed, the probability that the workpiece joint belongs to each category is output, the value with the maximum output is the final prediction category, and the cross entropy is used to calculate the loss of multi-classification, and the calculation formula is as follows: where J is the classification loss for a certain sample, p i is the predicted value (0, 1) for this sample, y i is the indicator function, which is 1 when the predicted class is correct, and 0 otherwise.

2. The image-based tube weld joint type recognition method according to claim 1, characterized in that: In step 3) above, the channel attention mechanism is to compress the feature map in the spatial dimension, and the results of average pooling and maximum pooling are mapped to 0-1 as the weight of the channel through MLP and sigmoid activation function, and the channel attention mechanism can be expressed as: In the formula, AvgPool is an average pooling operation, MaxPool is a maximum pooling operation, MLP is a fully connected network, and sigma is a sigmoid activation function.

Images