K-TIG penetration state identification method based on improved ResNet18 model

Abstract

The invention provides a K-TIG penetration state recognition method based on an improved ResNet18 model, and the method comprises the steps: obtaining a penetration image data set through a pre-built deep fusion arc welding experiment platform; the penetration image data set comprises molten pool images in different penetration states; according to the fusion penetration image data set, training the improved ResNet18 model to obtain a fusion penetration state recognition model; the improved ResNet18 model comprises a depth feature extraction module, a histogram feature extraction module, a feature fusion layer and a Softmax layer; the depth feature extraction module is an improved ResNet18 network; and the obtained to-be-recognized molten pool image is input into a penetration state recognition model for penetration state recognition, and a penetration state recognition result is obtained. According to the fusion penetration state recognition model obtained based on the improved ResNet18 network design, the feature extraction capability and the state recognition accuracy of the model can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of welding penetration state recognition, and in particular to a K-TIG penetration state recognition method based on an improved ResNet18 model. Background Art

[0002] As a new deep penetration welding technology, K-TIG (Keyhole Tungsten Inert Gas) welding can achieve single-sided welding and double-sided forming process without beveling and filling welding materials under certain thickness conditions, and the mechanical properties of the weld joint are good and the weld is beautiful. However, the penetration state during K-TIG welding directly affects the quality and performance of the weld, which is crucial to ensure the welding quality.

[0003] At present, the recognition of K-TIG welding penetration state mainly includes penetration recognition technology based on acoustic signals, electrical signals and visual signals. Although the existing penetration recognition technology has achieved certain results in identifying the penetration state, it still has some obvious shortcomings and application limitations: in the actual production process, noise sources such as mechanical vibration and electromagnetic interference often exist, which will interfere with the acoustic signal or electrical signal, causing signal distortion or masking. In this case, the penetration recognition technology based on acoustic signals or electrical signals is prone to misjudgment, which in turn affects the accurate evaluation of welding quality; at the same time, although the penetration recognition technology based on visual signals is less affected by environmental factors and can extract richer feature information, the design process of existing machine learning algorithms is often more complicated. It relies on manual feature extraction and requires a lot of time and resources for model training and optimization. In addition, under different materials, welding processes and parameters, the feature performance of the molten pool image may be significantly different, which also makes it difficult for existing machine learning algorithms to adapt to various complex situations to achieve real-time, fast and accurate penetration state recognition. Summary of the invention

[0004] The purpose of the present invention is to provide a K-TIG penetration state recognition method based on an improved ResNet18 model. The penetration state recognition model obtained by optimizing the residual structure of the existing ResNet18 network, introducing the attention mechanism and integrating the histogram feature extraction technology can efficiently and comprehensively capture the molten pool image features, and accurately obtain the welding penetration state in real time, thereby providing reliable guarantee for the K-TIG welding quality.

[0005] In order to achieve the above objectives, it is necessary to provide a K-TIG penetration state recognition method based on an improved ResNet18 model to address the above technical problems.

[0006] In a first aspect, an embodiment of the present invention provides a K-TIG penetration state recognition method based on an improved ResNet18 model, the method comprising the following steps:

[0007] A penetration image data set is obtained through a pre-built deep melting arc welding experimental platform; the penetration image data set includes molten pool images of different penetration states;

[0008] According to the melt penetration image data set, the improved ResNet18 model is trained to obtain a melt penetration state recognition model; the improved ResNet18 model includes a deep feature extraction module, a histogram feature extraction module, a feature fusion layer and a Softmax layer; the deep feature extraction module is an improved ResNet18 network;

[0009] The acquired molten pool image to be identified is input into the melt-through state identification model to perform melt-through state identification, and obtain a corresponding melt-through state identification result.

[0010] Furthermore, the deep melting arc welding experimental platform includes a welding robot system, a deep melting arc welding system and a passive visual sensing system; the welding robot system includes a welding robot and a robot control cabinet; the deep melting arc welding system includes a deep melting arc welding machine, a deep melting arc welding gun, an industrial water chiller and a protective gas cylinder; the passive visual sensing system includes a clamping device, and a front-end industrial camera and a rear-end industrial camera mounted on the welding robot through the clamping device; the front-end industrial camera obtains a front molten pool image; the rear-end industrial camera obtains a back molten pool image.

[0011] Furthermore, the step of obtaining a penetration image data set through a pre-built deep arc welding experimental platform includes:

[0012] Through the deep melting arc welding experimental platform, molten pool images of different penetration states are obtained; the penetration states include incomplete penetration state, full penetration state and over-penetration state;

[0013] Perform penetration state annotation and preprocessing operations on each molten pool image in turn to obtain a corresponding preprocessed molten pool image;

[0014] The melt penetration image data set is obtained according to the preprocessed molten pool image.

[0015] Furthermore, the deep feature extraction module includes an initial module, 1 first residual module, 3 second residual modules, 1 average pooling layer and 1 fully connected layer; the initial module includes 3 first convolutional layers, 1 batch normalization layer, 1 activation layer and 1 maximum pooling layer; the first residual module includes 2 identical residual blocks and 1 convolutional block attention module; the second residual module includes 1 identical residual block, 1 convolutional residual block and 1 convolutional block attention module.

[0016] Furthermore, the identical residual block includes two first convolution blocks, one first connection layer and one activation layer; the first convolution block includes one first convolution layer, one batch normalization layer and one activation layer; the first connection layer connects the input of the identical residual block with the output processed by the two first convolution blocks.

[0017] Furthermore, the convolution residual block includes two first convolution blocks, one second convolution block, one second connection layer and one activation layer; the first convolution block includes one first convolution layer, one batch normalization layer and one activation layer; the second convolution block includes one second convolution layer, one batch normalization layer and one activation layer; the second connection layer connects the output processed by the two first convolution blocks with the output processed by the one second convolution block.

[0018] Furthermore, the convolution kernel size of the first convolution layer is 3×3; the convolution kernel size of the second convolution layer is 1×1; and the activation layer is a PReLU activation function.

[0019] Furthermore, the feature fusion layer includes a Concat connection layer, an improved convolutional block attention module and a weighted fusion layer; the improved convolutional block attention module includes a parallel channel attention mechanism and a spatial attention mechanism.

[0020] Furthermore, the step of inputting the acquired molten pool image to be identified into the melt penetration state recognition model to perform melt penetration state recognition to obtain a corresponding melt penetration state recognition result includes:

[0021] Inputting the molten pool image to be identified into the depth feature extraction module to extract the depth feature to obtain a first image feature;

[0022] Inputting the molten pool image to be identified into the histogram feature extraction module to extract histogram features to obtain a second image feature;

[0023] Inputting the first image feature and the second image feature into the feature fusion layer for feature fusion to obtain a fused image feature;

[0024] The fused image features are input into the Softmax layer for classification and recognition to obtain the melt penetration state recognition result.

[0025] Furthermore, the step of inputting the molten pool image to be identified into the histogram feature extraction module to extract the histogram feature to obtain the second image feature includes:

[0026] Converting the molten pool image to be identified into a corresponding molten pool grayscale image, and performing Gamma correction on the molten pool grayscale image to obtain an image to be analyzed;

[0027] Calculating the gradient information of each pixel in the image to be analyzed; the gradient information includes the gradient magnitude and the gradient direction;

[0028] Dividing the image to be analyzed into cells according to a preset cell size, and constructing a corresponding gradient histogram according to the gradient information of the pixel points in each cell;

[0029] Combining a preset number of adjacent cells into a pixel block, normalizing all gradient histograms in the pixel block to obtain a block histogram feature descriptor;

[0030] All block histogram feature descriptors are fused to obtain the second image feature.

[0031] The above application provides a K-TIG penetration state recognition method based on an improved ResNet18 model, through which a penetration image data set including molten pool images of different penetration states is obtained through a pre-built deep arc welding experimental platform, and an improved ResNet18 model including a depth feature extraction module, a histogram feature extraction module, a feature fusion layer and a Softmax layer is trained according to the penetration image data set. After obtaining the penetration state recognition model, the obtained molten pool image to be identified is input into the penetration state recognition model for penetration state recognition, and the corresponding penetration state recognition result is obtained. Compared with the prior art, the K-TIG penetration state recognition method based on the improved ResNet18 model, through the optimization of the residual structure of the existing ResNet18 network, the introduction of the attention mechanism and the integration of the histogram feature extraction technology to obtain the penetration state recognition model, can effectively adapt to the differences in molten pool images of different welding scenes, efficiently and comprehensively capture the molten pool image features, and accurately obtain the welding penetration state in real time, thereby providing reliable guarantee for the K-TIG welding quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of a K-TIG penetration state recognition method based on an improved ResNet18 model in an embodiment of the present invention;

[0033] Figure 2 Schematic diagram of the deep melting arc welding experimental platform in an embodiment of the present invention;

[0034] Figure 3 is a schematic diagram of the structure of the constant residual block in the improved ResNet18 model in an embodiment of the present invention;

[0035] Figure 4 It is a schematic diagram of the structure of the convolution residual block in the improved ResNet18 model in an embodiment of the present invention;

[0036] Figure 5 It is a detailed network structure diagram of the improved ResNet18 model in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical scheme and beneficial effects of the present application clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the embodiments described below are part of the embodiments of the present invention and are only used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] The K-TIG penetration state recognition method based on the improved ResNet18 model provided by the present invention can be understood as an application defect based on the existing penetration recognition technology, such as insufficient real-time performance and low recognition accuracy. The proposed penetration state recognition model is designed based on the residual structure optimization of the existing ResNet18 network, the introduction of the attention mechanism, and the integration of the histogram feature extraction technology to analyze and process the front molten pool image, and can identify three different penetration states of K-TIG penetration state recognition method, such as incomplete penetration, full penetration, and over-penetration. The following embodiments will explain in detail the K-TIG penetration state recognition method based on the improved ResNet18 model of the present invention.

[0039] In one embodiment, Figure 1 As shown, a K-TIG penetration state recognition method based on an improved ResNet18 model is provided, comprising the following steps:

[0040] S11. Obtain a penetration image data set through a pre-built deep arc welding experimental platform; the penetration image data set includes molten pool images in different penetration states; wherein, the deep arc welding experimental platform can be built with reference to the existing welding experimental platform in principle, but in order to ensure that richer and more comprehensive molten pool images can be collected to improve the training effect of the penetration state recognition model and to ensure the robustness of the penetration state recognition model, this embodiment preferably designs a deep arc welding experimental platform that can simultaneously use two high-dynamic industrial cameras to collect molten pool images in different orientations for collecting molten pool images in different penetration states in welding operation scenarios.

[0041] Specifically, the deep melting arc welding experimental platform includes a welding robot system, a deep melting arc welding (K-TIG) system and a passive visual sensing system; the welding robot system includes a welding robot and a robot control cabinet, which are responsible for realizing precise control of the welding speed, and guiding the movement of the welding gun by precisely controlling the movement of the robot, thereby ensuring the smooth progress of the K-TIG welding experiment; the deep melting arc welding system includes a deep melting arc welding machine (K-TIG welding machine), a deep melting arc welding gun (K-TIG special welding gun), an industrial chiller and a protective gas cylinder, which are mainly responsible for conducting K-TIG experiments under different welding parameters; the passive visual sensing system includes a clamping device, and a front-end industrial camera and a rear-end industrial camera mounted on the welding robot through the clamping device, wherein the front-end industrial camera shoots from the front end of the front molten pool to obtain the front molten pool image, and the rear-end industrial camera shoots from the rear end of the front molten pool to obtain the back molten pool image, and then the front and rear molten pool images are transmitted to the industrial control computer in real time for fusion to obtain a complete K-TIG welding molten pool image, so as to better realize the recognition of the penetration state; the specific deep melting arc welding experimental platform architecture is as follows Figure 2 It should be noted that the devices / components that are not specifically designed in the deep melting arc welding experimental platform can be implemented by referring to the existing welding experimental platform, and will not be described in detail here.

[0042] Specifically, the step of obtaining a penetration image data set through a pre-built deep arc welding experimental platform includes:

[0043] Through the deep melting arc welding experimental platform, molten pool images of different penetration states are obtained; the penetration state includes an incomplete penetration state, a full penetration state, and an over-penetration state; the specific process of collecting molten pool images through the deep melting arc welding experimental platform is as follows:

[0044] Before the welding operation begins, the workpiece to be welded is fixed on the workbench to ensure stability during the welding process; during the welding operation, two high-dynamic cameras (front-end industrial camera and back-end industrial camera) and the welding gun are mounted on the end of the welding robot through a clamping device, and the three keep moving synchronously to ensure that a stable and continuous K-TIG molten pool image can be captured from the front of the workpiece to be welded.

[0045] The penetration state of each molten pool image is labeled and preprocessed in turn to obtain the corresponding preprocessed molten pool image; wherein, the process of obtaining the preprocessed molten pool image can be understood as: first, the incomplete penetration molten pool images, the full penetration molten pool images and the over-penetration molten pool images obtained by performing a series of welding experiments based on the deep melting arc welding experimental platform are respectively labeled according to the corresponding penetration state, that is, the penetration state label of each image is determined; then, the molten pool image data is preprocessed including resizing, standardization and cropping to ensure that the molten pool image data format meets the input requirements of the model training. The specific processing details can be set according to the actual application needs and are not specifically limited here.

[0046] According to the preprocessed molten pool image, the melt penetration image dataset is obtained; the melt penetration image dataset can be understood as a dataset for training and verifying the melt penetration state recognition model. In practical applications, the dataset can be divided into a training set, a verification set and a test set according to a preset ratio according to specific needs. The improved ResNet18 model is iteratively trained and verified according to the training set and the verification set. After the training is completed, the test set is used to test the effect of the model.

[0047] This embodiment constructs a deep melting arc welding experimental platform that can simultaneously capture the molten pool images at the front and rear ends to obtain a more comprehensive and complete K-TIG welding molten pool image to construct a training data set for melt penetration state recognition. That is, not only the molten pool images at the front and rear ends of the welding gun are captured by two high-dynamic industrial cameras to ensure the stability of the image acquisition process, but also the consistency of the shooting angle can be ensured. Moreover, by fusing the molten pool images at the front and rear ends of the welding gun, an image with a complete front molten pool and keyhole entrance can be obtained, so that a subsequent melt penetration recognition model can more effectively extract the melt penetration features in the molten pool image, thereby providing a high-quality data basis for the subsequent melt penetration recognition model training, thereby ensuring that more accurate recognition of the melt penetration state can be achieved.

[0048] S12. According to the penetration image data set, the improved ResNet18 model is trained to obtain a penetration state recognition model; wherein, the ResNet18 model is a deep convolutional neural network that introduces a residual connection mechanism, which can effectively solve the gradient vanishing and gradient explosion problems in deep network training, thereby improving the training efficiency and classification performance of the model; and the improved ResNet18 model in this embodiment can be understood as a network model that is targeted to meet the special needs of K-TIG welding penetration state recognition by optimizing the network structure, introducing an attention mechanism, and fusing HOG features, so as to further improve the feature extraction capability and recognition accuracy, including a deep feature extraction module, a histogram feature extraction module, a feature fusion layer and a Softmax layer, and the deep feature extraction module is an improved ResNet18 network; specifically, the deep feature extraction module includes an initial module, 1 first residual module, 3 second residual modules, 1 average pooling layer and 1 fully connected layer.

[0049] In this embodiment, the initial module includes 3 first convolutional layers, 1 batch normalization layer, 1 activation layer and 1 maximum pooling layer. Among them, the convolution kernel size of the first convolutional layer is 3×3, that is, 3 3×3 convolution kernels are used to replace the original 7×7 convolution kernel to reduce the number of parameters while ensuring that the receptive field remains unchanged. In practical applications, the stride of the first convolutional layer can be set to 2, and the strides of the second and third convolutional layers can be set to 1. The main function of the batch normalization (BatchNormalization, BN) layer is to normalize each batch of input data to make the distribution of data more stable, thereby accelerating the network training process and improving the performance of the model. The activation layer here uses the PReLU (Parametric Rectified Linear Unit) activation function to replace the ReLU activation function in the traditional ResNet18, and is applied to the output of the BN layer. Compared with the ReLU activation function, the PReLU function introduces a learnable parameter α, which allows it to retain more information, thereby improving the model's expressiveness and learning ability. At the same time, its parameter α can be automatically adjusted according to the data, which also enhances the model's robustness to noise and outliers, and can help the model better adapt to different data distributions and task requirements. The maximum pooling layer is used to downsample the output of the PReLU activation function. By taking the maximum value in each pooling window as the output, the spatial size of the data can be effectively reduced, thereby reducing the computational complexity and extracting more representative features.

[0050] Connected to the above initial module is the core component of the ResNet18 network, which is a stack of 4 residual modules responsible for further extracting and integrating features. Among them, the first residual module includes 2 identical residual blocks and 1 convolutional block attention module, and Figure 3 As shown, the identity residual block includes two first convolution blocks, one first connection layer and one activation layer; the first convolution block includes one 3×3 first convolution layer, one batch normalization layer and one PReLU activation layer; the first connection layer connects the input of the identity residual block with the output processed by the two first convolution blocks, that is, the input is directly connected to the output through a jump connection, so that the network can learn the identity transformation. The above convolutional block attention module can be understood as a CBAM (Convolutional Block Attention Module) module that applies the channel attention and spatial attention mechanisms in sequence.

[0051] The second residual module includes 1 identical residual block, 1 convolution residual block and 1 convolution block attention module, and the identical residual block is still Figure 3 The structure shown in the figure shows that the main structure of the convolutional residual block is the same as that of the identity residual block. The difference is that the number of channels of the input feature map needs to be changed through a 1×1 convolution operation on the residual path, and then added to the processed input, such as Figure 4 As shown, the convolution residual block includes 2 first convolution blocks, 1 second convolution block, 1 second connection layer and 1 activation layer; the first convolution block includes 1 first convolution layer, 1 batch normalization layer and 1 activation layer; the second convolution block includes 1 second convolution layer, 1 batch normalization layer and 1 activation layer; the second connection layer connects the output processed by the 2 first convolution blocks with the output processed by the 1 second convolution block; preferably, the convolution kernel size of the first convolution layer is 3×3; the convolution kernel size of the second convolution layer is 1×1; the activation layer is a PReLU activation function. Similarly, the convolution block attention module is also a CBAM (Convolutional Block Attention Module) module that applies channel attention and spatial attention mechanisms in sequence.

[0052] The network structure obtained by combining the above-mentioned initial module, the first residual module and the second residual module given in this embodiment can learn more nonlinear transformations while maintaining the advantages of residual connections; at the same time, through the introduction of the attention mechanism, the features that need to be paid attention to in the input image can be amplified and unnecessary features can be suppressed, so that the model can focus more on processing important features, obtain the optimal recognition ability, and increase the recognition accuracy of the network.

[0053] The average pooling layer and fully connected layer of the above-mentioned deep feature extraction module can be understood as the corresponding deep feature extraction layers. The function of the average pooling layer is to average pool the feature map output by the last residual module in the spatial dimension to obtain a feature vector. This feature vector contains the global information of the input image and has a low dimension, which is convenient for subsequent fully connected layer processing. At the same time, the fully connected layer is responsible for receiving the output of the average pooling layer and generating deep features.

[0054] The histogram feature extraction module in the improved ResNet18 model can be understood as a graph feature extraction module that constructs the HOG (Histogram of Oriented Gradient) feature by calculating the unit gradient and direction of the local area of ​​the molten pool image; since the HOG feature undergoes local histogram and normalization processing during the calculation process, it can maintain good invariance to the geometric and optical deformations of the image, which means that even if there are interference factors such as illumination changes and angle changes in the molten pool image during the acquisition process, the HOG feature can still be used for effective state recognition; at the same time, the HOG feature is quantized in the position and direction space, which can help suppress the influence of translation and rotation, that is, it can effectively deal with the situation where the molten pool position or angle in the acquired molten pool image changes due to movement during welding or changes in camera angle, and reduce the influence of these changing factors on the recognition of the penetration state, thereby providing a reliable guarantee for efficient recognition based on the penetration state recognition model to obtain accurate penetration state recognition results.

[0055] The feature fusion layer in this embodiment can be understood as a functional module that fuses the features extracted by the deep feature extraction module and the histogram feature extraction module to obtain new features for state recognition and classification; specifically, the feature fusion layer includes a Concat connection layer, an improved convolution block attention module and a weighted fusion layer; the improved convolution block attention module includes a parallel channel attention mechanism and a spatial attention mechanism. In practical applications, the Concat connection layer first performs a Concat connection on the two types of features extracted by the deep feature extraction module and the histogram feature extraction module, and then uses the channel attention mechanism and the spatial attention mechanism in the improved convolution block attention module to obtain the corresponding weights of the Concat connection generated features, and finally directly performs a weighted operation on the weights with the Concat connection generated features to obtain the output feature map, that is, the new features required in the end. It should be noted that the traditional convolutional block attention module first applies the channel attention mechanism, which sometimes limits the effect of the spatial attention mechanism on the extraction of subsequent features. In order to overcome this limitation, the improved convolutional block attention module used in this embodiment transforms the "serial" connection of the channel attention mechanism and the spatial attention mechanism in the traditional CBAM module into a "parallel" structure, which can ensure that both the channel attention and the spatial attention mechanism can directly act on the original features, thereby maximizing the effectiveness of both, and then enabling the network to autonomously adjust the weight of the HOG feature in the fusion of the new feature (overall feature map), thereby effectively strengthening the key information in the feature map while weakening redundant information.

[0056] It should be noted that after the improved ResNet18 model is designed by the above method, the model is iteratively trained and verified using the training set and the validation set, and the accuracy is selected as the main evaluation indicator of the model. The accuracy refers to the proportion of the number of samples correctly classified by the model to the total number of samples, and the calculation formula is: Accuracy = (number of correctly classified samples / total number of samples) × 100%. By monitoring the changes in the accuracy during the training process, the training effect of the model can be evaluated, and the model parameters can be adjusted or the algorithm can be optimized accordingly to further improve the performance; after the training is completed, the model with the highest recognition accuracy is saved and used as the final model, and the recognition accuracy of up to 98.5% can be obtained by using this model to test the test set.

[0057] This embodiment combines a histogram feature extraction module that can extract edge structure features describing the melt pool image and capture local shape information with an improved ResNet18 network that can extract high-level abstract features, and uses an improved convolutional block attention module for feature fusion adjustment design. Figure 5The shown penetration state recognition model can not only significantly reduce the number of parameters by replacing the initial 7×7 convolution layer with three 3×3 convolution layers in series, but also make the network have stronger feature extraction ability, which can better capture the details and high-level features in the molten pool image and improve the recognition accuracy of the model for the penetration state. In addition, by introducing the convolution block attention module in the improved ResNet18 network, the network's ability to extract penetration features is strengthened, and the robustness and generalization ability of the model in the K-TIG welding environment are enhanced. In the feature fusion part, by improving the connection structure of the channel and spatial attention mechanisms in CBAM, it is ensured that the channel attention and spatial attention mechanisms can directly act on the original features, so that the network can autonomously adjust the weight of the HOG feature in the overall feature map to ensure that the optimization effect of the HOG feature is optimal. By extracting and fusing HOG and deep features of the molten pool image, the advantages of the two features are fully utilized to efficiently mine richer and more comprehensive molten pool image information, improve the accuracy and robustness of penetration recognition, and provide reliable support for obtaining more accurate penetration state recognition results.

[0058] S13, inputting the acquired molten pool image to be identified into the melt penetration state recognition model for melt penetration state recognition, and obtaining a corresponding melt penetration state recognition result; wherein the molten pool image to be identified can be understood as a molten pool image that is collected in real time and needs to be melt penetration state recognized, and the corresponding melt penetration state recognition result can be any one of three different melt penetration states, namely, incomplete melt penetration, full melt penetration, and excessive melt penetration; specifically, the step of inputting the acquired molten pool image to be identified into the melt penetration state recognition model for melt penetration state recognition, and obtaining a corresponding melt penetration state recognition result comprises:

[0059] The molten pool image to be identified is input into the deep feature extraction module for deep feature extraction to obtain a first image feature; wherein, the process of obtaining the first image feature can refer to the relevant description of the aforementioned deep feature extraction module, which will not be repeated here.

[0060] The molten pool image to be identified is input into the histogram feature extraction module for histogram feature extraction to obtain a second image feature; wherein the second image feature can be understood as the aforementioned HOG feature that can describe the edge structure feature of the molten pool image and capture local shape information.

[0061] Specifically, the step of inputting the molten pool image to be identified into the histogram feature extraction module to extract the histogram feature to obtain the second image feature includes:

[0062] Convert the molten pool image to be identified into a corresponding molten pool grayscale image, and perform gamma correction on the molten pool grayscale image to obtain an image to be analyzed; wherein the image to be analyzed can be understood as a molten pool grayscale image that reduces the influence of external factors such as light;

[0063] Calculating the gradient information of each pixel in the image to be analyzed; the gradient information includes the gradient magnitude and the gradient direction;

[0064] Dividing the image to be analyzed into cells according to a preset cell size, and constructing a corresponding gradient histogram according to the gradient information of the pixel points in each cell;

[0065] Combining a preset number of adjacent cells into a pixel block, normalizing all gradient histograms in the pixel block to obtain a block histogram feature descriptor;

[0066] All block histogram feature descriptors are fused to obtain the second image feature.

[0067] It should be noted that the extraction process of the second image feature can be implemented by referring to the related technology of existing HOG feature extraction, which will not be described in detail here.

[0068] The first image feature and the second image feature are input into the feature fusion layer for feature fusion to obtain fused image features; wherein, the process of obtaining the fused image features can refer to the relevant description of the feature fusion layer in the aforementioned improved ResNet18 model, which will not be repeated here.

[0069] The fused image features are input into the Softmax layer for classification and recognition to obtain the melt penetration state recognition result.

[0070] The embodiment of the present application obtains a penetration image dataset including molten pool images with different penetration states through a pre-built deep melting arc welding experimental platform, and trains an improved ResNet18 model including a depth feature extraction module, a histogram feature extraction module, a feature fusion layer and a Softmax layer according to the penetration image dataset. After obtaining the penetration state recognition model, the obtained molten pool image to be identified is input into the penetration state recognition model for penetration state recognition, and a corresponding solution for penetration state recognition results is obtained. The solution can effectively adapt to the differences in molten pool images in different welding scenarios, efficiently and comprehensively capture molten pool image features, and accurately obtain welding penetration states in real time. The solution has wider applicability and flexibility in practical applications, can adapt to the penetration state recognition requirements under different materials, welding processes and parameters, and thus provide reliable guarantee for K-TIG welding quality.

[0071] It should be noted that although the steps in the above flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders.

[0072] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referenced to each other, and each embodiment focuses on the differences from other embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.

Claims

1. A K-TIG penetration state recognition method based on an improved ResNet18 model, characterized in that: The method comprises the following steps: A penetration image data set is obtained through a pre-built deep melting arc welding experimental platform; the penetration image data set includes molten pool images of different penetration states; According to the melt penetration image data set, the improved ResNet18 model is trained to obtain a melt penetration state recognition model; the improved ResNet18 model includes a deep feature extraction module, a histogram feature extraction module, a feature fusion layer and a Softmax layer; the deep feature extraction module is an improved ResNet18 network; The acquired molten pool image to be identified is input into the melt-through state identification model to perform melt-through state identification, and obtain a corresponding melt-through state identification result.

2. The K-TIG penetration state recognition method based on the improved ResNet18 model according to claim 1 is characterized in that: The deep melting arc welding experimental platform includes a welding robot system, a deep melting arc welding system and a passive visual sensing system; the welding robot system includes a welding robot and a robot control cabinet; the deep melting arc welding system includes a deep melting arc welding machine, a deep melting arc welding gun, an industrial water chiller and a protective gas cylinder; the passive visual sensing system includes a clamping device, and a front-end industrial camera and a rear-end industrial camera mounted on the welding robot through the clamping device; the front-end industrial camera obtains a front molten pool image; the rear-end industrial camera obtains a back molten pool image.

3. The K-TIG penetration state recognition method based on the improved ResNet18 model according to claim 1 is characterized in that: The step of obtaining a penetration image data set through a pre-built deep arc welding experimental platform includes: Through the deep melting arc welding experimental platform, molten pool images of different penetration states are obtained; the penetration states include incomplete penetration state, full penetration state and over-penetration state; Perform penetration state annotation and preprocessing operations on each molten pool image in turn to obtain a corresponding preprocessed molten pool image; The melt penetration image data set is obtained according to the preprocessed molten pool image.

4. The K-TIG penetration state recognition method based on the improved ResNet18 model according to claim 1 is characterized in that: The deep feature extraction module includes an initial module, 1 first residual module, 3 second residual modules, 1 average pooling layer and 1 fully connected layer; the initial module includes 3 first convolutional layers, 1 batch normalization layer, 1 activation layer and 1 maximum pooling layer; the first residual module includes 2 identical residual blocks and 1 convolutional block attention module; the second residual module includes 1 identical residual block, 1 convolutional residual block and 1 convolutional block attention module.

5. The K-TIG penetration state recognition method based on the improved ResNet18 model according to claim 4 is characterized in that: The identical residual block includes two first convolution blocks, one first connection layer and one activation layer; the first convolution block includes one first convolution layer, one batch normalization layer and one activation layer; the first connection layer connects the input of the identical residual block with the output processed by the two first convolution blocks.

6. The K-TIG penetration state recognition method based on the improved ResNet18 model according to claim 4 is characterized in that: The convolution residual block includes two first convolution blocks, one second convolution block, one second connection layer and one activation layer; the first convolution block includes one first convolution layer, one batch normalization layer and one activation layer; the second convolution block includes one second convolution layer, one batch normalization layer and one activation layer; the second connection layer connects the output processed by the two first convolution blocks with the output processed by the one second convolution block.

7. The K-TIG penetration state recognition method based on the improved ResNet18 model according to claim 6 is characterized in that: The convolution kernel size of the first convolution layer is 3×3; the convolution kernel size of the second convolution layer is 1×1; and the activation layer is a PReLU activation function.

8. The K-TIG penetration state recognition method based on the improved ResNet18 model according to claim 1 is characterized in that: The feature fusion layer includes a Concat connection layer, an improved convolutional block attention module and a weighted fusion layer; the improved convolutional block attention module includes a parallel channel attention mechanism and a spatial attention mechanism.

9. The K-TIG penetration state recognition method based on the improved ResNet18 model according to claim 1 is characterized in that: The step of inputting the acquired molten pool image to be identified into the melt penetration state recognition model to perform melt penetration state recognition to obtain the corresponding melt penetration state recognition result comprises: Inputting the molten pool image to be identified into the depth feature extraction module to extract the depth feature to obtain a first image feature; Inputting the molten pool image to be identified into the histogram feature extraction module to extract histogram features to obtain a second image feature; Inputting the first image feature and the second image feature into the feature fusion layer for feature fusion to obtain a fused image feature; The fused image features are input into the Softmax layer for classification and recognition to obtain the melt penetration state recognition result.

10. The K-TIG penetration state recognition method based on the improved ResNet18 model according to claim 9 is characterized in that: The step of inputting the molten pool image to be identified into the histogram feature extraction module to extract the histogram feature to obtain the second image feature comprises: Converting the molten pool image to be identified into a corresponding molten pool grayscale image, and performing Gamma correction on the molten pool grayscale image to obtain an image to be analyzed; Calculating the gradient information of each pixel in the image to be analyzed; the gradient information includes the gradient magnitude and the gradient direction; Dividing the image to be analyzed into cells according to a preset cell size, and constructing a corresponding gradient histogram according to the gradient information of the pixel points in each cell; Combining a preset number of adjacent cells into a pixel block, normalizing all gradient histograms in the pixel block to obtain a block histogram feature descriptor; All block histogram feature descriptors are fused to obtain the second image feature.

Images