High-precision and high-quality visual guidance robot automatic welding method based on deep learning

Abstract

The invention discloses a high-precision and high-quality visual guidance robot automatic welding method based on deep learning. The method comprises the steps that an image to be recognized is obtained; the image to be recognized is input into a segmentation network model, the welding seam position, the mass center of a molten pool and the number of spatter are recognized, the segmentation network model is an improved semi-supervised segmentation network, and the improved semi-supervised segmentation network is composed of an encoder and a decoder of an optimized teacher-student model; the welding seam position is sent to a robot controller, the position of a welding gun is adjusted in real time, and a robot is guided to conduct automatic welding; according to the encoder structure of the improved semi-supervised segmentation network, a multi-scale convolution structure, an affinity field loss function and a residual network are introduced, and the position of a welding seam, the mass center of a molten pool and the number of spatter are rapidly and accurately recognized. According to the method, the complexity of a detection system is reduced, meanwhile, high-precision and high-quality visual guidance robot automatic welding is achieved, and the intelligent level of the welding process is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot welding, and in particular relates to a high-precision and high-quality visual-guided robot automatic welding method based on deep learning. Background Art

[0002] Before using a welding robot for welding operations, technicians need to teach and program the welding path of the welding robot according to the weld path. "Teach-playback" mode and offline programming (OLP) are still the two main working modes of modern welding robots. When the welding robot changes the welding workpiece in "Teach-playback" mode and offline programming (OLP), the welding robot needs to be re-taught and reprogrammed. Both teaching and offline programming require professional technicians to spend a lot of time setting up the welding robot before welding. Although it can improve the welding quality and reduce the labor intensity of workers to a certain extent, the welding auxiliary time is long and the quality requirements of technical workers are high. Not only do technical workers need to have welding skills and rich experience, but they also need to have robot operation and programming skills. These weaknesses seriously hinder the widespread application of welding robots. Therefore, in order to meet the development needs of modern industrial automated welding, it is necessary to study the real-time tracking technology of welding robots.

[0003] Prior art 1 proposes a robot trajectory tracking and correction method during welding (CN116512270A), which uses a structured light scanning device to obtain the welding trajectory data on the workpiece in real time and uses the robot control end to synchronously obtain the real-time control data of the workpiece during welding, so as to realize the correction of the welding trajectory. However, this method realizes the correction of the welding trajectory on the basis of preliminary planning of the welding trajectory, and the system flexibility is poor.

[0004] Prior art 2 A welding robot visual control system based on deep learning CN116587288A, this method combines deep learning algorithm with visual detection to identify welds, including the type and position of welds, and adopts different welding processes for different types of welds. At the same time, the time series information and spatial sequence information of the welding position of the welding gun head are continuously learned during the welding process to optimize the welding process of the welding robot. The method proposed in this patent includes modules such as weld trajectory determination module, movement parameter determination module and welding parameter determination module. The combination of multiple models has high computational complexity and limited real-time performance.

[0005] Prior art 3 A gas shielded welding process parameter optimization system and method CN116511652A, by processing the welding process image, obtains the molten pool area data in the welding image, and optimizes the parameter optimization model. The welding process is monitored in real time, and the current welding parameters are optimized and adjusted according to the parameter optimization model. This method has high requirements on the quality of training data, sensor accuracy and anti-interference ability, and is insufficient to cope with abnormal working conditions.

[0006] Prior art 4 An automatic welding monitoring system, method, device and readable storage medium CN116586719A, including a welding control module, a welding quality detection module and a welding adjustment module, the welding control module selects welding parameters to weld the weldment, and the welding quality detection module integrates multiple sensor data, including welding gun sensor data, visual sensor data, ultrasonic sensor data and infrared thermal imager data. Through the fusion processing of multiple sensor data, it is determined whether the welding data should be adjusted and how to adjust it, and finally the welding adjustment module adjusts the welding parameters in real time. The method proposed in this patent has the problems of complex equipment, high cost and high maintenance and repair costs in the later stage.

[0007] Since the visual sensor is installed at the end of the welding gun, the weld image it recognizes is the image at the previous moment. During the welding process, due to factors such as robot errors and deformation of the weldment, the weld position recognized by the visual sensor is not accurate enough.

[0008] During the welding process, improper selection of welding parameters and poor weld processing accuracy lead to an increase in spatter. On the one hand, the increase in spatter affects the surface quality after welding, and has a serious impact on the environment and the health of welding operators. On the other hand, spatter increases stress concentration at the weld, affecting the service life of welded parts. Therefore, a high-precision and high-quality vision-guided robot automatic welding method based on deep learning is urgently needed. Summary of the invention

[0009] In order to solve the above technical problems, the present invention proposes a high-precision and high-quality vision-guided robot automatic welding method based on deep learning, which can realize the automation and intelligence of the welding process, improve the welding quality, and reduce the labor intensity of workers.

[0010] The present invention provides a high-precision and high-quality vision-guided robot automatic welding method based on deep learning, comprising:

[0011] Obtain an image to be recognized;

[0012] Inputting the image to be identified into a segmentation network model to identify the weld position, the centroid of the molten pool and the amount of spatter, wherein the segmentation network model is an improved semi-supervised segmentation network, and the improved semi-supervised segmentation network is composed of an encoder and a decoder of an optimized teacher-student model;

[0013] The weld position is sent to the robot controller, the welding gun position is adjusted in real time, and the robot is guided to weld automatically.

[0014] Optionally, the encoder and decoder of the optimized teacher-student model include: combining dilated convolution, multi-scale feature fusion and residual network to improve the encoder and decoder, using dilated convolution instead of traditional stride convolution in Layer2, Layer3 and Layer4 of the encoder structure, and outputting the features of Layer1, Layer2, Layer3 and Layer4 through The convolution layer adjusts the number of channels and finally splices the output, improves the batch normalization layers in the encoder and decoder, improves the loss function, and replaces the ResNet network structure in the decoder backbone network with the Xception network structure.

[0015] Optionally, the improved encoder structure includes: a plurality of convolutional layers, a maximum pooling layer, and a plurality of bottleneck structure hole convolutional layers, and the improved decoder structure includes: replacing the ResNet network structure in the decoder backbone network with the Xception network structure;

[0016] The convolution layer is used to extract image features and obtain feature maps;

[0017] The maximum pooling layer is used to reduce the spatial size of the feature map;

[0018] The bottleneck structure hole convolution layer is used to expand the receptive field, maintain the feature map resolution, and reduce parameters and calculation amount without increasing the number of model parameters.

[0019] Optionally, the improved decoder includes: a dilated spatial convolutional pooling pyramid, a head structure, an upsampling layer, a convolutional layer, a concatenation layer, and an output layer;

[0020] The dilated spatial convolution pooling pyramid is used to introduce multiple parallel branches to capture context information of different scales and obtain multi-level features;

[0021] The head structure is used to prevent overfitting;

[0022] The upsampling is used to fuse high-level features with low-level features and optimize the accuracy of segmentation boundaries;

[0023] The convolution layer is used to extract multi-scale features of the input image;

[0024] The splicing layer is used to splice the upsampling result and the convolution result;

[0025] The output layer is used to output the splicing result.

[0026] Optionally, the improved batch normalization layer includes: replacing the original batch normalization layer with a synchronous batch normalization method.

[0027] Optionally, the improved loss function includes: improving the original loss function by combining the affinity field loss and the weighted loss fusion method to obtain an improved loss function;

[0028] The improved loss function is:

[0029]

[0030] in, is the improved loss function, To monitor losses, is the unsupervised loss, is the contrast loss, , and is the weight coefficient of the corresponding item, For affinity field loss.

[0031] Optionally, sending the weld position to a robot controller and adjusting the welding gun position in real time includes:

[0032] The weld position is sent to the robot controller for welding. During welding, the improved semi-supervised segmentation network is used to identify the amount of spatter and the center of mass of the molten pool in the image, and the center of mass position of the molten pool is compared with the weld position, and the deviation is calculated, and the deviation is used as the weld position adjustment amount at the next moment.

[0033] Optionally, sending the weld position to the robot controller for welding includes:

[0034] The identified weld position coordinates are transformed according to a transformation matrix to obtain the coordinates in a robot coordinate system;

[0035] Welding is performed according to the coordinates in the robot coordinate system.

[0036] Optionally, the conversion matrix is:

[0037]

[0038] in, is the transformation matrix from the workpiece coordinate system to the six-axis robot base coordinate system, is the transformation matrix from the six-axis robot end coordinate system to the six-axis robot base coordinate system, is the transformation matrix from the workpiece coordinate system to the camera coordinate system, It is the transformation matrix from the six-axis robot end coordinate system to the camera coordinate system.

[0039] Optionally, the weld position is sent to a robot controller to adjust the welding gun position in real time, and further includes: comparing the current spatter amount with the spatter amount at the previous moment to adjust the welding process parameters;

[0040] Compare the current spatter quantity with the spatter quantity at the previous moment and adjust the welding process parameters including:

[0041] The number of spatters in the i-th frame and the i+1-th frame image is calculated. If the number of spatters decreases, the welding process parameters are adjusted positively; otherwise, the welding inclination angle and the nozzle height are adjusted negatively.

[0042] Compared with the prior art, the present invention has the following advantages and technical effects:

[0043] 1. The encoder structure of the improved semi-supervised segmentation network adopted in the present invention introduces a multi-scale convolution structure, affinity field loss function and residual network, innovatively designs a new feature extraction module, and simultaneously segments the weld, spatter, molten pool, distortion point and laser to reduce the complexity of the detection system.

[0044] 2. The present invention guides the robot to perform automatic welding by identifying the weld position, thereby avoiding manual teaching and improving efficiency.

[0045] 3. The present invention adjusts the deviation caused by the front placement of the visual sensor and the deformation of the weldment during the welding process according to the identified centroid of the welding pool, thereby improving the welding quality.

[0046] 4. The present invention adjusts welding parameters in real time by identifying the amount of spatter and the molten pool to obtain a better-formed weld. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0048] Figure 1 This is a flow chart of a high-precision and high-quality vision-guided robot automatic welding method based on deep learning in an embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of a system framework of an embodiment of the present invention;

[0050] Figure 3is a schematic diagram of weld tracking according to an embodiment of the present invention;

[0051] Figure 4 is a working principle diagram of a sensor according to an embodiment of the present invention;

[0052] Figure 5 is a welding deviation model diagram of an embodiment of the present invention;

[0053] Figure 6 is a schematic diagram of adjusting welding process parameters according to an embodiment of the present invention;

[0054] Figure 7 is a schematic diagram of the encoder network architecture structure of an embodiment of the present invention;

[0055] Figure 8 It is a schematic diagram of the decoder network architecture structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0056] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0057] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] The professional knowledge involved in the embodiment is described below:

[0059] The semi-supervised segmentation network model adopts the Momentum Teacher structure commonly seen in the self-supervised technology route. It consists of two branch networks with exactly the same structure, the teacher and the student. The teacher model is a model that maintains fixed parameters during the training process. The prediction results of the teacher model are regarded as a "soft label" corresponding to the hard label (i.e. the true label); the student model is a model that needs to be trained, and it is trained by minimizing the difference between the prediction and the teacher model. During the training process, for labeled samples, the true label is used to calculate the loss; for unlabeled samples, the prediction results of the teacher model are used to calculate the loss. The network weights of the teacher branch are modified by the exponential moving average (EMA) according to the network weights of the student branch, making full use of the unlabeled sample information, reducing model oscillation, and forcing the model's prediction on unlabeled samples to be close to the prediction of the teacher model, reducing overfitting. Both network branches consist of an encoder and a decoder. The encoder uses the residual network deep residual network ResNet101 model structure, and the decoder uses the deep learning image segmentation Deeplabv3+ model structure. The encoder is mainly responsible for feature extraction and dimensionality compression, removing redundancy and noise; the decoder is responsible for generating the final output sequence from the encoder output, including the category, position and pixel information of the required recognition features. In the encoder architecture, by integrating multi-scale convolution, the comprehensive consideration of local details and global information is achieved, providing a more flexible and comprehensive feature representation method.

[0060] This embodiment proposes a high-precision and high-quality vision-guided robot automatic welding method based on deep learning. Figure 1 As shown, the specific steps include:

[0061] Obtain an image to be recognized;

[0062] The image to be identified is input into the segmentation network model to identify the weld position, the centroid of the molten pool and the amount of spatter, wherein the segmentation network model is an improved semi-supervised segmentation network, which is composed of an encoder and a decoder of an optimized teacher-student model;

[0063] The weld position is sent to the robot controller, the welding gun position is adjusted in real time, and the robot is guided to weld automatically.

[0064] Specifically, the semi-supervised segmentation network model recognizes features such as welds, spatter, molten pool, distortion points and lasers. It mainly identifies distortion points to guide welding. However, sometimes the field of view is limited and the distortion points are not obvious. In this case, the intersection of the laser and the weld is identified to determine the position of the distortion points to guide welding.

[0065] Furthermore, the encoder and decoder of the optimized teacher-student model include: combining dilated convolution, multi-scale feature fusion and residual network to improve the encoder and decoder, using dilated convolution technology to expand the receptive field of the convolution kernel without increasing the parameter burden of the model, ensuring the accuracy of segmentation and maintaining the efficiency of the model, using dilated convolution instead of traditional stride convolution in Layer2, Layer3, and Layer4 in the encoder structure, and adjusting the number of channels of the features output by Layer1, Layer2, Layer3, and Layer4 through the convolution layer for final splicing and output, improving the batch normalization layer in the encoder and decoder, and improving the loss function, replacing the residual ResNet network structure in the decoder backbone network with the convolutional neural Xception network structure.

[0066] Specifically, Figure 2 As shown, it mainly consists of 100 welding power supply, 200 robot controller, 300 host computer, 400 structured light vision device, 500 welding execution device, wherein the welding execution device includes 501 six-axis robot arm, 502 welding gun. The structured light vision device 400 is installed at the end of the welding gun 502 and moves with the movement of the welding gun.

[0067] Furthermore, the improved encoder and decoder architectures are as follows: Figure 7 and Figure 8 The improved encoder structure includes: several convolutional layers, maximum pooling layers and several bottleneck structure hole convolutional layers;

[0068] Convolutional layers are used to extract image features. Shallow convolutional layers can reveal detailed features such as spatter and distortion points, while deep convolutional layers grasp broader contextual information and help identify the overall contours of welds, laser streaks, and molten pools.

[0069] The maximum pooling layer is used to reduce the spatial size of the feature map, reduce the amount of calculation and the number of parameters of the subsequent layers, and increase the receptive field;

[0070] The bottleneck structure of the atrous convolution layer is used to expand the receptive field, maintain the feature map resolution, and reduce parameters and computation without increasing the number of model parameters.

[0071] Specifically, in the encoder structure, the input layer receives the image information of the dataset; the basic building unit of the bottleneck structure module is the residual structure, and by adding cross-stage connections, the network learns the residual mapping between the input and output data, avoiding the problems of gradient vanishing and gradient exploding; the convolution layer is used to extract the features of the input data, such as spatter, laser, and the color and texture of the molten pool, and pass the output data to the next residual structure to form a layer-by-layer feature extraction process; the pooling layer uses the maximum pooling method to reduce the dimensionality of the feature map, reduce the amount of calculation while retaining the key information of the feature, suppress noise, reduce information redundancy, and make the model more robust to changes in the input feature position. The entire encoder ResNet101 network structure is regarded as a feature extractor.

[0072] Furthermore, the ResNet network structure in the decoder backbone network is replaced with the Xception network structure, including: replacing the maximum pooling layer with a depthwise separable convolution with a step size of 2 to reduce the number of parameters and calculations, and adding batch normalization and ReLU activation functions after the deep convolution to improve the learning ability of the model.

[0073] The improved decoder includes: dilated spatial convolutional pooling pyramid, head structure, upsampling layer, convolution layer, concatenation layer and output layer;

[0074] The dilated spatial convolutional pooling pyramid is used to introduce multiple parallel branches to capture contextual information of different scales and obtain multi-level features;

[0075] Head structure, used to prevent overfitting and improve the generalization ability of the model;

[0076] Upsampling is used to fuse high-level features with low-level features and optimize the accuracy of segmentation boundaries;

[0077] Convolutional layer, used to extract multi-scale features of the input image;

[0078] The concatenation layer is used to concatenate the upsampling result and the convolution result;

[0079] Output layer, used to output the splicing results.

[0080] Specifically, the Deeplabv3+ model structure serves as a mask predictor in the decoder, and its Atrous Spatial Convolution Pooling Pyramid (ASPP) module captures contextual information of different scales by introducing multiple parallel branches. Each branch uses different atrous convolution and pooling operations to obtain multi-level features. By using different atrous rates (or dilation rates) for atrous convolution, the receptive field can be effectively expanded, which is equivalent to capturing the contextual information of the welding image dataset at multiple scales, thereby processing objects and structures of different scales and increasing the network model's perception of welds, lasers, distortion points, spatter and molten pools. The spliced ​​output results are reduced to 256 channels through the Head layer, then upsampled and spliced ​​with the output results of the original input data through the convolution layer, and finally the output category features are mapped to a 256-dimensional representation space through the representation layer.

[0081] Furthermore, the improved batch normalization layer includes: replacing the original batch normalization layer with a synchronous batch normalization method to ensure that the parameters of the normalization layer on different devices remain consistent, reduce communication overhead, and improve the training effect, convergence speed and stability of the model.

[0082] Furthermore, the improved loss function includes: improving the original loss function by combining the affinity field loss and the weighted loss fusion method to obtain an improved loss function;

[0083] The improved loss function is:

[0084]

[0085] in, is the improved loss function, To monitor losses, is the unsupervised loss, is the contrast loss, , and is the weight coefficient of the corresponding item, For affinity field loss.

[0086] Specifically, the loss function of the original model combines supervised loss, unsupervised loss, and contrast loss. Used to calculate the difference between the model prediction and the true label, the formula is as follows:

[0087]

[0088] in, represents the number of samples in a labeled data batch, is the cross entropy loss function; represents the prediction function of the model, where Usually a feature extraction network, Usually a classification network, are the model parameters, including all weights and biases that need to be learned through training; is a sample in the labeled data, is the corresponding true label.

[0089] Unsupervised loss Used to process reliable prediction results in unlabeled data. Use the model's own predictions to generate pseudo labels, and use these pseudo labels to further train the model. The formula is as follows:

[0090]

[0091] in, represents the number of samples in the unlabeled data batch, is a sample in the unlabeled data, It is a pseudo label generated by the model’s own prediction.

[0092] Contrastive loss It is used to deal with unreliable prediction results in unlabeled data, and to encourage the model to learn feature representations that distinguish different categories by comparing positive and negative samples. The formula is as follows:

[0093]

[0094] in, and Represent the total number of categories and the total number of pixels of each sample respectively; Representative model for The categories of samples The feature representation of is the temperature parameter that controls the smoothness of the softmax function in the contrast loss. Each anchor pixel is followed by a positive sample and negative samples, respectively represented as and , is the cosine similarity between features from two different pixels and is constrained to range between -1 and 1.

[0095] The final loss function is expressed as:

[0096]

[0097] in, and is the weight coefficient of the corresponding item, which is updated in real time using a linear change strategy during training.

[0098] In order to further optimize the clarity of the weld edge and spatter boundary in the semantic segmentation of welding images by the semi-supervised segmentation model, an improved loss function integrating affinity field loss is proposed. This loss function strengthens the recognition of fine edges in the weld area and achieves more accurate pixel-level segmentation. The improved loss function consists of the following parts:

[0099]

[0100]

[0101] in, is the number of all pixel pairs in the image; is the set of all pixel pairs, where each pair of pixels All are adjacent pixels; is a function that calculates the affinity loss value for a given pixel pair, specifically:

[0102]

[0103] and It's pixels and True label or pseudo label of is the model prediction pixel and The probability of being of the same category.

[0104] Improved loss function By considering the spatial relationship and coherence between pixels, the performance of the model in complex welding scenes is enhanced. A constraint on the relationship between pixel pairs is introduced, which requires adjacent pixels to be semantically consistent, resulting in a finer segmentation boundary, encouraging the model to learn more accurate edges and boundaries, thereby effectively improving the segmentation quality of the model for the weld edge, especially at the irregular boundaries generated by the molten pool and spatter during the welding process.

[0105] Furthermore, the weld position is sent to the robot controller, and the welding gun position is adjusted in real time including:

[0106] The weld position is sent to the robot controller for welding. During welding, the improved semi-supervised segmentation network is used to identify the centroid of the molten pool in the image, and the centroid position of the molten pool is compared with the weld position to calculate the deviation, which is used as the weld position adjustment amount at the next moment.

[0107] Further, sending the weld position to the robot controller for welding includes:

[0108] The coordinates of the identified weld position are transformed according to the transformation matrix to obtain the coordinates in the robot coordinate system;

[0109] Welding is performed according to the coordinates in the robot coordinate system.

[0110] Specifically, the weights of the trained semi-supervised segmentation model are used to identify the weld position. For the identified weld position, the position of the weld position in the robot polar coordinate system is calculated using the hand-eye calibration matrix that has been obtained through calibration, thereby guiding the six-axis robot arm to drive the welding gun to weld the weld. The hand-eye calibration matrix is ​​as follows:

[0111]

[0112] in: The transformation matrix from the workpiece coordinate system to the six-axis robot base coordinate system;

[0113] The transformation matrix from the six-axis robot end coordinate system to the six-axis robot base coordinate system;

[0114] Transformation matrix from workpiece coordinate system to camera coordinate system;

[0115] It is the transformation matrix from the six-axis robot end coordinate system to the camera coordinate system.

[0116] Furthermore, if Figure 6 As shown, the weld position is sent to the robot controller, and the welding gun position is adjusted in real time, and also includes: comparing the current spatter amount with the spatter amount at the previous moment, and adjusting the welding process parameters;

[0117] Compare the current spatter quantity with the spatter quantity at the previous moment and adjust the welding process parameters including:

[0118] Calculate the number of spatters in the i-th frame and the i+1-th frame image. If the number of spatters decreases, adjust the welding process parameters in a positive direction; otherwise, adjust the welding process parameters in a negative direction, wherein the process parameters include: welding inclination angle and nozzle height. In addition, the positive adjustment referred to in this embodiment is to increase the value, and the negative adjustment is to decrease the value;

[0119] Specifically, after the welding arc is started, the image collected by the structured light vision device 400 contains not only the weld position information, but also the spatter information and the molten pool information. Therefore, after the welding arc is started, on the one hand, the weld position is continuously identified to guide the six-axis robot 501 to drive the welding gun 502 to weld the weld, and at the same time, the number of spatters and the center of mass of the molten pool in the image are identified through the trained deep learning algorithm, and the welding process parameters are optimized and adjusted based on the spatter number information and the molten pool information, so as to achieve the purpose of reducing the number of spatters and improving the quality of weld formation.

[0120] The present embodiment is described in detail below with reference to the accompanying drawings:

[0121] When using Figure 2 When the welding equipment shown performs welding operation, the weldment in the welding area is first photographed by the line structured light device 400 installed on the welding gun 502, and the acquired image is transmitted to the host computer 300. The trained semi-supervised segmentation model weights are used in the host computer 300 to identify the weld, laser, molten pool, distortion point and spatter features in the welding image.

[0122] By identifying the weld position and The coordinate system transformation matrix converts the weld position coordinates identified in the image into robot base coordinate system coordinates, and sends the coordinates to the robot controller 200, thereby guiding the six-axis robot arm 501 to move along the identified weld position.

[0123] After the welding arc is started, the structured light vision device 400 continuously transmits the captured welding image to the host computer 300, and the welding image is input into the trained deep learning algorithm in the host computer 300. The weld position is continuously identified by the deep learning algorithm, and the identified weld position is continuously input into the robot controller 200 after the coordinate system is transformed, and the six-axis robot arm 501 drives the welding gun 502 to weld, so as to realize the structured light vision device 400 to guide the welding. The center of mass of the molten pool in the welding image is identified, such as Figure 3-Figure 5 As shown, the plane where the welding gun is located is Perpendicular to the weld plane , the welding gun is used during the welding process The angle of rotation of the welding gun is called the welding gun inclination angle. During the rotation of the welding gun, the plane where the welding gun is located does not change. The plane of the weld Perpendicular relationship. Thermal deformation during welding can cause welding errors. In flat butt welding, this error is considered to be the distance between the end of the welding gun and the weld plane. Parallel The deviation in the plane is considered, so when adjusting the deviation, only the deviation in the x and y directions is considered, and the deviation in the z direction is not considered. The weld position identified by the laser vision system is the position of the current welding position at the moment before, and there is a certain tracking deviation in the robot movement. The current position of the welding gun has a certain deviation due to various reasons. and Weld position information identified by the laser vision system Combine and adjust the coordinate information Transmitted to the robot to achieve high-precision welding operations.

[0124] Identify the amount of splashes in the image, such as Figure 6 As shown, according to the identified amount of spatter in the welding process, the welding process parameters are adjusted to suppress the amount of spatter in the welding process.

[0125] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A high-precision and high-quality vision-guided robot automatic welding method based on deep learning, characterized in that: include: Obtain an image to be recognized; Inputting the image to be identified into a segmentation network model to identify the weld position, the centroid of the molten pool and the amount of spatter, wherein the segmentation network model is an improved semi-supervised segmentation network, and the improved semi-supervised segmentation network is composed of an encoder and a decoder of an optimized teacher-student model; The weld position is sent to the robot controller, the welding gun position is adjusted in real time, and the robot is guided to weld automatically.

2. According to the high-precision and high-quality vision-guided robot automatic welding method based on deep learning in claim 1, it is characterized in that: The encoder and decoder of the optimized teacher-student model include: combining dilated convolution, multi-scale feature fusion and residual network to improve the encoder and decoder. In the Layer2, Layer3 and Layer4 layers of the encoder structure, dilated convolution is used instead of traditional stride convolution. At the same time, the features output by Layer1, Layer2, Layer3 and Layer4 are transmitted through The convolution layer adjusts the number of channels and finally splices the output, improves the batch normalization layers in the encoder and decoder, improves the loss function, and replaces the ResNet network structure in the decoder backbone network with the Xception network structure.

3. According to claim 2, a high-precision and high-quality vision-guided robot automatic welding method based on deep learning is characterized in that: The improved encoder structure includes: several convolutional layers, maximum pooling layers and several bottleneck structure hole convolutional layers. The improved decoder structure includes: replacing the ResNet network structure in the decoder backbone network with the Xception network structure; The convolution layer is used to extract image features and obtain feature maps; The maximum pooling layer is used to reduce the spatial size of the feature map; The bottleneck structure hole convolution layer is used to expand the receptive field, maintain the feature map resolution, and reduce parameters and calculation amount without increasing the number of model parameters.

4. According to the high-precision and high-quality vision-guided robot automatic welding method based on deep learning in claim 2, it is characterized in that: The improved decoder comprises: a dilated spatial convolutional pooling pyramid, a head structure, an upsampling layer, a convolutional layer, a concatenation layer and an output layer; The atrous spatial convolutional pooling pyramid is used to introduce multiple parallel branches to capture context information of different scales and obtain multi-level features; The head structure is used to prevent overfitting; The upsampling is used to fuse high-level features with low-level features and optimize the accuracy of segmentation boundaries; The convolution layer is used to extract multi-scale features of the input image; The splicing layer is used to splice the upsampling result and the convolution result; The output layer is used to output the splicing result.

5. The high-precision and high-quality vision-guided robot automatic welding method based on deep learning according to claim 2 is characterized in that: The improved batch normalization layer includes: replacing the original batch normalization layer with a synchronous batch normalization method.

6. The high-precision and high-quality vision-guided robot automatic welding method based on deep learning according to claim 2 is characterized in that: The improved loss function includes: improving the original loss function by combining affinity field loss and weighted loss fusion method to obtain an improved loss function; The improved loss function is: in, is the improved loss function, To monitor losses, is the unsupervised loss, is the contrast loss, , and is the weight coefficient of the corresponding item, For affinity field loss.

7. The high-precision and high-quality vision-guided robot automatic welding method based on deep learning according to claim 1 is characterized in that: Sending the weld position to the robot controller and adjusting the welding gun position in real time include: The weld position is sent to the robot controller for welding. During welding, the improved semi-supervised segmentation network is used to identify the centroid of the molten pool in the image, and the centroid position of the molten pool is compared with the weld position, the deviation is calculated, and the deviation is used as the weld position adjustment amount at the next moment.

8. The high-precision and high-quality vision-guided robot automatic welding method based on deep learning according to claim 7 is characterized in that: Sending the welding gun position to the robot controller for welding includes: The identified weld position coordinates are transformed according to a transformation matrix to obtain the coordinates in a robot coordinate system; Welding is performed according to the coordinates in the robot coordinate system.

9. The high-precision and high-quality vision-guided robot automatic welding method based on deep learning according to claim 8 is characterized in that: The transformation matrix is: in, is the transformation matrix from the workpiece coordinate system to the six-axis robot base coordinate system, is the transformation matrix from the six-axis robot end coordinate system to the six-axis robot base coordinate system, is the transformation matrix from the workpiece coordinate system to the camera coordinate system, It is the transformation matrix from the six-axis robot end coordinate system to the camera coordinate system.

10. The high-precision and high-quality vision-guided robot automatic welding method based on deep learning according to claim 7, characterized in that: The weld position is sent to the robot controller to adjust the welding gun position in real time, and further includes: comparing the current spatter quantity with the spatter quantity at the previous moment to adjust the welding process parameters; Compare the current spatter quantity with the spatter quantity at the previous moment and adjust the welding process parameters including: The spatter amount of the i-th frame and the i+1-th frame image is calculated. If the current spatter amount decreases, the welding process parameters are adjusted positively; otherwise, the welding inclination angle and the nozzle height are adjusted negatively.

Images