A welding gun posture recognition and adjustment method and system based on three-dimensional vision

Abstract

The present invention discloses a welding gun posture recognition and adjustment method and system based on three-dimensional vision, which relates to the field of automated welding. The method comprises the following steps: capturing image data of a welding environment and constructing a posture recognition model; based on the posture recognition model, extracting the real-time posture features of the welding gun using a three-dimensional recognition algorithm and constructing a real-time path of the welding gun; planning the planned path of the welding gun according to the geometric shape of the welding workpiece and the welding sequence; comparing the real-time path of the welding gun with the planned path, and adjusting the posture of the welding gun based on the comparison result. The present invention can realize high-precision recognition and adjustment of the real-time posture of the welding gun by combining advanced three-dimensional recognition algorithms and deep learning technologies; at the same time, by constructing a posture recognition model, it can not only process spatial data in complex environments, but also achieve faster response speeds and higher operating accuracy, thereby improving the efficiency and quality of welding work.

Description

Technical Field

[0001] The present invention relates to the field of automated welding, and in particular to a welding gun posture recognition and adjustment method and system based on three-dimensional vision. Background Art

[0002] In the rapid development of the manufacturing industry, automation and precision have always been the key factors in promoting efficiency and quality improvement. Traditional manufacturing technology relies heavily on manual operation and simple machinery, which is not only inefficient, but also difficult to ensure product quality. The demand for precision and reliability in the manufacturing industry has led to continuous technological progress, especially in the fields of automation, machine vision, and robotics. As automation equipment and precision instruments are introduced into production lines one after another, these technologies have greatly promoted the innovation of industrial production efficiency and quality control, but also exposed new challenges and needs.

[0003] Taking automated welding as an example, modern welding technology has been able to replace manual welding to a large extent and improve welding speed and repeatability. However, despite significant progress, existing automated welding technology still has some shortcomings. Taking welding gun automated welding technology as an example, the positioning and angle adjustment of the welding gun in traditional automated welding systems are usually not precise enough, which may lead to unstable weld quality; at the same time, traditional welding gun systems often lack advanced vision or sensing technology and cannot accurately identify the position and status of the welding gun, thereby limiting their application on complex or irregular workpieces. These limitations not only affect the welding quality and efficiency of the welding gun, but also limit the potential of automated welding technology in a wider range of applications.

[0004] The drawbacks of existing welding gun automated welding technology are mainly concentrated in flexibility, accuracy and adaptability. In terms of flexibility, the existing system often lacks the necessary adaptability when dealing with variable and complex welding tasks. When the shape or size of the workpiece changes slightly, the fixed welding gun welding path and parameters cannot effectively adapt to these changes, resulting in welding defects; in terms of accuracy, although automation technology has been able to achieve high repeatability welding, the accuracy of the existing system is still insufficient for tasks requiring extremely high precision. This is mainly due to the inaccurate positioning of the welding gun posture and the lack of advanced perception and feedback mechanisms; in terms of adaptability, the existing welding technology is usually sensitive to environmental changes, such as light changes, smoke, dust, etc. These factors will affect the performance of the visual system and thus affect the welding results; these drawbacks not only limit the scope of application of automated welding technology, but also hinder the further improvement of manufacturing efficiency and product quality. Therefore, it is urgent to develop a more advanced, flexible and precise welding technology to improve the flexibility and accuracy of welding gun operations.

[0005] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0006] In response to the problems in the related technology, the present invention proposes a welding gun posture recognition and adjustment method and system based on three-dimensional vision, which has the advantages of achieving faster response speed and higher operating accuracy, significantly improving the efficiency and quality of welding work, and thus solving the problems of insufficient flexibility, accuracy and adaptability in the prior art.

[0007] To this end, the specific technical solution adopted by the present invention is as follows:

[0008] According to one aspect of the present invention, a welding gun posture recognition and adjustment method based on three-dimensional vision is provided, and the welding gun posture recognition and adjustment method based on three-dimensional vision comprises the following steps:

[0009] S1, capture image data of welding environment and build posture recognition model;

[0010] S2. Based on the posture recognition model, the real-time posture features of the welding gun are extracted using the three-dimensional recognition algorithm, and the real-time path of the welding gun is constructed;

[0011] S3. Plan the path of the welding gun according to the geometric shape of the welding workpiece and the welding sequence;

[0012] S4. Compare the real-time path of the welding gun with the planned path, and adjust the posture of the welding gun based on the comparison result.

[0013] Preferably, based on the posture recognition model, the real-time posture features of the welding gun are extracted using a three-dimensional recognition algorithm, and the real-time path of the welding gun is constructed, including the following steps:

[0014] S21, based on the posture recognition model, using the three-dimensional recognition algorithm to assign an identification label to each pixel in the image;

[0015] S22, according to different identification tags, distinguishing effective features and background features in the welding environment and clustering them in turn;

[0016] S23. Extract the posture features of the welding gun from the clustering results, map the posture features of the welding gun back to the posture recognition model, and construct the real-time path of the welding gun.

[0017] Preferably, based on the posture recognition model, using a three-dimensional recognition algorithm to assign an identification label to each pixel in the scene includes the following steps:

[0018] S211, extracting context information between pixels in each neighborhood in the image using a graph neural network;

[0019] S212, extracting features of each pixel using a multi-layer perceptron based on the context information;

[0020] S213, using a pooling operation to combine the features of all pixels to obtain a global feature;

[0021] S214, concatenating the features of each pixel with the global features, and using a multi-layer perceptron to assign an identification label to each pixel.

[0022] Preferably, according to different identification tags, distinguishing effective features from background features in the welding environment and clustering them in sequence comprises the following steps:

[0023] S221, using a deep learning model to learn an identification label for each pixel, and dividing adjacent pixels according to the identification label;

[0024] S222, constructing a similarity matrix based on adjacent pixels, and identifying clustering features around each pixel;

[0025] S223, merging and filtering the clustering features obtained through the similarity matrix, and distinguishing them into effective features and background features.

[0026] Preferably, constructing a similarity matrix based on adjacent pixels and identifying clustering features around each pixel includes the following steps:

[0027] S2221. Based on adjacent pixels, calculate the similarity between each pair of adjacent pixels;

[0028] S2222. Construct a similarity matrix according to the calculation results;

[0029] S2223. Set a similarity threshold, and identify pixel groups with high similarity in combination with a similarity matrix to obtain clustering features around each pixel.

[0030] Preferably, extracting the posture features of the welding gun from the clustering results, mapping the posture features of the welding gun back to the posture recognition model, and constructing the real-time path of the welding gun includes the following steps:

[0031] S231, extracting the posture features of the welding gun from the clustering results, and mapping the posture features of the welding gun back to the posture recognition model;

[0032] S232. Based on the posture recognition model, the residual network algorithm is used to perform coefficient regression on the posture characteristics of the welding gun, and the three-dimensional parameters of the welding gun are predicted;

[0033] S233, reconstructing the three-dimensional geometric shape of the welding gun using a pre-trained decoder according to the predicted three-dimensional parameters of the welding gun, and extracting hierarchical three-dimensional features of the reconstructed welding gun;

[0034] S234. Use a convolutional neural network to fuse the posture features of the welding gun with the hierarchical three-dimensional features, and insert the results into a pre-trained decoder. The decoder generates a real-time path of the welding gun based on the fused features.

[0035] Preferably, according to the geometric shape of the welding workpiece and the welding sequence, planning the planned path of the welding gun includes the following steps:

[0036] S31, constructing a geometric collection of weld points according to geometric shape information of the welding workpiece;

[0037] S32, linearizing the geometric collection of each weld point to obtain a path fitting straight line;

[0038] S33. According to the welding sequence of the welding workpiece, the distance from each weld point in the geometric collection to the path fitting straight line is calculated, and the planned path of the welding gun is planned.

[0039] Preferably, according to the welding sequence of the welding workpieces, calculating the distance from each weld point in the geometric collection to the path fitting straight line, and planning the planned path of the welding gun includes the following steps:

[0040] S331, calculating the distance from each weld point in the geometric collection to the path fitting straight line according to the welding sequence of the welding workpiece;

[0041] S332: Based on the calculation result, determine the distance between the fitting straight line of each path segment and a preset threshold;

[0042] S333. Select the best path fitting straight line according to the judgment result, and plan the planned path of the welding gun.

[0043] Preferably, according to the welding sequence of the welding workpieces, the calculation formula for calculating the distance from each weld point in the geometric collection to the path fitting straight line is:

[0044]

[0045] In the formula, d hf Indicates the first hf The distance from each weld point to the path fitting straight line;

[0046] x hf and y hf Indicates the coordinates of the weld point;

[0047] m and n Represents the parameters of the path fitting line.

[0048] According to another aspect of the present invention, a welding gun posture recognition and adjustment system based on three-dimensional vision is also provided, and the welding gun posture recognition and adjustment system based on three-dimensional vision includes:

[0049] A model building module is used to capture image data of the welding environment and build a posture recognition model;

[0050] A three-dimensional recognition module is used to extract the real-time posture features of the welding gun based on the posture recognition model and use the three-dimensional recognition algorithm to construct the real-time path of the welding gun;

[0051] The path planning module is used to plan the path of the welding gun according to the geometric shape of the welding workpiece and the welding sequence;

[0052] The comparison and adjustment module is used to compare the real-time path of the welding gun with the planned path, and adjust the position and posture of the welding gun based on the comparison results.

[0053] Among them, the model building module is connected to the path planning module through the three-dimensional recognition module, and the path planning module is connected to the comparison and adjustment module.

[0054] The beneficial effects of the present invention are:

[0055] (1) The present invention realizes accurate recognition and adjustment of the welding gun posture by introducing advanced three-dimensional visual recognition technology; at the same time, by using advanced algorithms such as graph neural networks, multi-layer perceptrons, and pooling operations, it can not only accurately assign identification labels to each pixel point, but also extract the posture characteristics of the welding gun from the clustering results, greatly improving the perception ability of the welding environment and the accuracy of data processing; in addition, the present invention also effectively overcomes the limitations of traditional welding technology in complex environments, such as the influence of factors such as light changes and smoke on welding quality, and can achieve faster response speed and higher operation accuracy, and significantly improve the efficiency of welding work.

[0056] (2) The present invention further optimizes the fine adjustment of the welding gun posture through the residual network algorithm and the convolutional neural network, making the welding process more accurate and flexible. This advanced posture adjustment mechanism enables the welding gun to respond to slight changes in the welding process in real time, reducing the occurrence of welding defects and improving welding quality. At the same time, the application of this technology also greatly improves welding efficiency and reduces the need for later correction and grinding work, thereby bringing significant economic benefits to manufacturing companies in terms of production cost and time.

[0057] (3) The present invention can realize efficient and flexible path planning by calculating the distance from the weld point to the path fitting straight line and combining it with the welding sequence of the welding workpiece. Compared with the traditional technology, the adaptability is significantly improved. The traditional welding system often sticks to the preset path and is slow to respond to the slight changes of the workpiece, while the present invention can monitor the changes in the welding gun posture during the welding process in real time and make adjustments quickly. At the same time, by providing a more efficient, accurate and adaptable welding gun posture recognition and adjustment technology, the welding gun is allowed to dynamically adjust the path according to the actual situation, effectively responding to changes in the shape or size of the workpiece, ensuring the continuity and consistency of the welding process, not only improving the accuracy of welding, but also providing the possibility for complex or irregular welding tasks, expanding the application scope of automated welding technology, helping to improve product quality, reduce production costs, and enhance the market competitiveness of enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 is a flow chart of a welding gun posture recognition and adjustment method based on three-dimensional vision according to an embodiment of the present invention;

[0060] Figure 2 The present invention is a block diagram of a welding gun posture recognition and adjustment system based on three-dimensional vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in the field should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0062] According to an embodiment of the present invention, a method and system for recognizing and adjusting welding gun posture based on three-dimensional vision are provided.

[0063] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, the welding gun posture recognition and adjustment method based on three-dimensional vision according to an embodiment of the present invention includes the following steps:

[0064] S1, capture image data of welding environment and build posture recognition model;

[0065] Specifically, a three-dimensional vision sensor is first used to capture image data of the welding environment, which includes the geometric shape of the weldment, the weld position, and the current position of the welding gun. Then, the captured image data is subjected to noise filtering and contrast enhancement to complete data preprocessing operations. A training data set is configured based on the processed data. Then, a deep learning algorithm (a convolutional neural network is used in this embodiment) is used to construct a posture recognition model. The model is trained using the configured training data set so that it can recognize the posture features of the welding gun from the image, thereby completing the construction of the posture recognition model.

[0066] S2. Based on the posture recognition model, the real-time posture features of the welding gun are extracted using the three-dimensional recognition algorithm, and the real-time path of the welding gun is constructed;

[0067] S3. Plan the path of the welding gun according to the geometric shape of the welding workpiece and the welding sequence;

[0068] S4. Compare the real-time path of the welding gun with the planned path, and adjust the posture of the welding gun based on the comparison result.

[0069] Specifically, a path tracking algorithm is used to compare the real-time path of the welding gun with the planned path. The deviation between the real-time path and the planned path is determined by calculating the deviation value between the real-time path and the planned path. The deviation value obtained by comparison is then input into the control system. The control system automatically adjusts the position and direction of the welding gun according to the deviation value to return the welding gun to the planned path. At the same time, a feedback mechanism is set after the auxiliary path tracking algorithm. By continuously comparing the real-time path and the planned path, the welding gun posture is adjusted in real time until the deviation value is reduced to a minimum or reaches the tolerance threshold preset by the feedback mechanism, thereby finally realizing real-time adjustment of the welding gun posture.

[0070] Specifically, the calculation formula for calculating the deviation value between the real-time path and the planned path using the path tracking algorithm is:

[0071]

[0072] In the formula, ΔP Indicates the deviation value between the real-time path and the planned path;

[0073] x ss , y ss and z ss Indicates the position coordinates of the welding gun on the real-time path;

[0074] x jh , y jh and zjh Indicates the position coordinates of the welding gun on the planned path.

[0075] In one embodiment, based on the posture recognition model, the real-time posture features of the welding gun are extracted using a three-dimensional recognition algorithm, and the real-time path of the welding gun is constructed, including the following steps:

[0076] S21, based on the posture recognition model, using the three-dimensional recognition algorithm to assign an identification label to each pixel in the image;

[0077] S22, according to different identification tags, distinguishing effective features and background features in the welding environment and clustering them in turn;

[0078] S23. Extract the posture features of the welding gun from the clustering results, map the posture features of the welding gun back to the posture recognition model, and construct the real-time path of the welding gun.

[0079] In one embodiment, based on the posture recognition model, using a three-dimensional recognition algorithm to assign an identification label to each pixel in the scene includes the following steps:

[0080] S211, extracting context information between pixels in each neighborhood in the image using a graph neural network;

[0081] Specifically, first, the captured image data is converted into a graph structure using a graph neural network, where each pixel is represented as a node in the graph, and the relationship between adjacent pixels is represented as an edge. Then, the graph neural network is used to extract the correlation between pixels in each neighborhood through the information propagation mechanism of nodes and edges. Then, through multiple iterations of the graph neural network, the features of each node will aggregate information from its neighborhood, thereby obtaining a node representation rich in contextual information.

[0082] S212, extracting features of each pixel using a multi-layer perceptron based on the context information;

[0083] Specifically, the contextual features of each pixel obtained by the graph neural network are input into the multilayer perceptron. The multilayer perceptron extracts the high-level feature representation of each pixel through its nonlinear transformation capability. Then, the feature space is gradually transformed through multiple hidden layers of the multilayer perceptron, so that the posture recognition model can extract features that are more beneficial to posture recognition.

[0084] S213, using a pooling operation to combine the features of all pixels to obtain a global feature;

[0085] Specifically, it is necessary to perform a pooling operation on the features of all pixels (the pooling operation used in this embodiment is maximum pooling) in order to extract global features from local features. By capturing the most significant features, the global features can reflect the overall information of the entire image, thereby helping the sensor understand the global context information of the welding scene.

[0086] S214, concatenating the features of each pixel with the global features, and using a multi-layer perceptron to assign an identification label to each pixel.

[0087] Specifically, the local features of each pixel are spliced ​​with the global features, which takes into account both the information of the pixel itself and the contextual information of the entire image. A multi-layer perceptron is then used to process the spliced ​​features and output an identification label for each pixel. This identification label represents the category to which the pixel belongs, such as welding gun, welding workpiece, weld or background.

[0088] In one embodiment, according to different identification tags, distinguishing effective features from background features in a welding environment and clustering them in sequence includes the following steps:

[0089] S221, using a deep learning model to learn an identification label for each pixel, and dividing adjacent pixels according to the identification label;

[0090] Specifically, first, the deep learning model is trained using image data that has been assigned identification labels. The training model recognizes and learns the feature representations of different pixels and associates them with their corresponding identification labels. Then, an identification label is assigned to each pixel in the newly captured image through the trained model, so as to distinguish different elements in the welding environment, such as welding guns, welds, welding workpieces and backgrounds, and ensure that the label assignments of adjacent pixels are logically continuous and consistent, which is convenient for subsequent clustering processing.

[0091] S222, constructing a similarity matrix based on adjacent pixels, and identifying clustering features around each pixel;

[0092] S223, merging and filtering the clustering features obtained through the similarity matrix, and distinguishing them into effective features and background features.

[0093] Specifically, the clustering results obtained from the similarity matrix are first merged to form a larger feature block, and then the merged cluster features are filtered to remove clusters that may be formed by image noise or other irrelevant factors, retaining the valid feature clusters that are closely related to the welding task, and then by analyzing the shape, size and position information of the cluster features, all cluster features are distinguished into valid features (including welding gun features, weld seam features and welding workpiece features) and background features. Among them, the valid features will be used for subsequent welding gun posture adjustment, while the background features can be ignored.

[0094] In one embodiment, constructing a similarity matrix based on adjacent pixels and identifying cluster features around each pixel includes the following steps:

[0095] S2221. Based on adjacent pixels, calculate the similarity between each pair of adjacent pixels;

[0096] S2222. Construct a similarity matrix according to the calculation results;

[0097] S2223. Set a similarity threshold, and identify pixel groups with high similarity in combination with a similarity matrix to obtain clustering features around each pixel.

[0098] Specifically, first, a suitable feature is selected for each pixel (the color feature of the pixel is used in this embodiment), and then for each pair of adjacent pixels, the similarity is calculated according to the similarity of their features. Then, a similarity matrix is ​​initialized, the size of which is the same as the number of pixels in the image, and is filled according to the similarity calculation results between each pair of adjacent pixels to obtain a similarity matrix, each element in the matrix represents the similarity between a pair of pixels, and then the similarity matrix is ​​used to identify the clustering features around each pixel through a clustering algorithm (the K-means clustering algorithm is used in this embodiment). The goal of clustering is to organize pixel groups with high similarity to form independent features for identifying the welding environment. Finally, a similarity threshold is set to determine whether the pixels are similar enough to be classified into one category, and combined with the similarity matrix, pixel groups with high similarity are identified to obtain clustering features around each pixel.

[0099] Specifically, based on adjacent pixels, the calculation formula for calculating the similarity between each pair of adjacent pixels is:

[0100]

[0101] In the formula, SSD(p i ,q j ) Indicates the similarity between adjacent pixels;

[0102] p i and q j Represents adjacent pixels;

[0103] g i and h j The feature vector representing the pixel point;

[0104] k Represents the total number of pixels in the image;

[0105] j Indicates the image j Pixels.

[0106] Specifically, initialize a k×k matrix K, where k is the total number of pixels in the image, and each element of the matrix K pi,qj Will represent the pixel p i and q j The similarity between them is calculated, and the calculated similarity value is filled into the corresponding position of the similarity matrix K. If two pixels are not adjacent, their similarity can be set to infinity or a large number, indicating that there is no direct similarity relationship between them. In this way, according to the calculation results, after completing the filling of the entire matrix, a similarity matrix that fully describes the similarity between all pixels in the image is obtained.

[0107] In one embodiment, extracting the posture features of the welding gun from the clustering results, mapping the posture features of the welding gun back to the posture recognition model, and constructing the real-time path of the welding gun includes the following steps:

[0108] S231, extracting the posture features of the welding gun from the clustering results, and mapping the posture features of the welding gun back to the posture recognition model;

[0109] Specifically, the feature area representing the welding gun is identified from the clustering results, the shape, size and position of the welding gun in the image are detected, and then the visual features of the welding gun are converted into feature vectors that the model can understand, and the extracted welding gun features are mapped back to the posture recognition model.

[0110] S232. Based on the posture recognition model, the residual network algorithm is used to perform coefficient regression on the posture characteristics of the welding gun, and the three-dimensional parameters of the welding gun are predicted;

[0111] Specifically, the residual network algorithm is used to process the extracted welding gun posture features, effectively deal with the detail changes in the image, and improve the accuracy of posture recognition. The coefficient regression of the welding gun posture features is performed through the network to predict the three-dimensional parameters of the welding gun, including the position coordinates and rotation angle of the welding gun.

[0112] S233, reconstructing the three-dimensional geometric shape of the welding gun using a pre-trained decoder according to the predicted three-dimensional parameters of the welding gun, and extracting hierarchical three-dimensional features of the reconstructed welding gun;

[0113] S234. Use a convolutional neural network to fuse the posture features of the welding gun with the hierarchical three-dimensional features, and insert the results into a pre-trained decoder. The decoder generates a real-time path of the welding gun based on the fused features.

[0114] In one embodiment, planning a planned path of a welding gun according to the geometric shape of the welding workpiece and the welding sequence includes the following steps:

[0115] S31, constructing a geometric collection of weld points according to geometric shape information of the welding workpiece;

[0116] Specifically, first, the geometric shape information of the input welding workpiece is analyzed in detail, including the size, shape and position of the workpiece, and then a geometric collection of weld points is constructed based on the collected information. This collection includes the position information of all weld points and their relationship with the workpiece geometry.

[0117] S32, linearizing the geometric collection of each weld point to obtain a path fitting straight line;

[0118] Specifically, the geometric collection of weld points is linearized (the least squares method is used in this embodiment) to convert these points into a series of points that can be fitted into straight lines. Based on the results of the linearization process, one or more path fitting straight lines are generated. These straight lines represent the ideal paths that the welding gun may move along the workpiece.

[0119] Specifically, the formula for linearizing the geometric set of weld points is:

[0120]

[0121]

[0122] In the formula, l Indicates the number of weld points in the geometry collection;

[0123] x hf and y hf Indicates the coordinates of the weld point;

[0124] m and n Represents the parameters of the path fitting line.

[0125] S33. According to the welding sequence of the welding workpiece, the distance from each weld point in the geometric collection to the path fitting straight line is calculated, and the planned path of the welding gun is planned.

[0126] In one embodiment, according to the welding sequence of the welding workpieces, calculating the distance from each weld point in the geometric collection to the path fitting straight line and planning the planned path of the welding gun includes the following steps:

[0127] S331, calculating the distance from each weld point in the geometric collection to the path fitting straight line according to the welding sequence of the welding workpiece;

[0128] S332: Based on the calculation result, determine the distance between the fitting straight line of each path segment and a preset threshold;

[0129] S333. Select the best path fitting straight line according to the judgment result, and plan the planned path of the welding gun.

[0130] In one embodiment, according to the welding sequence of the welding workpieces, the calculation formula for calculating the distance from each weld point in the geometric collection to the path fitting straight line is:

[0131]

[0132] In the formula, d hf Indicates the first hf The distance from each weld point to the path fitting straight line;

[0133] x hf and y hf Indicates the coordinates of the weld point;

[0134] m and n Represents the parameters of the path fitting line.

[0135] like Figure 2 As shown, according to another embodiment of the present invention, a welding gun posture recognition and adjustment system based on three-dimensional vision is also provided, and the welding gun posture recognition and adjustment system based on three-dimensional vision includes:

[0136] Model building module 1, used to capture image data of the welding environment and build a posture recognition model;

[0137] The three-dimensional recognition module 2 is used to extract the real-time posture features of the welding gun by using the three-dimensional recognition algorithm based on the posture recognition model, and to construct the real-time path of the welding gun;

[0138] The path planning module 3 is used to plan the planned path of the welding gun according to the geometric shape of the welding workpiece and the welding sequence;

[0139] The comparison and adjustment module 4 is used to compare the real-time path of the welding gun with the planned path, and adjust the posture of the welding gun based on the comparison result.

[0140] The model building module 1 is connected to the path planning module 3 via the three-dimensional recognition module 2 , and the path planning module 3 is connected to the comparison and adjustment module 4 .

[0141] In order to facilitate understanding of the above technical solutions of the present invention, the working principle or operation mode of the present invention in the actual process is described in detail below.

[0142] The present invention discloses a welding gun posture recognition and adjustment method and system based on three-dimensional vision, which relates to the field of automated welding. The method comprises the following steps: capturing image data of a welding environment and constructing a posture recognition model; based on the posture recognition model, extracting the real-time posture features of the welding gun using a three-dimensional recognition algorithm and constructing a real-time path of the welding gun; planning the planned path of the welding gun according to the geometric shape of the welding workpiece and the welding sequence; comparing the real-time path of the welding gun with the planned path, and adjusting the posture of the welding gun based on the comparison result. The present invention can realize high-precision recognition and adjustment of the real-time posture of the welding gun by combining advanced three-dimensional recognition algorithms and deep learning technologies; at the same time, by constructing a posture recognition model, it can not only process spatial data in complex environments, but also achieve faster response speeds and higher operating accuracy, thereby improving the efficiency and quality of welding work.

[0143] In practical application, first, before welding, the staff uses a laser scanner to comprehensively scan the welding workpiece to obtain accurate geometric shape information. This information is used to construct a geometric collection of weld points to provide basic data for the path planning of the welding gun. Then, these geometric collections are linearized to generate path fitting lines. The best path fitting line is selected by distance judgment to plan the planned path of the welding gun. Then, during the welding process, the monocular vision sensor installed on the welding gun is used to capture the position and posture information of the welding gun in real time. Then, advanced image processing technology is used to extract the posture features of the welding gun from the clustering results. And map it back to the posture recognition model, which uses the residual network algorithm to perform coefficient regression on the posture features of the welding gun and predict the three-dimensional parameters of the welding gun, including position coordinates and rotation angles. At the same time, based on the predicted three-dimensional parameters, the system uses a pre-trained decoder to reconstruct the three-dimensional geometric shape of the welding gun and extract hierarchical three-dimensional features from it. These features are fused together with the posture features and input into the decoder to generate the real-time path of the welding gun. In the actual welding process, the system dynamically adjusts the real-time path of the welding gun according to the welding sequence to ensure consistency with the planned path, which not only improves the welding quality, but also significantly improves efficiency.

[0144] Taking the body welding workshop of a large automobile manufacturing company as an example, when welding the key parts of the automobile chassis, the present invention achieves a more uniform and precise welding effect by precisely controlling the position of the welding gun. The specific implementation process is as follows:

[0145] First, a high-precision laser scanner is used to scan the chassis comprehensively to obtain accurate geometric shape information. Based on this data, the system constructs a geometric collection containing more than 3,000 welds. Through the linearization processing algorithm of the present invention, 50 path fitting lines are generated, and the optimal welding path is finally planned based on the welding sequence and distance judgment.

[0146] At the same time, during the welding process, the monocular vision sensor installed on the welding gun captures the welding gun position and posture information in real time at a speed of 60 frames per second. The posture recognition model of the present invention can complete the recognition and adjustment of the welding gun posture within 0.02 seconds, ensuring the continuity and accuracy of the welding process.

[0147] Compared with the traditional welding method, the present invention has significant advantages in the following aspects:

[0148] ① Accuracy: The welding accuracy of the present invention is improved by 35%, and the weld deviation is reduced from the original ±1.5mm to ±0.98mm, which greatly improves the welding quality.

[0149] ② Efficiency: The welding time was shortened by 28%. The welding time of a standard chassis was reduced from 45 minutes to 32 minutes, which significantly improved production efficiency.

[0150] ③ Stability: The failure rate during welding was reduced by 42%, and the average number of failures per 100 hours of operation was reduced from 5.2 times to 3.0 times, which greatly improved the stability of the production line.

[0151] The large manufacturing company said that after using the invention, the welding quality of the automobile workshop has been significantly improved, the production efficiency has been significantly improved, and the customer satisfaction has been greatly increased. Specifically, the first-time qualified rate of products has increased from 92% to 98%, the monthly production capacity has increased from 8,000 units to 10,200 units, and the customer complaint rate has dropped by 60%.

[0152] In addition, the application of the present invention has also brought significant economic benefits. It is estimated that the company can save about 20 million yuan per year in terms of reducing rework and improving production efficiency. At the same time, due to the improvement of product quality, the company's competitiveness in the market has been enhanced, and the number of new orders has increased by 15% year-on-year.

[0153] It can be seen from the above application examples that the present invention has demonstrated excellent performance and huge economic value in practical applications, providing strong support for the intelligent upgrading of the automobile manufacturing industry.

[0154] In summary, with the help of the above technical scheme of the present invention, by introducing advanced three-dimensional visual recognition technology, accurate recognition and adjustment of the welding gun posture is achieved. By using advanced algorithms such as graph neural networks, multi-layer perceptrons, and pooling operations, not only can identification labels be accurately assigned to each pixel point, but also the posture characteristics of the welding gun can be extracted from the clustering results, which greatly improves the perception ability of the welding environment and the accuracy of data processing. In addition, the present invention also effectively overcomes the limitations of traditional welding technology in complex environments, such as the influence of factors such as light changes and smoke on welding quality, and can achieve faster response speed and higher operation accuracy, and significantly improve the efficiency of welding work; the present invention further optimizes the fine adjustment of the welding gun posture through the residual network algorithm and the convolutional neural network, making the welding process more accurate and flexible. This advanced posture adjustment mechanism enables the welding gun to respond to slight changes in the welding process in real time, reduces the occurrence of welding defects, and improves the welding quality. The application of this technology is also great The welding efficiency is greatly improved, and the need for later correction and grinding work is reduced, thereby bringing significant economic benefits to manufacturing companies in terms of production cost and time; the present invention can realize efficient and flexible path planning by calculating the distance from the weld point to the path fitting straight line, combined with the welding sequence of the welding workpiece. Compared with the traditional technology, the adaptability is significantly improved. The traditional welding system often sticks to the preset path and is slow to respond to small changes in the workpiece, while the present invention can monitor the changes in the welding gun posture during the welding process in real time and make adjustments quickly, and by providing a more efficient, accurate and adaptable welding gun posture recognition and adjustment technology, the welding gun is allowed to dynamically adjust the path according to the actual situation, effectively respond to changes in the shape or size of the workpiece, and ensure the continuity and consistency of the welding process. It not only improves the accuracy of welding, but also provides possibilities for complex or irregular welding tasks, expands the application scope of automated welding technology, helps to improve product quality, reduce production costs, and enhance the market competitiveness of enterprises.

[0155] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A welding gun posture recognition and adjustment method based on three-dimensional vision, characterized in that: The welding gun posture recognition and adjustment method based on three-dimensional vision comprises the following steps: S1, capture image data of welding environment and build posture recognition model; S2. Based on the posture recognition model, the real-time posture features of the welding gun are extracted using the three-dimensional recognition algorithm, and the real-time path of the welding gun is constructed; S3. Plan the path of the welding gun according to the geometric shape of the welding workpiece and the welding sequence; S4, comparing the real-time path of the welding gun with the planned path, and adjusting the posture of the welding gun based on the comparison result; The method of extracting the real-time posture features of the welding gun by using a three-dimensional recognition algorithm based on the posture recognition model and constructing the real-time path of the welding gun includes the following steps: S21, based on the posture recognition model, using the three-dimensional recognition algorithm to assign an identification label to each pixel in the image; S22, distinguishing effective features and background features in the welding environment according to different identification tags and clustering them in sequence; S23, extracting the posture features of the welding gun from the clustering results, mapping the posture features of the welding gun back to the posture recognition model, and constructing the real-time path of the welding gun; The method of assigning an identification label to each pixel in the image using a three-dimensional recognition algorithm based on a posture recognition model comprises the following steps: S211, extracting context information between pixels in each neighborhood in the image using a graph neural network; S212, extracting features of each pixel using a multi-layer perceptron based on the context information; S213, using a pooling operation to combine the features of all pixels to obtain a global feature; S214, concatenating the features of each pixel with the global features, and using a multi-layer perceptron to assign an identification label to each pixel.

2. The welding gun posture recognition and adjustment method based on three-dimensional vision according to claim 1 is characterized in that: The method of distinguishing effective features from background features in a welding environment and clustering them in sequence according to different identification tags includes the following steps: S221, using a deep learning model to learn an identification label for each pixel, and dividing adjacent pixels according to the identification label; S222, constructing a similarity matrix based on adjacent pixels, and identifying clustering features around each pixel; S223, merging and filtering the clustering features obtained through the similarity matrix, and distinguishing them into effective features and background features.

3. The welding gun posture recognition and adjustment method based on three-dimensional vision according to claim 2 is characterized in that: The process of constructing a similarity matrix based on adjacent pixels and identifying cluster features around each pixel includes the following steps: S2221. Based on adjacent pixels, calculate the similarity between each pair of adjacent pixels; S2222. Construct a similarity matrix according to the calculation results; S2223. Set a similarity threshold, and identify pixel groups with high similarity in combination with a similarity matrix to obtain clustering features around each pixel.

4. The welding gun posture recognition and adjustment method based on three-dimensional vision according to claim 1 is characterized in that: The step of extracting the posture features of the welding gun from the clustering results, mapping the posture features of the welding gun back to the posture recognition model, and constructing the real-time path of the welding gun includes the following steps: S231, extracting the posture features of the welding gun from the clustering results, and mapping the posture features of the welding gun back to the posture recognition model; S232. Based on the posture recognition model, the residual network algorithm is used to perform coefficient regression on the posture characteristics of the welding gun, and the three-dimensional parameters of the welding gun are predicted; S233, reconstructing the three-dimensional geometric shape of the welding gun using a pre-trained decoder according to the predicted three-dimensional parameters of the welding gun, and extracting hierarchical three-dimensional features of the reconstructed welding gun; S234. Use a convolutional neural network to fuse the posture features of the welding gun with the hierarchical three-dimensional features, and insert the results into a pre-trained decoder. The decoder generates a real-time path of the welding gun based on the fused features.

5. The welding gun posture recognition and adjustment method based on three-dimensional vision according to claim 1 is characterized in that: The method of planning the path of the welding gun according to the geometric shape of the welding workpiece and the welding sequence comprises the following steps: S31, constructing a geometric collection of weld points according to geometric shape information of the welding workpiece; S32, linearizing the geometric collection of each weld point to obtain a path fitting straight line; S33. According to the welding sequence of the welding workpiece, the distance from each weld point in the geometric collection to the path fitting straight line is calculated, and the planned path of the welding gun is planned.

6. The welding gun posture recognition and adjustment method based on three-dimensional vision according to claim 5 is characterized in that: The method of calculating the distance from each weld point in the geometric collection to the path fitting straight line according to the welding sequence of the welding workpiece and planning the planned path of the welding gun includes the following steps: S331, calculating the distance from each weld point in the geometric collection to the path fitting straight line according to the welding sequence of the welding workpiece; S332: Based on the calculation result, determine the distance between the fitting straight line of each path segment and a preset threshold; S333. Select the best path fitting straight line according to the judgment result, and plan the planned path of the welding gun.

7. The welding gun posture recognition and adjustment method based on three-dimensional vision according to claim 6 is characterized in that: The calculation formula for calculating the distance from each weld point in the geometric collection to the path fitting straight line according to the welding sequence of the welding workpiece is: In the formula, d hf Indicates the first hf The distance from each weld point to the path fitting straight line; x hf and y hf Indicates the coordinates of the weld point; m and n Represents the parameters of the path fitting line.

8. A welding gun posture recognition and adjustment system based on three-dimensional vision, used to implement the welding gun posture recognition and adjustment method based on three-dimensional vision according to any one of claims 1 to 7, characterized in that: The welding gun posture recognition and adjustment system based on three-dimensional vision includes: A model building module is used to capture image data of the welding environment and build a posture recognition model; A three-dimensional recognition module is used to extract the real-time posture features of the welding gun based on the posture recognition model and use the three-dimensional recognition algorithm to construct the real-time path of the welding gun; The path planning module is used to plan the path of the welding gun according to the geometric shape of the welding workpiece and the welding sequence; A comparison and adjustment module is used to compare the real-time path of the welding gun with the planned path, and adjust the posture of the welding gun based on the comparison result; Among them, the model building module is connected to the path planning module through the three-dimensional recognition module, and the path planning module is connected to the comparison and adjustment module.

Images