Robot and welding machine set automatic control system and method

By using a complete automated control system for robots and welding machines, and employing deep learning technology to correct welding deviations in real time, the health threats and quality instability issues of traditional manual welding have been resolved, achieving efficient and stable welding results.

CN118990513AInactive Publication Date: 2024-11-22KINGREE WELDING TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411402503.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-09
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional manual welding poses health threats, high-temperature radiation, and difficulty in maintaining consistent welding quality, failing to meet the high demands of modern manufacturing.

Method used

An automated control system integrating robots and welding machines is adopted. The system acquires images of the welded products and the position coordinates of the robotic arm through cameras, and uses deep learning technology for feature extraction and correlation analysis to correct welding deviations in real time and optimize welding quality and consistency.

Benefits of technology

Improve welding quality and consistency, reduce human intervention and errors, and enhance overall production quality and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118990513A_ABST
    Figure CN118990513A_ABST
Patent Text Reader

Abstract

The invention relates to the field of welding machine complete-set robots, and particularly discloses a robot and welding machine complete-set automatic control system and method.The method comprises the steps that firstly, a welding product image collected by a camera and welding position coordinates of a mechanical arm of a welding machine complete-set robot are obtained; and feature extraction and correlation analysis are carried out on the two parameters, and finally, a decoding value is obtained through a decoder so as to obtain a welding coordinate at the next moment, so that the deviation in the welding process is corrected in time, the welding robot set can dynamically adjust and optimize the welding result in the actual operation, and the overall welding quality and consistency are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of complete sets of robots for welding machines, and more specifically, to a complete set of automatic control systems and methods for robots and welding machines. Background Art

[0002] Welding is a process that combines metals or thermoplastic materials by applying heat, pressure, or a combination of both. This process typically uses a filler material such as welding rod to create a molten weld pool that cools to form a strong connection. Welding technology is widely used in manufacturing, construction and maintenance industries to connect, repair parts or create new products.

[0003] The high temperature and strong radiation in the welding environment pose a threat to workers' health. In addition, traditional manual welding has low work efficiency and difficult to stabilize welding quality, which can no longer meet the high requirements of modern manufacturing industry for welding quality.

[0004] Therefore, a complete set of automated control systems and methods for robots and welding machines are desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a robot and welding machine complete set automatic control system and method, which first obtains the welding product image collected by the camera and the welding position coordinates of the welding machine complete set robot arm, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally obtains the decoding value through the decoder to obtain the welding coordinates at the next moment, so as to correct the deviation in the welding process in time, so that the welding machine complete set robot can be dynamically adjusted in actual operation, optimize the welding results, and improve the overall welding quality and consistency.

[0006] According to one aspect of the present application, a complete set of automatic control systems for a robot and a welding machine is provided, comprising:

[0007] A welding machine robot data acquisition module is used to acquire the welding product image collected by the camera and the welding position coordinates of the welding machine robot arm;

[0008] A welding robot data processing module is used to extract a welding product weld area feature vector and a robot arm welding position coordinate semantic feature vector from the welding product image captured by the camera and the welding position coordinate of the robot arm of the welding robot;

[0009] The welding coordinate decoding module of the welding robot set is used to obtain the welding coordinates at the next moment based on the weld area feature vector of the welding product and the semantic feature vector of the welding position coordinates of the robot arm.

[0010] According to another aspect of the present application, a complete set of automated control methods for a robot and a welding machine is provided, comprising:

[0011] Obtain the welding product image captured by the camera and the welding position coordinates of the welding machine robot arm;

[0012] Extracting a welding product weld area feature vector and a robot arm welding position coordinate semantic feature vector from the welding product image captured by the camera and the welding position coordinates of the welding machine robot robot arm;

[0013] Based on the weld area feature vector of the welding product and the semantic feature vector of the welding position coordinate of the robot arm, the welding coordinates at the next moment are obtained.

[0014] Compared with the prior art, the present application provides a complete set of robot and welding machine automation control system and method, which first obtains the welding product image captured by the camera and the welding position coordinates of the welding machine robot arm, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally obtains the decoding value through the decoder to obtain the welding coordinates at the next moment, so as to correct the deviation in the welding process in time, so that the welding machine robot can be dynamically adjusted in actual operation, optimize the welding results, and improve the overall welding quality and consistency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 It is a block diagram of a complete set of automated control systems for robots and welding machines according to an embodiment of the present application.

[0017] Figure 2 The block diagram is a data processing module of a complete set of robots for welding machines in a complete set of automation control systems for robots and welding machines according to an embodiment of the present application.

[0018] Figure 3 It is a block diagram of a welding product feature extraction unit in a complete set of robot and welding machine automation control systems according to an embodiment of the present application.

[0019] Figure 4 It is a block diagram of a welding coordinate decoding module of a complete set of robots for welding machines in a complete set of automation control systems for robots and welding machines according to an embodiment of the present application.

[0020] Figure 5It is a flow chart of a complete set of automated control methods for robots and welding machines according to an embodiment of the present application.

[0021] Figure 6 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0023] Figure 1 : is a block diagram of a robot and welding machine complete set of automatic control systems according to an embodiment of the present application. Figure 1 As shown, according to the robot and welding machine complete set automation control system 100 of the embodiment of the present application, it includes: a welding machine complete set robot data acquisition module 110, which is used to obtain the welding product image collected by the camera and the welding position coordinates of the welding machine complete set robot mechanical arm; a welding machine complete set robot data processing module 120, which is used to extract the welding product weld area feature vector and the mechanical arm welding position coordinate semantic feature vector from the welding product image collected by the camera and the welding position coordinates of the welding machine complete set robot mechanical arm; a welding coordinate decoding module 130 of the welding machine complete set robot, which is used to obtain the welding coordinates at the next moment based on the welding product weld area feature vector and the mechanical arm welding position coordinate semantic feature vector.

[0024] In the above-mentioned robot and welding machine complete set automation control system 100, the welding machine complete set robot data acquisition module 110 is used to obtain the welding product image collected by the camera and the welding position coordinates of the welding machine complete set robot arm. It should be understood that welding is a process for connecting metals or thermoplastic materials together by heating, applying pressure or a combination of the two. In this process, filler materials such as welding rods are usually used to form a molten welding pool, which is cooled to form a strong connection. Welding technology is widely used in manufacturing, construction and maintenance industries, mainly for connecting, repairing parts or producing new products. However, the welding environment is usually accompanied by high temperature and strong radiation, which poses a potential threat to the health of workers. In addition, the traditional manual welding method has low working efficiency, and the welding quality is difficult to maintain consistency, which cannot meet the strict requirements of modern manufacturing industry for welding quality. Therefore, in the technical solution of the present application, by acquiring the welding product image captured by the camera and the welding position coordinates of the welding machine robot arm, and combining with deep learning technology, the welding coordinates at the next moment are obtained, so as to timely correct the deviation in the welding process, so that the welding robot can make dynamic adjustments in actual operation, thereby optimizing the welding effect and improving the overall welding quality and consistency.

[0025] Specifically, the welding robot is an automation solution that integrates welding robots, welding equipment and control systems, aiming to improve the efficiency and consistency of welding operations. The robot automatically completes the welding task by precisely controlling the welding path and parameters, while the welding equipment provides the required power and materials, and the control system is responsible for monitoring and adjusting the various parameters in the welding process to ensure welding quality and production stability. Among them, obtaining the welding product image captured by the camera and the welding position coordinates of the welding robot arm are crucial for modern automated welding systems. First, the images captured by the camera provide the system with real-time visual data of the welding product, which contains important information such as the appearance of the welding product, the position of the weld and its status. On the other hand, the welding position coordinates of the robot arm provide the precise position data of the welding robot in space. By tracking and recording the welding position of the robot arm in real time, the welding path can be adjusted in time to avoid welding deviation, thereby improving welding accuracy and efficiency and achieving a higher level of automated control and optimization.

[0026] In the above-mentioned robot and welding machine complete set automation control system 100, the welding machine complete set robot data processing module 120 is used to extract the welding product weld area feature vector and the mechanical arm welding position coordinate semantic feature vector from the welding product image collected by the camera and the welding position coordinate of the mechanical arm of the welding machine complete set robot. It should be understood that the use of the welding product weld area feature vector and the mechanical arm welding position coordinate semantic feature vector in combination can achieve more intelligent welding control. The system can perform real-time analysis and decision-making based on the actual state of the weld area and the current position of the mechanical arm, so that the system can complete the welding task with higher efficiency and accuracy, while reducing human intervention and errors, and improving the overall production quality and benefits.

[0027] In summary, extracting feature vectors from welding product images and robot arm position coordinates is the core step to realize the intelligent and automated welding process, which not only improves the accuracy and consistency of welding, but also optimizes the efficiency and quality of the entire welding operation.

[0028] Figure 2 FIG. 1 is a block diagram of a data processing module of a complete set of robots and welding machines in a complete set of automatic control systems according to an embodiment of the present application. Figure 2 As shown, in a specific embodiment of the present application, the welding robot data processing module 120 includes: a welding product feature extraction unit 121, which is used to extract features from the welding product image captured by the camera to obtain the welding product weld area feature vector; a welding position coordinate feature encoding unit 122, which is used to feature encode the welding position coordinates of the welding robot manipulator arm to obtain the manipulator arm welding position coordinate semantic feature vector.

[0029] It should be understood that the welded product images contain rich visual information about the weld, including the location, shape, size, and possible defects or inconsistencies of the weld. Through feature extraction technology, feature vectors of the weld area can be extracted from these images. The feature extraction process usually includes steps such as image denoising, edge detection, and region segmentation. These operations are designed to enhance the weld information in the image and remove irrelevant noise and interference. Through these processes, clear images of the weld area can be obtained, and relevant features such as the weld contour, width, location, and contrast with the surrounding area can be extracted from them.

[0030] Furthermore, the robot needs to move and position accurately in three-dimensional space during the welding process to ensure that the welding head can accurately reach the predetermined welding position. The main purpose of feature coding of welding position coordinates is to convert these position data into semantic feature vectors, which can better describe the spatial position, posture and motion trajectory of the robot. Feature coding digitizes the position coordinates and simplifies complex spatial data into a form that can be directly used for analysis and decision-making. These semantic feature vectors contain information such as the position, angle, speed, etc. of the robot in space, providing key information for subsequent automated control and optimization.

[0031] Figure 3 FIG. 1 is a block diagram of a welding product feature extraction unit in a complete set of robot and welding machine automation control systems according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the welding product feature extraction unit 121 includes: a welding product image preprocessing subunit 1211, used to perform data preprocessing on the welding product image collected by the camera to obtain a welding product denoising and enhanced initial feature map; a welding product feature encoding subunit 1212, used to perform feature encoding on the welding product denoising and enhanced initial feature map to obtain the welding product weld area feature vector.

[0032] It should be understood that the images of welding products captured by the camera usually contain various noises and interferences, such as lighting changes, background clutter, and image blur, which may affect the recognition and analysis of weld features. Data preprocessing of welding product images includes denoising, which aims to remove these unnecessary noises to ensure that the weld area in the image is clearer. Commonly used denoising techniques include filtering algorithms (such as Gaussian filtering, median filtering, etc.), which can effectively reduce random noise and interference in the image and retain important structures and features in the image.

[0033] Furthermore, the core purpose of feature coding is to convert the preprocessed image information into a simplified and informative representation, so that the system can analyze and utilize the data more effectively. In the preprocessing stage of the welded product image, the interference and noise in the original image are significantly reduced, and the contrast and details of the weld area are improved. The purpose of this stage is to ensure the clarity of the image and the discernibility of the features, laying a solid foundation for subsequent feature extraction and coding. However, although the denoising and enhancement processing improves the quality of the image, the original image still has a large amount of data and complexity, and directly using these images for analysis and decision-making may not be efficient enough. Through feature coding, the system converts the preprocessed image into feature vectors, which are a more compact and informative representation. Feature coding technology (such as the feature extraction layer in the convolutional neural network) can extract the key information in the image into a set of digital features, which can effectively represent various attributes of the weld area, including the shape, size, position and texture characteristics of the weld. The benefit of feature coding is that it can significantly reduce the dimension of the data while retaining the core feature information of the weld area. This makes subsequent analysis and processing more efficient.

[0034] In a specific embodiment of the present application, the welding product image preprocessing subunit 1211 includes: passing the welding product image captured by the camera through a welding product image denoising module based on an autoencoder to obtain a welding product denoising and enhanced image; passing the welding product denoising and enhanced image through a welding product denoising and enhanced convolutional feature extractor to obtain the welding product denoising and enhanced initial feature map.

[0035] It should be understood that the autoencoder consists of two main parts: the encoder and the decoder. The encoder's task is to compress the input high-dimensional image data into a low-dimensional latent feature representation, while the decoder reconstructs these latent features back to a form close to the original image. In the denoising task, the autoencoder is trained to recover a clean image from a noisy image. The basic principle is to continuously optimize the network parameters by comparing the difference between the input image and the target denoised image, thereby improving the clarity and quality of the image. Specifically, the denoising module of the autoencoder first maps the input noisy image through the encoder to a low-dimensional latent space, in which the noise information is minimized and the main features of the welded product are retained. Then, the decoder reconstructs these latent features into a denoised image, restoring the details of the real welded product as much as possible. This process not only removes random noise in the image, but also improves the overall quality of the image, making the details of the weld area more obvious. The autoencoder can self-adjust according to the noise characteristics and image details in the training data, and automatically learn the most suitable denoising strategy. This method is more flexible and effective than traditional denoising techniques (such as simple filters), and can significantly reduce noise interference while retaining important features of the image. Specifically, the welding product image captured by the camera is input into the encoder of the welding product image denoising module based on the autoencoder, wherein the encoder uses a convolution layer to perform explicit spatial encoding on the welding product image captured by the camera to obtain image features; and the image features are input into the decoder of the welding product image denoising module based on the autoencoder, wherein the decoder uses a deconvolution layer to perform deconvolution processing on the image features to obtain the welding product denoising enhanced image.

[0036] Furthermore, in the processing flow of welding product images, the denoising and enhanced images have been processed by the autoencoder, which has significantly reduced the noise and improved the clarity of the image. However, in order to further analyze the weld features in the image, it is still necessary to extract more representative features through a convolutional feature extractor. The convolutional feature extractor processes the image layer by layer through multiple convolutional layers, and can automatically identify and extract key features in the image, such as edges, textures, shapes, etc. These features are crucial for subsequent welding quality inspection and defect analysis. The working principle of the convolutional feature extractor is to slide the convolution kernel (filter) on the image, calculate the convolution operation and generate a feature map. Each convolution kernel can extract specific features in the image, such as edges or corners, thereby converting the original image into a feature map containing important features. This process can convert complex image data into a form that is easier to analyze, making feature recognition more accurate and efficient. Specifically, each layer of the welding product denoising and enhancement convolutional feature extractor is used to perform convolution processing, mean pooling processing based on the local feature matrix, and nonlinear activation processing on the input data in the forward pass of the layer, so that the last layer of the welding product denoising and enhancement convolutional feature extractor outputs the welding product denoising and enhancement initial feature map, wherein the input of the welding product denoising and enhancement convolutional feature extractor is the welding product denoising and enhancement image.

[0037] In a specific embodiment of the present application, the welding product feature encoding subunit 1212 includes: passing the welding product denoising and enhanced initial feature map through a welding product weld target detection network module to obtain a welding product weld region of interest; passing the welding product weld region of interest through a welding product weld feature encoder based on a deep residual network to obtain a welding product weld region feature vector.

[0038] It should be understood that when processing welding product images, the denoising and enhanced initial feature maps have significantly improved the image quality and extracted key information through the denoising and feature extraction steps. However, these feature maps still contain a lot of background information and irrelevant details, so it is necessary to further screen and locate the region of interest of the weld through the target detection network. The main task of the target detection network is to identify the areas related to the weld from these feature maps, which usually include the weld itself and possible defects or inconsistencies. The target detection network module usually includes two main components: the region proposal network (RPN) and the target classification and localization network. The region proposal network is responsible for generating candidate regions on the feature map, which may contain target objects (in this case, welds). These candidate regions are further processed and classified by the network to determine whether each region contains a weld, and accurately locate the position, size and specific shape of the weld. Through the target detection network module, the weld area in the welding product image can be separated from other irrelevant parts, so that the system can focus on analyzing the quality and characteristics of the weld. Accordingly, in a specific example of the present application, the welding product weld target detection network module is Fast R-CNN, Faster R-CNN or RetinaNet. Specifically, the welding product weld target detection network module is used to process the welding product denoising and enhanced initial feature map using the following target detection formula to obtain the welding product weld region of interest;

[0039] Among them, the target detection formula is:

[0040] ROI=(cls(ψ det ,B),Regr(ψ det ,B))

[0041] Among them, ψ det is the initial feature map for welding product denoising and enhancement, B is the anchor frame, ROI is the weld region of interest for welding products, cls(ψ det ,B) indicates classification, Regr(ψ det ,B) indicates regression.

[0042] Furthermore, in the process of welding product image processing, the target detection network module has identified and located the regions of interest of the weld. Although these regions have been extracted from the image, in order to further analyze and understand the quality and characteristics of the weld, in-depth feature encoding of these regions is required. The role of the feature encoder is to convert the key information in the weld area into a high-dimensional feature vector, which can be used for subsequent analysis and decision-making. As an advanced deep learning architecture, the deep residual network has significant advantages. In ResNet, the network solves the gradient vanishing and gradient exploding problems faced by traditional deep networks during training by introducing residual blocks. The residual block enables the network to learn the residual between the input data and the output data by adding skip connections. This structure not only improves the training efficiency of the network, but also enhances the feature extraction ability of the network. Specifically, the region of interest of the weld of the welding product is input into the feature encoder based on the deep residual network, and the network can extract and integrate the features of the weld area at multiple levels. The deep residual network gradually extracts the detailed information of the weld area through layers of deep convolution and residual connection, including the shape, texture, boundary features, etc. of the weld. These feature information are integrated into a feature vector, which contains the key features of the weld area and can comprehensively and efficiently represent the state of the weld. More specifically, each layer of the welding product weld feature encoder based on the deep residual network performs the following operations on the input data in the forward pass of the layer: convolution processing is performed on the input data based on the convolution kernel to generate a convolution feature map; global mean pooling processing is performed on the convolution feature map based on the feature matrix to generate a pooling feature map; and nonlinear activation is performed on the feature values ​​at each position in the pooling feature map to generate an activation feature augmentation map; wherein the output of the last layer of the welding product weld feature encoder based on the deep residual network is the feature vector of the welding product weld area, the input of the second layer to the last layer of the welding product weld feature encoder based on the deep residual network is the output of the previous layer, and the input of the welding product weld feature encoder based on the deep residual network is the welding product weld region of interest.

[0043] In a specific embodiment of the present application, the welding position coordinate feature encoding unit 122 includes: passing the welding position coordinates of the welding machine robot arm through a robot arm welding position coordinate semantic encoder to obtain a plurality of robot arm welding position coordinate feature vectors; and splicing the plurality of robot arm welding position coordinate feature vectors into the robot arm welding position coordinate semantic feature vector.

[0044] It should be understood that in the welding robot system, the position coordinate data of the robot arm contains the specific position and posture information of each robot arm during the welding process. These coordinate data are usually in the original numerical form. Although they can reflect the position of the robot arm, directly using these data for subsequent analysis and decision-making may not be efficient and accurate enough. In order to better understand and utilize these data, they need to be converted into feature vectors, so that the position information can be effectively integrated with other system parameters to improve the overall control and optimization capabilities of the welding process. The role of the robot arm welding position coordinate semantic encoder is to map these original position coordinate data to a higher-dimensional feature space. The semantic encoder can extract feature vectors representing the position characteristics of the robot arm by performing complex transformations and processing on the input coordinate data. These feature vectors not only retain the spatial position information of the robot arm, but also integrate and express the position relationship, motion mode and welding task requirements between the robot arms. Specifically, the welding position coordinates of the welding machine robot arm are segmented to obtain a welding position coordinate word sequence; the embedding layer of the welding position coordinate semantic encoder of the robot arm is used to map each welding position coordinate word in the welding position coordinate word sequence into a word embedding vector to obtain a sequence of welding position coordinate word embedding vectors; the converter-based Bert model of the robot arm welding position coordinate semantic encoder is used to perform global context semantic encoding on the sequence of welding position coordinate word embedding vectors to obtain multiple feature vectors.

[0045] Furthermore, through the concatenation operation, the feature vectors of all the robotic arms are connected into a longer feature vector, which can simultaneously retain the position data of each robotic arm and fuse them into a high-dimensional feature representation. This comprehensive feature vector can reflect the relative position, motion coordination and overall working status of the robotic arms.

[0046] In the above-mentioned robot and welding machine complete set automation control system 100, the welding machine complete set robot welding coordinate decoding module 130 is used to obtain the welding coordinates at the next moment based on the welding product weld area feature vector and the robotic arm welding position coordinate semantic feature vector. It should be understood that the welding product weld area feature vector provides a high-level summary of key information such as weld shape, position, texture, etc. These feature vectors can reveal detailed information about the weld area, such as the actual contour of the weld, possible defects, and the degree of matching with the welding standard. The robotic arm welding position coordinate semantic feature vector reflects the specific position, posture, and coordination state of the robotic arm in the current welding task. Through these two feature vectors, the system can comprehensively consider the actual needs of the welding product and the operating state of the robotic arm. The weld area feature vector and the robotic arm position coordinate feature vector are input into the prediction model, usually using machine learning or optimization algorithms, and the system can calculate the optimal welding coordinates at the next moment. These models take into account the weld characteristics and the current state of the robotic arm, and adjust the motion path and posture of the robotic arm through the optimization algorithm to ensure the continuity and accuracy of the welding process. Further significantly improve welding quality and efficiency. Predicting the welding coordinates at the next moment allows the robot arm to accurately position and adjust within each welding cycle, thereby achieving high-quality welding operations and reducing the possibility of defects and rework.

[0047] Figure 4 FIG. 1 is a block diagram of a welding coordinate decoding module of a complete set of robots and welding machines in a complete set of automatic control systems for robots and welding machines according to an embodiment of the present application. Figure 4 As shown, in a specific embodiment of the present application, the welding coordinate decoding module 130 of the welding robot assembly includes: a welding robot assembly feature association unit 131, which is used to perform local structural collapse adjustment on the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector based on feature alignment to obtain the welding coordinate decoding feature vector at the next moment; a welding coordinate decoding unit 132, which is used to pass the welding coordinate decoding feature vector at the next moment through a decoder to obtain a decoding value, and the decoding value is used to represent the welding coordinates at the next moment.

[0048] It should be understood that by correlating feature vectors, a comprehensive model can be constructed to predict the welding coordinates at the next moment. This model usually uses machine learning algorithms or optimization methods to calculate the optimal welding position and path based on the actual requirements of the weld and the current state of the robot arm. This prediction capability can significantly improve welding accuracy and efficiency.

[0049] In particular, in the technical solution of the present application, the weld area feature vector of the welding product is obtained by image processing technology, involving image denoising, convolution feature extraction and target detection steps. These features have high-dimensional and complex spatial structure information. On the other hand, the semantic feature vector of the robot arm welding position coordinate is generated by the position coordinate data, which is processed by the position coordinate semantic encoder, focusing on the specific position and motion trajectory of the robot arm in space, emphasizing the spatiotemporal position information of the robot arm. Its dimension and representation are different from the image features, and are usually simple and numerical. This difference leads to fine-grained dimensional offset when fusing the weld area feature vector of the welding product and the semantic feature vector of the robot arm welding position coordinate, that is, the asynchrony between the spatial features and the position features contained in the image feature vector may make the features unable to be effectively aligned when simply fused, and unable to accurately reflect the key information of the two data sources. In this way, the effectiveness of some areas is reduced, which affects the performance of the decoder, making the decoded value unable to accurately predict the welding coordinates at the next moment, thereby affecting the precise control of the welding process. Therefore, in the technical solution of the present application, a local structural collapse adjustment based on feature alignment is performed on the weld area feature vector of the welding product and the semantic feature vector of the welding position coordinate of the robot arm to obtain the welding coordinate decoding feature vector at the next moment.

[0050] Among them, the local structure collapse adjustment based on feature alignment is performed on the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector to obtain the welding coordinate decoding feature vector at the next moment, including: calculating the position difference between the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector to obtain the feature alignment differential feature vector; calculating the position point multiplication between the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector to obtain the feature alignment position point multiplication feature vector; calculating the position addition between the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector to obtain the feature alignment position addition feature vector; adding the feature alignment differential feature vector, the feature The aligned positional dot product feature vector and the feature alignment positional addition feature vector are input into a one-dimensional convolution layer and maximum pooling is performed to obtain a feature alignment convolutional coding pooling feature vector; the square root of the two norm of the feature alignment convolutional coding pooling feature vector is weighted and added with the one norm of the feature alignment convolutional coding pooling feature vector to obtain a feature alignment factor; the feature alignment positional dot product feature vector and the square root of the length of the weld area feature vector of the welding product are added positionally to obtain a feature alignment intermediate vector; the feature alignment factor is used as the first weighting coefficient and the second weighted hyperparameter is used as the second weighting coefficient to calculate the positional weighted sum of the feature alignment positional addition feature vector and the feature alignment intermediate vector to obtain the welding coordinate decoding feature vector at the next moment.

[0051] Among them, the local structure collapse adjustment based on feature alignment is performed on the weld area feature vector of the welding product and the semantic feature vector of the welding position coordinate of the robot arm to obtain the welding coordinate decoding feature vector at the next moment, including:

[0052]

[0053]

[0054] Among them, V1 represents the weld area feature vector of the welding product, V2 represents the semantic feature vector of the welding position coordinates of the robot arm, represents positional subtraction, ⊙ represents positional multiplication, represents positional addition, conv1D represents a one-dimensional convolutional layer, MaxPool represents maximum pooling, V μ represents the feature alignment convolutional coding pooling feature vector, ||·||1 represents the first norm, ||·||2 represents the second norm, α represents the first weighted hyperparameter, τ represents the feature alignment factor, L represents the length of the weld area feature vector of the welding product, β represents the second weighted hyperparameter, V f Represents the decoding feature vector of the welding coordinates at the next moment.

[0055] In the technical solution of the present application, in the process of fusing the welding product weld area feature vector and the robotic arm welding position coordinate semantic feature vector, the fine-grained feature alignment between the welding product weld area feature vector and the robotic arm welding position coordinate semantic feature vector is taken into consideration, which may cause the fused welding coordinate decoding feature vector at the next moment to have local structural collapse. Based on this, in the technical solution of the present application, the welding product weld area feature vector and the robotic arm welding position coordinate semantic feature vector are adjusted for local structural collapse based on feature alignment.

[0056] Specifically, based on the restricted expression of the foreground-background structure distinction of the high-dimensional manifold of the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector by the amplitude of the semantic difference vector between the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector, the key feature fine-grained correspondence between the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector is modeled, and then the active fine-grained correspondence adjustment propagation is performed through the back-propagation feature alignment information between the corresponding eigenvalues ​​of the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector, so as to avoid the imbalance at the feature vector level of the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector in a focusing manner, thereby synchronizing the micro-dimensions of the two feature vectors, ensuring that in the fusion process, the information between the feature vectors can be effectively aligned and combined, thereby avoiding the abnormal accumulation of local structures.

[0057] Furthermore, the decoder is a specially designed model that maps high-dimensional feature vectors to actual welding coordinates. Through the decoder, the abstract information in the feature vector is converted into specific numerical values, which are the decoded values. The decoded values ​​usually include key information such as the target position and posture angle of the robot arm at the next moment, which can clearly indicate where the robot arm needs to move to for welding, so as to significantly improve the accuracy and consistency of the welding process and reduce human errors and operational deviations.

[0058] In another specific embodiment of the present application, a welding robot set is provided. It should be understood that the welding robot set is an industrial robot engaged in welding (including cutting and spraying). The industrial robot is a multi-purpose, reprogrammable automatic control manipulator (Manipulator) with three or more programmable axes, used in the field of industrial automation.

[0059] Among them, in another specific embodiment of the present application, the control mode of the welding machine robot is mainly point motion control and trajectory motion control; the number of memory points is 160,000 points; the number of programs is 9999 (each program can have 999 statements); the memory content mainly includes points, straight lines, arcs, conditional commands (welding conditions, swing arc conditions, time limits, I / O, etc.); the external memory is through the control cabinet USB memory card socket, which can be used for program backup; it has four coordinate systems (joint, right angle, tool and workpiece coordinate system), and has the function of setting any coordinate system, which is convenient for users to use; and it has the functions of editing, copying, inserting, correcting, removing, checking, renaming, etc., and has the functions of program translation, program jumping, transfer, etc., which is convenient to use; further, TCP constant speed control is adopted. In this way, it can reduce operating costs, improve company management, further improve employee work quality, reduce personnel turnover, and enhance production flexibility, save production space, and at the same time, improve product quality and consistency, meet safety regulations, and further expand production and reduce investment costs.

[0060] In summary, the embodiment of the present application first obtains the welding product image collected by the camera and the welding position coordinates of the welding robot arm, and then uses deep learning technology to perform feature extraction and correlation analysis on the two. Finally, a decoder is used to obtain a decoding value to obtain the welding coordinates at the next moment, thereby promptly correcting deviations in the welding process, so that the welding robot can be dynamically adjusted in actual operation, optimize welding results, and improve overall welding quality and consistency.

[0061] As described above, the robot and welding machine complete set of automation control system 100 according to the embodiment of the present application can be implemented in various terminal devices. In one example, the robot and welding machine complete set of automation control system 100 can be integrated into the terminal device as a software module and / or a hardware module. For example, the robot and welding machine complete set of automation control system 100 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the robot and welding machine complete set of automation control system 100 can also be one of the many hardware modules of the terminal device.

[0062] Alternatively, in another example, the robot and welding machine complete automation control system 100 and the terminal device may also be separate devices, and the robot and welding machine complete automation control system 100 may be connected to the terminal device via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0063] Figure 5 Flow chart of the complete set of automated control methods for robots and welding machines according to an embodiment of the present application. Figure 5As shown, according to the robot and welding machine complete automation control method of the embodiment of the present application, it includes: S110, acquiring the welding product image captured by the camera and the welding position coordinates of the welding machine complete robot arm; S120, extracting the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector from the welding product image captured by the camera and the welding position coordinates of the welding machine complete robot arm; S130, based on the welding product weld area feature vector and the robot arm welding position coordinate semantic feature vector, obtaining the welding coordinates at the next moment.

[0064] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned robot and welding machine complete set of automated control methods have been described in detail above. Figures 1 to 4 The robot and welding machine complete automation control system are described in detail, and therefore, their repeated description will be omitted.

[0065] Below, reference Figure 6 To describe an electronic device according to an embodiment of the present application.

[0066] Figure 6 is a block diagram of an electronic device according to an embodiment of the present application. Figure 6 , which shows a schematic diagram of the structure of an electronic device suitable for implementing a terminal device or a server in an embodiment of the present application.

[0067] like Figure 6As shown, the electronic device includes one or more processors, a communication unit, etc. The one or more processors are, for example: one or more central processing units (CPUs), and / or one or more graphics processing units (GPUs), etc. The processor can perform various appropriate actions and processes according to executable instructions stored in a read-only memory (ROM) or executable instructions loaded from a storage part to a random access memory (RAM). The communication unit may include but is not limited to a network card, and the network card may include but is not limited to an IB (Infiniband) network card. The processor may communicate with a read-only memory and / or a random access memory to execute executable instructions, and is connected to the communication unit through a bus and communicates with other target devices through the communication unit, thereby completing the operation corresponding to any method provided in the embodiments of the present application, for example, the primary shard manager shards the features in the feature queue; for any shard, the primary shard manager stores the any shard in the corresponding shard feature library, and records the mapping relationship between the any shard and the corresponding shard feature library in a mapping table; in response to a failure of the primary shard manager, a shard manager is reselected from at least one slave shard manager as the new primary shard manager; the new primary shard manager obtains shard information of the shard currently being written from all shard feature libraries, and recreates the mapping relationship of the shard currently being written based on the shard information of the shard currently being written in all shard feature libraries.

[0068] In addition, various programs and data required for the operation of the device can also be stored in the RAM. The CPU, ROM and RAM are connected to each other via a bus. In the case of RAM, the ROM is an optional module. The RAM stores executable instructions, or writes executable instructions into the ROM at runtime, and the executable instructions enable the processor to perform operations corresponding to any of the above methods of the present invention. The input / output (I / O) interface is also connected to the bus. The communication unit can be integrated or can be configured to have multiple submodules (e.g., multiple IB network cards) and be linked on the bus.

[0069] The following components are connected to the I / O interface: an input section including a keyboard, a mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a LAN card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that a computer program read therefrom is installed into the storage section as needed.

[0070] If necessary, Figure 6The architecture shown is only an optional implementation method. In the specific practice, the above can be modified according to actual needs. Figure 6 The number and type of components can be selected, deleted, added or replaced; different functional components can be set up by separate settings or integrated settings, for example, the GPU and CPU can be set up separately or the GPU can be integrated on the CPU, the communication unit can be set up separately or integrated on the CPU or GPU, etc. These replaceable implementations all fall within the protection scope disclosed in the present invention.

[0071] In addition, an embodiment of the present invention further provides a computer program, including computer-readable code. When the computer-readable code runs on a device, a processor in the device executes instructions for implementing each step in the data processing method of any of the above embodiments of the present invention.

[0072] In addition, an embodiment of the present invention further provides a computer-readable storage medium for storing computer-readable instructions, which, when executed, implement the operations of each step in the data processing method of any of the above embodiments of the present invention.

[0073] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0074] The method and apparatus of the present invention may be implemented in many ways. For example, the method and apparatus of the present invention may be implemented by software, hardware, firmware, or any combination of software, hardware, firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present invention are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present invention may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0075] The description of the present invention is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the invention to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for particular uses.

Claims

1. A complete set of automatic control systems for robots and welding machines, It is characterized in that include: A welding machine robot data acquisition module is used to acquire the welding product image collected by the camera and the welding position coordinates of the welding machine robot arm; A welding robot data processing module is used to extract a welding product weld area feature vector and a robot arm welding position coordinate semantic feature vector from the welding product image captured by the camera and the welding position coordinate of the robot arm of the welding robot; The welding coordinate decoding module of the welding robot set is used to obtain the welding coordinates at the next moment based on the weld area feature vector of the welding product and the semantic feature vector of the welding position coordinates of the robot arm.

2. According to claim 1, the robot and welding machine complete set of automatic control system, It is characterized in that The welding machine complete robot data processing module comprises: A welding product feature extraction unit, used for performing feature extraction on the welding product image acquired by the camera to obtain a feature vector of the weld area of ​​the welding product; The welding position coordinate feature encoding unit is used to perform feature encoding on the welding position coordinates of the welding machine robot arm to obtain a semantic feature vector of the welding position coordinates of the robot arm.

3. The robot and welding machine complete automation control system according to claim 2, It is characterized in that The welding product feature extraction unit comprises: A welding product image preprocessing subunit, used for performing data preprocessing on the welding product image acquired by the camera to obtain a denoising and enhanced initial feature map of the welding product; The welding product feature encoding subunit is used to perform feature encoding on the denoising and enhanced initial feature map of the welding product to obtain a feature vector of the weld area of ​​the welding product.

4. The robot and welding machine complete automation control system according to claim 3, It is characterized in that The welding product image preprocessing subunit comprises: Passing the welding product image captured by the camera through a welding product image denoising module based on an autoencoder to obtain a welding product denoising enhanced image; The welding product denoising and enhanced image is passed through a welding product denoising and enhanced convolutional feature extractor to obtain the welding product denoising and enhanced initial feature map.

5. According to claim 4, the robot and welding machine complete set of automatic control system, It is characterized in that The welding product feature coding subunit includes: The denoised and enhanced initial feature map of the welding product is passed through a welding product weld target detection network module to obtain a welding product weld region of interest; The welding product weld region of interest is passed through a welding product weld feature encoder based on a deep residual network to obtain a welding product weld region feature vector.

6. The robot and welding machine complete set of automatic control system according to claim 5, It is characterized in that The welding position coordinate feature encoding unit comprises: The welding position coordinates of the robot arm of the welding machine complete set are passed through a robot arm welding position coordinate semantic encoder to obtain a plurality of robot arm welding position coordinate feature vectors; The multiple robot arm welding position coordinate feature vectors are spliced ​​into the robot arm welding position coordinate semantic feature vector.

7. The robot and welding machine complete automation control system according to claim 6, It is characterized in that The welding coordinate decoding module of the welding machine complete robot comprises: The welding robot feature association unit is used to perform local structural collapse adjustment based on feature alignment on the weld area feature vector of the welding product and the semantic feature vector of the welding position coordinate of the robot arm to obtain the decoding feature vector of the welding coordinate at the next moment; The welding coordinate decoding unit is used to pass the welding coordinate decoding feature vector at the next moment through a decoder to obtain a decoding value, and the decoding value is used to represent the welding coordinate at the next moment.

8. The robot and welding machine complete set of automatic control system according to claim 7, It is characterized in that The welding machine robot feature association unit comprises: Calculating the positional difference between the weld area feature vector of the welding product and the semantic feature vector of the welding position coordinates of the robot arm to obtain a feature alignment differential feature vector; Calculating the position point multiplication between the weld area feature vector of the welding product and the semantic feature vector of the welding position coordinates of the robot arm to obtain a feature alignment position point multiplication feature vector; Calculate the positional addition between the weld area feature vector of the welding product and the semantic feature vector of the welding position coordinate of the robot arm to obtain a feature alignment positional addition feature vector; Inputting the feature alignment differential feature vector, the feature alignment positional dot product feature vector and the feature alignment positional addition feature vector into a one-dimensional convolutional layer and performing maximum pooling to obtain a feature alignment convolutional coding pooling feature vector; The square root value of the two-norm of the feature-aligned convolutional coding pooling feature vector is weighted and added to the one-norm of the feature-aligned convolutional coding pooling feature vector to obtain a feature alignment factor; The feature alignment is performed by adding the square root of the feature vector length of the weld area feature vector of the welding product according to the position point multiplication to obtain a feature alignment intermediate vector; The feature alignment factor is used as the first weighting coefficient, the second weighted hyperparameter is used as the second weighting coefficient, and the position-weighted sum of the feature alignment position-added feature vector and the feature alignment intermediate vector is calculated to obtain the welding coordinate decoding feature vector at the next moment.

9. A complete set of automatic control methods for robots and welding machines, It is characterized in that include: Obtain the welding product image captured by the camera and the welding position coordinates of the welding machine robot arm; Extracting a welding product weld area feature vector and a robot arm welding position coordinate semantic feature vector from the welding product image captured by the camera and the welding position coordinates of the welding machine robot robot arm; Based on the weld area feature vector of the welding product and the semantic feature vector of the welding position coordinate of the robot arm, the welding coordinates at the next moment are obtained.

10. The robot and welding machine complete set of automated control methods according to claim 9, It is characterized in that Extracting a welding product weld area feature vector and a robot arm welding position coordinate semantic feature vector from the welding product image captured by the camera and the welding position coordinate of the welding machine robot robot arm, including: Performing feature extraction on the welding product image captured by the camera to obtain a feature vector of the weld area of ​​the welding product; The welding position coordinates of the robot arm of the welding machine are feature-encoded to obtain a semantic feature vector of the welding position coordinates of the robot arm.

Citation Information

Cited By

  • Precise operation method and system of explosion-proof spraying robot driven by space-time coding

    CN120941425A