Welding robot design method and system for steel structure workpiece welding seam tracking

Abstract

The invention provides a welding robot design method and system for steel structure workpiece welding seam tracking, and relates to the technical field of steel structure workpiece welding seam visual tracking and automatic welding. Steel structure workpiece image data collected by an industrial camera in real time is divided into two groups; training a weld groove detection network based on YOLOv3 by using the marked image data to obtain a weld groove real-time detection model based on YOLOv3, and optimizing and deploying the model; and the online group uses the model to infer the position of the weld groove, determines the coordinates of the inferred weld groove and the welding gun of the welding robot under the base coordinate system through sensor position calibration, and uses a spline interpolation method to generate the movement track of the welding gun of the welding robot in real time. By means of the process, the teaching-reproduction mode of an existing robot is broken, the problem that a robot pre-programming mode deals with complex changes of field working conditions is solved, and high automation of real-time tracking tasks of the weld groove is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of visual tracking of weld seams of steel structure workpieces and automated welding, and in particular to a welding robot design method and system for tracking weld seams of steel structure workpieces. Background Art

[0002] Steel structure workpieces are widely used in large factories, stadiums, super high-rise buildings and other fields. The welding accuracy of steel structure workpieces is an important indicator for evaluating welding quality. With the continuous improvement of industrial production technology and its degree of automation, traditional manual welding technology can no longer meet the precision requirements in the actual production of steel structure workpieces. At present, robotic automated welding is mainly implemented in a "teach-reproduce" manner. The staff needs to compile a control program according to the robot's tasks and motion trajectories in advance, and input the control program into the controller. The controller controls the robot to complete the specified task according to the instruction sequence predetermined by the control program. When the welding task changes, the control program needs to be modified or recompiled. The on-site welding conditions of steel structure workpieces are constantly changing, and the automation and intelligence of robotic welding face huge challenges. Researching a welding robot design method and system for steel structure workpiece weld tracking is an effective means to achieve automation and intelligence of steel structure weld groove welding.

[0003] Therefore, it is necessary to provide an improved technical solution to the above-mentioned deficiencies in the prior art. Summary of the Invention

[0004] The purpose of this application is to provide a welding robot design method and system for steel structure workpiece weld tracking to solve or alleviate the problems existing in the above-mentioned prior art.

[0005] In order to achieve the above objectives, this application provides the following technical solutions: This application provides a welding robot design method for tracking weld seams of steel structure workpieces, including: The steel structure weld groove image data collected by industrial cameras in real time is divided into two groups for processing: the offline group is manually marked with weld grooves, and the online group is processed in real time by the subsequent YOLOv3-based weld groove real-time detection model. The industrial camera acquires the steel structure workpiece weld groove image data in real time online; The manually labeled steel structure workpiece weld groove image data is used to train the weld groove detection network based on YOLOv3, and a real-time weld groove detection model based on YOLOv3 is obtained; Optimize and deploy a real-time weld groove detection model based on YOLOv3; Using the real-time weld groove detection model based on YOLOv3, the weld groove position is inferred through online group reasoning; Through sensor position calibration, the coordinates of the inferred weld groove and the welding robot's welding gun in the base coordinate system are determined; The spline interpolation method is used to generate the moving trajectory of the welding robot's welding gun in real time.

[0006] Preferably, the YOLOv3-based weld groove real-time detection model is optimized, including: The trained YOLOv3-based weld groove real-time detection model is a float32 precision model. Float32 precision means that the model weights are stored in the float32 data type and quantized to int8 precision using the int8 precision quantization method. Identifying redundant operators generated by the YOLOv3-based weld groove real-time detection model during the training phase, and performing layer fusion processing on adjacent layers according to the computational graph dependency of the redundant operators to obtain a fusion processing result; The default network structure of the YOLOv3-based weld groove real-time detection model uses general computing operators. The general computing operator structure is reconstructed and a graphics card-level multi-threaded parallel optimization method is implemented through manual programming. The resulting computing operator is adapted to graphics card-level hardware.

[0007] Preferably, the reasoning of the weld groove position includes: The collected steel structure weld groove image data is used as input data, and the input data is normalized and preprocessed. The preprocessed input data is input into a real-time weld groove detection model based on YOLOv3. The weld groove positions of flat welds and fillet welds in the steel structure weld groove image data are inferred to obtain real-time weld groove detection results.

[0008] Preferably, the spline interpolation method is used to generate the moving trajectory of the welding robot's welding gun in real time, specifically: The real-time weld groove detection result and the relative coordinates of the welding robot's welding gun in the base coordinate system are used as input data, and the spline interpolation method is used to generate the welding robot's welding gun movement trajectory in real time.

[0009] Preferably, the weld groove detection network based on YOLOv3 is constructed as follows: For the weld groove detection scenario of steel structure workpieces, a weld groove detection network based on YOLOv3 is designed to optimize the weld groove features of flat welds and fillet welds. To meet the demand for accurate recognition of flat and fillet weld grooves, we used steel structure weld groove image data collected by a large number of industrial cameras as input, introduced a residual network structure, and obtained a sub-network of the YOLOv3-based weld groove detection network, namely a dedicated backbone feature extraction network for the welding field. Aiming at the complex and dynamic on-site working environment in the welding field, a multi-feature graph scale loss optimization function is set for the positions of sensors, robots and robot end flange tools relative to steel structure workpieces to detect the features extracted by the dedicated backbone feature extraction network.

[0010] Preferably, the dedicated backbone feature extraction network includes: By increasing the network depth, the residual network structure is introduced to construct a dedicated backbone feature extraction network for the welding field.

[0011] Preferably, the multi-feature graph scale loss optimization function includes: The positive sample weld coordinate loss function indicates the accuracy of the weld detection position: , in, represents the coordinate loss function, and Represents the x coordinates of the center points of the predicted box and the real box respectively, and Represents the y coordinates of the center points of the predicted box and the real box respectively, and Represent the width of the predicted box and the real box respectively, and Represent the height of the predicted box and the real box respectively, B represents the number of detection boxes corresponding to each point in the prediction matrix, S 2 Represents the number of elements in the prediction matrix, i represents traversing the prediction matrix, and j represents traversing all detection boxes of each node in the prediction matrix; The positive sample weld confidence loss function indicates the accuracy of the weld prediction confidence: , in, represents the confidence loss function, Indicates the confidence level of the box predicted as a positive sample, B indicates the number of prediction boxes corresponding to each point in the prediction matrix, S 2 Represents the number of elements in the prediction matrix, i represents traversing the prediction matrix, and j represents traversing all detection boxes of each node in the prediction matrix; The positive sample weld classification loss function indicates the accuracy of weld prediction classification: , in, represents the classification loss function, and Represent the classification probabilities of the predicted box and the true box respectively, B represents the number of predicted boxes corresponding to each point in the prediction matrix, S 2Represents the number of elements in the prediction matrix, i represents traversing the prediction matrix, j represents traversing all detection boxes of each node in the prediction matrix, n represents the number of prediction types, and k represents the total number of traversed detection types; The overall loss function is expressed as: , The overall loss function is used as the loss optimization function for multiple feature map scales, and multi-scale training strategies are constructed on three feature map sizes: 13×13, 26×26, and 52×52.

[0012] The present disclosure also provides a welding robot system for tracking weld seams of steel structure workpieces. The system is used to execute the steps of the welding robot design method for tracking weld seams of steel structure workpieces, including: Weld groove data acquisition module, used to obtain the weld groove of steel structure workpiece in real time, including: offline weld groove data collection module and real-time weld groove data acquisition module; The weld groove marking and review module is used to manually mark and review the image data collected by the offline weld groove data collection module; Real-time detection model training module, used to train the real-time weld groove detection model based on YOLOv3, including: feature extraction network submodule, weld detection network submodule and detection network output decoupling submodule; The model optimization and deployment module is used to optimize and stabilize the YOLOv3-based real-time weld groove detection model, including: a public general operator optimization and deployment submodule and a weld detection-specific operator optimization and deployment submodule; The model reasoning and detection module is used to reason about the real-time acquired steel structure workpiece weld images, including: a steel structure workpiece weld preprocessing submodule, a weld groove detection submodule, and a weld detection post-processing submodule; The welding gun trajectory planning module is used to generate the welding gun motion trajectory according to the weld groove output by the detection model and the relative position of the welding robot welding gun in the base coordinate system, including: a sensor position calibration submodule and a robot welding gun path planning submodule.

[0013] Compared with the closest prior art, the technical solution of the embodiment of the present application has the following beneficial effects: 1. This invention combines offline and online data collection, from real-time image data transmission back to a cloud server, to manual weld bevel annotation, to real-time weld bevel detection model training based on YOLOv3, and finally to automated model deployment. This significantly reduces the development cycle and difficulty of the detection algorithm, effectively improving detection efficiency. Continuous manual annotation and model training continuously optimize the model and enhance detection accuracy.

[0014] 2. The YOLOv3-based real-time weld groove detection model of the present invention can provide high-precision visual capabilities for weld groove detection. After integrating it into the real-time autonomous tracking model of weld grooves, it can calculate the movement trajectory of the robot's end welding gun online, solve system abnormalities caused by changes in the on-site environment or emergencies, and improve the stability of the model.

[0015] 3. This application breaks the existing robot "teaching-reproduction" method from data collection and manual labeling to model training and deployment, and then to model detection and robot trajectory planning. It solves the problem of robots' pre-programmed methods coping with complex changes in on-site working conditions and realizes the high degree of automation of the real-time tracking task of weld grooves. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings and descriptions that constitute part of this application are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. Among them: Figure 1 This is a flow chart of a welding robot design method for tracking weld seams of steel structure workpieces provided according to an embodiment of the present application.

[0017] Figure 2 The present invention provides a flow chart of a training method for a welding robot design method for tracking weld seams of steel structure workpieces according to an embodiment of the present application.

[0018] Figure 3 The present invention provides a flow chart of a reasoning method for designing a welding robot for tracking weld seams of steel structure workpieces according to an embodiment of the present application.

[0019] Figure 4 This is a schematic diagram of module training of a welding robot system for tracking weld seams of steel structure workpieces provided according to an embodiment of the present application.

[0020] Figure 5 The present invention provides a schematic diagram of module reasoning for a welding robot system for tracking weld seams of steel structure workpieces according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments. Each example is provided by way of explanation of the present application and does not limit the present application. In fact, it will be clear to those skilled in the art that modifications and variations can be made in the present application without departing from the scope or spirit of the present application. For example, a feature shown or described as part of one embodiment can be used in another embodiment to produce yet another embodiment. Therefore, it is expected that the present application includes such modifications and variations within the scope of the appended claims and their equivalents.

[0022] In the following description, the terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first / second / third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art in the art of the present disclosure. The terms used herein are only for the purpose of describing the embodiments of the present disclosure and are not intended to limit the present disclosure.

[0024] In the description of this application, the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing this application and do not require that this application must be constructed and operated in a specific orientation. Therefore, they should not be understood as limitations on this application. The terms "connected", "connected", and "set" used in this application should be understood in a broad sense. For example, they can be fixed connections or detachable connections; they can be directly connected or indirectly connected through intermediate components; they can be wired electrical connections, radio connections, or wireless communication signal connections. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0025] Reference Figure 1 As shown, the embodiment of the present application provides a welding robot design method for tracking weld seams of steel structure workpieces, which specifically includes: The steel structure weld groove image data collected by industrial cameras in real time is divided into two groups for processing: the offline group is manually marked with weld grooves, and the online group is processed in real time by the subsequent YOLOv3-based weld groove real-time detection model. The industrial camera acquires the steel structure workpiece weld groove image data in real time online; This welding robot design method for steel structure workpiece weld tracking is built based on the weld groove detection model of YoLov3.

[0026] Real-time data collection is performed through an industrial camera. For example, the industrial camera may be an area array color industrial camera.

[0027] The steel structure workpiece is transported to the welding station, and the steel structure workpiece located in the welding station is photographed by an area array color industrial camera to obtain the weld groove image of the steel structure workpiece. The collected steel structure workpiece weld groove image data is divided into two groups for processing: one group is an offline group and the other group is an online group. The steel structure workpiece weld groove image data in the offline group is added to the training image pool. When the number of steel structure images in the training image pool reaches a certain number, the steel structure image data in the training image pool is uploaded to the cloud server. Then, the steel structure workpiece weld groove data is obtained by manually annotating and reviewing the types and locations of weld grooves such as flat welds and fillet welds in the steel structure workpiece images. The steel structure workpiece weld groove data is then used as input to train a YOLOv3-based real-time detection model for steel structure workpiece weld grooves. The trained YOLOv3-based real-time detection model for steel structure workpiece weld grooves is optimized and deployed on an industrial computer. Among them, the real-time detection model of weld grooves for steel structure workpieces based on YOLOv3 is a real-time detection model of weld grooves based on YOLOv3.

[0028] The online steel workpiece weld groove image data is collected as input data. The images are first normalized and preprocessed. These preprocessed images are then fed into a YOLOv3-based real-time weld groove detection model for steel workpieces deployed on a central control computer. The YOLOv3-based real-time weld groove detection model is updated in real time using the online, real-time weld groove image data. It infers the locations of flat welds, fillet welds, and other weld grooves in the weld groove images, filtering out low-confidence weld grooves to obtain high-confidence weld groove locations. After sensor position calibration, the relative positions of the weld groove and the robot's welding gun tip in the base coordinate system are calculated. Using a spline interpolation algorithm, the robot's welding gun tip movement trajectory is generated in real time.

[0029] From acquiring steel workpiece images to training, deploying, and inferencing a YOLOv3-based real-time weld groove detection model for steel workpieces, and then to planning the robot's welding gun trajectory, the system follows a pre-programmed process, enabling it to effectively and autonomously track weld grooves in steel workpieces. This programmed, automated, and intelligent real-time weld groove tracking effectively replaces the "teach-and-play" approach used in robotic welding automation. By generating real-time welding gun trajectories based on changing task objectives and field conditions during steel workpiece welding, it helps improve welding precision and ensures better control of welding quality.

[0030] Further, Figure 2A flow chart of a training method for a welding robot design method for tracking weld seams of steel structure workpieces according to an embodiment of this specification is shown. The method includes the following steps: Step S110: When a steel structure workpiece passes through a welding station, an area array color industrial camera is used to obtain a weld image of the steel structure workpiece in real time, and image data of the steel structure workpiece is collected offline; In a specific embodiment, a color industrial camera is used to take pictures of a steel structure workpiece, and images of the weld grooves of the steel structure workpiece are obtained from an acquisition card for offline collection. The collected images of the weld grooves of the steel structure workpiece are placed in a training image data pool, waiting to be uploaded to a cloud storage server.

[0031] Step S120: uploading the steel structure workpiece image training data to the cloud server in real time, manually marking the weld grooves in the steel structure workpiece images, and training a weld groove real-time detection network model based on YOLOv3; In a specific embodiment, when the offline collected steel workpiece weld groove images reach the data pool capacity threshold, the steel workpiece weld groove images are uploaded to the cloud storage server in real time. The steel workpiece weld groove images reaching the cloud storage server need to be manually labeled and the type and position of the weld grooves are reviewed, which is of great help to the subsequent model training. The manually labeled steel workpiece weld groove data is used to train a weld groove detection network based on YOLOv3. This network is an improved network under the YOLOv3 general network and is suitable for the detection of weld grooves such as fillet welds and flat welds. The weld groove detection network based on YOLOv3 is constructed as follows: For the weld groove detection scenario of steel structure workpieces, a weld groove detection network based on YOLOv3 is designed to optimize the weld groove features of flat welds and fillet welds. In response to the demand for accurate recognition of flat weld and fillet weld grooves, the steel structure weld groove image data collected by a large number of industrial cameras is used as input, and a residual network structure is introduced to obtain a sub-network of the YOLOv3-based weld groove detection network, that is, a dedicated backbone feature extraction network for the welding field. The dedicated backbone feature extraction network is constructed by introducing a residual network structure in a way that increases the network depth.

[0032] In view of the complex and dynamic on-site operation environment in the welding field, a multi-feature map scale loss optimization function is set for the position of sensors, robots and robot end flange tools relative to steel structure workpieces to detect the features extracted by the dedicated backbone feature extraction network, which solves the problem of changes in the relative spatial position of the tool and the target and improves the algorithm's adaptability.

[0033] The multi-feature map scale loss optimization functions used mainly include the following: The positive sample weld coordinate loss function indicates the accuracy of the weld detection position: , in, represents the coordinate loss function, and Represents the x coordinates of the center points of the predicted box and the real box respectively, and Represents the y coordinates of the center points of the predicted box and the real box respectively, and Represent the width of the predicted box and the real box respectively, and Represent the height of the predicted box and the real box respectively, B represents the number of detection boxes corresponding to each point in the prediction matrix, S 2 Represents the number of elements in the prediction matrix, i represents traversing the prediction matrix, and j represents traversing all detection boxes of each node in the prediction matrix; The positive sample weld confidence loss function indicates the accuracy of the weld prediction confidence: , in, represents the confidence loss function, Indicates the confidence level of the box predicted as a positive sample, B indicates the number of prediction boxes corresponding to each point in the prediction matrix, S 2 Represents the number of elements in the prediction matrix, i represents the traversal of the prediction matrix, and j represents all detection boxes of each node in the prediction matrix. The accuracy of the model's inference of the weld position is evaluated by the positive sample weld coordinate loss function. The back propagation of the positive sample weld coordinate loss function can improve the model's positioning accuracy for the weld. The reliability of the model's inference of the weld is evaluated by the positive sample weld confidence loss function. The back propagation of the positive sample weld confidence loss function can improve the model's prediction accuracy, define low-confidence welds as untrustworthy inferences, and reduce the weld false detection rate. The positive sample weld classification loss function indicates the accuracy of weld prediction classification: , in, represents the classification loss function, and Represent the classification probabilities of the predicted box and the true box respectively, B represents the number of predicted boxes corresponding to each point in the prediction matrix, S 2represents the number of elements in the prediction matrix, i represents the traversal of the prediction matrix, j represents the traversal of all detection boxes of each node in the prediction matrix, n represents the number of prediction types, and k represents the total number of traversed detection types. The reliability of the weld classification reasoned by the model is evaluated by the positive sample weld classification loss function. The back propagation of the positive sample weld classification loss function can improve the classification accuracy of the model and reduce the robot operation risk caused by weld classification errors. The overall loss function is expressed as: , Taking the overall loss function as the loss optimization function of multiple feature map scales, a multi-scale training strategy is constructed on three feature map sizes of 13×13, 26×26, and 52×52, respectively. By comprehensively considering the influence of the three loss functions, the weld positioning accuracy, weld classification accuracy and reasoning reliability can be taken into account, providing high-precision and high-reliability input for robot autonomous weld tracking.

[0034] Furthermore, in the loss function of this embodiment, S represents the number of rows and columns of the matrix, where the number of rows is equal to the number of columns, so S 2 Indicates the number of elements in the prediction matrix.

[0035] Step S130: Optimizing a proprietary operator for the steel structure workpiece weld groove detection scenario to obtain a detection deployment model; In a specific embodiment, the trained YOLOv3-based real-time detection model for weld grooves of steel structure workpieces is optimized for public general operators on the industrial computer hardware platform through a deployment tool. The trained YOLOv3-based real-time detection model for weld grooves of steel structure workpieces is a float32 precision model. The float32 precision mentioned here means that the weights of the YOLOv3-based real-time detection model for weld grooves of steel structure workpieces are stored in float32 data type, and the int8 precision quantization method is used to quantize the float32 precision to int8 precision without losing the model precision, thereby achieving model lightweighting and improving the model. Reasoning performance; Identify the redundant operators generated by the YOLOv3-based real-time detection model for weld grooves of steel structure workpieces during the training phase, use layer fusion technology to perform layer fusion processing on adjacent layers according to the computational graph dependency of the redundant operators, obtain the fusion processing results, and improve the overall computing efficiency; The default network structure of the real-time detection model for weld grooves based on YOLOv3 uses general computing operators, and the general computing operator structure is reconstructed. The graphics card-level multi-threaded parallel optimization method is implemented by manually programming the industrial computer hardware platform to obtain the computing operator adapted to the graphics card-level hardware, which can shorten the computing time and improve the inference rate of the weld groove detection deployment model based on YOLOv3.

[0036] Figure 3The following is a flow chart of a reasoning method for designing a welding robot for tracking weld seams of steel workpieces according to an embodiment of this specification. The method specifically includes the following steps: Step S210: When the steel structure workpiece is transferred to the welding robot station, an area array color industrial camera is used to collect images of the steel structure workpiece in real time online; In a specific embodiment, on the assembly line of a steel structure workpiece production factory, when the current steel structure workpiece is transferred to the welding robot station, the area array color industrial camera takes a picture of the current steel structure workpiece and collects the photographed steel structure workpiece image into an acquisition card device. The current steel structure workpiece image can be obtained from the hardware memory address provided by the acquisition card device.

[0037] Step S220: using the collected real-time image of the steel structure workpiece as input, and using the YOLOv3-based weld real-time detection network to infer the weld groove to accurately locate the weld position; In a specific embodiment, the collected steel structure weld groove image data is used as input data, the input data is normalized and preprocessed, and the preprocessed data is input into the YOLOv3-based weld groove real-time detection model deployed in step S130, and the YOLOv3-based weld groove real-time detection model can be updated in real time through the steel structure workpiece weld groove image data obtained online in real time, and the weld groove positions of flat welds and fillet welds in the steel structure weld groove image data are inferred. Through hand-eye calibration, the relative position relationship between the robot end and the camera is obtained, and the position relationship between the robot base and the robot end, and the camera and the weld groove is input. The position of the weld groove in the robot base coordinate system, that is, the relative coordinate, is calculated through the transformation matrix to obtain the real-time weld groove detection result.

[0038] The above embodiment is further described. Specifically, sensor position calibration is performed to determine the relative position of the robot's welding gun end in a base coordinate system. The sensor can be a visual sensor, which is the eye in hand-eye calibration. Hand-eye calibration can be used to determine the rotation matrix of the operating tool on the robot end flange relative to the visual sensor. The rotation matrix of the operating tool on the robot end flange relative to the robot base is known, and the rotation matrix from the visual sensor to the weld groove is provided by the visual sensor. The rotation matrix of the weld groove relative to the robot base is calculated using the rotation matrix of the operating tool on the robot end flange relative to the robot base, the rotation matrix of the visual sensor relative to the robot end, and the rotation matrix of the weld relative to the visual sensor. The rotation matrix can be understood as a method for representing relative position.

[0039] Step S230: Using the spline interpolation method to generate the movement trajectory of the welding robot's welding gun in real time, so as to achieve effective tracking of the weld groove of the steel structure workpiece; In a specific embodiment, during the deployment stage, the weld groove position is inferred by using the YOLOv3-based weld groove real-time detection model in step S220, and the position is converted into the relative coordinates of the robot. The continuously inferred weld groove position coordinate point set is input into the YOLOv3-based weld groove real-time detection model deployed to the industrial computer, and the robot terminal path planning is calculated using the cubic spline interpolation method. The spatial points formed by the weld groove positions obtained through multiple inferences are used to optimize the welding robot terminal motion trajectory in real time to obtain a smooth terminal trajectory path, thereby realizing adaptive tracking of the weld groove of the steel structure.

[0040] This application breaks the existing robot-based "teaching-reproduction" method from data collection and manual labeling to model training and deployment, and then to model detection and robot trajectory planning. It solves the problem of robot pre-programming methods responding to complex changes in on-site working conditions and achieves a high degree of automation in the real-time tracking of weld grooves.

[0041] Figure 4 and Figure 5 A welding robot system for tracking weld seams of steel structure workpieces is provided according to some embodiments of the present application; wherein, Figure 4 It represents the model training process of the system. Figure 5 It represents the model reasoning process of the system, which includes: The weld groove data acquisition module is used to obtain steel structure weld groove image data in real time, providing input data for offline manual weld groove marking and online weld groove real-time monitoring. The weld groove data acquisition module is further divided into an offline weld groove data collection module 410 and a real-time weld groove data acquisition module 510. The offline weld groove data collection module 410 is used to collect steel structure workpiece weld groove images, collect steel structure workpiece weld groove images offline, and upload the collected images to the cloud server in real time, and then manually mark the weld groove. The real-time weld groove data acquisition module 510 is used to collect steel structure weld groove images online in real time, and input the collected images into the weld groove real-time detection model based on YOLOv3.

[0042] The weld groove marking and review module 420 collects steel structure workpiece images captured by the area array color industrial camera offline into a training image pool. When the number of steel structure workpiece images in the image training pool reaches a certain number, they are uploaded from the industrial computer device to the cloud server and reach the weld groove marking and review module, which manually marks the type and position of the flat weld, fillet weld, etc. weld grooves in the steel structure workpiece images, and reviews the correctness of the flat weld, fillet weld, etc. weld groove markings.

[0043] The real-time detection model training module 430 trains the manually labeled steel structure workpiece weld groove image into a Yolov3-based depth detection model for the steel structure workpiece weld groove scenario.

[0044] In a specific embodiment, the real-time detection model training module includes: The feature extraction network submodule is configured with a deep feature extraction network, a residual structure connection layer, and a feature pyramid. The deep feature extraction network is used to extract rich weld groove feature information. The residual structure connection layer is used to solve the gradient vanishing problem caused by the deep feature extraction network. The feature pyramid is used to extract weld groove features of different scales and resolutions from steel workpiece images to detect weld grooves of different sizes, such as flat welds and fillet welds in the image. The weld detection network submodule is configured for upsampling, splicing and fusing weld groove features of steel structure workpieces at different scales and resolutions to predict the type and location of the weld groove; The detection network output decoupling submodule converts the predicted weld groove center point coordinate offset and size scaling factor into a prediction frame relative to the prior frame and calculates the weld groove position loss function.

[0045] The model optimization and deployment module 440 is used to optimize and stabilize the YOLOv3-based weld groove real-time detection model, perform specific hardware platform level optimization on the common general operators and proprietary operators in the weld groove detection model, and deploy the optimized YOLOv3-based weld groove real-time detection model.

[0046] In a specific embodiment, the model optimization deployment module includes: The public general operator optimization deployment submodule optimizes the public general operators in the weld detection model and realizes the deployment optimization of the deep network model for the proprietary hardware platform; The weld detection proprietary operator optimization deployment sub-module optimizes the dedicated operators in the weld detection model and implements proprietary optimization of the weld detection deep network model for the deployment platform.

[0047] The model reasoning detection module 520 inputs the steel structure workpiece image as input to the real-time deployment detection model of the weld groove, and infers the weld groove positions of flat welds, fillet welds, etc. in the steel structure workpiece image.

[0048] In a specific embodiment, the model reasoning detection module includes: The steel structure workpiece image preprocessing submodule performs normalization processing on the steel structure workpiece image and inputs the preprocessing results into the weld groove detection model; The weld groove detection submodule infers the weld image of steel structure workpieces and predicts the weld groove positions of flat welds, fillet welds, etc. The weld detection post-processing submodule filters the low-confidence weld groove positions and obtains the high-confidence weld groove detection model inference results.

[0049] The welding gun trajectory planning module 530 uses the inference results of the deployed weld groove real-time detection model and the real-time position of the robot welding gun end as input, uses sensor calibration to calculate the weld groove position of flat welds, fillet welds, and the relative position of the robot welding gun end in the base coordinate system, and uses the spline interpolation algorithm to generate the robot welding gun end motion trajectory in real time; In a specific embodiment, the welding gun trajectory planning module includes: The sensor position calibration submodule takes the inference results of the real-time deployment model of weld groove detection as input, and calculates the relative position of the weld groove in the base coordinate system through sensor position calibration and transformation matrix; it also takes the end of the robot welding gun as input, and calculates the relative position of the robot welding gun in the base coordinate system through sensor position calibration.

[0050] The robot welding gun path planning submodule takes the relative positions of the weld groove and the robot welding gun end in the base coordinate system as input, and uses the spline interpolation algorithm to generate the movement trajectory of the robot welding gun end in real time.

[0051] As can be seen from the above, a welding robot system for real-time weld tracking of steel workpieces can, after acquiring an image of the steel workpiece, upload the image to a cloud server in real time according to certain rules. It then manually annotates and verifies the locations of weld grooves, such as flat welds and fillet welds, in the steel workpiece image. It then trains a YOLOv3-based real-time weld groove detection model, optimizes the YOLOv3-based real-time weld groove detection model, and deploys the optimized YOLOv3-based real-time weld groove detection model to an industrial computer, completing the training, optimization, and deployment of the real-time weld groove detection model. When the steel workpiece arrives at the welding station, an area array color industrial camera captures the workpiece image in real time. The YOLOv3-based real-time weld groove detection model deployed on the industrial computer infers the locations of weld grooves, such as flat welds and fillet welds, using sensor position calibration to calculate the relative positions of the weld groove and the robot's welding torch in the base coordinate system. A spline interpolation algorithm is then used to generate the robot's welding torch trajectory in real time. From acquiring steel workpiece images to training, deploying, and inferencing a YOLOv3-based weld groove detection model for steel workpieces, and then to planning the robot's welding gun trajectory, all steps are performed according to pre-set procedures, systematically and rationally completing real-time autonomous tracking of steel workpiece weld grooves. This programmed, automated, and intelligent real-time autonomous weld groove tracking effectively replaces the "teach-and-play" approach used in robotic welding automation. By generating real-time welding gun trajectories based on changing task objectives and field conditions during the steel workpiece welding process, it helps improve welding precision and better control welding quality.

[0052] The foregoing description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A welding robot design method for tracking weld seams of steel structure workpieces, characterized in that: include: The steel structure weld groove image data collected by industrial cameras in real time is divided into two groups for processing: the offline group is manually marked with weld grooves, and the online group is processed in real time by the subsequent YOLOv3-based weld groove real-time detection model. The industrial camera acquires the steel structure workpiece weld groove image data in real time online; The manually labeled steel structure workpiece weld groove image data is used to train the weld groove detection network based on YOLOv3, and a real-time weld groove detection model based on YOLOv3 is obtained; Optimize and deploy a real-time weld groove detection model based on YOLOv3; Using the real-time weld groove detection model based on YOLOv3, the weld groove position is inferred through online group reasoning; Through sensor position calibration, the coordinates of the inferred weld groove and the welding robot's welding gun in the base coordinate system are determined; The spline interpolation method is used to generate the moving trajectory of the welding robot's welding gun in real time.

2. A welding robot design method for steel structure workpiece weld tracking according to claim 1, characterized in that: The YOLOv3-based weld groove real-time detection model is optimized, including: The trained YOLOv3-based weld groove real-time detection model is a float32 precision model. Float32 precision means that the model weights are stored in the float32 data type and quantized to int8 precision using the int8 precision quantization method. Identifying redundant operators generated by the YOLOv3-based weld groove real-time detection model during the training phase, and performing layer fusion processing on adjacent layers according to the computational graph dependency of the redundant operators to obtain a fusion processing result; The default network structure of the YOLOv3-based weld groove real-time detection model uses general computing operators. The general computing operator structure is reconstructed and a graphics card-level multi-threaded parallel optimization method is implemented through manual programming. The resulting computing operator is adapted to graphics card-level hardware.

3. The method for designing a welding robot for tracking weld seams of steel structure workpieces according to claim 1, characterized in that: The reasoning of the weld groove position includes: The collected steel structure weld groove image data is used as input data, and the input data is normalized and preprocessed. The preprocessed input data is input into a real-time weld groove detection model based on YOLOv3. The weld groove positions of flat welds and fillet welds in the steel structure weld groove image data are inferred to obtain real-time weld groove detection results.

4. A welding robot design method for steel structure workpiece weld tracking according to claim 3, characterized in that: The spline interpolation method is used to generate the moving trajectory of the welding robot's welding gun in real time, specifically: The real-time weld groove detection result and the relative coordinates of the welding robot's welding gun in the base coordinate system are used as input data, and the spline interpolation method is used to generate the welding robot's welding gun movement trajectory in real time.

5. The method for designing a welding robot for tracking weld seams of steel structure workpieces according to claim 1, characterized in that: The weld groove detection network based on YOLOv3 is constructed as follows: For the weld groove detection scenario of steel structure workpieces, a weld groove detection network based on YOLOv3 is designed to optimize the weld groove features of flat welds and fillet welds. To meet the demand for accurate recognition of flat and fillet weld grooves, we used steel structure weld groove image data collected by a large number of industrial cameras as input, introduced a residual network structure, and obtained a sub-network of the YOLOv3-based weld groove detection network, namely a dedicated backbone feature extraction network for the welding field. Aiming at the complex and dynamic on-site working environment in the welding field, a multi-feature graph scale loss optimization function is set for the positions of sensors, robots and robot end flange tools relative to steel structure workpieces to detect the features extracted by the dedicated backbone feature extraction network.

6. The method for designing a welding robot for tracking weld seams of steel structure workpieces according to claim 5, characterized in that: The dedicated backbone feature extraction network includes: By increasing the network depth, the residual network structure is introduced to construct a dedicated backbone feature extraction network for the welding field.

7. The method for designing a welding robot for tracking weld seams of steel structure workpieces according to claim 5, characterized in that: The multi-feature map scale loss optimization function includes: The positive sample weld coordinate loss function indicates the accuracy of the weld detection position: ， in, represents the coordinate loss function, and Represents the x coordinates of the center points of the predicted box and the real box respectively, and Represents the y coordinates of the center points of the predicted box and the real box respectively, and Represent the width of the predicted box and the real box respectively, and Represent the height of the predicted box and the real box respectively, B represents the number of detection boxes corresponding to each point in the prediction matrix, S 2 Represents the number of elements in the prediction matrix, i represents traversing the prediction matrix, and j represents traversing all detection boxes of each node in the prediction matrix; The positive sample weld confidence loss function indicates the accuracy of the weld prediction confidence: ， in, represents the confidence loss function, Indicates the confidence level of the box predicted as a positive sample, B indicates the number of prediction boxes corresponding to each point in the prediction matrix, S 2 Represents the number of elements in the prediction matrix, i represents traversing the prediction matrix, and j represents traversing all detection boxes of each node in the prediction matrix; The positive sample weld classification loss function indicates the accuracy of weld prediction classification: ， in, represents the classification loss function, and Represent the classification probabilities of the predicted box and the true box respectively, B represents the number of predicted boxes corresponding to each point in the prediction matrix, S 2 Represents the number of elements in the prediction matrix, i represents traversing the prediction matrix, j represents traversing all detection boxes of each node in the prediction matrix, n represents the number of prediction types, and k represents the total number of traversed detection types; The overall loss function is expressed as: , The overall loss function is used as the loss optimization function for multiple feature map scales, and multi-scale training strategies are constructed on three feature map sizes: 13×13, 26×26, and 52×52.

8. A welding robot system for tracking weld seams of steel structure workpieces, the system being configured to execute the steps of a welding robot design method for tracking weld seams of steel structure workpieces as recited in any one of claims 1 to 7, characterized in that: include: Weld groove data acquisition module, used to obtain the weld groove of steel structure workpiece in real time, including: offline weld groove data collection module and real-time weld groove data acquisition module; The weld groove marking and review module is used to manually mark and review the image data collected by the offline weld groove data collection module; Real-time detection model training module, used to train the real-time weld groove detection model based on YOLOv3, including: feature extraction network submodule, weld detection network submodule and detection network output decoupling submodule; The model optimization and deployment module is used to optimize and stabilize the YOLOv3-based real-time weld groove detection model, including: a public general operator optimization and deployment submodule and a weld detection-specific operator optimization and deployment submodule; The model reasoning and detection module is used to reason about the real-time acquired steel structure workpiece weld images, including: a steel structure workpiece weld preprocessing submodule, a weld groove detection submodule, and a weld detection post-processing submodule; The welding gun trajectory planning module is used to generate the welding gun motion trajectory according to the weld groove output by the detection model and the relative position of the welding robot welding gun in the base coordinate system, including: a sensor position calibration submodule and a robot welding gun path planning submodule.

Images