Welding control method, device and system of welding robot

Abstract

The invention provides a welding control method, device and system for a welding robot, and the method comprises the steps: collecting the point cloud data of a to-be-welded workpiece in real time, and obtaining a first point cloud; constructing an encoder based on sparse convolution, and at least combining the encoder, a decoder and a filter into an instance segmentation model; processing the first point cloud by adopting an instance segmentation model to obtain a plurality of weld instance objects; extracting a plurality of skeleton points of each welding seam instance object through a point cloud skeleton extraction algorithm to obtain a welding point of each welding seam; all the welding points of all the welding seams are connected, connecting lines between the welding points are smoothed, target welding tracks of all the welding seams are obtained, operation parameters of all the welding points in the target welding tracks are determined, and a welding gun is controlled to weld all the welding seams in sequence according to the target welding tracks and the operation parameters. The method solves the problems that in the prior art, a welding track generation method is low in precision, and the requirement for large-scale automatic welding cannot be met.

Description

Technical Field

[0001] The present invention relates to the technical field of welding robot control, and in particular to a welding control method, device, computer-readable storage medium and welding control system of a welding robot. Background Art

[0002] With the development of industrialization, welding robots are increasingly used in the manufacturing industry. The welding trajectory of welding robots directly affects welding quality and efficiency. Traditional welding trajectory generation methods rely on manual teaching through online programming. However, when faced with complex workpieces and complex welding scenarios involving multiple workpiece types, manual teaching cannot guarantee welding trajectory accuracy and is inefficient.

[0003] The existing technology uses offline programming to realize welding trajectory planning in a virtual environment. However, offline programming relies on the precise construction of CAD models and predefined rules. In scenarios where the workpiece is more complex or irregular, it is difficult to ensure the accuracy of the CAD model and the matching of predefined rules, and the accuracy of the welding trajectory cannot be guaranteed.

[0004] To address the above issues, the existing technology proposes a point cloud-based welding trajectory generation method to directly extract weld features from the workpiece's point cloud data, reducing the reliance on manual labor and CAD models in the welding trajectory generation process. However, the point cloud processing method in the existing technology relies on manual features and cannot guarantee that every complex workpiece is covered. The generalization is poor, and when faced with complex workpieces not covered by manual features, it is difficult to ensure the accuracy of the welding trajectory.

[0005] In summary, the welding trajectory generation technology in the prior art is difficult to ensure accurate generation of welding trajectories when faced with irregular welds on complex workpieces, which leads to a decrease in welding accuracy. Summary of the Invention

[0006] The main purpose of this application is to provide a welding control method, device, computer-readable storage medium and welding control system for a welding robot, so as to at least solve the problem that the welding trajectory generation method in the prior art has low accuracy and cannot meet the needs of large-scale automated welding.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a welding control method for a welding robot is provided, including: real-time acquisition of point cloud data of a workpiece to be welded to obtain a first point cloud; constructing an encoder based on sparse convolution, and combining at least the encoder, decoder and filter into an instance segmentation model, and the filter is used to screen the weld point cloud in the first point cloud; using the instance segmentation model to process the first point cloud to obtain multiple weld instance objects, and one weld instance object includes a point cloud corresponding to a weld; extracting multiple skeleton points of each weld instance object through a point cloud skeleton extraction algorithm to obtain welding points of each weld; connecting all welding points of each weld and smoothing the connecting lines between the welding points to obtain a target welding trajectory of each weld, determining the operating parameters of each welding point in the target welding trajectory, and controlling the welding gun to weld each weld in sequence according to the target welding trajectory and the operating parameters, and the operating parameters include the welding gun angle, the welding gun movement direction and the welding gun swing amplitude.

[0008] Optionally, an encoder is constructed based on sparse convolution, and at least the encoder, decoder and filter are combined into an instance segmentation model, including: replacing the two-dimensional convolution in the ResNet50 network with sparse convolution to construct an encoder, and constructing a decoder corresponding to the encoder based on the U-Net architecture; the encoder and decoder are combined into a point cloud feature extractor, and the point cloud feature extractor, the multi-layer perceptron and the fully connected layer are combined in sequence to obtain a semantic segmentation module; a filter is constructed, and a clusterer is constructed based on a point cloud density clustering algorithm with normal vector constraints, and the filter and the clusterer are combined in sequence to obtain an instance clustering module, and the clusterer is used to segment the weld point cloud into weld instance objects; the semantic segmentation module and the instance clustering module are combined into an alternative segmentation model; deep learning is performed on the alternative segmentation model to obtain an instance segmentation model.

[0009] Optionally, deep learning is performed on the alternative segmentation model to obtain an instance segmentation model, including: obtaining point cloud data of the workpiece to be welded to obtain a second point cloud, determining the semantic label and instance label of each data point in the second point cloud, the semantic label is used to characterize the type of physical object to which the data point belongs, and the instance label includes the semantic label and is used to uniquely identify the physical object to which the data point belongs; inputting the second point cloud into the semantic segmentation module to obtain a semantic segmentation result, the semantic segmentation result includes the probability of each data point belonging to each physical object type; calculating the loss function value between the semantic segmentation result and the semantic label through the cross entropy loss function; when the loss function value is less than a first threshold, backpropagating the semantic segmentation module according to the loss function value until the loss function value is greater than the first threshold; constructing multiple groups of grid parameters corresponding to the parameters of the point cloud density clustering algorithm with normal vector constraints by the grid search method, and inputting the semantic segmentation results output by the trained semantic segmentation module into the instance clustering modules configured with each group of grid parameters to obtain multiple predicted weld instance objects; configuring the instance clustering module according to the grid parameters with the smallest deviation between the corresponding predicted weld instance object and the instance label to obtain an instance segmentation model.

[0010] Optionally, multiple skeleton points of each weld instance object are extracted by a point cloud skeleton extraction algorithm, including: selecting multiple data points in the weld instance object by uniform sampling to determine as candidate skeleton points; initializing multiple clusters with each candidate skeleton point as the cluster center, and dividing each data point in the weld instance object into the cluster corresponding to the candidate skeleton point with the smallest distance to the data point to obtain multiple sub-point clouds; a replacement step of replacing the corresponding candidate skeleton point with the centroid point corresponding to each sub-point cloud to obtain a candidate skeleton point; a merging step of merging the sub-point clouds until the distance between the candidate skeleton points corresponding to any two sub-point clouds is greater than a second threshold; repeating the replacement step and the merging step at least once in sequence until the number of candidate skeleton points between two adjacent iterations is the same, and all candidate skeleton points are determined as welding points.

[0011] Optionally, all welding points of each weld are connected, and the connecting lines between the welding points are smoothed to obtain a target welding trajectory of each weld, including: connecting the welding points by a minimum spanning tree method to obtain a first trajectory; sorting the welding points in the first trajectory by a depth-first search algorithm to obtain a second trajectory; and smoothing the second trajectory using a non-uniform rational B-spline curve to obtain a target welding trajectory.

[0012] Optionally, each welding point is connected by a minimum spanning tree method to obtain a first trajectory, including: taking the welding point as a node, connecting each node to all other nodes through an edge, and obtaining a welding point graph; calculating the Euclidean distance between each node in the welding point graph, and determining the Euclidean distance as the weight of the edge to obtain a weighted graph; determining any node as the current node of the spanning tree, and repeatedly adding the nodes connected to the current node and corresponding to the minimum weight to the spanning tree until the spanning tree includes all the nodes in the weighted graph; and connecting each welding point in sequence according to the spanning tree to obtain the first trajectory.

[0013] Optionally, determining the operating parameters of each welding point in the target welding trajectory includes: determining two target planes corresponding to the weld of each welding point based on the neighborhood data points of each welding point in the target welding trajectory, adding and normalizing the normal vectors corresponding to the two target planes to obtain the z-axis basis vector corresponding to each welding point; determining the tangent and the tangent direction corresponding to each welding point based on the target welding trajectory, and generating the x-axis basis vector based on the tangent and the tangent direction; determining the y-axis basis vector based on the z-axis basis vector and the x-axis basis vector; determining the welding gun angle corresponding to each welding point based on the z-axis basis vector, determining the welding gun movement direction corresponding to each welding point based on the x-axis basis vector, and determining the welding gun swing amplitude corresponding to each welding point based on the y-axis basis vector to obtain the operating parameters of each welding point.

[0014] According to another aspect of the present application, an intelligent welding trajectory generation device for a welding robot is provided, the device including: a first acquisition unit for collecting point cloud data of a workpiece to be welded in real time to obtain a first point cloud; a first processing unit for constructing an encoder based on sparse convolution, and combining at least the encoder, decoder and filter into an instance segmentation model, the filter being used to screen the weld point cloud in the first point cloud; a second acquisition unit for processing the first point cloud using the instance segmentation model to obtain multiple weld instance objects, where one weld instance object includes a point cloud corresponding to one weld; a second processing unit for extracting multiple skeleton points of each weld instance object through a point cloud skeleton extraction algorithm to obtain welding points of each weld; a generation unit for connecting all welding points of each weld and smoothing the connecting lines between the welding points to obtain a target welding trajectory of each weld, determining the operating parameters of each welding point in the target welding trajectory, and controlling the welding gun to weld each weld in sequence according to the target welding trajectory and the operating parameters, wherein the operating parameters include the welding gun angle, the welding gun movement direction and the welding gun swing amplitude.

[0015] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described above.

[0016] According to another aspect of the present application, a welding control system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the described methods.

[0017] Applying the technical solution of the present application, in the welding control method of the above-mentioned welding robot, first, point cloud data of the workpiece to be welded is collected in real time to obtain a first point cloud; then, an encoder is constructed based on sparse convolution, and at least the encoder, decoder and filter are combined into an instance segmentation model, and the filter is used to filter the weld point cloud in the first point cloud; then, the first point cloud is processed using the instance segmentation model to obtain multiple weld instance objects, where each weld instance object includes a point cloud corresponding to a weld; then, multiple skeleton points of each weld instance object are extracted using a point cloud skeleton extraction algorithm to obtain welding points of each weld; finally, all welding points of each weld are connected and the connecting lines between the welding points are smoothed to obtain a target welding trajectory for each weld, and the operating parameters of each welding point in the target welding trajectory are determined. The welding gun is controlled to weld each weld in sequence according to the target welding trajectory and the operating parameters, and the operating parameters include welding gun angle, welding gun movement direction and welding gun swing amplitude. The present application uses sparse convolution to construct an encoder, learns and extracts features of different types of welds in an end-to-end deep learning manner, and improves the accuracy of feature extraction in complex scenarios. Furthermore, the welding points in the weld instance object after model processing are obtained through the point cloud skeleton extraction algorithm, which realizes the accurate extraction of the welding trajectory of irregular welds. Compared with the point cloud processing method that relies on manual feature processing in the existing technology, the recognition accuracy and trajectory planning accuracy in complex weldments and multi-type weld scenarios are enhanced, so as to solve the problem that the welding trajectory generation method in the existing technology has low accuracy and cannot meet the needs of large-scale automated welding. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The following is a hardware structure diagram of a mobile terminal for controlling welding of a welding robot provided in an embodiment of the present application;

[0019] Figure 2 A schematic flow chart of a welding control method for a welding robot according to an embodiment of the present application is shown;

[0020] Figure 3 A flow chart of generating a welding trajectory according to an embodiment of the present application is shown;

[0021] Figure 4 A schematic diagram of a model architecture of an instance segmentation model provided according to an embodiment of the present application is shown;

[0022] Figure 5 A schematic diagram of a process of extracting a point cloud skeleton according to an embodiment of the present application is shown;

[0023] Figure 6 A schematic diagram of a welding trajectory of a welding workpiece provided according to an embodiment of the present application is shown;

[0024] Figure 7 A schematic diagram of a specific welding trajectory generation process according to another embodiment of the present application is shown;

[0025] Figure 8 A structural block diagram of a welding control device of a welding robot provided according to an embodiment of the present application is shown.

[0026] The above drawings include the following reference numerals:

[0027] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. DETAILED DESCRIPTION

[0028] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0029] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] As introduced in the background technology, the welding trajectory generation technology in the prior art is difficult to ensure the accurate generation of welding trajectories when facing irregular welds of complex workpieces, which in turn leads to a decrease in welding accuracy. In order to solve the problem that the welding trajectory generation method in the prior art has low accuracy and cannot meet the needs of large-scale automated welding, the embodiments of the present application provide a welding control method, device, computer-readable storage medium and welding control system for a welding robot.

[0032] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0033] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal for a welding control method of a welding robot according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0034] The memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the welding control method for the welding robot in the embodiment of the present invention. The processor 102 executes the computer programs stored in the memory 104 to execute various functional applications and data processing, thereby implementing the above-mentioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The transmission device 106 is used to receive or transmit data via a network. Specific examples of such networks may include a wireless network provided by the mobile terminal's telecommunications provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0035] In this embodiment, a welding control method for a welding robot running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] Figure 2 FIG. 1 is a flow chart of a welding control method for a welding robot according to an embodiment of the present application. Figure 2 As shown, the method includes the following steps:

[0037] Step S201, collecting point cloud data of the workpiece to be welded in real time to obtain a first point cloud;

[0038] Specifically, if Figure 3 As shown, the three-dimensional point cloud data of the workpiece to be welded is collected in real time using a 3D camera or laser sensor around the workpiece to obtain the above-mentioned first point cloud.

[0039] It can be understood that the geometric details of the workpiece surface are captured by collecting point cloud data, which is applicable to workpieces of various shapes and sizes and provides original three-dimensional geometric information for subsequent weld detection.

[0040] Step S202: constructing an encoder based on sparse convolution, combining at least the encoder, decoder, and filter into an instance segmentation model, wherein the filter is used to filter the weld point cloud in the first point cloud;

[0041] Specifically, an instance segmentation model consisting of at least an encoder, a decoder, and a filter is constructed. The key is to introduce coefficient convolution to construct the encoder. Compared with traditional dense convolution, sparse convolution can retain more geometric information when dealing with sparse data such as point clouds. The encoder is used to extract high-level features from point cloud data, and the decoder is used to map the features back to the point cloud space for instance-level segmentation. The filter is used to filter the point cloud data belonging to the weld after the model predicts the instance to which the data point in the point cloud belongs.

[0042] Step S203: Processing the first point cloud using an instance segmentation model to obtain a plurality of weld instance objects, where each weld instance object includes a point cloud corresponding to a weld;

[0043] Specifically, if Figure 3 As shown in FIG, after performing instance segmentation on the first point cloud using the strength segmentation model, a weld instance object is obtained.

[0044] It is understandable that during the instance segmentation process, the model will output prediction results for different weld instances. Each weld instance object corresponds to an independent weld and its nearby point cloud.

[0045] Step S204, extracting multiple skeleton points of each weld instance object using a point cloud skeleton extraction algorithm to obtain welding points of each weld;

[0046] Step S205: Connect all welding points of each weld and smooth the connecting lines between the welding points to obtain the target welding trajectory of each weld. Determine the operating parameters of each welding point in the target welding trajectory. Control the welding gun to weld each weld in sequence according to the target welding trajectory and the operating parameters. The operating parameters include the welding gun angle, the welding gun movement direction, and the welding gun swing amplitude.

[0047] Specifically, if Figure 3As shown in the figure, for each weld instance, a point cloud skeleton extraction algorithm is used to find the skeleton points representing the weld direction. Then, by determining the starting and ending points of the weld, the other weld points are sorted to ensure that the welding gun can weld in the correct order. To make the trajectory closer to the actual weld, the weld points are smoothed using non-uniform rational B-spline curves to eliminate welding trajectory fluctuations caused by point cloud noise and obtain the final welding trajectory. By analyzing the point cloud data near the skeleton points, the local tangent direction (as the x-axis) and welding direction (as the z-axis) of the weld point are calculated. The y-axis direction is then determined according to the right-hand rule, thereby obtaining the complete pose information of the weld point. Combining the sorting and smoothing results of the weld points, as well as operating parameters such as the welding gun angle, movement direction, and swing amplitude, precise control of the welding gun can be achieved, completing the welding operation along the target welding trajectory.

[0048] According to this embodiment, first, point cloud data of the workpiece to be welded is collected in real time to obtain a first point cloud; then, an encoder is constructed based on sparse convolution, and at least the encoder, decoder, and filter are combined into an instance segmentation model, and the filter is used to filter the weld point cloud in the first point cloud; then, the first point cloud is processed using the instance segmentation model to obtain multiple weld instance objects, where each weld instance object includes a point cloud corresponding to a weld; then, multiple skeleton points of each weld instance object are extracted using a point cloud skeleton extraction algorithm to obtain welding points of each weld; finally, all welding points of each weld are connected and the connecting lines between the welding points are smoothed to obtain a target welding trajectory for each weld, and the operating parameters of each welding point in the target welding trajectory are determined. The welding gun is controlled to weld each weld in sequence according to the target welding trajectory and the operating parameters, and the operating parameters include welding gun angle, welding gun movement direction, and welding gun swing amplitude. This application uses sparse convolution to construct an encoder, learns and extracts features of different types of welds in an end-to-end deep learning manner, and improves the accuracy of feature extraction in complex scenarios. Furthermore, the welding points in the weld instance object after model processing are obtained through the point cloud skeleton extraction algorithm, which realizes the accurate extraction of the welding trajectory of irregular welds. Compared with the point cloud processing method that relies on manual feature processing in the existing technology, the recognition accuracy and trajectory planning accuracy in complex weldments and multi-type weld scenarios are enhanced, so as to solve the problem that the welding trajectory generation method in the existing technology has low accuracy and cannot meet the needs of large-scale automated welding.

[0049] In order to construct the above instance segmentation model, in an optional implementation, the above step S202 includes:

[0050] Step S2021: replace the two-dimensional convolution in the ResNet50 network with sparse convolution to construct an encoder, and construct a decoder corresponding to the encoder based on the U-Net architecture;

[0051] Specifically, if Figure 4 As shown in the figure, the encoder is constructed by replacing the 2D convolutions in the ResNet50 network with the sparse convolution functionality provided by the MinkowskiEngine. The decoder is built on the U-Net architecture and gradually restores the spatial resolution of the point cloud based on the high-level features extracted by the encoder to generate point-by-point feature vectors.

[0052] The encoder is used to extract features from the input welding workpiece point cloud (the first point cloud, n*f), achieving efficient extraction of high-level semantic features. The decoder is used to restore the resolution of the features and output a point-by-point feature vector (n*512).

[0053] Step S2022: Combining the encoder and decoder into a point cloud feature extractor, and sequentially combining the point cloud feature extractor, the multi-layer perceptron, and the fully connected layer to obtain a semantic segmentation module;

[0054] Specifically, if Figure 4 As shown in the figure, the point cloud feature extractor is composed of an encoder and a decoder, which is used to output the original point cloud data as a feature vector containing semantic information. Furthermore, based on the point cloud feature extractor, a multi-layer perceptron (MLP) and a fully connected layer (FC) are connected in sequence to form a semantic segmentation module.

[0055] Among them, MLP is used to further refine features (n*64), while FC is responsible for mapping features to classification labels (n*c, c is the preset number of label types), realizing the classification of point cloud semantic information and obtaining semantic segmentation results.

[0056] In one embodiment, the MLP consists of two fully connected layers, which process the features into n*128 and n*64 respectively.

[0057] Step S2023: construct a filter, construct a clusterer based on the point cloud density clustering algorithm with normal vector constraints, combine the filter and the clusterer in sequence to obtain an instance clustering module, and the clusterer is used to segment the weld point cloud into weld instance objects;

[0058] Specifically, if Figure 4 As shown in the figure, the filter is designed to filter out weld point clouds from the semantic segmentation results, while the clusterer uses a point cloud density clustering algorithm with normal vector constraints to achieve weld instance clustering and segment the weld point cloud into different weld instance objects.

[0059] Step S2024, combining the semantic segmentation module and the instance clustering module into an alternative segmentation model;

[0060] Specifically, the semantic segmentation module and instance clustering module are combined to form an alternative segmentation model. This model first identifies the weld point cloud through the semantic segmentation module and then subdivides it into different weld instance objects through the instance clustering module.

[0061] Step S2025: Perform deep learning on the candidate segmentation model to obtain an instance segmentation model.

[0062] Specifically, the constructed model is trained until the performance of the model meets the usage requirements, thereby obtaining the above-mentioned instance segmentation model.

[0063] Through the above-described embodiments, the use of coefficient convolution to process workpiece point cloud data improves instance segmentation accuracy compared to traditional dense convolution or two-dimensional convolution. The U-Net architecture, through the combination of an encoder and decoder, enables feature abstraction and detailed recovery to achieve instance-level segmentation tasks. By combining sparse convolution with the U-Net model, the instance segmentation model can adapt to different types of welds and workpiece shapes without the need for manually predefined rules or reliance on CAD models, thereby improving trajectory generation accuracy in complex welding scenarios.

[0064] In order to improve the accuracy of the instance segmentation model, in an optional implementation, the above step S2025 includes:

[0065] Step S20251: Acquire point cloud data of the workpiece to be welded to obtain a second point cloud, and determine a semantic label and an instance label for each data point in the second point cloud. The semantic label is used to characterize the type of physical object to which the data point belongs, and the instance label includes the semantic label and is used to uniquely identify the physical object to which the data point belongs.

[0066] Specifically, pre-collected point cloud data of the workpiece to be welded is obtained to form the aforementioned second point cloud. Each point cloud data is annotated with a semantic label (e.g., weld, non-weld, or other object) and an instance label. The instance label uniquely identifies a specific weld or object, even if the same object exists in multiple instances.

[0067] Step S20252: Input the second point cloud into a semantic segmentation module to obtain a semantic segmentation result, where the semantic segmentation result includes the probability of each data point belonging to each object type.

[0068] Specifically, the annotated second point cloud is input into the semantic segmentation module, and the deep learning model is used to predict the classification probability of each point cloud data point to obtain the semantic segmentation result.

[0069] Step S20253, calculating the loss function value between the semantic segmentation result and the semantic label using a cross entropy loss function;

[0070] Specifically, the difference between the predicted probability and the actual semantic label is calculated, and the cross entropy loss function is used as the evaluation indicator. It can be understood that the use of weighted cross entropy loss function can better deal with the imbalance problem of point clouds of different categories.

[0071] Step S20254: When the loss function value is greater than the first threshold, backpropagation is performed on the semantic segmentation module according to the loss function value until the loss function value is less than the first threshold;

[0072] Specifically, when the loss function value is lower than the set first threshold, it indicates that the difference between the model prediction and the true label is within an acceptable range. At this time, backpropagation is stopped to avoid overfitting. Otherwise, backpropagation is continued and model parameters are adjusted until the loss function value no longer decreases or the preset maximum number of iterations is reached.

[0073] It is understandable that the setting of the first threshold depends on the accuracy requirement that is desired to be achieved. The higher the accuracy requirement, the smaller the first threshold is set.

[0074] Step S20255: construct multiple groups of grid parameters corresponding to the parameters of the point cloud density clustering algorithm with normal vector constraints using a grid search method, input the semantic segmentation results output by the trained semantic segmentation module into the instance clustering module configured with each group of grid parameters, and obtain multiple predicted weld instance objects;

[0075] Specifically, through the grid search method, multiple groups of grid parameters (including but not limited to the normal vector angle threshold, the number of neighborhood points, the neighborhood radius, etc.) are explored. The semantic segmentation results output by the trained semantic segmentation model are used, combined with the point cloud density and normal vector information, and the point cloud density clustering algorithm with normal vector constraints is used to segment the weld point cloud into different weld instance objects to obtain the above-mentioned predicted weld instance objects.

[0076] Step S20256: configure the instance clustering module according to the grid parameters that correspond to the smallest deviation between the predicted weld instance object and the instance label to obtain an instance segmentation model.

[0077] Specifically, the clustering results under different parameter configurations are compared with the manually marked instance labels, and the parameter set with the smallest deviation between the segmentation results and the labels is selected as the final configuration parameters of the instance clustering module.

[0078] In the specific implementation, after completing the above training and parameter optimization, the instance segmentation model also needs to be verified on an independent test set to ensure the robustness and generalization performance of the instance segmentation model. It can be deployed only when the instance segmentation model performs equally well on the test set.

[0079] Through the above-described embodiment, by optimizing clustering parameters, the instance segmentation model can effectively remove noise points and maintain the purity of weld instance objects, thereby improving the stability of weld trajectory generation. Optimizing the semantic segmentation module ensures that the instance segmentation model accurately distinguishes different weld types, maintaining high semantic segmentation accuracy even in extreme cases.

[0080] In order to extract welding points, in an optional embodiment, the above step S204 includes:

[0081] Step S2041, using uniform sampling to select multiple data points in the weld instance object to determine as candidate skeleton points;

[0082] Specifically, if Figure 5 As shown, the generation of candidate skeleton points includes: for each weld instance object obtained from the instance segmentation model, a number of data points are selected as the above-mentioned candidate skeleton points in the internal object by uniform sampling.

[0083] It can be understood that uniform sampling ensures that the skeleton points are evenly distributed along the length of the weld, avoiding oversampling in local areas.

[0084] Step S2042: Initialize multiple clusters using each candidate skeleton point as a cluster center, and divide each data point in the weld instance object into the cluster corresponding to the candidate skeleton point with the smallest distance to the data point, thereby obtaining multiple sub-point clouds.

[0085] Step S2043, a replacement step, replacing the corresponding candidate skeleton point with the centroid point corresponding to each sub-point cloud to obtain a candidate skeleton point;

[0086] Specifically, updating the candidate skeleton points includes: taking the above-mentioned alternative skeleton point as the center, allocating all data points in the weld instance to the cluster formed by the alternative skeleton point closest to it, thereby dividing the weld instance into multiple sub-point clouds. Each sub-point cloud represents the spatial distribution information near the alternative skeleton point. For each sub-point cloud, its centroid point is calculated (the average value of the coordinates of all data points, which reflects the central tendency and center of gravity position of the sub-point cloud and is more robust than a single alternative skeleton point), and the centroid point of the sub-point cloud is used to replace the original alternative skeleton point to obtain the candidate skeleton point.

[0087] Step S2044, a merging step, merging the sub-point clouds until the distance between the candidate skeleton points corresponding to any two sub-point clouds is greater than a second threshold;

[0088] Specifically, the candidate skeleton point processing includes: candidate skeleton point thinning, that is, when the candidate skeleton points corresponding to two or more sub-point clouds are very close (that is, less than the second threshold), a merging operation is performed to reduce the number of skeleton points and avoid the path being too dense, which affects the operating efficiency and welding quality of the welding robot.

[0089] Step S2045 : Repeat the replacing step and the merging step at least once in sequence until the number of candidate skeleton points between two adjacent iterations is the same, and all candidate skeleton points are determined as welding points.

[0090] Specifically, the merging step and the replacing step constitute an iterative process, which is repeated until the number of skeleton points between two adjacent iterations no longer changes. The remaining skeleton points constitute the basic framework of the welding trajectory, and the above candidate skeleton points are determined as welding points.

[0091] Through the above embodiment, the point cloud skeleton extraction algorithm is adopted to ensure the accuracy and continuity of the welding trajectory, ensure the adaptation to complex weld morphology, and generate a smooth and compliant welding trajectory. Among them, the iterative centroid calculation and skeleton point sparsification can effectively remove redundant skeleton points caused by point cloud noise or local density, thereby ensuring the accuracy of the welding trajectory.

[0092] In order to generate the target welding trajectory, in an optional embodiment, the above step S205 includes:

[0093] Step S2051, connecting each welding point by a minimum spanning tree method to obtain a first trajectory;

[0094] Specifically, the minimum spanning tree method is used in graph theory to find the tree structure with the shortest connections between all vertices in an unweighted graph. This application is used to connect all welding points in a weld instance to form a preliminary welding path to ensure that all welding points are connected while minimizing the total path length and avoiding redundancy and repeated trajectories.

[0095] Step S2052, sorting the welding points in the first trajectory by a depth-first search algorithm to obtain a second trajectory;

[0096] Specifically, the depth-first search algorithm is used to traverse or search a tree or graph. This application uses this algorithm to sequentially sort welding points in welding trajectory generation, ensuring that the operation sequence of the welding robot is reasonable, avoiding jumps or backoffs during welding, and improving the continuity and efficiency of the welding process.

[0097] Step S2053: Use a non-uniform rational B-spline curve to smooth the second trajectory to obtain a target welding trajectory.

[0098] Specifically, non-uniform rational B-spline curve smoothing technology is an application of curve fitting, which can generate a smooth and continuous curve from a given set of points. In welding trajectory generation, this application fits and smoothes the welding points, eliminating trajectory fluctuations caused by irregularities or noise in the point cloud data, ensuring the smoothness and accuracy of the welding trajectory.

[0099] Through the above embodiments, the smoothed welding trajectory is more consistent with the weld path, avoiding the decline in welding quality caused by irregular point cloud data. By sorting the welding points through the depth-first search algorithm and smoothing the non-uniform rational B-spline curve, the welding robot can weld according to the optimized path, reducing unnecessary movement and improving production efficiency.

[0100] In order to obtain the first trajectory, in an optional implementation, step S2051 includes:

[0101] Step S20511: Take the welding points as nodes, connect each node with all other nodes through edges, and obtain a welding point graph;

[0102] Specifically, the welding points are regarded as nodes in the graph, and any two nodes are connected by edges to form a complete graph, namely the welding point graph, which provides the basic framework of the minimum spanning tree algorithm.

[0103] Step S20512, calculating the Euclidean distance between each pair of nodes in the solder joint graph, and determining the Euclidean distance as the weight of the edge to obtain a weighted graph;

[0104] Specifically, for each edge in the weld point graph, the Euclidean distance between the two corresponding weld points is calculated as the weight of the edge, which is used to measure the cost of the welding robot moving during the welding process.

[0105] Step S20513, determining any node as the current node of the spanning tree, and repeatedly adding the nodes connected to the current node and corresponding to the minimum weight to the spanning tree until the spanning tree includes all the nodes in the weighted graph;

[0106] Specifically, any welding point is selected as the starting node of the spanning tree, that is, the current node. Starting from the current node, a node adjacent to it but not yet included in the spanning tree is found, and the edge weight connecting the two nodes is the smallest. This node is added to the spanning tree, and the current node is updated. The above process is repeated until all welding points are added to the spanning tree. The spanning tree obtained at this time is the minimum spanning tree, which will contain all welding points and achieve connections with the lowest total weight.

[0107] Step S20514: Connect each welding point in sequence according to the spanning tree to obtain a first trajectory.

[0108] Specifically, according to the connection order of the edges in the minimum spanning tree, all welding points are connected in sequence to form a preliminary welding trajectory of the welding robot, that is, the first trajectory.

[0109] Through the above embodiment, the preliminary welding trajectory generated by the minimum spanning tree method can ensure that the welding robot traverses all welding points in the most economical path, reducing ineffective movement during the welding process and improving welding efficiency.

[0110] In order to ensure the accuracy of welding, in an optional embodiment, the above step S205 further includes:

[0111] Step S2054: determining two target planes corresponding to the weld seam of each welding point based on the neighborhood data points of each welding point in the target welding trajectory, adding and normalizing the normal vectors corresponding to the two target planes to obtain the z-axis basis vector corresponding to each welding point;

[0112] Specifically, for each weld point on the target weld trajectory, point cloud data points within its neighborhood are selected and statistical analysis is performed to determine two target planes corresponding to the weld seam. The normal vectors of these two planes are added and normalized to obtain the z-axis basis vector of the weld point. This vector points in the direction of the weld seam normal and is the key reference direction for the welding gun to be perpendicular to the weld surface.

[0113] Step S2055, determining the tangent line and tangent line direction corresponding to each welding point according to the target welding trajectory, and generating an x-axis basis vector according to the tangent line and tangent line direction;

[0114] Specifically, according to the tangent direction of the welding point on the target welding trajectory, the x-axis basis vector is defined, which determines the movement direction of the welding gun and ensures that the welding process proceeds along the correct path.

[0115] Step S2056, determining the y-axis basis vector according to the z-axis basis vector and the x-axis basis vector;

[0116] Specifically, when the z-axis basis vector and the x-axis basis vector have been determined, the y-axis basis vector can be defined according to the right-hand screw rule.

[0117] Step S2057: Determine the welding gun angle corresponding to each welding point based on the z-axis basis vector, determine the welding gun movement direction corresponding to each welding point based on the x-axis basis vector, and determine the welding gun swing amplitude corresponding to each welding point based on the y-axis basis vector to obtain the operating parameters of each welding point.

[0118] Specifically, defining the operation parameters according to the coordinate axis includes:

[0119] Welding gun angle calculation: The z-axis basis vector determines the vertical angle between the welding gun and the workpiece surface during welding, that is, the direction of the welding gun coincides with the z-axis basis vector.

[0120] Welding gun movement direction setting: The x-axis basis vector defines the movement direction of the welding gun, ensuring that the welding gun moves smoothly along the welding trajectory.

[0121] Determination of the swing amplitude of the welding gun: The y-axis basis vector is perpendicular to the weld and orthogonal to the x-axis. The swing amplitude of the welding gun during welding can be determined by analyzing the distribution of neighborhood data points of the welding point to adapt to the width change of the weld.

[0122] In one embodiment, the above coordinate axes and welding trajectories are as follows: Figure 6 shown.

[0123] The above-described embodiment accurately calculates the coordinate system of the welding point, ensuring proper torch posture control during welding. This significantly improves welding accuracy and reduces welding defects caused by improper torch angles. Optimizing the torch's movement direction and swing amplitude reduces unnecessary movement during welding, speeding up the welding process and improving overall welding efficiency.

[0124] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the welding control method of the welding robot of the present application will be described in detail below with reference to specific embodiments.

[0125] This embodiment relates to a specific method for generating a welding trajectory of a welding robot. Figure 7 As shown, the following steps are included:

[0126] Step S1: Independent weld segment point cloud: collect point cloud data of the workpiece to be welded and process it through the instance segmentation model to obtain an independent weld segment point cloud corresponding to each weld;

[0127] Step S2: point cloud skeleton point extraction: using the point cloud skeleton extraction algorithm, the skeleton points of the geographic weld segment point cloud are obtained as weld points, realizing the calculation of multiple types of weld points;

[0128] Step S3: Welding point sorting: take the skeleton points of the independent weld segment point cloud as welding points, define the head and tail welding points, and sort the welding points using the ascending-first search algorithm;

[0129] Step S4: welding point smoothing: smoothing the sorted welding points using a non-uniform rational B-spline curve;

[0130] Step S5: Welding point pose calculation: Based on the welding points of the independent weld segments, a coordinate system with each welding point as the origin is defined in the point cloud to obtain the welding point pose;

[0131] Step S6: Welding trajectory: Generate welding trajectory according to the welding point posture.

[0132] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0133] The embodiments of the present application also provide a welding control device for a welding robot. It should be noted that the welding control device for the welding robot of the embodiments of the present application can be used to execute the welding control method for a welding robot provided in the embodiments of the present application. The device is used to implement the above-mentioned embodiments and preferred embodiments, and the details that have been described will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0134] The following introduces the welding control device of the welding robot provided in the embodiment of the present application.

[0135] Figure 8 FIG. 1 is a structural block diagram of a welding control device of a welding robot according to an embodiment of the present application. Figure 8 As shown, the device includes:

[0136] A first acquisition unit 10 is used to collect point cloud data of the workpiece to be welded in real time to obtain a first point cloud;

[0137] A first processing unit 20 is configured to construct an encoder based on sparse convolution, and to combine at least the encoder, the decoder, and the filter into an instance segmentation model, wherein the filter is configured to filter the weld point cloud in the first point cloud;

[0138] A second acquisition unit 30 is configured to process the first point cloud using an instance segmentation model to obtain a plurality of weld instance objects, where each weld instance object includes a point cloud corresponding to a weld;

[0139] The second processing unit 40 is used to extract multiple skeleton points of each weld instance object using a point cloud skeleton extraction algorithm to obtain welding points of each weld;

[0140] The generation unit 50 is used to connect all the welding points of each weld and smooth the connecting lines between the welding points to obtain the target welding trajectory of each weld, determine the operating parameters of each welding point in the target welding trajectory, and control the welding gun to weld each weld in sequence according to the target welding trajectory and the operating parameters. The operating parameters include the welding gun angle, the welding gun movement direction, and the welding gun swing amplitude.

[0141] According to this embodiment, a first acquisition unit collects point cloud data of a workpiece to be welded in real time to obtain a first point cloud; a first processing unit constructs an encoder based on sparse convolution, combining at least an encoder, a decoder, and a filter into an instance segmentation model, wherein the filter is used to filter the weld point cloud in the first point cloud; a second acquisition unit processes the first point cloud using the instance segmentation model to obtain multiple weld instance objects, each weld instance object including a point cloud corresponding to a weld; a second processing unit extracts multiple skeleton points of each weld instance object using a point cloud skeleton extraction algorithm to obtain welding points of each weld; a generation unit connects all welding points of each weld and smoothes the connecting lines between the welding points to obtain a target welding trajectory for each weld, determines the operating parameters of each welding point in the target welding trajectory, and controls the welding gun to weld each weld in sequence according to the target welding trajectory and the operating parameters, wherein the operating parameters include the welding gun angle, the welding gun movement direction, and the welding gun swing amplitude. This application uses sparse convolution to construct an encoder, learns and extracts features of different types of welds in an end-to-end deep learning manner, and improves the accuracy of feature extraction in complex scenarios. Furthermore, the welding points in the weld instance object after model processing are obtained through the point cloud skeleton extraction algorithm, which realizes the accurate extraction of the welding trajectory of irregular welds. Compared with the point cloud processing method that relies on manual feature processing in the existing technology, the recognition accuracy and trajectory planning accuracy in complex weldments and multi-type weld scenarios are enhanced, so as to solve the problem that the welding trajectory generation method in the existing technology has low accuracy and cannot meet the needs of large-scale automated welding.

[0142] In order to construct the above instance segmentation model, in an optional embodiment, the above first processing unit includes:

[0143] The first building block is used to replace the two-dimensional convolution in the ResNet50 network with sparse convolution to build an encoder, and to build a decoder corresponding to the encoder based on the U-Net architecture;

[0144] The second building block is used to combine the encoder and decoder into a point cloud feature extractor, and sequentially combine the point cloud feature extractor, the multi-layer perceptron, and the fully connected layer to obtain a semantic segmentation module;

[0145] The third construction module is used to construct a filter, and a clusterer is constructed based on the point cloud density clustering algorithm with normal vector constraints. The filter and the clusterer are combined in sequence to obtain an instance clustering module. The clusterer is used to segment the weld point cloud into weld instance objects.

[0146] a fourth building module for combining the semantic segmentation module and the instance clustering module into an alternative segmentation model;

[0147] The first training module is used to perform deep learning on the alternative segmentation model to obtain an instance segmentation model.

[0148] In order to improve the accuracy of the instance segmentation model, in an optional embodiment, the first training module includes:

[0149] A first training submodule is configured to acquire point cloud data of a workpiece to be welded, obtain a second point cloud, and determine a semantic label and an instance label for each data point in the second point cloud, wherein the semantic label is used to characterize the type of physical object to which the data point belongs, and the instance label includes the semantic label and is used to uniquely identify the physical object to which the data point belongs;

[0150] The second training submodule is used to input the second point cloud into the semantic segmentation module to obtain a semantic segmentation result, where the semantic segmentation result includes the probability of each data point belonging to each physical object type;

[0151] The third training submodule is used to calculate the loss function value between the semantic segmentation result and the semantic label through the cross entropy loss function;

[0152] a fourth training submodule, configured to, when the loss function value is greater than the first threshold, perform backpropagation on the semantic segmentation module according to the loss function value until the loss function value is less than the first threshold;

[0153] The fifth training submodule is used to construct multiple groups of grid parameters corresponding to the parameters of the point cloud density clustering algorithm with normal vector constraints using a grid search method, and input the semantic segmentation results output by the trained semantic segmentation module into the instance clustering module configured with each group of grid parameters to obtain multiple predicted weld instance objects;

[0154] The sixth training submodule is used to configure the instance clustering module according to the grid parameters that have the smallest deviation between the corresponding predicted weld instance object and the instance label to obtain an instance segmentation model.

[0155] In order to extract welding points, in an optional embodiment, the second processing unit includes:

[0156] A first processing module is used to select multiple data points in the weld instance object by uniform sampling to determine them as candidate skeleton points;

[0157] The second processing module is used to initialize multiple clusters with each candidate skeleton point as the cluster center, and divide each data point in the weld instance object into the cluster corresponding to the candidate skeleton point with the smallest distance to the data point, thereby obtaining multiple sub-point clouds;

[0158] The third processing module is used to perform a replacement step to replace the corresponding candidate skeleton point with the centroid point corresponding to each sub-point cloud to obtain a candidate skeleton point;

[0159] a fourth processing module, configured to execute a merging step to merge the sub-point clouds until the distance between the candidate skeleton points corresponding to any two sub-point clouds is greater than a second threshold;

[0160] The repetition module is used to sequentially repeat the replacing step and the merging step at least once until the number of candidate skeleton points between two adjacent iterations is the same, and all candidate skeleton points are determined as welding points.

[0161] In order to generate the target welding trajectory, in an optional embodiment, the generating unit includes:

[0162] A fifth processing module is used to connect the welding points using a minimum spanning tree method to obtain a first trajectory;

[0163] a sixth processing module, configured to sort the welding points in the first trajectory by a depth-first search algorithm to obtain a second trajectory;

[0164] The seventh processing module is used to smooth the second trajectory using a non-uniform rational B-spline curve to obtain a target welding trajectory.

[0165] In order to obtain the first trajectory, in an optional embodiment, the fifth processing module includes:

[0166] The first processing submodule is used to treat the welding points as nodes, connect each node with all other nodes through edges, and obtain a welding point graph;

[0167] The second processing submodule is used to calculate the Euclidean distance between each node in the solder joint graph and determine the Euclidean distance as the weight of the edge to obtain a weighted graph;

[0168] a third processing submodule, configured to determine any node as a current node of the spanning tree, and repeatedly add nodes connected to the current node and corresponding to the minimum weight to the spanning tree until the spanning tree includes all nodes in the weighted graph;

[0169] The fourth processing submodule is configured to connect the welding points in sequence according to the spanning tree to obtain a first trajectory.

[0170] In order to ensure the accuracy of welding, in an optional embodiment, the generating unit further includes:

[0171] A first determination module is configured to determine two target planes corresponding to the weld of each welding point based on the neighborhood data points of each welding point in the target welding trajectory, and to add and normalize the normal vectors corresponding to the two target planes to obtain a z-axis basis vector corresponding to each welding point;

[0172] The second determination module is used to determine the tangent line and the tangent line direction corresponding to each welding point according to the target welding trajectory, and generate an x-axis basis vector according to the tangent line and the tangent line direction;

[0173] A third determining module is used to determine the y-axis basis vector according to the z-axis basis vector and the x-axis basis vector;

[0174] The fourth determination module is used to determine the welding gun angle corresponding to each welding point based on the z-axis basis vector, determine the welding gun movement direction corresponding to each welding point based on the x-axis basis vector, and determine the welding gun swing amplitude corresponding to each welding point based on the y-axis basis vector to obtain the operating parameters of each welding point.

[0175] The welding control device of the welding robot includes a processor and a memory. The first acquisition unit, first processing unit, second acquisition unit, second processing unit, and generation unit are all stored as program units in the memory. The processor executes the program units stored in the memory to implement the corresponding functions. The modules are all located in the same processor; alternatively, the modules can be located in different processors in any combination.

[0176] The processor contains a kernel, which calls the corresponding program unit from the memory. The kernel can be set to one or more, and the welding accuracy can be improved by adjusting the kernel parameters.

[0177] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0178] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program. When the program is run, the device where the computer-readable storage medium is located is controlled to execute the welding control method of the welding robot.

[0179] An embodiment of the present invention provides a processor, which is used to run a program, wherein the welding control method of the welding robot is executed when the program is run.

[0180] An embodiment of the present invention provides a communication system, which includes a primary communication domain, a secondary communication domain processor, a memory, and a program stored in the memory and runnable on the processor. When the processor executes the program, at least the steps of the welding control method of the above-mentioned welding robot are implemented.

[0181] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes at least the steps of the welding control method of the welding robot described above.

[0182] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0183] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0184] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0185] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0187] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0188] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0189] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0190] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0191] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0192] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0193] 1) The welding control method of the welding robot of the present application first collects point cloud data of the workpiece to be welded in real time to obtain a first point cloud; then, an encoder is constructed based on sparse convolution, and at least the encoder, decoder and filter are combined into an instance segmentation model, and the filter is used to filter the weld point cloud in the first point cloud; then, the first point cloud is processed using the instance segmentation model to obtain multiple weld instance objects, where each weld instance object includes a point cloud corresponding to a weld; then, multiple skeleton points of each weld instance object are extracted using a point cloud skeleton extraction algorithm to obtain the welding points of each weld; finally, all welding points of each weld are connected and the connecting lines between the welding points are smoothed to obtain the target welding trajectory of each weld, and the operating parameters of each welding point in the target welding trajectory are determined. The welding gun is controlled to weld each weld in sequence according to the target welding trajectory and the operating parameters, and the operating parameters include the welding gun angle, welding gun movement direction and welding gun swing amplitude. The present application uses sparse convolution to construct an encoder, and learns and extracts features of different types of welds in an end-to-end deep learning manner, thereby improving the accuracy of feature extraction in complex scenarios. Furthermore, the welding points in the weld instance object after model processing are obtained through the point cloud skeleton extraction algorithm, which realizes the accurate extraction of the welding trajectory of irregular welds. Compared with the point cloud processing method that relies on manual feature processing in the existing technology, the recognition accuracy and trajectory planning accuracy in complex weldments and multi-type weld scenarios are enhanced, so as to solve the problem that the welding trajectory generation method in the existing technology has low accuracy and cannot meet the needs of large-scale automated welding.

[0194] 2) The welding control device of the welding robot of the present application comprises a first acquisition unit that collects point cloud data of the workpiece to be welded in real time to obtain a first point cloud; a first processing unit that constructs an encoder based on sparse convolution, and combines at least the encoder, decoder, and filter into an instance segmentation model, wherein the filter is used to filter the weld point cloud in the first point cloud; a second acquisition unit that processes the first point cloud using the instance segmentation model to obtain multiple weld instance objects, wherein each weld instance object includes a point cloud corresponding to a weld; a second processing unit that extracts multiple skeleton points of each weld instance object using a point cloud skeleton extraction algorithm to obtain welding points of each weld; a generation unit that connects all welding points of each weld and smoothes the connecting lines between the welding points to obtain a target welding trajectory for each weld, determines the operating parameters of each welding point in the target welding trajectory, and controls the welding gun to weld each weld in sequence according to the target welding trajectory and the operating parameters, wherein the operating parameters include the welding gun angle, the welding gun movement direction, and the welding gun swing amplitude. The present application uses sparse convolution to construct an encoder, and learns and extracts features of different types of welds in an end-to-end deep learning manner, thereby improving the accuracy of feature extraction in complex scenarios. Furthermore, the welding points in the weld instance object after model processing are obtained through the point cloud skeleton extraction algorithm, which realizes the accurate extraction of the welding trajectory of irregular welds. Compared with the point cloud processing method that relies on manual feature processing in the existing technology, the recognition accuracy and trajectory planning accuracy in complex weldments and multi-type weld scenarios are enhanced, so as to solve the problem that the welding trajectory generation method in the existing technology has low accuracy and cannot meet the needs of large-scale automated welding.

[0195] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A welding control method for a welding robot, characterized in that: include: Collect point cloud data of the workpiece to be welded in real time to obtain the first point cloud; constructing an encoder based on sparse convolution, combining at least the encoder, the decoder, and the filter into an instance segmentation model, wherein the filter is used to filter the weld point cloud in the first point cloud; Processing the first point cloud using the instance segmentation model to obtain a plurality of weld instance objects, where each weld instance object includes a point cloud corresponding to a weld; Extracting multiple skeleton points of each weld instance object using a point cloud skeleton extraction algorithm to obtain welding points of each weld; All the welding points of each weld are connected, and the connecting lines between the welding points are smoothed to obtain a target welding trajectory of each weld, and operating parameters of each welding point in the target welding trajectory are determined. The welding gun is controlled to weld each weld in sequence according to the target welding trajectory and the operating parameters, wherein the operating parameters include a welding gun angle, a welding gun movement direction, and a welding gun swing amplitude.

2. The method according to claim 1, characterized in that Constructing an encoder based on sparse convolution, and combining at least the encoder, decoder, and filter into an instance segmentation model, including: The encoder is constructed by replacing the two-dimensional convolution in the ResNet50 network with the sparse convolution, and a decoder corresponding to the encoder is constructed based on the U-Net architecture; Combining the encoder and decoder into a point cloud feature extractor, and sequentially combining the point cloud feature extractor, the multi-layer perceptron, and the fully connected layer to obtain a semantic segmentation module; Constructing the filter, constructing a clusterer based on a point cloud density clustering algorithm with normal vector constraints, combining the filter and the clusterer in sequence to obtain an instance clustering module, wherein the clusterer is used to segment the weld point cloud into the weld instance objects; combining the semantic segmentation module and the instance clustering module into an alternative segmentation model; Deep learning is performed on the candidate segmentation model to obtain the instance segmentation model.

3. The method according to claim 2, characterized in that Performing deep learning on the candidate segmentation model to obtain the instance segmentation model includes: Acquiring point cloud data of the workpiece to be welded to obtain a second point cloud, and determining a semantic label and an instance label for each data point in the second point cloud, wherein the semantic label is used to characterize the type of physical object to which the data point belongs, and the instance label includes the semantic label and is used to uniquely identify the physical object to which the data point belongs; Inputting the second point cloud into the semantic segmentation module to obtain a semantic segmentation result, wherein the semantic segmentation result includes a probability that each data point belongs to each physical object type; Calculating the loss function value between the semantic segmentation result and the semantic label by using a cross entropy loss function; When the loss function value is greater than a first threshold, performing backpropagation on the semantic segmentation module according to the loss function value until the loss function value is less than the first threshold; Constructing multiple groups of grid parameters corresponding to the parameters of the point cloud density clustering algorithm with normal vector constraints using a grid search method, inputting the semantic segmentation results output by the trained semantic segmentation module into the instance clustering module configured with each group of grid parameters, respectively, to obtain multiple predicted weld instance objects; The instance clustering module is configured according to the grid parameters corresponding to the minimum deviation between the predicted weld instance object and the instance label to obtain the instance segmentation model.

4. The method according to claim 1, wherein Extracting multiple skeleton points of each weld instance object using a point cloud skeleton extraction algorithm includes: uniform sampling is used to select multiple data points in the weld instance object to determine as candidate skeleton points; Initializing multiple clusters with each candidate skeleton point as a cluster center, and dividing each data point in the weld instance object into a cluster corresponding to the candidate skeleton point having the smallest distance to the data point, to obtain multiple sub-point clouds; a replacement step of replacing the corresponding candidate skeleton point with the centroid point corresponding to each of the sub-point clouds to obtain a candidate skeleton point; a merging step of merging the sub-point clouds until the distance between the candidate skeleton points corresponding to any two of the sub-point clouds is greater than a second threshold; The replacing step and the merging step are repeated at least once in sequence until the number of the candidate skeleton points between two adjacent iterations is the same, and all the candidate skeleton points are determined as the welding points.

5. The method according to claim 1, wherein Connecting all the welding points of each weld and smoothing the connection lines between the welding points to obtain a target welding trajectory of each weld, including: Connecting the welding points by a minimum spanning tree method to obtain a first trajectory; Sorting the welding points in the first trajectory by a depth-first search algorithm to obtain a second trajectory; The second trajectory is smoothed using a non-uniform rational B-spline curve to obtain the target welding trajectory.

6. The method according to claim 5, characterized in that Connecting the welding points by a minimum spanning tree method to obtain a first trajectory includes: The welding points are taken as nodes, and each node is connected to all other nodes through edges to obtain a welding point graph; Calculating the Euclidean distance between each pair of nodes in the solder joint graph, and determining the Euclidean distance as the weight of the edge to obtain a weighted graph; Determine any one node as a current node of a spanning tree, and repeatedly add nodes connected to the current node and corresponding to minimum weights to the spanning tree until the spanning tree includes all nodes in the weighted graph; The welding points are connected in sequence according to the spanning tree to obtain the first trajectory.

7. The method according to claim 1, characterized in that Determining the operating parameters of each welding point in the target welding trajectory includes: Determining two target planes corresponding to the weld of each welding point based on neighborhood data points of each welding point in the target welding trajectory, adding and normalizing the normal vectors corresponding to the two target planes to obtain a z-axis basis vector corresponding to each welding point; Determining the tangent line and the tangent line direction corresponding to each welding point according to the target welding trajectory, and generating an x-axis basis vector according to the tangent line and the tangent line direction; Determine a y-axis basis vector based on the z-axis basis vector and the x-axis basis vector; The welding gun angle corresponding to each welding point is determined based on the z-axis basis vector, the welding gun movement direction corresponding to each welding point is determined based on the x-axis basis vector, and the welding gun swing amplitude corresponding to each welding point is determined based on the y-axis basis vector to obtain the operating parameters of each welding point.

8. An intelligent welding trajectory generating device for a welding robot, characterized in that: The device comprises: A first acquisition unit is used to collect point cloud data of the workpiece to be welded in real time to obtain a first point cloud; a first processing unit, configured to construct an encoder based on sparse convolution, and to combine at least the encoder, the decoder, and the filter into an instance segmentation model, wherein the filter is configured to filter the weld point cloud in the first point cloud; a second acquisition unit, configured to process the first point cloud using the instance segmentation model to obtain a plurality of weld instance objects, wherein each weld instance object includes a point cloud corresponding to a weld; A second processing unit is configured to extract multiple skeleton points of each weld instance object by using a point cloud skeleton extraction algorithm to obtain welding points of each weld; A generating unit is configured to connect all the welding points of each weld and smooth the connecting lines between the welding points to obtain a target welding trajectory of each weld, determine operating parameters of each welding point in the target welding trajectory, and control a welding gun to weld each weld in sequence according to the target welding trajectory and the operating parameters, wherein the operating parameters include a welding gun angle, a welding gun movement direction, and a welding gun swing amplitude.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

10. A welding control system, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method of any one of claims 1 to 7.

Images