Automatic welding method, system, central control device and computer readable storage medium

Abstract

The application is suitable for the technical field of welding, and provides an automatic welding method, system, central control equipment and computer readable storage medium. The automatic welding method comprises the following steps: when it is detected that a welding matrix is placed on a welding platform and welding accessories matched with the welding matrix are placed on a material platform, determining a accessory stacking state of the material platform according to first RGB images and first point cloud data of the material platform collected by a first shooting component on a clamp of a mechanical arm; determining a grabbing priority of each welding accessory according to the accessory stacking state; determining a welding sequence of each welding accessory according to the grabbing priority; and in sequence for each welding accessory according to the welding sequence, controlling the mechanical arm to grab the welding accessory from the material platform and move the welding accessory to a target welding position of the welding matrix, and controlling a welding machine to weld the target welding position based on target values of various welding parameters, so that full-automatic welding can be realized and the labor cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of welding, and particularly relates to an automatic welding method, system, central control device and computer readable storage medium. BACKGROUND

[0002] At present, welding robots are widely used in hardware welding work. The traditional hardware welding robot usually welds according to the fixed trajectory of teaching, which causes the hardware welding robot to be unable to grab the welding accessories at different positions, and the welding accessories need to be clamped on the mechanical arm clamp of the welding robot by artificial one by one, that is, the whole welding process needs to be assisted by artificial. In addition, when welding different hardware workpieces, professional personnel need to reprogram and teach the hardware welding robot. It can be seen that the traditional welding method needs to be assisted by artificial in multiple welding links, and cannot realize full-automatic welding. SUMMARY

[0003] Therefore, the embodiments of the present application provide an automatic welding method, system, central control device and computer readable storage medium to solve the technical problem that the existing welding method needs to be assisted by artificial in multiple welding links and cannot realize full-automatic welding.

[0004] In a first aspect, the embodiments of the present application provide an automatic welding method, comprising:

[0005] When it is detected that a welding platform is placed with a welding matrix and a material platform is placed with welding accessories matched with the welding matrix, determining a stack state of the welding accessories of the material platform according to first RGB images and first point cloud data of the material platform collected by a first shooting assembly on a clamp of a mechanical arm;

[0006] Determining a grabbing priority of each welding accessory according to the stack state of the welding accessories;

[0007] Determining a welding sequence of each welding accessory according to the grabbing priority;

[0008] According to the welding sequence, the mechanical arm is controlled to grab the welding accessory from the material platform and move the welding accessory to a target welding position of the welding matrix for each welding accessory, and a welding machine is controlled to weld the target welding position based on target values of various welding parameters.

[0009] In an optional implementation manner of the first aspect, the stack state of the welding accessories includes: existence of stacked accessories and non-existence of stacked accessories; correspondingly, determining the stack state of the welding accessories of the material platform according to the first RGB images and the first point cloud data of the material platform collected by the first shooting assembly on the clamp of the mechanical arm, comprises:

[0010] According to the corresponding segmentation regions of each of the welding fittings in the first RGB image, an intersection-over-union of the corresponding segmentation regions of each two welding fittings is calculated, and each two welding fittings with the intersection-over-union greater than a first preset intersection-over-union are determined as a group of candidate fittings respectively; wherein the segmentation region corresponding to the welding fitting is obtained by processing the first RGB image by using a preset instance segmentation model;

[0011] Two point cloud clusters corresponding to each group of the candidate fittings are determined from the first point cloud data, and a centroid height difference and a spatial overlap rate of the two point cloud clusters corresponding to each group of the candidate fittings are calculated;

[0012] For the two point cloud clusters corresponding to each group of the candidate fittings, if the centroid height difference of the two point cloud clusters is greater than a preset height difference, and the spatial overlap rate of the two point cloud clusters is greater than a preset overlap rate, then the corresponding two welding fittings are determined as a group of stacked fittings; if the centroid height difference of the two point cloud clusters is less than or equal to the preset height difference, or the spatial overlap rate of the two point cloud clusters is less than or equal to the preset overlap rate, then it is determined that the corresponding two welding fittings are not stacked.

[0013] In an optional implementation of the first aspect, determining a grasping priority of each of the welding fittings according to the fitting stacking state comprises:

[0014] In a case where the fitting stacking state is that there are stacked fittings, an upper welding fitting and a lower welding fitting in each group of stacked fittings are determined;

[0015] The grasping priority of all the upper welding fittings is determined as a first priority;

[0016] The grasping priority of the welding fittings that are not stacked and all the lower welding fittings is determined as a second priority; wherein the first priority is higher than the second priority.

[0017] In an optional implementation of the first aspect, determining a welding sequence of each of the welding fittings according to the grasping priority comprises:

[0018] For each welding fitting under the first priority, a weighted sum of a maximum height of a point cloud cluster corresponding to the welding fitting and a centroid height of the point cloud cluster corresponding to the welding fitting is calculated according to a first weighting coefficient corresponding to the maximum height and a second weighting coefficient corresponding to the centroid height, and a welding sequence of each of the welding fittings is determined according to the weighted sum; wherein the greater the weighted sum is, the earlier the welding sequence of the welding fitting is.

[0019] randomly assigning a welding sequence to each welding accessory under the second priority;

[0020] wherein the welding sequence of each welding accessory under the second priority is later than the latest welding sequence under the first priority.

[0021] In an optional implementation of the first aspect, sequentially for each welding accessory according to the welding sequence, the robot arm is controlled to pick up the welding accessory from the material platform and move the welding accessory to a target welding position of the welding body, and the welding machine is controlled to weld the target welding position based on target welding parameters, including:

[0022] sequentially for each welding accessory according to the welding sequence, determining a first actual pose of the welding accessory according to the first point cloud data and a 3D model of the welding accessory;

[0023] generating a picking motion trajectory according to the first actual pose and a second actual pose of the robot arm at present, and controlling the robot arm to pick up the welding accessory based on the picking motion trajectory;

[0024] determining a third actual pose and an actual welding trajectory of the target welding position corresponding to the welding accessory according to second point cloud data, a 3D model of the welding body and a preset welding trajectory;

[0025] generating a moving trajectory according to the third actual pose and a fourth actual pose of the robot arm after picking up the welding accessory, and controlling the robot arm to move the picked welding accessory to the target welding position based on the moving trajectory;

[0026] controlling the welding machine to weld the target welding position along the actual welding trajectory based on target values of the welding parameters.

[0027] In an optional implementation of the first aspect, determining a third actual pose and an actual welding trajectory of the target welding position corresponding to the welding accessory according to second point cloud data, a 3D model of the welding body and a preset welding trajectory, includes:

[0028] registering the second point cloud data with the 3D model of the welding body to determine an attitude difference vector of an actual pose of the welding body and a standard pose corresponding to the 3D model;

[0029] determining the actual welding trajectory corresponding to the target welding position according to the attitude difference vector and a preset welding trajectory corresponding to the target welding position;

[0030] According to a model of the welding accessory, a target point cloud cluster corresponding to the target welding position is determined from the second point cloud data, and the target point cloud cluster is registered with the 3D model of the welding matrix to obtain a third actual pose of the target welding position; wherein the model of the welding accessory is obtained by processing the first RGB image using a preset instance segmentation model.

[0031] In an optional implementation of the first aspect, the control of the welding machine to weld the target welding position along the actual welding trajectory based on the target values of the welding parameters comprises:

[0032] The actual welding trajectory is divided into a plurality of straight-line welding segments of equal length, and a fifth actual pose of each straight-line welding segment is determined;

[0033] The welding machine is controlled to be in each fifth actual pose in sequence, and each straight-line welding segment is spot-welded using the target value of each welding parameter.

[0034] In a second aspect, the embodiments of the present application provide an automatic welding system, comprising a central control device, a mechanical arm, a first shooting assembly, a second shooting assembly, and a welding machine; the mechanical arm, the first shooting assembly, the second shooting assembly, and the welding machine are all controlled by the central control device;

[0035] The first shooting assembly is installed on a clamp at the end of the mechanical arm, and is used to collect a first RGB image and first point cloud data of a material platform when the mechanical arm is in a preset pose;

[0036] The second shooting assembly is installed above a welding platform, and is used to collect a second RGB image and second point cloud data of the welding platform;

[0037] The central control device is used to execute the method of any optional implementation of the first aspect.

[0038] In a third aspect, the embodiments of the present application provide another central control device, comprising a memory and a computer program stored in the memory and executable on a processor, and the processor executes the computer program to implement the method of any optional implementation of the first aspect.

[0039] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the automatic welding method of any optional implementation of the first aspect.

[0040] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on the central control device, causes the central control device to implement the automatic welding method in any optional implementation manner of the first aspect.

[0041] The automatic welding method, system, central control device, computer readable storage medium and computer program product provided by the embodiments of the present application have the following beneficial effects:

[0042] The automatic welding method provided by the embodiments of the present application determines the part stacking state of the material platform according to the first RGB image and the first point cloud data of the material platform, determines the grabbing priority of each welding part according to the part stacking state, and automatically determines the welding sequence of each welding part according to the grabbing priority, so as to facilitate the control of the robot to automatically grab the corresponding welding part from the material platform according to the welding sequence of each welding part and deliver it to the target welding position of the welding matrix for welding, thereby realizing the full automation of the welding process without manual assistance. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0044] Figure 1 A structural schematic diagram of an automatic welding system provided by an embodiment of the present application;

[0045] Figure 2 A schematic flowchart of an automatic welding method provided by an embodiment of the present application;

[0046] Figure 3 An implementation flowchart of S201 in an automatic welding method provided by an embodiment of the present application;

[0047] Figure 4 An implementation flowchart of S202 in an automatic welding method provided by an embodiment of the present application;

[0048] Figure 5 An implementation flowchart of S203 in an automatic welding method provided by an embodiment of the present application;

[0049] Figure 6 An implementation flowchart of S204 in an automatic welding method provided by an embodiment of the present application;

[0050] Figure 7 A structural schematic diagram of a central control device provided by an embodiment of the present application;

[0051] Figure 8 A structural schematic diagram of a central control device provided for another embodiment of the present application is shown. DETAILED DESCRIPTION

[0052] The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and thus only serve as examples, but cannot be used to limit the protection scope of the present application.

[0053] In the description of the embodiments of the present application, the technical terms "comprise", "contain", "have" and any variants thereof mean "include but not limited to", unless otherwise specifically emphasized. In the description of the embodiments of the present application, unless otherwise specified, the technical term "multiple" means two or more than two, and the technical terms "at least one" and "one or more" mean one, two or more than two. The technical terms "first", "second" and the like are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. The technical term "and / or" is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A existing alone, A and B existing together, and B existing alone. In addition, the character " / " in this paper generally represents an "or" relationship between the front and rear associated objects.

[0054] The embodiments of the present application first provide an automatic welding system. Figure 1 A structural schematic diagram of an automatic welding system provided for an embodiment of the present application is shown. As shown in the figure, the automatic welding system can include a central control device 11, a mechanical arm 12, a first shooting assembly 13, a second shooting assembly 14 and a welding machine 15. Among them, the mechanical arm 12, the welding machine 15, the first shooting assembly 13 and the second shooting assembly are directly controlled by the central control device 11. Figure 1

[0055] Exemplarily, the mechanical arm 12 can be a six-axis mechanical arm. The six-axis mechanical arm refers to a mechanical arm capable of moving in the directions of x axis, y axis and z axis, and capable of rotating with x axis, y axis and z axis as the rotation axis. Optionally, the end of the mechanical arm 12 can be provided with a clamp. The first shooting assembly 13 can be installed on the clamp in a way that the lens faces outward, forming a structure of "eyes on the hands".

[0056] Exemplarily, the first shooting assembly 13 can include a first RGB camera and a first point cloud camera.

[0057] ​Optionally, during the grasping preparation stage of the welding accessory, the central control device 11 can control the robot arm 12 to be in a preset pose. When the robot arm 12 is in the preset pose, the gripper at the end of the robot arm 12 is located directly above the material platform, and the first shooting assembly 13 installed on the gripper is directly opposite the material platform, so that the first RGB camera in the first shooting assembly 13 can collect a first RGB image of the material platform, and the first point cloud camera can collect first point cloud data of the material platform. The material platform refers to a platform for placing welding accessories.

[0058] Exemplarily, the welding accessories on the material platform can be conveyed by the first conveying belt. The timing of the first conveying belt conveying the welding accessories to the material platform can be controlled by the central control device 11. For example, when the central control device 11 detects that the welding platform is placed with a welding matrix, the central control device 11 can send a start running instruction to the controller of the first conveying belt to control the first conveying belt to convey each welding accessory corresponding to the welding matrix to the material platform. For another example, when all the welding accessories corresponding to the welding matrix are conveyed to the material platform, the central control device 11 can send a stop running instruction to the controller of the first conveying belt to control the first conveying belt to stop running. By automatically controlling the first conveying belt to run when it is needed to convey the welding accessories and to stop running when it is not needed to convey the welding accessories, the idle time of the first conveying belt can be reduced.

[0059] Optionally, the second shooting assembly 14 can be installed directly above the welding platform, and the lens faces the welding platform. The welding platform refers to a platform for placing a welding matrix.

[0060] Exemplarily, the second shooting assembly 14 can include a second RGB camera and a second point cloud camera. The second RGB camera can be used to collect a second RGB image of the welding platform (or the welding matrix). The second point cloud camera can be used to collect second point cloud data of the welding platform (or the welding matrix).

[0061] Exemplarily, the welding machine 15 can be a five-axis motion welding machine. The five-axis motion welding machine refers to a welding machine that can move in x axis direction, y axis direction, and z axis direction, and can rotate around x axis and z axis. The welding machine 15 can automatically weld each welding accessory to the corresponding welding position on the welding matrix based on the target value of each welding parameter under the control of the central control device 11.

[0062] The target value of each welding parameter can refer to the optimal value of each welding parameter that can make the quality of the welding finished product meet the expected requirements.

[0063] Exemplarily, the welding parameters can include welding current, welding voltage, welding speed, and the like.

[0064] In actual application, the central control device can control the welding machine to perform welding based on values of a plurality of different welding parameters in advance, detect welding quality of welding products corresponding to the values of the different welding parameters, and determine target values of the welding parameters according to values of the welding parameters corresponding to welding products with qualified quality.

[0065] In actual application, before the automatic welding system starts work, the central control device 11 can be imported with the model, 3D model, preset welding track, and list of accessories of each welding body to be welded in advance.

[0066] The preset welding track can be used to describe an expected welding path corresponding to each welding position on the welding body. Exemplarily, the preset welding track can be manually drawn on the 3D model of the welding body in advance. Since each welding position of the welding body can be used to weld welding accessories of different models, different welding positions of the welding body can correspond to different preset welding tracks. For example, for a kettle, the welding body can be a kettle body, the welding accessories can include a spout and a handle, the preset welding track corresponding to the spout can be a circular track, and the welding track corresponding to the handle can be a rectangular track.

[0067] Exemplarily, the list of accessories of the welding body can include the model and 3D model of the welding accessory corresponding to each welding position on the welding body.

[0068] Optionally, the central control device 11 can also be configured with a preset instance segmentation model. The preset instance segmentation model can be used to identify each target object (for example, a welding accessory) in an RGB image and perform region segmentation on each target object in the RGB image to obtain the model and segmentation region of each target object.

[0069] Exemplarily, the preset instance segmentation model can adopt a mask region-based convolutional neural network (Mask R-CNN) architecture, or can adopt other architectures. The specific architecture of the preset instance segmentation model is not limited in the present application.

[0070] The central control device 11 can specifically perform full-automatic welding control on the automatic welding system based on the automatic welding method in the subsequent embodiments. It should be noted that the specific process of the central control device 11 performing full-automatic welding control on the automatic welding system can refer to the related description in the subsequent embodiments, which will not be described in detail here.

[0071] The embodiment of the application further provides an automatic welding method, and an execution subject of the automatic welding method can be a central control device 11 in the automatic welding system as shown in the figure. Exemplarily, Figure 1 An exemplary flow chart of an automatic welding method provided by the embodiment of the application is shown in the figure. As shown in the figure, Figure 2 An exemplary flow chart of an automatic welding method provided by the embodiment of the application is shown in the figure. As shown in the figure, Figure 2 The automatic welding method can include S201-S204, and details are as follows:

[0072] S201, when it is detected that a welding platform is placed with a welding body and a material platform is placed with a welding accessory matched with the welding body, determining a stacking state of the welding accessory on the material platform according to first RGB images and first point cloud data of the material platform collected by a first shooting assembly on a clamp of a mechanical arm.

[0073] Optionally, the central control device can control the second RGB camera to collect a second RGB image of the welding platform every first time length, and identify whether there is a welding body to be welded in the second RGB image. The first time length can be set according to actual requirements, and the embodiment of the application does not limit it.

[0074] Exemplarily, the central control device can process the second RGB image by using a preset instance segmentation model to obtain a model and a segmentation area of each target object in the second RGB image. Based on this, when the model of at least one target object in the second RGB image is the same as the model of the welding body to be welded, the central control device can determine that the welding platform is placed with the welding body. When there is no target object with the same model as the welding body in the second RGB image, the central control device can determine that the welding platform is not placed with the welding body.

[0075] Optionally, when the central control device detects that the welding platform is placed with the welding body, it can determine whether the material platform is placed with the welding accessory matched with the welding body through S2011-S2016 as shown in the figure, Figure 3

[0076] S2011, controlling the mechanical arm to be in a preset pose.

[0077] The preset pose can make the clamp at the end of the mechanical arm be located directly above the material platform, and the first shooting assembly installed on the clamp faces the material platform.

[0078] S2012, controlling the first shooting assembly to collect first RGB images and first point cloud data of the material platform, and controlling the second shooting assembly to collect second RGB images and second point cloud data of the welding body.

[0079] ​The first point cloud data can include point cloud clusters corresponding to respective welding fittings on the material platform. Each point cloud cluster can include a plurality of point clouds, each of which can correspond to a three-dimensional coordinate point in a three-dimensional space, and the plurality of point clouds are used to describe the actual pose of the corresponding welding fitting in the three-dimensional space.

[0080] The second point cloud data can include a point cloud cluster corresponding to the welding matrix, and the point cloud cluster corresponding to the welding matrix can include a plurality of point clouds, each of which can correspond to a three-dimensional coordinate point in a three-dimensional space, and the plurality of point clouds are used to describe the actual pose of the welding matrix in the three-dimensional space.

[0081] S2013, a preset instance segmentation model is used to process the first RGB image to obtain the model and the corresponding segmentation region of each welding fitting on the material platform, and a preset instance segmentation model is used to detect the target of the second RGB image to obtain the model and the corresponding segmentation region of the welding matrix.

[0082] The segmentation region corresponding to the welding fitting can refer to the region occupied by the welding fitting in the first RGB image.

[0083] The segmentation region corresponding to the welding matrix can refer to the region occupied by the welding matrix in the second RGB image.

[0084] S2014, determining the fitting list of the welding matrix according to the model of the welding matrix.

[0085] S2015, when the model of one or more welding fittings on the material platform is included in the fitting list of the welding matrix, it is determined that the welding fitting matching the welding matrix is placed on the material platform.

[0086] S2016, when the model of any welding fitting on the material platform is not included in the first fitting list, it is determined that the welding fitting matching the welding matrix is not placed on the material platform.

[0087] When the central control device detects that the welding fitting matching the welding matrix is placed on the material platform, the central control device can determine the fitting stacking state of the material platform according to the first RGB image and the first point cloud data.

[0088] The fitting stacking state can be used to describe whether there is a stacked fitting on the material platform. Illustratively, the fitting stacking state can include two states of existing stacked fittings and non-existing stacked fittings.

[0089] Optionally, the central control device can determine the fitting stacking state of the material platform through S2017-S2019 in the following Figure 3

[0090] ​S2017, calculate an intersection-over-union of the segmentation region corresponding to each two welding fittings according to the segmentation region corresponding to each two welding fittings in the first RGB image, and determine each two welding fittings with the intersection-over-union greater than a first preset intersection-over-union as a group of candidate fittings respectively.

[0091] Optionally, the central control device can determine a ratio of an intersection area to a union area of the segmentation region corresponding to each two welding fittings as the intersection-over-union of the segmentation region corresponding to each two welding fittings respectively.

[0092] The first preset intersection-over-union can be set according to actual needs, and embodiments of the present application do not limit it.

[0093] Embodiments of the present application can roughly screen the stacked fittings on the material platform by determining each two welding fittings with the intersection-over-union of the segmentation region greater than the first preset intersection-over-union as a group of candidate fittings respectively.

[0094] S2018, determine two point cloud clusters corresponding to each group of candidate fittings from the first point cloud data, and calculate a centroid height difference and a spatial overlap rate of the two point cloud clusters corresponding to each group of candidate fittings.

[0095] It can be understood that since each group of candidate fittings includes two welding fittings, each group of candidate fittings corresponds to two point cloud clusters. Optionally, for any one point cloud cluster corresponding to each group of candidate fittings, the central control device can determine an average value of the vertical height (i.e. z axis value) of all point clouds included in the point cloud cluster as the centroid height of the point cloud cluster, and determine a difference value of the centroid heights of the two point cloud clusters corresponding to each group of candidate fittings as the centroid height difference of the point cloud clusters corresponding to each group of candidate fittings.

[0096] Optionally, the central control device can specifically calculate the spatial overlap rate of the point cloud clusters corresponding to each group of candidate fittings through steps 1.1-1.3, which are described in detail as follows:

[0097] Step 1.1, determine a three-dimensional boundary of a target voxel grid according to the first point cloud data of the material platform, and construct the target voxel grid according to the three-dimensional boundary of the target voxel grid and a preset voxel resolution.

[0098] Exemplarily, the three-dimensional boundary of the target voxel grid can be represented by an origin point of the target voxel grid, an x-axis boundary value, a y-axis boundary value, and a z-axis boundary value. Optionally, the central control device can determine the minimum x axis value x min , the minimum y axis value y min , and the minimum z axis value zmin Corresponding three-dimensional coordinate points ( x min , y min , z min The origin of the target voxel mesh is determined by this point. The central control device can then determine the largest value in the first point cloud data. x Axis value x max ,maximum y Axis value y max and the largest z Axis value z max The target voxel meshes were determined respectively. x Axis boundary values, y Axis boundary values ​​and z Axis boundary values.

[0099] A preset voxel resolution can be used to describe the variable length of each voxel. The preset voxel resolution can be set according to actual needs, and this application embodiment does not limit it.

[0100] Optionally, after determining the three-dimensional boundary of the target voxel mesh, the central control equipment can determine the target voxel mesh's resolution at each location using the following formula, based on the target voxel mesh's three-dimensional boundary and the preset voxel resolution. x axis, y shaft and z Number of voxels on the axis:

[0101]

[0102] in, N x For the target voxel mesh in x Number of voxels on the axis N y For the target voxel mesh in y Number of voxels on the axis N z For the target voxel mesh in z Number of voxels on the axis r The preset voxel resolution, This is the floor function.

[0103] Optionally, the central control device can determine the origin of the target voxel mesh and the target voxel mesh location based on the target voxel mesh location. x The number of voxels on the axis, the target voxel mesh in y The number of voxels on the axis and the target voxel mesh in zThe number of voxels on the axis is used to construct a target voxel grid. Each voxel in the target voxel grid can be uniquely identified by a three-dimensional voxel index (x, y, z) for example. i x , i y , i z wherein 1≤x≤X, 1≤y≤Y and 1≤z≤Z are all integers. i x ≤X, 1≤Y and 1≤Z are all integers. N x , 1≤x≤X, 1≤y≤Y and 1≤z≤Z are all integers. i y ≤X, 1≤Y and 1≤Z are all integers. N y , 1≤x≤X, 1≤y≤Y and 1≤z≤Z are all integers. i z ≤X, 1≤Y and 1≤Z are all integers. N z , i x , i y and i z are all integers.

[0104] Step 1.2, determining two voxel sets in the global voxel network to which the two point cloud clusters corresponding to each group of candidate accessories are mapped.

[0105] Optionally, for any one point cloud cluster corresponding to each group of candidate accessories, the central control device can traverse each point cloud in the point cloud cluster, calculate the voxel index corresponding to the point cloud according to the three-dimensional coordinates of the point cloud, the origin coordinates of the target voxel grid and the preset voxel resolution, and determine the set of voxel indexes corresponding to all point clouds included in the point cloud cluster as the voxel set in the global voxel network to which the point cloud cluster is mapped.

[0106] Specifically, for any one point cloud cluster corresponding to each group of candidate accessories, the central control device can calculate the voxel index corresponding to each point cloud in the point cloud cluster by the following formula:

[0107]

[0108] wherein, v xi is the x-axis voxel index corresponding to the i-th point cloud in the point cloud cluster, i xi is the y-axis voxel index corresponding to the i-th point cloud in the point cloud cluster, x xi is the z-axis voxel index corresponding to the i-th point cloud in the point cloud cluster, y i y z i z x ​​​​​​i is the z-axis coordinate of the i-th point cloud in the point cloud cluster, i is the z-axis coordinate of the i-th point cloud in the point cloud cluster, x is the z-axis coordinate of the i-th point cloud in the point cloud cluster, y i is the z-axis coordinate of the i-th point cloud in the point cloud cluster, i is the z-axis coordinate of the i-th point cloud in the point cloud cluster, y is the z-axis coordinate of the i-th point cloud in the point cloud cluster, z i is the z-axis coordinate of the i-th point cloud in the point cloud cluster, i is the z-axis coordinate of the i-th point cloud in the point cloud cluster, z is the z-axis coordinate of the i-th point cloud in the point cloud cluster, is a floor function.

[0109] Step 1.3, the ratio of the intersection voxel number of the voxel sets of the two point cloud clusters corresponding to each candidate assembly group to the union voxel number is determined as the spatial overlap rate of the two point cloud clusters corresponding to each candidate assembly group.

[0110] Optionally, for the two point cloud clusters corresponding to each candidate assembly group, the central control device can determine the number of the same voxel indexes in the voxel sets corresponding to the two point cloud clusters as the intersection voxel number of the voxel sets of the two point cloud clusters. The central control device can determine the difference between the sum of the voxel numbers included in the two voxel sets corresponding to the two point cloud clusters and the intersection voxel number as the union voxel number of the voxel sets of the two point cloud clusters.

[0111] S2019, for the two point cloud clusters corresponding to each candidate assembly group, if the centroid height difference of the two point cloud clusters is greater than a preset height difference, and the spatial overlap rate of the two point cloud clusters is greater than a preset overlap rate, the two welding assemblies corresponding to the two point cloud clusters are determined as a group of stacked assemblies; if the centroid height difference of the two point cloud clusters is less than or equal to the preset height difference, or the spatial overlap rate of the two point cloud clusters is less than or equal to the preset overlap rate, it is determined that the two welding assemblies corresponding to the two point cloud clusters are not stacked.

[0112] The preset height difference and the preset overlap rate can be set according to actual needs, and the embodiments of the present application do not limit them.

[0113] It can be understood that, in the case that the centroid height difference of the two point cloud clusters corresponding to a certain candidate assembly group is greater than the preset height difference, it is indicated that the two welding assemblies corresponding to the two point cloud clusters are not on the same horizontal plane, that is, it is indicated that the possibility of stacking one welding assembly on another welding assembly is greater. On this basis, if the spatial overlap rate of the two point cloud clusters is greater than the preset threshold, it is indicated that the two welding assemblies corresponding to the two point cloud clusters have a certain contact, so it can be determined that the two welding assemblies corresponding to the two point cloud clusters are indeed stacked together.

[0114] It can also be understood that, in the case that the height difference of the two point cloud clusters corresponding to a certain group of candidate fittings is less than or equal to the preset height difference, it is indicated that the two welding fittings corresponding to the group are probably in the same horizontal plane, so it can be determined that the two welding fittings corresponding to the group are not stacked together.

[0115] It can also be understood that, in the case that the spatial overlap rate of the two point cloud clusters corresponding to a certain group of candidate fittings is less than or equal to the preset height difference, it is indicated that the two welding fittings corresponding to the group do not have a contact surface, so it can also be determined that the two welding fittings corresponding to the group are not stacked together.

[0116] S202, determining the grabbing priority of each welding fitting on the material platform according to the fitting stacking state of the material platform.

[0117] It can be understood that, in the case that there are stacked fittings on the material platform, if the fitting grabbing is directly performed in a disordered manner, there may be a case that a lower welding fitting in the stacked fittings is grabbed first and an upper fitting in the stacked fittings is grabbed later. Since the upper welding fitting in the stacked fittings will partially shield the lower welding fitting, the difficulty of grabbing the lower welding fitting is higher, therefore, the grabbing manner of grabbing the lower welding fitting first and then grabbing the upper welding fitting will not only reduce the success rate of grabbing the lower welding fitting, but also change the pose of the upper welding fitting, so that the pose of each welding fitting needs to be identified again before each welding fitting is grabbed subsequently, which not only increases the calculation complexity of the automatic welding method, but also reduces the overall welding efficiency.

[0118] Therefore, in order to improve the success rate of grabbing the welding fitting, reduce the calculation complexity of the automatic welding method, and improve the overall welding efficiency, the central control device in the embodiments of the present application can determine the grabbing priority of each welding fitting on the material platform according to the fitting stacking state of the material platform.

[0119] Optionally, the central control device can determine the grabbing priority of each welding fitting on the material platform through S2021-S2023 as shown in the figure. Figure 4 The details are as follows:

[0120] S2021, in the case that the fitting stacking state of the material platform is that there are stacked fittings, determining the upper welding fitting and the lower welding fitting in each group of stacked fittings.

[0121] Optionally, for each group of stacked fittings, the central control device can determine the welding fitting with a larger center of mass height of the corresponding point cloud cluster as the upper welding fitting, and determine the welding fitting with a smaller center of mass height of the corresponding point cloud cluster as the lower welding fitting.

[0122] S2022, determining the grabbing priority of all upper welding fittings as a first priority.

[0123] S2023, determine the grabbing priority of the unstacked welding accessory and all underlying welding accessories as the second priority; wherein the first priority is higher than the second priority.

[0124] S203, determine the welding sequence of each welding accessory according to the grabbing priority of each welding accessory.

[0125] It can be understood that the above S202 only roughly divides the grabbing priorities of the upper and lower welding accessories on the material platform, and since the first priority and the second priority can each include multiple welding accessories, it is further necessary to accurately determine the welding sequence (i.e. the grabbing sequence) of each welding accessory under each grabbing priority.

[0126] Optionally, the central control device can determine the welding sequence of each welding accessory through S2031-S2032 as shown in the following: Figure 5

[0127] S2031, for each welding accessory under the first priority, calculate the weighted sum of the maximum height of the point cloud cluster corresponding to the welding accessory and the centroid height of the point cloud cluster according to the maximum height, the centroid height, a first weighting coefficient corresponding to the maximum height, and a second weighting coefficient corresponding to the centroid height, and determine the welding sequence of each welding accessory according to the weighted sum.

[0128] Among them, the maximum height of the point cloud cluster corresponding to the welding accessory can refer to the maximum vertical height (i.e. the z-axis value) of all point clouds included in the point cloud cluster.

[0129] Optionally, since the centroid height of the point cloud cluster is used to reflect the overall height of the welding accessory corresponding to the point cloud cluster, and the maximum height of the point cloud cluster is used to reflect the height of the highest point of the welding accessory corresponding to the point cloud cluster, in order to improve the accuracy of the welding sequence determination, the first weighting coefficient can be set to be greater than the second weighting coefficient, thereby further improving the success rate of grabbing the welding accessory.

[0130] For each welding accessory under the first priority, the central control device can sort the welding accessories under the first priority in order of size after calculating the weighted sum of the maximum height and the centroid height of the point cloud cluster corresponding to each welding accessory, and determine the arrangement serial number corresponding to each welding accessory as the welding sequence of the welding accessory. Among them, the larger the weighted sum, the earlier the welding sequence of the welding accessory; the smaller the weighted sum, the later the welding sequence of the welding accessory.

[0131] ​Exemplarily, assuming that there are three welding fittings under the first priority, namely, welding fitting 1, welding fitting 2 and welding fitting 3, the above-mentioned weighted sum of the welding fitting 1 is greater than the above-mentioned weighted sum of the welding fitting 2, and the above-mentioned weighted sum of the welding fitting 3 is greater than the above-mentioned weighted sum of the welding fitting 1, then the three welding fittings can be sorted in the order of the weighted sum from large to small as welding fitting 3, welding fitting 1 and welding fitting 2, that is, the arrangement serial number (i.e., the welding sequence) corresponding to the welding fitting 3 is 1, the arrangement serial number (i.e., the welding sequence) corresponding to the welding fitting 1 is 2, and the arrangement serial number (i.e., the welding sequence) corresponding to the welding fitting 2 is 3. Then the central control device can determine the welding sequence of the welding fitting 3 as 1, the welding sequence of the welding fitting 1 as 2, and the welding sequence of the welding fitting 2 as 3.

[0132] S2032, randomly assigning a welding sequence to each welding fitting under the second priority; the welding sequence of all welding fittings under the second priority is later than the latest welding sequence under the first priority.

[0133] Exemplarily, assuming that the welding sequences of the welding fitting 3, the welding fitting 1 and the welding fitting 2 under the first priority are 1, 2 and 3 respectively, then the welding sequences of each welding fitting under the second priority can be 5, 6, …, M. Wherein, M is the number of welding fittings under the second priority.

[0134] S204, the welding sequence is sequentially for each welding fitting, controlling the robot arm to grab the welding fitting from the material platform and move the welding fitting to the target welding position of the welding parent body, and controlling the welding machine to weld the target welding position based on the target value of each welding parameter.

[0135] It should be noted that the specific acquisition method of the target value of each welding parameter can refer to the related description in the above-mentioned system embodiment, which will not be described here.

[0136] Optionally, S204 can include S2041-S2045 as shown in Figure 6 , which are described in detail as follows:

[0137] S2041, according to the welding sequence, sequentially for each welding fitting, determining the first actual pose of the welding fitting according to the first point cloud data and the 3D model of the welding fitting.

[0138] Optionally, the central control device can first determine the point cloud cluster corresponding to each welding accessory from the first point cloud data according to the segmentation area of each welding accessory obtained in S2013, and then sequentially for each welding accessory according to the welding sequence, register the point cloud cluster corresponding to the welding accessory with the 3D model of the welding accessory to obtain the first rotation matrix and the first translation matrix from the model coordinate system to the first camera coordinate system, and determine the first actual pose of the welding accessory according to the first rotation matrix and the first translation matrix.

[0139] The first camera coordinate system can refer to the camera coordinate system corresponding to the first shooting assembly.

[0140] The first rotation matrix and the first translation matrix are both three-dimensional matrices. The rotation matrix is used to describe the rotation amount around the x axis, y axis, and z axis. The translation matrix is used to describe the position on the x axis, y axis, and z axis.

[0141] Exemplarily, the central control device can use an iterative closest point (ICP) or a scale iterative closest point (SICP) algorithm to register the point cloud cluster corresponding to the welding accessory with the 3D model of the welding accessory.

[0142] Exemplarily, the first actual pose of the welding accessory can be a six-dimensional space pose ( x 1, y 1, z 1, α 1, β 1, θ 1). Wherein, ( x 1, y 1, z 1) can be used to describe the first position of the welding accessory in the three-dimensional space, and ( α 1, β 1, θ 1) can be used to describe the first rotation pose of the welding accessory around the x axis, y axis, and z axis in the three-dimensional space, respectively.

[0143] The embodiment of the present application can roughly determine the pose range of each welding accessory by identifying the segmentation area corresponding to each welding accessory from the first RGB image, can accurately locate the point cloud cluster of each welding accessory by determining the point cloud cluster corresponding to each welding accessory from the first point cloud data according to the segmentation area corresponding to each welding accessory, and can shorten the matching time of the point cloud data and the 3D model, improve the pose determination efficiency of the welding accessory, and further improve the overall welding efficiency by registering the point cloud cluster corresponding to each welding accessory with the 3D model thereof respectively.

[0144] S2042, generating a grabbing motion trajectory according to the first actual pose and the second actual pose of the mechanical arm, and controlling the mechanical arm to grab the welding accessory based on the grabbing motion trajectory.

[0145] Exemplarily, the second actual pose can be a six-dimensional space pose ( x 2, y 2, z 2, α 2, β 2, θ 2). Wherein, ( x 2, y 2, z 2) can be used to describe the second position of the mechanical arm in the three-dimensional space at present, ( α 2, β 2, θ 2) can be used to describe the second rotation attitude of the mechanical arm in the three-dimensional space at present respectively around x axis, y axis and z axis.

[0146] It can be understood that since the mechanical arm is in a preset pose when the first shooting assembly collects the first RGB image and the first point cloud data, the second actual pose is the preset pose.

[0147] It should be noted that the specific way of determining the grabbing motion trajectory according to the first actual pose and the second actual pose can refer to the related description in the prior art, and the embodiment of the present application does not make a detailed description.

[0148] S2043, determining the third actual pose of the target welding part corresponding to the welding accessory and the actual welding trajectory according to the second point cloud data of the welding matrix, the 3D model and the preset welding trajectory.

[0149] Exemplarily, the third actual pose can be a six-dimensional space pose ( x 3, y 3, z 3, α 3, β 3, θ3). Wherein, x 3, y 3, z 3) can be used to describe the third position of the target welding part in the three-dimensional space, α 3, β 3, θ 3) can be used to describe the third rotation posture of the target welding part in the three-dimensional space around x axis, y axis, and z axis, respectively.

[0150] Optionally, S2042 can be implemented through steps 2.1-2.3, which are described in detail as follows:

[0151] Step 2.1, register the second point cloud data with the 3D model of the welding matrix to determine the attitude difference vector between the actual pose of the welding matrix and the standard pose corresponding to the 3D model.

[0152] Optionally, the central control device can use an iterative closest point (ICP) or scale iterative closest point (SICP) algorithm to register the second point cloud data with the 3D model of the welding matrix, and the attitude difference vector between the actual pose of the welding matrix and the standard pose corresponding to the 3D model. Wherein, the attitude difference vector can be represented by a second rotation matrix and a second translation matrix. Exemplarily, the second rotation matrix and the second translation matrix are both three-dimensional matrices.

[0153] Step 2.2, according to the attitude difference vector and the preset welding trajectory corresponding to the target welding part, determine the actual welding trajectory corresponding to the target welding part.

[0154] Optionally, the central control device can correct the preset welding trajectory corresponding to the target welding part using the attitude difference vector, thereby obtaining the actual welding trajectory corresponding to the target welding part.

[0155] Step 2.3, according to the model of the welding accessory, determine the target point cloud cluster corresponding to the target welding part from the second point cloud data, and register the target point cloud cluster with the 3D model of the welding matrix to obtain the third actual pose of the target welding part.

[0156] It can be understood that since different welding parts on the welding matrix can be used to weld different welding accessories, the central control device can first determine the target welding part matching the model of the welding accessory from the second RGB image of the welding matrix according to the model of the welding accessory identified in the above S2013, and then determine the target point cloud cluster corresponding to the target welding part from the second point cloud data.

[0157] Optionally, the central control device can register the target point cloud cluster with the 3D model of the welding mother body to obtain the third rotation matrix and the third translation matrix from the model coordinate system to the second camera coordinate system, and determine the third actual pose of the target welding part based on the third rotation matrix and the third translation matrix.

[0158] The first camera coordinate system can refer to the camera coordinate system corresponding to the third shooting component.

[0159] Both the third rotation matrix and the third translation matrix are three-dimensional matrices.

[0160] S2044: Based on the third actual pose and the fourth actual pose after the robotic arm has grasped the welding parts, a movement trajectory is generated, and the robotic arm is controlled to move the grasped welding parts to the target welding position based on the movement trajectory.

[0161] For example, the fourth actual pose can be a six-dimensional spatial pose ( x 4, y 4, z 4, α 4, β 4, θ 4).

[0162] in,( x 4, y 4, z 4) It can be used to describe the fourth position in three-dimensional space after the robotic arm has grasped the welding component. α 4, β 4, θ 4) It can be used to describe how the robotic arm moves around the welding parts in three-dimensional space after grasping them. x axis, y shaft and z The fourth rotational orientation of the axis.

[0163] It should be noted that the specific method for determining the movement trajectory based on the third and fourth actual poses can be found in the relevant descriptions in the prior art, and will not be described in detail in the embodiments of this application.

[0164] S2045 controls the welding machine to weld the target welding part along the actual welding trajectory based on the target values ​​of various welding parameters.

[0165] Optionally, in order to accurately control the welding machine to weld the target welding area along the actual welding trajectory, S2045 may include steps 3.1 to 3.2, detailed below:

[0166] Step 3.1: Divide the actual welding trajectory into multiple straight welding segments of equal length, and determine the fifth actual pose of each straight welding segment.

[0167] The fifth actual pose of each straight welding segment can be a five-dimensional spatial pose ( x m , y m , z m , α m , β m ).

[0168] in,( x m , y m , z m ) can be used to describe the target position of each straight welded segment in three-dimensional space, α m , β m This can be used to describe how each straight welded segment is wound around... x shaft and z The fifth rotational orientation of the axis.

[0169] The length of each straight welding segment can be set according to actual needs, for example, it can be 0.1mm.

[0170] Step 3.2: Control the welding machine to be in each of the fifth actual positions in sequence, and use the target values ​​of each welding parameter to spot weld each straight welding segment in sequence.

[0171] Optionally, after each welding component is completed, the central control equipment can control the robotic arm to move to the position of the next welding component and pick it up.

[0172] As can be seen from the above, the automatic welding method provided in this application determines the component stacking state of the material platform based on the first RGB image and the first point cloud data of the material platform, determines the grasping priority of each welding component based on the component stacking state, and automatically determines the welding sequence of each welding component based on the grasping priority. This facilitates the control of the robotic arm to automatically grasp the corresponding welding components from the material platform in sequence according to the welding sequence of each welding component and transport them to the target welding part of the welding matrix for welding, thus eliminating the need for manual assistance and realizing the full automation of the welding process.

[0173] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0174] Based on the automatic welding method provided in the above embodiments, this application further provides embodiments of a central control device for implementing the above method embodiments. Please refer to... Figure 7 This is a schematic diagram of a central control device provided in an embodiment of this application. For ease of explanation, only the parts relevant to this embodiment are shown. Figure 7 As shown, the central control device 70 may include: a first determining unit 701, a second determining unit 702, a third determining unit 703, and a welding control unit 704. Wherein:

[0175] The first determining unit 701 is used to determine the component stacking state of the material platform based on the first RGB image and first point cloud data of the material platform collected by the first imaging component on the gripper of the robotic arm when a welding mother body is detected on the welding platform and a welding accessory matching the welding mother body is placed on the material platform.

[0176] The second determining unit 702 is used to determine the picking priority of each of the welding components based on the stacking state of the components.

[0177] The third determining unit 703 is used to determine the welding sequence of each of the welding components according to the grasping priority.

[0178] The welding control unit 704 is used to control the robotic arm to pick up the welding component from the material platform and move the welding component to the target welding position of the welding matrix in sequence for each of the welding components according to the welding sequence, and to control the welding machine to weld the target welding position based on the target values ​​of each welding parameter.

[0179] Optionally, the component stacking state includes: components are stacked and components are not stacked; correspondingly, the first determining unit 701 is specifically used for:

[0180] Based on the segmented regions corresponding to each of the welding components in the first RGB image, the intersection-union ratio (CIU) of the segmented regions corresponding to each pair of welding components is calculated, and each pair of welding components with a CIU greater than a first preset CIU is determined as a group of candidate components; wherein, the segmented regions corresponding to the welding components are obtained by processing the first RGB image using a preset instance segmentation model;

[0181] From the first point cloud data, determine the two point cloud clusters corresponding to each group of candidate accessories, and calculate the centroid height difference and spatial overlap rate of the two point cloud clusters corresponding to each group of candidate accessories.

[0182] For each group of the candidate fittings corresponding to two point cloud clusters, if a height difference of the two point cloud clusters is greater than a preset height difference, and a spatial overlap rate of the two point cloud clusters is greater than a preset overlap rate, the two welding fittings corresponding to the two point cloud clusters are determined as a group of stacked fittings; if the height difference of the two point cloud clusters is less than or equal to the preset height difference, or the spatial overlap rate of the two point cloud clusters is less than or equal to the preset overlap rate, it is determined that the two welding fittings corresponding to the two point cloud clusters are not stacked.

[0183] Optionally, the second determination unit 702 is specifically configured to:

[0184] In a case where the fitting stacking state is that there are stacked fittings, determine an upper welding fitting and a lower welding fitting in each group of stacked fittings;

[0185] Determine a grabbing priority of all the upper welding fittings as a first priority;

[0186] Determine a grabbing priority of the non-stacked welding fitting and all the lower welding fittings as a second priority; and the first priority is higher than the second priority.

[0187] Optionally, the third determination unit 703 is specifically configured to:

[0188] For each welding fitting under the first priority, calculate a weighted sum of a maximum height of a point cloud cluster corresponding to the welding fitting and a centroid height of the point cloud cluster corresponding to the welding fitting according to the maximum height, the centroid height, a first weighting coefficient corresponding to the maximum height, and a second weighting coefficient corresponding to the centroid height, and determine a welding sequence of each welding fitting according to the weighted sum; and the greater the weighted sum is, the earlier the welding sequence of the welding fitting is.

[0189] Randomly assign a welding sequence to each welding fitting under the second priority;

[0190] All welding sequences of the welding fittings under the second priority are later than a latest welding sequence under the first priority.

[0191] Optionally, the welding control unit 704 is specifically configured to:

[0192] According to the welding sequence, sequentially for each welding fitting, determine a first actual pose of the welding fitting according to the first point cloud data and a 3D model of the welding fitting;

[0193] Generate a grabbing motion trajectory according to the first actual pose and a second actual pose of the mechanical arm at present, and control the mechanical arm to grab the welding fitting based on the grabbing motion trajectory;

[0194] determine a third actual pose and an actual welding trajectory of the target welding position corresponding to the welding accessory according to the second point cloud data of the welding base body, the 3D model and the preset welding trajectory;

[0195] generate a moving trajectory according to the third actual pose and a fourth actual pose of the mechanical arm after the mechanical arm grasps the welding accessory, and control the mechanical arm to move the grasped welding accessory to the target welding position based on the moving trajectory;

[0196] control the welding machine to weld the target welding position along the actual welding trajectory based on the target value of each welding parameter.

[0197] Optionally, the welding control unit 704 is specifically configured to:

[0198] register the second point cloud data with the 3D model of the welding base body to determine an attitude difference vector of an actual pose of the welding base body and a standard pose corresponding to the 3D model;

[0199] determine an actual welding trajectory corresponding to the target welding position according to the attitude difference vector and a preset welding trajectory corresponding to the target welding position;

[0200] determine a target point cloud cluster corresponding to the target welding position from the second point cloud data according to the model of the welding accessory, and register the target point cloud cluster with the 3D model of the welding base body to obtain a third actual pose of the target welding position; wherein the model of the welding accessory is obtained by processing the first RGB image by using a preset instance segmentation model.

[0201] Optionally, the welding control unit 704 is specifically configured to:

[0202] divide the actual welding trajectory into a plurality of straight-line welding segments of equal length, and determine a fifth actual pose of each straight-line welding segment;

[0203] control the welding machine to be in each fifth actual pose in turn, and perform spot welding on each straight-line welding segment by using the target value of each welding parameter in turn.

[0204] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units is taken as an example, and in actual application, the above functions can be completed by different functional units according to needs, that is, the internal structure of the central control device is divided into different functional units to complete all or part of the above described functions. Each functional unit in the embodiment can be integrated in one processing unit, or each unit can be physically independent, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific name of each functional unit is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of each unit in the central control device can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0205] Please refer to Figure 8 , Figure 8 The structure schematic diagram of a central control device provided by another embodiment of the present application is shown. As shown in Figure 8 , the central control device 8 provided by the embodiment can include a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80, for example, a program corresponding to the automatic welding method. The processor 80 implements the steps in the above automatic welding method embodiments when executing the computer program 82, for example, S201-S204 as shown in Figure 2 . Alternatively, the processor 80 implements the functions of each module / unit in the above central control device embodiments when executing the computer program 82, for example, the functions of units 701-704 as shown in Figure 7 .

[0206] For example, the computer program 82 can be divided into one or more modules / units, one or more modules / units are stored in the memory 81 and executed by the processor 80 to complete the present application. One or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which is used to describe the execution process of the computer program 82 in the central control device 8. For example, the computer program 82 can be divided into a first determining unit, a second determining unit, a third determining unit, and a welding control unit. The specific functions of each unit can refer to the related description in the corresponding embodiment, which will not be described here. Figure 5

[0207] Those skilled in the art can understand that Figure 8 the central control device 8 is only an example and does not constitute a limitation on the central control device 8, and can include more or fewer components than shown, or combine certain components, or different components.

[0208] ​The processor 80 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0209] The memory 81 can be an internal storage unit of the control device 8, for example, a hard disk or a memory of the control device 8. The memory 81 can also be an external storage device of the control device 8, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card or a flash card, etc. equipped on the control device 8. Further, the memory 81 can include both the internal storage unit and the external storage device of the control device 8. The memory 81 is used to store computer programs and other programs and data required by the control device. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0210] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement each step of the automatic welding method in the method embodiment.

[0211] The embodiment of the present application provides a computer program product. When the computer program product is run on the control device, the control device implements the steps in each method embodiment.

[0212] In the above embodiments, the description of each embodiment has its own focus. The parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0213] It should be noted that, unless otherwise specified, all technical terms used in the embodiments of the present application have the same meanings as those commonly understood by the person skilled in the art to which the present application belongs. The technical terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0214] The phrase "in one embodiment" in the description of the embodiments of the present application means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that the embodiments described herein are merely example and that a specific example, as described herein, can be combined with other examples, even though the other examples may not have been described in connection with the specific example.

[0215] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0216] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; even though the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An automatic welding method characterized by, The application comprises the following steps: When it is detected that a welding matrix is placed on a welding platform and a welding accessory matching the welding matrix is placed on a material platform, a first RGB image and first point cloud data of the material platform collected by a first shooting component on a clamp of a mechanical arm are used to determine a stacking state of the welding accessories on the material platform; A grabbing priority of each welding accessory is determined according to the stacking state of the welding accessories; A welding sequence of each welding accessory is determined according to the grabbing priority; According to the welding sequence, the mechanical arm is controlled to grab the welding accessory from the material platform and move the welding accessory to a target welding position of the welding matrix for each welding accessory, and a welding machine is controlled to weld the target welding position based on a target value of each welding parameter. The stacking state of the welding accessories includes stacked accessories and non-stacked accessories, and the stacking state of the welding accessories on the material platform is determined according to the first RGB image and the first point cloud data collected by the first shooting component on the clamp of the mechanical arm. The intersection-over-union of the corresponding segmentation regions of each two welding accessories is calculated according to the segmentation regions of each two welding accessories in the first RGB image, and each two welding accessories with an intersection-over-union greater than a first preset intersection-over-union are determined as a group of candidate accessories, wherein the segmentation regions of the welding accessories are obtained by processing the first RGB image by using a preset instance segmentation model. Two point cloud clusters corresponding to each group of candidate accessories are determined from the first point cloud data, and the centroid height difference and the spatial overlap rate of the two point cloud clusters corresponding to each group of candidate accessories are calculated. For the two point cloud clusters corresponding to each group of candidate accessories, if the centroid height difference of the two point cloud clusters is greater than a preset height difference and the spatial overlap rate of the two point cloud clusters is greater than a preset overlap rate, the two welding accessories corresponding to the two point cloud clusters are determined as a group of stacked accessories; if the centroid height difference of the two point cloud clusters is less than or equal to the preset height difference or the spatial overlap rate of the two point cloud clusters is less than or equal to the preset overlap rate, it is determined that the two welding accessories corresponding to the two point cloud clusters are not stacked.

2. The method of claim 1, wherein, The grabbing priority of each welding accessory is determined according to the stacking state of the welding accessories, including: In the case that the stacking state of the welding accessories is stacked accessories, the upper welding accessory and the lower welding accessory in each group of stacked accessories are determined; The grabbing priority of all the upper welding accessories is determined as a first priority; The grabbing priority of the non-stacked welding accessories and all the lower welding accessories is determined as a second priority; wherein the first priority is higher than the second priority.

3. The method of claim 2, wherein, The welding sequence of each welding accessory is determined according to the grabbing priority, including: For each welding component under the first priority, a weighted sum of the maximum height and the centroid height is calculated based on the maximum height of the point cloud cluster corresponding to the welding component, the centroid height of the point cloud cluster corresponding to the welding component, the first weighting coefficient corresponding to the maximum height, and the second weighting coefficient corresponding to the centroid height. The welding order of each welding component is determined based on the weighted sum. The larger the weighted sum, the earlier the welding order of the welding component is. Randomly assign a welding sequence to each welding component under the second priority; In this case, the welding sequence of all welding components under the second priority is later than the latest welding sequence under the first priority.

4. The method according to any one of claims 1 to 3, characterized in that, Following the welding sequence, for each of the welding components, the robotic arm is controlled to pick up the welding component from the material platform and move it to the target welding position on the welding matrix. The welding machine is then controlled to weld the target welding position based on the target welding parameters, including: According to the welding sequence, for each of the welding components, the first actual pose of the welding component is determined based on the first point cloud data and the 3D model of the welding component. Based on the first actual pose and the current second actual pose of the robotic arm, a grasping motion trajectory is generated, and the robotic arm is controlled to grasp the welding component based on the grasping motion trajectory; Based on the second point cloud data, 3D model and preset welding trajectory of the welding matrix, the third actual pose and actual welding trajectory of the target welding part corresponding to the welding accessory are determined. Based on the third actual pose and the fourth actual pose after the robotic arm has grasped the welding component, a movement trajectory is generated, and the robotic arm is controlled to move the grasped welding component to the target welding position based on the movement trajectory. The welding machine is controlled to weld the target welding area along the actual welding trajectory based on the target values ​​of various welding parameters.

5. The method of claim 4, wherein, Based on the second point cloud data, 3D model, and preset welding trajectory of the welding matrix, the third actual pose and actual welding trajectory of the target welding part corresponding to the welding accessory are determined, including: The second point cloud data is registered with the 3D model of the welding mother body to determine the pose difference vector between the actual pose of the welding mother body and the standard pose corresponding to the 3D model. Based on the attitude difference vector and the preset welding trajectory corresponding to the target welding part, the actual welding trajectory corresponding to the target welding part is determined; Based on the model of the welding accessory, the target point cloud cluster corresponding to the target welding part is determined from the second point cloud data, and the target point cloud cluster is registered with the 3D model of the welding parent body to obtain the third actual pose of the target welding part; wherein, the model of the welding accessory is obtained by processing the first RGB image using a preset instance segmentation model.

6. The method of claim 4, wherein, Controlling the welding machine to weld the target welding area along the actual welding trajectory based on target values ​​of various welding parameters includes: The actual welding track is divided into a plurality of equal-length straight welding segments, and a fifth actual pose of each of the straight welding segments is determined; The welding machine is controlled to be in each of the fifth actual poses in turn, and each of the straight welding segments is welded in turn by using a target value of each welding parameter.

7. An automatic welding system characterized by comprising: The system comprises a central control device, a mechanical arm, a first shooting assembly, a second shooting assembly, and a welding machine; the mechanical arm, the first shooting assembly, the second shooting assembly, and the welding machine are all controlled by the central control device; The first shooting assembly is installed on a clamp at the end of the mechanical arm, and is used to collect a first RGB image and first point cloud data of a material platform when the mechanical arm is in a preset pose; The second shooting assembly is installed above a welding platform, and is used to collect a second RGB image and second point cloud data of the welding platform; The central control device is used to execute the method according to any one of claims 1-6.

8. A control device characterized by comprising: The system comprises a memory and a computer program stored in the memory and executable on a processor; when the processor executes the computer program, the method according to any one of claims 1-6 is implemented.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executable on the processor to implement the automatic welding method according to any one of claims 1-6.

Images