Welding method and device based on point cloud data, electronic equipment and medium

Abstract

The invention provides a welding method and device based on point cloud data, electronic equipment and a medium, and relates to the field of automatic welding of plate ribs, welding tasks can be automatically and efficiently executed on the plate ribs of to-be-welded steel plates, various welding tasks can be automatically identified, corresponding welding tasks can be matched for different to-be-welded steel plates, manual intervention is not needed, and the welding efficiency is improved. And the welding effect is accurate and efficient. In the welding process, whether the welding precondition is met or not can be automatically checked, and convenience can be provided for manually checking whether the position is accurate or not. And specifically, automatic treatment of steel plate and plate rib welding is achieved. A plurality of welding tasks can be executed in a mixed sequence, and manual intervention and switching are not needed. The execution speed is high, and no complex operation exists. And in the execution process, welding preconditions can be automatically checked, early warning is conducted in advance, and material waste is avoided.

Description

Technical Field

[0001] The present application relates to the field of automatic welding of plate reinforcements, and more specifically, to a welding method, device, electronic equipment and medium based on point cloud data. Background Art

[0002] In the shipbuilding process, a large number of steel plate parts with welded plate ribs are used, including large-sized steel plates (such as 8000mm*2000mm, hereinafter referred to as steel plates) and several small steel plates (such as 500mm*100mm, hereinafter referred to as plate ribs) vertically welded on one side of the rectangular steel plate. At present, shipyards use manual welding to weld such workpieces. This type of welding requires high positioning accuracy, and manual welding is labor-intensive and very demanding of personnel skills. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a welding method, device, electronic device and medium based on point cloud data, so as to solve the above-mentioned problems existing in the prior art and improve welding efficiency and quality.

[0004] In a first aspect, a welding method based on point cloud data is provided, which may include:

[0005] Acquire a point cloud of a workbench including a steel plate to be welded, wherein the point cloud includes a plurality of points and corresponding point data;

[0006] Determining a plurality of boundary points of the steel plates to be welded in the point cloud based on the point cloud densities of the plurality of configured regions;

[0007] Divide multiple boundary points to obtain boundary points in different directions;

[0008] Determine target parameters of the outer rectangle of the steel plate to be welded based on point data corresponding to boundary points in different directions;

[0009] Based on the correspondence between different configured parameters and different welding tasks, determine the target welding task corresponding to the target parameters;

[0010] The welding robot is controlled to weld the steel plate to be welded according to the target welding task; the welding task includes the number, position and size of the plate reinforcements welded to the steel plate to be welded.

[0011] In a possible implementation, the area is a fixed neighborhood;

[0012] Determining a plurality of boundary points in the point cloud based on the point cloud densities of the plurality of configured regions includes:

[0013] For any point, a fixed neighborhood is constructed with the point as the first center point and the preset distance as the radius;

[0014] Determining a point cloud density of the fixed neighborhood based on the area and the number of points of the fixed neighborhood;

[0015] Based on the point cloud densities of multiple fixed neighborhoods, multiple boundary points are determined.

[0016] In a possible implementation, multiple boundary points are divided to obtain boundary points in different directions, including:

[0017] Based on the multiple points, determine the long side direction vector and the short side direction vector of the outer bounding rectangle;

[0018] Based on the long side direction vector and the short side direction vector, a plurality of boundary points are divided to obtain boundary points in different directions.

[0019] In a possible implementation, determining the long side direction vector and the short side direction vector of the bounding rectangle based on multiple points includes:

[0020] Calculate the point data corresponding to multiple points to determine the centroid of the point cloud;

[0021] For any point, the point data of the point and the centroid of the point cloud are calculated to obtain an offset;

[0022] Calculate multiple offsets to obtain a covariance matrix;

[0023] Decomposing the covariance matrix to obtain multiple eigenvalues ​​and corresponding eigenvectors;

[0024] Determine the eigenvector corresponding to the maximum eigenvalue among the multiple eigenvalues ​​as the long-side direction vector;

[0025] And, an eigenvector corresponding to the minimum eigenvalue among the multiple eigenvalues ​​is determined as the short-side direction vector.

[0026] In a possible implementation, the target parameters include: the boundary size of the enclosing rectangle;

[0027] Determining target parameters of the bounding rectangle based on point data corresponding to boundary points in different directions includes:

[0028] For any direction, based on the point data corresponding to the boundary points in the direction, the boundary size of the bounding rectangle in the direction is determined.

[0029] In a possible implementation, the target welding task includes a welding task combination;

[0030] The welding task combination includes: a first welding task, a second welding task and a ratio of the number of steel plate welds;

[0031] The first welding task corresponds to the number of first plate ribs, the position of first plate ribs and the size of first plate ribs; the second welding task corresponds to the number of second plate ribs, the position of second plate ribs and the size of second plate ribs;

[0032] The first welding task has a higher priority than the second welding task.

[0033] In a possible implementation, controlling the welding robot to weld the steel plate to be welded according to the target welding task includes:

[0034] Splitting the configured first welding steel plate quantity to obtain a plurality of first sub-welding steel plate quantities;

[0035] Determining the second sub-welding steel plate quantity for the second welding task based on the first sub-welding steel plate quantity and the ratio of the steel plate welding quantity;

[0036] Controlling the welding robot to weld the steel plates to be welded according to the first welding task to obtain first welded steel plates of a first number of sub-welded steel plates;

[0037] Controlling the welding robot to weld the steel plates to be welded according to the second welding task to obtain second welded steel plates of a second number of sub-welded steel plates;

[0038] Return to the execution step: control the welding robot to weld the steel plates to be welded according to the first welding task, and obtain the first number of welded steel plates until the first number of welded steel plates and the corresponding second number of welded steel plates are completed.

[0039] In a second aspect, a welding device based on point cloud data is provided, which may include:

[0040] An acquisition unit, used for acquiring a point cloud of a workbench including a steel plate to be welded, wherein the point cloud includes a plurality of points and corresponding point data;

[0041] A determination unit, configured to determine a plurality of boundary points of the steel plates to be welded in the point cloud based on the point cloud densities of the configured plurality of regions;

[0042] A division unit, used for dividing a plurality of boundary points to obtain boundary points in different directions;

[0043] The determination unit is further used to determine target parameters of the outer rectangle of the steel plate to be welded based on point data corresponding to boundary points in different directions;

[0044] and, based on the correspondence between different configured parameters and different welding tasks, determining a target welding task corresponding to the target parameters;

[0045] A control unit is used to control the welding robot to weld the steel plate to be welded according to the target welding task; the welding task includes the number, position and size of the plate reinforcements welded to the steel plate to be welded.

[0046] In a third aspect, an electronic device is provided, the electronic device comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0047] Memory, used to store computer programs;

[0048] The processor is used to implement any method step described in the first aspect when executing the program stored in the memory.

[0049] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any method step described in the first aspect is implemented.

[0050] The present application provides a welding method based on point cloud data, which includes: obtaining a point cloud of a workbench including a steel plate to be welded, determining a plurality of boundary points of the steel plate to be welded in the point cloud based on the point cloud density of a plurality of configured regions; dividing a plurality of boundary points to obtain boundary points in different directions; determining the target parameters of the outer rectangle of the steel plate to be welded based on the point data corresponding to the boundary points in different directions; determining the target welding task corresponding to the target parameter based on the corresponding relationship between different configured parameters and different welding tasks; controlling the welding robot to weld the steel plate to be welded according to the target welding task; the present application can automatically and efficiently perform welding tasks on the plate reinforcement of the steel plate to be welded, and can automatically identify a variety of welding tasks, match the corresponding welding tasks to different steel plates to be welded, without manual intervention, and the welding effect is accurate and efficient. During the welding process, it can automatically check whether the welding prerequisites are met, and it can also provide convenience for manual inspection of whether the position is accurate. Specifically, the automatic processing of steel plate and plate reinforcement welding is realized. Multiple welding tasks can be executed in mixed order without manual intervention and switching. The execution speed is fast and there is no complicated operation. During the execution process, the welding preconditions can be automatically checked, early warning can be given, and material waste can be avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A system architecture diagram for a welding method based on point cloud data provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of a welding method based on point cloud data provided in an embodiment of the present application;

[0054] Figure 3 A schematic diagram of the structure of a welding device based on point cloud data provided in an embodiment of the present application;

[0055] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] A welding method based on point cloud data provided in an embodiment of the present application can be applied to Figure 1 In the system architecture shown in Figure 1 As shown, the system may include: a host computer and a welding system. The server may be a physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0058] The welding system is used to collect point clouds and send them to the host computer.

[0059] The welding system includes: two industrial robots (grasping robot and welding robot), 3D industrial camera and workbench;

[0060] A workbench may include multiple calibration objects.

[0061] The 3D industrial camera takes pictures to generate 3D point cloud data of the steel plate to be welded. The position of the 3D industrial camera generally does not need to be fixed, as long as it can capture the complete point cloud data of the steel plate to be welded. In order to ensure the accuracy of the picture and the convenience of post-processing, the 3D industrial camera is generally set directly above the steel plate to be welded.

[0062] Around the rectangular workbench, square or cylindrical calibration objects are added at preset intervals and fixed near the workbench (as close as possible without affecting the workbench transmission). A calibration object is set at each vertex and the midpoint of the long side of the workbench. Use the robot arm to measure the coordinates of the center of the calibration object in the robot's geodetic coordinate system. This coordinate is subsequently used for the mutual conversion between the point cloud coordinates and the robot arm's geodetic coordinates.

[0063] With the workbench with transmission function as the center, the welding robot is placed on the long side of the workbench, and the grabbing robot is placed on the other long side. The external axis of each robot is parallel to the long side of the workbench and the two ends are flush. At least one welding robot is set. When there is only one welding robot, the welding robot first performs spot welding to fix the plate reinforcement, and then performs actual weld length welding. When there are multiple welding robots, the preceding welding robot and the grabbing robot perform spot welding to fix the radius, and the subsequent welding robot performs weld length welding, which can improve production efficiency.

[0064] The edge of the steel plate to be welded can be a polygon composed of any straight lines or curves, but the width and length of the outer rectangle on its outer plane cannot exceed the limit of the workbench.

[0065] One or more of the multiple industrial robots with external axes carry welding guns to perform welding tasks; a load gripper tool is used to grab the plate reinforcement and place the plate reinforcement on the steel plate to be welded.

[0066] The host computer is used to obtain the point cloud and execute a welding method based on point cloud data provided in this application.

[0067] The host computer can manage the welding task management system, control industrial robots, control the switch of 3D industrial cameras, etc. Among them, the welding task management system includes: a task database module and a scheduling execution module. The task database module stores all welding tasks. A welding task includes the length and width dimensions, external contour of a base plate (steel plate), and the model, size, shape, and position of the plate reinforcement welded on it. In a welding task, there is only one base plate (steel plate to be welded), and there can be several plate reinforcements on it. A base plate of a certain size and shape can correspond to multiple welding tasks. The main difference is the number, position, and size of the plate reinforcements welded on it. The scheduling execution module can set the welding tasks of the current execution batch, and the current batch of welding tasks can include multiple different types of welding tasks.

[0068] In the shipbuilding process, a large number of steel plate parts with welded plate ribs are used, including large-sized steel plates (such as 8000mm*2000mm, hereinafter referred to as steel plates) and several small steel plates (such as 500mm*100mm, hereinafter referred to as plate ribs) vertically welded on one side of the rectangular steel plate. At present, shipyards use manual welding to weld such workpieces. This type of welding requires high positioning accuracy, and manual welding is labor-intensive and very demanding of personnel skills.

[0069] Determining the welding trajectory is a necessary condition for the operation of the automatic laser welding system, and it is also the most important step in automatic welding. When welding plate ribs on large steel plates, the position of the steel plate is not fixed after each loading, and the deviation is large. The movement trajectory and posture determined by the user through teaching are very different from the actual trajectory of the steel plate that needs to be welded, and welding cannot be performed. If teaching is performed for each steel plate, the positioning accuracy is low and the labor intensity is high when the welding trajectory is determined manually, which leads to low welding efficiency and uneconomical. When welding plate ribs on large steel plates, the number and batches of welded products are not fixed. Manual teaching and welding are required, and the teaching complexity is high.

[0070] Therefore, the purpose of the embodiments of the present application is to provide a welding method based on point cloud data, so as to solve the above-mentioned problems existing in the prior art and improve welding efficiency and quality.

[0071] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0072] Figure 2 A schematic diagram of a welding method based on point cloud data provided in an embodiment of the present application. Figure 2 As shown, the method may include:

[0073] Step S210: Obtain a point cloud of a workbench including steel plates to be welded.

[0074] Among them, the point cloud is a collection of points corresponding to objects such as the workbench, the steel plate to be welded, and the calibration object;

[0075] Specifically, the operator places the steel plate to be welded on the workbench, and the steel plate to be welded is placed as much as possible inside the polygon formed by the calibration object, and the short side of the steel plate to be welded is as perpendicular as possible to the short side of the rectangular workbench;

[0076] The host computer controls the 3D industrial camera to shoot the point cloud of the workbench containing the steel plates to be welded. The 3D industrial camera is a structured light camera, which vertically shoots the three-dimensional point cloud of the steel plates to be welded. In order to improve the accuracy, multiple shots can be taken at the same position, and the points of each shot are directly superimposed. After processing, the point cloud accuracy can reach the mm level.

[0077] First, find the coordinates of the center point of the calibration object in the point cloud, and then transform the coordinates of each point in the point cloud from the camera coordinate system to the robot's earth coordinate system according to the coordinate transformation matrix. This operation can be performed by writing a program or manually selecting the center point of the calibration object.

[0078] Secondly, based on the center coordinates of the calibration object, points outside the workbench can be removed. When the calibration object is far from the work area of ​​the workbench, the center points of the calibration object are formed into a polygon, offset inward, and the point cloud outside the edge of the workbench is automatically cut off. The area obtained at this time is the area where the steel plate to be welded is located in the workbench area. Just focus on this area for internal processing.

[0079] Afterwards, record the distance from the top plane of the calibration object to the upper surface of the workbench as h, which needs to be measured before welding; again cut off the points below the calibration object plane whose elevation difference with the plane is greater than h. This operation is to cut off the points on and below the workbench plane. The remaining multiple points are the points of the steel plate to be welded, which should be the internal plane data of the space.

[0080] Finally, plane fitting is performed on the remaining multiple points to determine the optimal fitting plane: the vertical distances from the remaining multiple points to the plane are calculated; the points (these points are noise points and vertical section points of the steel plate) in the multiple vertical distances that are greater than the preset threshold are deleted to obtain the current multiple points, and the current multiple points are plane fitted again to obtain the plane data of the steel plate to be welded. According to the fitting plane and the error value, it can be checked whether the base plate meets the flatness specification. When welding a base plate that does not meet the flatness standard, the error is large and the welding quality cannot be guaranteed.

[0081] Step S220: Determine target parameters of the outer rectangle of the steel plates to be welded based on the point cloud of the steel plates to be welded.

[0082] Specifically, A. determining the boundary points of the steel plates to be welded based on the point cloud density of the configured area;

[0083] Among them, this area is a fixed neighborhood.

[0084] For any point in the point cloud, a circular fixed neighborhood is constructed with the point as the first center point and a preset distance as the radius; or a rectangular fixed neighborhood is constructed with a preset distance.

[0085] Determine the point cloud density of the fixed neighborhood based on the area and the number of points of the fixed neighborhood;

[0086] Based on the point cloud density of each fixed neighborhood, multiple boundary points are determined. In this way, the density difference between each point and other points in its neighborhood can be calculated. If the density difference between a point and its neighboring points exceeds a certain threshold, the point is considered to be on the boundary; that is, since the points inside the steel plate to be welded are relatively uniform, the density of the points fluctuates slightly, and the point cloud density at the boundary of the steel plate to be welded is from high to low, and the point with the lowest density is the boundary point; the boundary points can also be identified by analyzing the spatial change rate (i.e. gradient) of the point cloud density. The boundary usually corresponds to a place with a large density gradient. The edge detection algorithm in image processing (such as the Canny operator) can also be used to convert multiple points into a density map before performing edge detection.

[0087] B. Determine the long side direction vector and short side direction vector of the outer rectangle of the steel plate to be welded;

[0088] The process may include: calculating the point data to determine the centroid of the point cloud; specifically, the centroid of the point cloud, that is, the average value of the coordinates of all points, that is, cx = (x1+x2+...+xn) / n; cy = (y1+y2+...+yn) / n; cz = (z1+z2+...+zn) / n, where (x1, y1, z1), (x2, y2, z2), ..., (xn, yn, zn) are the coordinates of all points, and n is the total number of points in the point cloud.

[0089] The point data corresponding to any point and the point cloud centroid are calculated to obtain the offset; specifically, for each point (xi, yi, zi), its offset with the point cloud centroid is calculated, that is, dx = xi-cx; dy = yi-cy; dz = zi-cz.

[0090] Multiple offsets are calculated to obtain a covariance matrix; specifically, it is calculated according to the following formula:

[0091] Cxx=(dx1*dx1+dx2*dx2+...+dxn*dxn) / n;

[0092] Cxy=(dx1*dy1+dx2*dy2+...+dxn*dyn) / n;

[0093] Cxz=(dx1*dz1+dx2*dz2+...+dxn*dzn) / n;

[0094] Cyx=(dy1*dx1+dy2*dx2+...+dyn*dxn) / n;

[0095] Cyy=(dy1*dy1+dy2*dy2+...+dyn*dyn) / n;

[0096] Cyz=(dy1*dz1+dy2*dz2+...+dyn*dzn) / n;

[0097] Czx=(dz1*dx1+dz2*dx2+...+dzn*dxn) / n;

[0098] Czy=(dz1*dy1+dz2*dy2+...+dzn*dyn) / n;

[0099] Czz=(dz1*dz1+dz2*dz2+...+dzn*dzn) / n;where Cxx, Cxy, Cxz, Cyx, Cyy, Cyz, Czx, Czy, Czz are the elements of the covariance matrix respectively. The final covariance matrix is:

[0100]

[0101] The covariance matrix is ​​converted into a plane point cloud covariance matrix, which can be expressed as:

[0102] Decompose the covariance matrix of the plane point cloud to obtain multiple eigenvalues ​​and corresponding eigenvectors;

[0103] Determine the eigenvector corresponding to the maximum eigenvalue among the multiple eigenvalues ​​as the long-side direction vector;

[0104] And, an eigenvector corresponding to the minimum eigenvalue among the multiple eigenvalues ​​is determined as the short-side direction vector.

[0105] In this method, the eigenvalue represents the distribution of the point data in the direction of the corresponding eigenvector. A larger eigenvalue corresponds to the direction with the largest data change, while a smaller eigenvalue corresponds to the direction with the smallest change.

[0106] C. Based on the long side direction vector, the short side direction vector and the second center point in the point cloud, multiple boundary point clouds are divided to obtain boundary point clouds in different directions.

[0107] Specifically, based on the long side direction vector and the short side direction vector, the upper, lower, left and right edge points of the steel plate to be welded are determined; the four edge points are fitted (straight line or curve) to determine the outer rectangle of the steel plate to be welded, and then the center point (second center point) of the outer rectangle and the target parameters (length and width) of the outer rectangle can be determined. After that, a target coordinate system is constructed based on the long side direction vector, the short side direction vector and the second center point to adjust the positions of the outer rectangle and each point in the corresponding point cloud of the steel plate to be welded (the overall direction of the outer rectangle and the corresponding point cloud of the steel plate to be welded) based on the target coordinate system.

[0108] In another embodiment, the boundary points of the steel plates to be welded can be determined by applying density-based spatial clustering to determine multiple clusters; for any cluster, first, normal estimation is performed: the local surface normal is calculated for each point, which helps to identify planes, edges, and corners. Secondly, edge detection is performed: potential boundary points are detected based on normal information or by analyzing the distance changes between points. Finally, contour extraction can be performed: the boundary points are connected to form a complete contour or boundary line to determine the boundary size and boundary shape.

[0109] Step S230: Determine the target welding task based on the correspondence between different configured parameters and different welding tasks.

[0110] Among them, the different parameters configured are different steel plate parameters (boundary shape and boundary size of the steel plate). The welding task management system configures different outer rectangle parameters corresponding to different steel plate parameters; since different shapes can correspond to the same outer rectangle, one outer rectangle can correspond to multiple steel plate parameters.

[0111] Match the target parameter with multiple configuration wrapping rectangle parameters:

[0112] If the matched target outer rectangle corresponds to a steel plate parameter, then the welding task corresponding to the steel plate parameter is determined as the target welding task.

[0113] If the matched target enclosing rectangle corresponds to multiple steel plate parameters, the target steel plate parameters determined in step S220 are matched with the multiple steel plate parameters to determine the target welding task corresponding to the target steel plate parameters.

[0114] Furthermore, the target steel plate parameters can be determined according to the boundary points in different directions. Specifically, the target steel plate parameters can be determined by:

[0115] Determining the boundary shape of the steel plate may include: for boundary points in any direction, fitting all points in the direction to obtain the boundary shape of the steel plate.

[0116] Based on the boundary shape and the corresponding boundary points, a boundary size is determined.

[0117] In this method, through the two-layer screening process configured, the outer rectangle of the steel plate to be welded is first matched with multiple configured outer rectangles. When the matched target outer rectangle corresponds to a steel plate parameter, it indicates that a steel plate parameter in the welding task management system is determined, and the target welding task is determined based on the steel plate parameter. When the matched target outer rectangle corresponds to multiple steel plate parameters, it indicates that further matching process is required; therefore, the target steel plate parameter of the steel plate to be welded is matched with multiple steel plate parameters, thereby determining a steel plate parameter in the welding task management system.

[0118] In some embodiments, if after the target steel plate parameter is matched with multiple steel plate parameters, the target steel plate parameter also corresponds to multiple steel plate parameters, the boundary points of the steel plate to be welded are matched one by one with the boundary points of the determined multiple steel plate parameters, thereby determining a steel plate parameter corresponding to the target steel plate parameter in the welding task management system, and further determining the target welding task corresponding to the target steel plate parameter.

[0119] Step S240: Control the welding robot to weld the steel plate to be welded according to the target welding task.

[0120] Specifically, a welding task may consist of multiple tasks in the database. At this time, the operator selects one or more tasks under the current size and boundary type to add to the current welding batch task. This is the first screening. During specific welding, after visually identifying the steel plate size, the system automatically selects the welding task in the current batch of welding tasks. When there is only one task corresponding to this size, the system directly determines this task as the welding task for the current steel plate. When a steel plate size in the current welding task batch corresponds to multiple tasks, the following processing methods can be used:

[0121] 1. According to the task priority, you can select the number of welding tasks with the highest priority. After completing all the welding tasks with the highest priority, select the second highest priority task to execute, complete the corresponding steel plate welding data, and so on to execute the welding tasks with lower priority.

[0122] 2. The target welding task includes a welding task combination;

[0123] The welding task combination includes: the first welding task, the second welding task and the proportion of the number of steel plate welds;

[0124] The first welding task corresponds to the number of first plate ribs, the position of first plate ribs and the size of first plate ribs; the second welding task corresponds to the number of second plate ribs, the position of second plate ribs and the size of second plate ribs;

[0125] The first welding task has a higher priority than the second welding task.

[0126] Controlling the welding robot to weld the steel plate to be welded according to the target welding task may specifically include:

[0127] Splitting the configured first welding steel plate quantity to obtain a plurality of first sub-welding steel plate quantities;

[0128] Determine the second sub-welding steel plate quantity for the second welding task based on the first sub-welding steel plate quantity and the ratio of the steel plate welding quantity;

[0129] Controlling the welding robot to weld the steel plates to be welded according to the first welding task to obtain first welded steel plates of the first sub-welded steel plate quantity;

[0130] In some embodiments, before welding the steel plate to be welded, the method also includes: according to the position of the first plate rib in the first welding task (the position is the straight line or curve information of the coordinate system with the center point of the outer rectangle as the coordinate origin, the long side direction vector as the X-axis, and the short side direction vector as the Y-axis), the position of the first plate rib is converted to the actual point position; finally, the actual point position is mapped to the robot's geodetic coordinate system so that the welding robot can perform welding operations on the first plate rib.

[0131] Furthermore, before the welding robot performs the welding operation, the method may include: controlling the welding robot to draw a plate rib position line at the first plate rib position with a low-power laser line, only burning through the surface of the steel plate to be welded at the plate rib position without damaging the inside, and the drawn line (called a schematic line) is used to check whether the position is correct. After the welding robot draws all the position lines, the grasping robot grasps the first plate rib and places it on the plate rib welding line. The current image information can be captured to remotely check whether the direction and position of the first plate rib are pressed on the schematic line and whether the cutting angle direction of the upper part of the plate rib meets the requirements.

[0132] The welding operation process of the welding robot may include: when it is determined that all the spacings of the plurality of first plate ribs on the steel plate to be welded meet the requirements, spot welding the first plate ribs and the steel plate to be welded at a fixed spacing distance to fix the first plate ribs. When the flatness of the steel plate to be welded is not enough or the bottom of the first plate rib is not flat, the spacing between the first plate rib and the steel plate will exceed the threshold, the host computer will issue an alarm, and the user will replace the plate rib or process the steel plate to be welded. After that, after all the first plate ribs are fixed by spot welding, the welding robot welds all the first plate ribs. After all the first plate ribs are welded, the workbench transfers the completed first welded steel plate.

[0133] The welding robot is controlled to weld the steel plates to be welded according to the second welding task to obtain second welded steel plates of a second number of sub-welded steel plates; it should be noted that the method of welding the plate reinforcement in each welding task is the same.

[0134] Return to the execution step: control the welding robot to weld the steel plates to be welded according to the first welding task, and obtain the first number of welded steel plates until the first number of welded steel plates (the first welded steel plate and the second welded steel plate) are completed.

[0135] The process of executing welding task combinations can also be understood as: setting the execution group, and executing in a loop according to the order within the group. Set the execution order and corresponding quantity of different welding task types in the group, and execute the group in a loop. For example, set the execution group sequence ((task 1, quantity 2), (task 2, quantity 1)), when encountering base plates with the same size and boundaries, the 1st and 2nd plates execute task 1, the 3rd plate executes task 2, the 4th and 5th plates execute task 1, the 6th plate executes task 2, and so on.

[0136] The present application provides a welding method based on point cloud data, the method comprising: obtaining a point cloud of a workbench including a steel plate to be welded, determining a plurality of boundary points of the steel plate to be welded in the point cloud based on the point cloud density of a plurality of configured regions; dividing a plurality of boundary points to obtain boundary points in different directions; determining the target parameters of the outer rectangle of the steel plate to be welded based on the point data corresponding to the boundary points in different directions; determining the target welding task corresponding to the target parameter based on the correspondence between different configured parameters and different welding tasks; controlling the welding robot to weld the steel plate to be welded according to the target welding task; the present application can automatically and efficiently perform welding tasks on the plate reinforcement of the steel plate to be welded, and can automatically identify a plurality of welding tasks, match the corresponding welding tasks to different steel plates to be welded, without manual intervention, and the welding effect is accurate and efficient. During the welding process, it can automatically check whether the welding prerequisites are met, and it can also provide convenience for manual inspection of whether the position is accurate.

[0137] Corresponding to the above method, the embodiment of the present application also provides a welding device based on point cloud data, such as Figure 3 As shown, the device comprises:

[0138] An acquisition unit 310 is used to acquire a point cloud of a workbench including a steel plate to be welded, wherein the point cloud includes a plurality of points and corresponding point data;

[0139] A determination unit 320, configured to determine a plurality of boundary points of the steel plates to be welded in the point cloud based on the point cloud densities of the configured plurality of regions;

[0140] A division unit 330, configured to divide the plurality of boundary points to obtain boundary points in different directions;

[0141] The determination unit 320 is further used to determine target parameters of the outer rectangle of the steel plate to be welded based on point data corresponding to boundary points in different directions;

[0142] and, based on the correspondence between different configured parameters and different welding tasks, determining a target welding task corresponding to the target parameters;

[0143] The control unit 340 is used to control the welding robot to weld the steel plate to be welded according to the target welding task; the welding task includes the number, position and size of the plate reinforcements welded to the steel plate to be welded.

[0144] The functions of the various functional units of a welding device based on point cloud data provided in the above-mentioned embodiment of the present application can be realized through the above-mentioned method steps. Therefore, the specific working process and beneficial effects of each unit in a welding device based on point cloud data provided in the embodiment of the present application are not repeated here.

[0145] The present application also provides an electronic device, such as Figure 4 As shown, it includes a processor 410 , a communication interface 420 , a memory 430 and a communication bus 440 , wherein the processor 410 , the communication interface 420 , and the memory 430 communicate with each other via the communication bus 440 .

[0146] Memory 430, for storing computer programs;

[0147] The processor 410 is used to execute the program stored in the memory 430 to implement the following steps:

[0148] Acquire a point cloud of a workbench including a steel plate to be welded, wherein the point cloud includes a plurality of points and corresponding point data;

[0149] Determining a plurality of boundary points of the steel plates to be welded in the point cloud based on the point cloud densities of the plurality of configured regions;

[0150] Divide multiple boundary points to obtain boundary points in different directions;

[0151] Determine target parameters of the outer rectangle of the steel plate to be welded based on point data corresponding to boundary points in different directions;

[0152] Based on the correspondence between different configured parameters and different welding tasks, determine the target welding task corresponding to the target parameters;

[0153] The welding robot is controlled to weld the steel plate to be welded according to the target welding task; the welding task includes the number, position and size of the plate reinforcements welded to the steel plate to be welded.

[0154] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0155] The communication interface is used for communication between the above electronic device and other devices.

[0156] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0157] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0158] The implementation methods and beneficial effects of the components of the electronic device in the above embodiments to solve the problems can be seen in Figure 2 The various steps in the illustrated embodiment are implemented, therefore, the specific working process and beneficial effects of the electronic device provided by the embodiment of the present application are not repeated here.

[0159] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes a welding method based on point cloud data as described in any of the above embodiments.

[0160] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute a welding method based on point cloud data as described in any one of the above embodiments.

[0161] Those skilled in the art will appreciate that the embodiments in the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt a complete hardware embodiment, a complete software embodiment, or a form of an embodiment combining software and hardware. Moreover, the present application may adopt a form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0163] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0165] Unless otherwise defined, the technical terms or scientific terms used in this application should be understood by people with ordinary skills in the field to which the present invention belongs. "First", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect", "couple" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0166] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the present application embodiments are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application embodiments.

[0167] Obviously, those skilled in the art can make various changes and modifications to the embodiments in the present application without departing from the spirit and scope of the embodiments in the present application. Thus, if these modifications and variations of the embodiments in the present application are within the scope of the embodiments in the present application and their equivalents, the embodiments in the present application are also intended to include these modifications and variations.

Claims

1. A welding method based on point cloud data, characterized in that: The method comprises: Acquire a point cloud of a workbench including a steel plate to be welded, wherein the point cloud includes a plurality of points and corresponding point data; Determining a plurality of boundary points of the steel plates to be welded in the point cloud based on the point cloud densities of the plurality of configured regions; Divide multiple boundary points to obtain boundary points in different directions; Determine target parameters of the outer rectangle of the steel plate to be welded based on point data corresponding to boundary points in different directions; Based on the correspondence between different configured parameters and different welding tasks, determine the target welding task corresponding to the target parameters; The welding robot is controlled to weld the steel plate to be welded according to the target welding task; the welding task includes the number, position and size of the plate reinforcements welded to the steel plate to be welded.

2. The method according to claim 1, characterized in that The area is a fixed neighborhood; Determining a plurality of boundary points in the point cloud based on the point cloud densities of the plurality of configured regions includes: For any point, a fixed neighborhood is constructed with the point as the first center point and the preset distance as the radius; Determining a point cloud density of the fixed neighborhood based on the area and the number of points of the fixed neighborhood; Based on the point cloud densities of multiple fixed neighborhoods, multiple boundary points are determined.

3. The method according to claim 1, characterized in that Multiple boundary points are divided to obtain boundary points in different directions, including: Based on the multiple points, determine the long side direction vector and the short side direction vector of the outer bounding rectangle; Based on the long side direction vector and the short side direction vector, a plurality of boundary points are divided to obtain boundary points in different directions.

4. The method according to claim 3, characterized in that Determining the long side direction vector and the short side direction vector of the bounding rectangle based on the multiple points includes: Calculate the point data corresponding to multiple points to determine the centroid of the point cloud; For any point, the point data of the point and the centroid of the point cloud are calculated to obtain an offset; Calculate multiple offsets to obtain a covariance matrix; Decomposing the covariance matrix to obtain multiple eigenvalues ​​and corresponding eigenvectors; Determine the eigenvector corresponding to the maximum eigenvalue among the multiple eigenvalues ​​as the long-side direction vector; And, an eigenvector corresponding to the minimum eigenvalue among the multiple eigenvalues ​​is determined as the short-side direction vector.

5. The method according to claim 1, characterized in that The target parameters include: the boundary size of the outer rectangle; Determining target parameters of the bounding rectangle based on point data corresponding to boundary points in different directions includes: For any direction, based on the point data corresponding to the boundary points in the direction, the boundary size of the bounding rectangle in the direction is determined.

6. The method according to claim 1, characterized in that The target welding task includes a welding task combination; The welding task combination includes: a first welding task, a second welding task and a ratio of the number of steel plate welds; The first welding task corresponds to the number of first plate ribs, the position of first plate ribs and the size of first plate ribs; the second welding task corresponds to the number of second plate ribs, the position of second plate ribs and the size of second plate ribs; The first welding task has a higher priority than the second welding task.

7. The method according to claim 6, characterized in that Controlling the welding robot to weld the steel plate to be welded according to the target welding task includes: Splitting the configured first welding steel plate quantity to obtain a plurality of first sub-welding steel plate quantities; Determining the second sub-welding steel plate quantity for the second welding task based on the first sub-welding steel plate quantity and the ratio of the steel plate welding quantity; Controlling the welding robot to weld the steel plates to be welded according to the first welding task to obtain first welded steel plates of a first number of sub-welded steel plates; Controlling the welding robot to weld the steel plates to be welded according to the second welding task to obtain second welded steel plates of a second number of sub-welded steel plates; Return to the execution step: control the welding robot to weld the steel plates to be welded according to the first welding task, and obtain the first number of welded steel plates until the first number of welded steel plates and the corresponding second number of welded steel plates are completed.

8. A welding device based on point cloud data, characterized in that: The device comprises: An acquisition unit, used for acquiring a point cloud of a workbench including a steel plate to be welded, wherein the point cloud includes a plurality of points and corresponding point data; A determination unit, configured to determine a plurality of boundary points of the steel plates to be welded in the point cloud based on the point cloud densities of the configured plurality of regions; A division unit, used for dividing a plurality of boundary points to obtain boundary points in different directions; The determination unit is further used to determine target parameters of the outer rectangle of the steel plate to be welded based on point data corresponding to boundary points in different directions; and, based on the correspondence between different configured parameters and different welding tasks, determining a target welding task corresponding to the target parameters; A control unit is used to control the welding robot to weld the steel plate to be welded according to the target welding task; the welding task includes the number, position and size of the plate reinforcements welded to the steel plate to be welded.

9. An electronic device, characterized in that: The electronic device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.

Images