Point cloud identification method, device and equipment for automatic welding of fillet weld of lifting lug workpiece

Abstract

The invention relates to a point cloud recognition method, device and equipment for automatic welding of a fillet weld of a lifting lug workpiece, and the method comprises the steps: obtaining a scene point cloud of an actual workpiece placement scene, and extracting an actual workpiece point cloud from the scene point cloud; according to the actual workpiece point cloud, a registration algorithm is adopted to calculate a target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud; correcting a preset scanning track of the actual workpiece based on the target homogeneous change matrix to obtain an adjusted scanning track; and according to the adjusted scanning track, welding seam area point cloud information is obtained, and welding seam points are recognized based on the welding seam area point cloud information and the straight line intersection point angle welding seam recognition method. According to the whole scheme, the welding seam point can be accurately recognized, and finally accurate point cloud recognition of automatic welding of the lifting lug workpiece fillet welding seam is achieved.

Description

Technical Field

[0001] The present application relates to the field of point cloud recognition technology, and in particular to a point cloud recognition method, device, computer equipment, storage medium and computer program product for automatic welding of fillet welds of lifting lug workpieces. Background Art

[0002] With the development of industrial automation and intelligent manufacturing, welding technology has been widely used in fields such as mechanical engineering, construction, shipbuilding, and automotive manufacturing. While traditional manual welding methods offer advantages such as high flexibility and adaptability, they suffer from low welding efficiency, unstable quality, and susceptibility to damage, making them difficult to meet the demands of high-volume, high-speed production.

[0003] To improve welding efficiency and quality, robotic welding technology has been gradually adopted. However, robotic welding requires pre-programming and configuration, which can require extensive adjustments and adaptation for workpieces with complex shapes and varying sizes. This can also lead to issues such as lengthy manual instruction times and inaccurate instruction positions, making it difficult to adapt to the welding environment and the workpiece in real time. Against this backdrop, automated welding technology based on point cloud recognition has emerged. Point cloud recognition technology offers the advantages of being non-contact, fast, and highly accurate. It enables rapid scanning and 3D reconstruction of workpieces, opening up new possibilities for automated welding.

[0004] However, existing point cloud recognition technology still has some problems when processing complex workpieces. In particular, in the automatic welding of fillet welds on lug workpieces, due to the special structure of the workpiece and the complex weld characteristics, how to accurately identify the weld points is a technical challenge that continues to be solved. Summary of the Invention

[0005] Based on this, it is necessary to provide an accurate point cloud recognition method, device, computer equipment, storage medium and computer program product for automatic welding of fillet welds of lifting lug workpieces to address the above technical problems.

[0006] In a first aspect, the present application provides a point cloud recognition method for automatic welding of fillet welds of lifting lug workpieces.

[0007] The method comprises:

[0008] Acquire a scene point cloud of an actual workpiece placement scene, and extract the actual workpiece point cloud from the scene point cloud;

[0009] Calculating a target homogeneous change matrix between the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm based on the actual workpiece point cloud;

[0010] Correcting the preset scanning trajectory of the actual workpiece based on the target homogeneous change matrix to obtain an adjusted scanning trajectory;

[0011] The weld area point cloud information is acquired according to the adjusted scanning trajectory, and the weld points are identified based on the weld area point cloud information and a fillet weld identification method of straight line intersections.

[0012] In one embodiment, obtaining a scene point cloud of an actual workpiece placement scene and extracting an actual workpiece point cloud from the scene point cloud includes:

[0013] Scan the actual workpiece;

[0014] Convert each frame of line scan point cloud to the robot base coordinate system and merge them to obtain the scene point cloud of the actual workpiece placement scene;

[0015] Obtaining a known position corresponding to the workbench where the actual workpiece is placed;

[0016] Based on the known position, irrelevant point clouds in the scene point cloud are filtered through to extract the actual workpiece point cloud.

[0017] In one embodiment, the step of calculating a target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm according to the actual workpiece point cloud comprises:

[0018] Reading a template workpiece point cloud corresponding to the actual workpiece point cloud from a preset database;

[0019] Extracting the lifting ear hole feature from the actual workpiece point cloud, and identifying the center of the projected point cloud of the lifting ear hole on the mainboard and the mainboard normal vector based on the lifting ear hole feature;

[0020] According to the center of the projected point cloud of the lifting ear hole on the main board in the actual workpiece point cloud and the main board normal vector, and the center of the projected point cloud of the lifting ear hole on the main board in the template workpiece point cloud and the main board normal vector, the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud is calculated.

[0021] In one embodiment, extracting the ear hole feature from the actual workpiece point cloud and identifying the center of the projected point cloud of the ear hole on the mainboard and the mainboard normal vector based on the ear hole feature includes:

[0022] Determine the mainboard plane and mainboard normal vector based on random sampling consistency algorithm;

[0023] Removing the plane point cloud of the main plate from the actual workpiece point cloud, and performing plane extraction again on the remaining point cloud to obtain a composite plate point cloud;

[0024] Projecting the complex plate point cloud onto the main plate plane to obtain a projection result;

[0025] Extracting the point cloud boundary of the mainboard plane using the Alpha Shapes algorithm;

[0026] Obtaining inner hole boundary points and outer boundary points of the composite plate projection point cloud according to the projection result and the point cloud boundary of the main plate plane;

[0027] Clustering the inner hole boundary points and the outer boundary points of the composite plate projection point cloud, and classifying based on the clustering results to extract the boundary point cloud data of the ear hole;

[0028] Based on the random sampling consistency algorithm, the boundary point cloud of the ear hole is fitted with a spatial circle to determine the center of the projected point cloud of the ear hole on the mainboard and the mainboard normal vector.

[0029] In one embodiment, determining the mainboard plane and the mainboard normal vector based on a random sampling consistency algorithm includes:

[0030] Randomly select three points from the actual workpiece point cloud and calculate the plane equation determined by the three selected points;

[0031] According to the plane equation, calculating the distance errors between all points in the actual workpiece point cloud and the plane, and judging whether the corresponding points are interior points based on the errors;

[0032] Find the plane containing the most internal points, use the found plane as the main board plane, and determine the main board normal vector.

[0033] In one embodiment, calculating the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud based on the center of the projected point cloud of the lifting ear hole on the main board in the actual workpiece point cloud and the main board normal vector, and the center of the projected point cloud of the lifting ear hole on the main board in the template workpiece point cloud and the main board normal vector includes:

[0034] Determine the rotational relationship between the actual workpiece point cloud and the template workpiece point cloud by aligning the mainboard normal vector in the actual workpiece point cloud with the mainboard normal vector in the template workpiece point cloud;

[0035] Determine the translation relationship between the actual workpiece point cloud and the template workpiece point cloud by aligning the center of the projected point cloud of the lifting ear hole on the main board in the actual workpiece point cloud with the center of the projected point cloud of the lifting ear hole on the main board in the template workpiece point cloud;

[0036] Constructing an initial homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud according to the rotation relationship and the translation relationship;

[0037] Determining the optimal rotation angle based on the rotation relationship and using an optimization method of rotation angle traversal;

[0038] Obtaining a coarse registration homogeneous change matrix according to the optimal rotation angle and the initial homogeneous change matrix;

[0039] The coarse registration homogeneous change matrix is ​​finely registered using the ICP algorithm to obtain a target homogeneous change matrix.

[0040] In one embodiment, the step of obtaining weld area point cloud information according to the adjusted scanning trajectory and identifying weld points using the fillet weld identification method based on the weld area point cloud information and straight line intersections includes:

[0041] According to the adjusted scanning trajectory, the point cloud information of the weld area is obtained;

[0042] According to the point cloud information of the weld area, four straight lines are extracted based on the consistency of random sampling, and the intersection points between any two straight lines are calculated to obtain four intersection points;

[0043] Eliminate the two intersection points with larger vertical coordinates among the four intersection points;

[0044] Find the point closest to the rough positioning weld point among the remaining two intersection points to obtain the weld point.

[0045] In a second aspect, the present application also provides a point cloud recognition device for automatic welding of fillet welds of lifting lug workpieces. The device comprises:

[0046] A point cloud acquisition module is used to acquire a scene point cloud of an actual workpiece placement scene and extract the actual workpiece point cloud from the scene point cloud;

[0047] A homogeneous transformation module is used to calculate a target homogeneous change matrix between the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm according to the actual workpiece point cloud;

[0048] A scanning trajectory correction module, configured to correct a preset scanning trajectory of the actual workpiece based on the target homogeneous change matrix to obtain an adjusted scanning trajectory;

[0049] The weld point recognition module is used to obtain weld area point cloud information according to the adjusted scanning trajectory, and identify weld points based on the weld area point cloud information and the fillet weld recognition method of the straight line intersection.

[0050] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0051] Acquire a scene point cloud of an actual workpiece placement scene, and extract the actual workpiece point cloud from the scene point cloud;

[0052] Calculating a target homogeneous change matrix between the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm based on the actual workpiece point cloud;

[0053] Correcting the preset scanning trajectory of the actual workpiece based on the target homogeneous change matrix to obtain an adjusted scanning trajectory;

[0054] The weld area point cloud information is acquired according to the adjusted scanning trajectory, and the weld points are identified based on the weld area point cloud information and a fillet weld identification method of straight line intersections.

[0055] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0056] Acquire a scene point cloud of an actual workpiece placement scene, and extract the actual workpiece point cloud from the scene point cloud;

[0057] Calculating a target homogeneous change matrix between the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm based on the actual workpiece point cloud;

[0058] Correcting the preset scanning trajectory of the actual workpiece based on the target homogeneous change matrix to obtain an adjusted scanning trajectory;

[0059] The weld area point cloud information is acquired according to the adjusted scanning trajectory, and the weld points are identified based on the weld area point cloud information and a fillet weld identification method of straight line intersections.

[0060] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0061] Acquire a scene point cloud of an actual workpiece placement scene, and extract the actual workpiece point cloud from the scene point cloud;

[0062] Calculating a target homogeneous change matrix between the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm based on the actual workpiece point cloud;

[0063] Correcting the preset scanning trajectory of the actual workpiece based on the target homogeneous change matrix to obtain an adjusted scanning trajectory;

[0064] The weld area point cloud information is acquired according to the adjusted scanning trajectory, and the weld points are identified based on the weld area point cloud information and a fillet weld identification method of straight line intersections.

[0065] The above-mentioned point cloud recognition method, device, computer equipment, storage medium and computer program product for automatic welding of fillet welds of lifting ear workpieces obtain a scene point cloud of the actual workpiece placement scene and extract the actual workpiece point cloud from the scene point cloud; based on the actual workpiece point cloud, a registration algorithm is used to calculate the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud; based on the target homogeneous change matrix, the preset scanning trajectory of the actual workpiece is corrected to obtain an adjusted scanning trajectory; based on the adjusted scanning trajectory, weld area point cloud information is obtained, and the fillet weld recognition method based on the weld area point cloud information and the straight line intersection point identifies the weld points. Throughout the entire process, the registration algorithm is first used to correct the preset scanning trajectory of the actual workpiece, which can effectively avoid scanning trajectory deviation or equipment collision caused by workpiece offset, while improving the quality of precise positioning imaging to a certain extent. In addition, the fillet weld point positioning algorithm based on the straight line intersection point can accurately identify the weld points, and ultimately achieve accurate point cloud recognition for automatic welding of fillet welds of lifting ear workpieces. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a top view of various lifting lug workpieces;

[0067] Figure 2 A diagram illustrating an application environment of a point cloud recognition method for automatic welding of fillet welds of lifting lug workpieces according to an embodiment;

[0068] Figure 3 Schematic diagram of a flow chart of a point cloud recognition method for automatic welding of a fillet weld of a lifting lug workpiece according to one embodiment;

[0069] Figure 4 A schematic flow chart of a point cloud recognition method for automatic welding of a fillet weld of a lifting lug workpiece in another embodiment;

[0070] Figure 5 Schematic diagram of point cloud imaging at different processing stages;

[0071] Figure 6 This is a schematic diagram of the point cloud imaging of the fillet weld of the lifting ear workpiece;

[0072] Figure 7 A structural block diagram of a point cloud recognition device for automatic welding of fillet welds of lifting lug workpieces according to one embodiment;

[0073] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0075] In order to explain in detail the point cloud recognition method for automatic welding of fillet welds of lifting lug workpieces in this application, the following will first introduce the structure of the lifting lug workpiece. Figure 1 The commonly used lifting lug (top view) shown in the figure is generally composed of three parts: the main plate, the doubler plate, and the stiffener plate. The main plate is the main load-bearing structure of the lifting lug; the doubler plate is used to strengthen the strength and rigidity of the main plate; and the stiffener plate plays a reinforcing role.

[0076] There are lifting lug holes between the main plate and the backing plate for inserting hooks, slings or other lifting equipment. All lifting lug workpieces that need to be welded include at least the main plate, backing plate and the lifting lug holes.

[0077] The point cloud recognition method for automatic welding of fillet welds of lifting lug workpieces provided in the embodiment of the present application can be applied to Figure 2 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 sends a point cloud recognition request to the server 104. The server 104 responds to the point cloud recognition request, extracts the scene point cloud of the actual workpiece placement scene carried therein, and extracts the actual workpiece point cloud from the scene point cloud; based on the actual workpiece point cloud, a registration algorithm is used to calculate the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud; based on the target homogeneous change matrix, the preset scanning trajectory of the actual workpiece is corrected to obtain an adjusted scanning trajectory; based on the adjusted scanning trajectory, the weld area point cloud information is obtained, and the weld points are identified based on the weld area point cloud information and the fillet weld recognition method of the straight line intersection. Furthermore, the server 104 can feed back the identified weld points to the terminal 102 to guide the terminal 102 or the external device controlled by it to complete the subsequent automatic welding operation. Terminal 102 may include, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart car devices. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers.

[0078] In one embodiment, Figure 3 As shown in the figure, a point cloud recognition method for automatic welding of fillet welds of lifting lug workpieces is provided. Figure 2 Taking the server 104 in the example as an example, the following steps are included:

[0079] S200: Acquire a scene point cloud of an actual workpiece placement scene, and extract an actual workpiece point cloud from the scene point cloud.

[0080] Use a 3D laser scanner or structured light scanner to scan the actual workpiece placement scene and obtain 3D point cloud data of the scene. During the scanning process, ensure that the scanner parameters are set appropriately to obtain high-quality point cloud data. Use point cloud processing software (such as CloudCompare, PCL library, etc.) to preprocess the acquired scene point cloud, including denoising, filtering, segmentation, and other operations. By setting appropriate segmentation thresholds and algorithms, the actual workpiece point cloud is extracted from the scene point cloud. During the extraction process, it is necessary to ensure the integrity and accuracy of the actual workpiece point cloud.

[0081] S400: Calculating a target homogeneous change matrix between the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm according to the actual workpiece point cloud.

[0082] The template workpiece point cloud refers to the point cloud data collected for the template workpiece. Specifically, based on the standard model or design drawing of the known lifting lug workpiece, a three-dimensional model of the template workpiece can be created using three-dimensional modeling software (such as SolidWorks, UG, etc.) and converted into a point cloud data format. The template workpiece point cloud should contain all the key features and size information of the workpiece. The iterative closest point (ICP) algorithm or other registration algorithms are used to align the actual workpiece point cloud and the template workpiece point cloud. During the registration process, the position and posture of the actual workpiece point cloud need to be continuously adjusted to align it with the template workpiece point cloud. The goal of the registration algorithm is to find an optimal homogeneous change matrix so that the error between the actual workpiece point cloud and the template workpiece point cloud is minimized. After calculation by the registration algorithm, the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud is obtained. This matrix contains the position and posture information of the actual workpiece relative to the template workpiece, providing a basis for subsequent scanning trajectory correction.

[0083] S600: Correcting the preset scanning trajectory of the actual workpiece based on the target homogeneous change matrix to obtain an adjusted scanning trajectory.

[0084] On the template workpiece, a scanning trajectory is pre-planned based on the weld's location and shape. This trajectory should cover the entire weld area and ensure stability and accuracy during scanning. The resulting target homogeneous change matrix is ​​used to perform deflection correction on the pre-set scanning trajectory. Specifically, each point on the pre-set scanning trajectory is multiplied by the inverse of the target homogeneous change matrix to obtain a deflected scanning trajectory. This deflected scanning trajectory accurately reflects the weld's location on the actual workpiece.

[0085] S800: Acquire point cloud information of the weld area according to the adjusted scanning trajectory, and identify weld points based on the weld area point cloud information and a fillet weld identification method of straight line intersections.

[0086] Using a device such as a 3D laser scanner or structured light scanner, rescan the actual workpiece along the corrected scanning trajectory to obtain 3D point cloud data for the weld area. During the scanning process, ensure that the scanner parameter settings are consistent with those used to obtain the scene point cloud to obtain comparable point cloud data. The obtained weld area point cloud undergoes preprocessing, including denoising, filtering, and segmentation. By setting an appropriate segmentation threshold and algorithm, the weld point cloud is extracted from the weld area point cloud. During the extraction process, the integrity and accuracy of the weld point cloud must be ensured. Here, a fillet weld recognition method based on line intersections is used to identify the weld point cloud. Specifically, several lines are first fitted to the weld point cloud based on the direction and shape of the weld. The intersections of these lines are then calculated and used as candidate weld points. Finally, the candidate points are screened and verified based on the weld's geometric characteristics and dimensional requirements to obtain the final weld point.

[0087] The above-mentioned point cloud recognition method for automatic welding of fillet welds of lifting ear workpieces obtains a scene point cloud of the actual workpiece placement scene and extracts the actual workpiece point cloud from the scene point cloud; based on the actual workpiece point cloud, a registration algorithm is used to calculate the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud; based on the target homogeneous change matrix, the preset scanning trajectory of the actual workpiece is corrected to obtain an adjusted scanning trajectory; based on the adjusted scanning trajectory, point cloud information of the weld area is obtained, and the fillet weld recognition method based on the weld area point cloud information and the straight line intersection points identifies the weld points. Throughout the entire process, the registration algorithm is first used to correct the preset scanning trajectory of the actual workpiece, which can effectively avoid scanning trajectory deviation or equipment collision caused by workpiece offset, while improving the quality of precise positioning imaging to a certain extent. In addition, the fillet weld point positioning algorithm based on the straight line intersection points can accurately identify the weld points, and ultimately achieve accurate point cloud recognition for automatic welding of fillet welds of lifting ear workpieces.

[0088] Specifically, the point cloud recognition method for automatic welding of fillet welds of lifting ear workpieces in this application obtains the scene point cloud through three-dimensional scanning, and then uses straight-through filtering to extract the workpiece point cloud to ensure that subsequent processing focuses on the target area. In the registration stage, the homogeneous change matrix is ​​calculated by extracting the lifting ear hole features, and the standard geometry of the template workpiece is mapped to the actual workpiece to eliminate the trajectory deviation caused by position offset. The scanning trajectory after correction guides the sensor to collect the point cloud of the weld area, and the straight line features at the junction of the composite plate and the main plate are obtained by projection and boundary extraction algorithms, and the final weld point is determined by intersection screening. This process achieves precise positioning of welds under complex structures through the dual constraints of point cloud registration and geometric features. Compared with the existing technology, the traditional method relies on manual teaching to adjust the trajectory, which is difficult to deal with the deviation of the workpiece placement. The present application automatically aligns the template and the actual workpiece through point cloud registration, reducing manual intervention. The existing point cloud recognition technology is susceptible to noise interference when directly extracting welds. The present application improves the robustness of feature extraction by combining projection and straight line intersection calculation with geometric constraints.

[0089] In one embodiment, Figure 4 As shown, S200 includes:

[0090] S220: Scanning the actual workpiece.

[0091] In this embodiment, a line laser profilometer (line scanner) is used to scan the actual lifting lug workpiece. Before scanning, ensure that the line scanner is correctly installed and calibrated, and that communication with the robot system is complete to ensure the accuracy and stability of the scan data. The robot drives the line scanner to perform a full-scale scan of the lifting lug workpiece according to the preset initial scanning path. During the scanning process, the line scanner continuously acquires point cloud data of the workpiece surface at a certain frame rate (for example, 100 frames / second). Each frame of point cloud data contains the shape and position information of the scanned area of ​​the workpiece at that moment.

[0092] S240: Convert each frame of line scan point cloud into the robot base coordinate system and merge them to obtain the scene point cloud of the actual workpiece placement scene.

[0093] Since the line scanning device is installed on the end effector of the robot, each frame of point cloud data collected by it is based on the coordinate system of the line scanning device itself. In order to obtain the accurate position and posture information of the workpiece in the robot workspace, it is necessary to convert each frame of line scanning point cloud to the robot base coordinate system. Through the robot kinematic model, the position transformation matrix of the line scanning device relative to the robot base coordinate system at the moment of each frame scanning is obtained. Using this transformation matrix, the coordinates of each frame of line scanning point cloud are transformed, and it is converted from the line scanning device coordinate system to the robot base coordinate system. All the converted frame line scanning point clouds are merged to obtain the scene point cloud of the actual workpiece placement scene. During the merging process, a point cloud splicing algorithm (such as a simplified version of the ICP algorithm based on feature matching) is used to align and fuse adjacent frame point clouds to eliminate overlaps and errors between point clouds and ensure the integrity and accuracy of the scene point cloud.

[0094] S260: Obtain the known position corresponding to the actual workpiece placement workbench.

[0095] In actual production environments, lifting lug workpieces are typically placed on a specific workbench. During the installation and commissioning of the robot system, the workbench's known position in the robot's base coordinate system is acquired through precise measurement and calibration. This known position information can be expressed as the coordinates of the workbench's four corner points or key feature points in the robot's base coordinate system. This known position information can be stored in the robot's control system for subsequent point cloud processing.

[0096] S280: Performing direct filtering on irrelevant point clouds in the scene point cloud based on known positions to extract actual workpiece point clouds.

[0097] Through-filtering is a simple and effective point cloud filtering method that screens and filters the point cloud based on the range of coordinate values ​​of each point in the point cloud in a certain coordinate axis direction. In this embodiment, the known position information of the workbench is used to determine the spatial range of the workbench in the robot base coordinate system. Specifically, based on the coordinates of the four corner points or key feature points of the workbench, the minimum and maximum values ​​of the workbench in the X, Y, and Z coordinate axes are calculated to form a rectangular spatial area. For each point in the scene point cloud, it is determined whether its coordinate values ​​in the X, Y, and Z coordinate axes are within the rectangular spatial area corresponding to the workbench. If the coordinate value of a point is within the area, it is considered that the point belongs to the workbench or an irrelevant point cloud near the workbench, and it is removed from the scene point cloud; if the coordinate value of a point is not within the area, it is considered that the point belongs to the actual workpiece point cloud and is retained. Through straight-through filtering processing, point cloud data irrelevant to the workpiece in the scene point cloud, such as the workbench surface and debris in the surrounding environment, can be effectively removed, thereby extracting accurate actual workpiece point clouds and providing a reliable data basis for subsequent workpiece identification and welding trajectory planning.

[0098] In one embodiment, Figure 4 As shown, S400 includes:

[0099] S420: Reading a template workpiece point cloud corresponding to the actual workpiece point cloud from a preset database.

[0100] In this embodiment, a database containing template point clouds for various types of lifting lug workpieces is pre-built. The template point clouds in this database are acquired through high-precision scanning of standard lifting lug workpieces and stored after preprocessing (e.g., denoising, streamlining, etc.). Once the actual workpiece point cloud is acquired, the corresponding template workpiece point cloud is retrieved from the pre-set database based on the actual workpiece model information (this model information can be pre-acquired through other means, such as image recognition or manual input). For example, if the actual workpiece is a lifting lug model A, the template workpiece point cloud for model A is retrieved from the database.

[0101] S440: Extracting the features of the lifting ear hole from the actual workpiece point cloud, and identifying the center of the projected point cloud of the lifting ear hole on the mainboard and the mainboard normal vector based on the features of the lifting ear hole.

[0102] A shape-based method is used to extract the features of the lifting lug holes from the actual workpiece point cloud. Because lifting lug holes have distinct circular features, the actual workpiece point cloud is first segmented to extract regions that may contain the lifting lug holes. Then, the Hough circle transform algorithm is used to detect circularity in these regions, identifying the position and shape of the lifting lug holes in the point cloud. After identifying the lifting lug holes, the projected area of ​​the lifting lug holes on the mainboard is determined. Based on the mainboard's geometric characteristics (e.g., the mainboard is typically a relatively flat surface), the center of mass of the points within the projected area is calculated to determine the center position pt of the projected point cloud of the lifting lug holes on the mainboard. Specifically, the coordinates of all point clouds within the projected area are averaged to obtain the center coordinates. The random sampling consensus algorithm (RANSAC) is used to determine the mainboard plane and then calculate the mainboard normal vector nt. The RANSAC algorithm fits a plane model by randomly selecting points from the point cloud. All points are then classified according to the fitted model, and the inliers that meet the model are used to determine the final mainboard plane. After obtaining the mainboard plane equation, the mainboard normal vector is calculated using the coefficients of the plane equation.

[0103] S460: Calculate the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud based on the center of the projected point cloud of the lifting ear hole on the main board in the actual workpiece point cloud and the main board normal vector, and the center of the projected point cloud of the lifting ear hole on the main board in the template workpiece point cloud and the main board normal vector.

[0104] First, the center position of the projection point cloud of the ear hole on the mainboard and the mainboard normal vector are extracted from the actual workpiece point cloud, and the corresponding features are obtained from the template workpiece point cloud. Next, the mainboard normal vector is used to calculate the rotation matrix to align the template workpiece mainboard orientation with the actual workpiece. The translation vector is then calculated based on the center position to achieve position matching between the two and obtain a preliminary coarse registration transformation matrix. Finally, combined with fine registration algorithms such as iterative closest point (ICP), the transformation matrix is ​​further optimized to eliminate local errors and obtain the final target homogeneous change matrix, achieving precise alignment of the actual workpiece point cloud and the template workpiece point cloud.

[0105] In a specific application example, since the motherboard normal vector n t and the template workpiece mainboard normal vector n s Only two degrees of freedom of rotation can be constrained. First, the normal vector alignment is used to determine the rotation relationship. By calculating the angle between the two normal vectors and the rotation axis, a preliminary rotation matrix R is obtained. This rotation matrix can rotate the template workpiece's mainboard normal vector to the same direction as the actual workpiece mainboard normal vector. According to the center position p of the projected point cloud on the mainboard of the ear hole, t The center position p of the projection point cloud of the template workpiece ear hole on the main board s , calculate the translation vector t = p t -Rp s The translation vector can translate the center position of the template workpiece to a position consistent with the center position of the actual workpiece. The rotation matrix R and the translation vector t are combined into a coarse registration matrix T c ,Right now

[0106]

[0107] Furthermore, even if the normal vectors are perfectly matched, there is still uncertainty in the rotation around the normal vector axis. In order to confirm the rotation angle around the normal vector axis, based on the rough registration, the center p of the actual workpiece point cloud is rotated. t and the normal vector n t Rotate. By setting different rotation angles, the template workpiece point cloud is transformed and the degree of overlap between the transformed template workpiece point cloud and the actual workpiece point cloud is calculated. Using an iterative method, the rotation angle is continuously adjusted until the angle with the highest overlap is found, resulting in the final coarse registration adjustment matrix.

[0108] On the basis of coarse registration (including coarse registration after rotation adjustment around the normal axis), the iterative closest point (ICP) algorithm is used for fine registration. The ICP algorithm seeks the optimal alignment between the actual workpiece point cloud and the template workpiece point cloud through continuous iteration, so as to minimize the distance between the corresponding points. Specifically, firstly, some corresponding points in the template workpiece point cloud and the actual workpiece point cloud are selected as the initial matching point pairs, and then the initial homogeneous change matrix is ​​calculated based on these matching point pairs. Based on the initial homogeneous change matrix, the template workpiece point cloud is transformed, and the error between the transformed template workpiece point cloud and the actual workpiece point cloud is calculated. According to the error value, the matching point pairs are adjusted, and the homogeneous change matrix is ​​recalculated until the error meets the preset convergence condition to obtain the fine registration matrix. The fine registration matrix is ​​multiplied by the coarse registration adjustment matrix to obtain the final homogeneous change matrix (target homogeneous change matrix) T from the template point cloud to the actual workpiece point cloud.

[0109] In one embodiment, extracting the lifting ear hole feature from the actual workpiece point cloud, and identifying the center of the projected point cloud of the lifting ear hole on the mainboard and the mainboard normal vector based on the lifting ear hole feature includes:

[0110] Step 1: Determine the mainboard plane and the mainboard normal vector based on the random sampling consistency algorithm.

[0111] After obtaining the actual workpiece point cloud data, the random sampling consensus algorithm (RANSAC) is first used to determine the mainboard plane. Specifically, a certain number of points are randomly selected from the actual workpiece point cloud, a plane model is constructed, and the fitting error between the model and other points is calculated. After multiple iterations, the plane model with the smallest fitting error is selected as the mainboard plane. At the same time, according to the plane equation parameters Ax+By+Cz+D=0 output by the RANSAC algorithm, the mainboard normal vector n=(A, B, C) is obtained. Since the workpiece needs to be placed upward in the robot coordinate system, the above features are used to ensure the direction of the normal vector. If the obtained plane normal vector is not facing the Z axis upward, the plane equation parameters can be inverted, that is, (-A, -B, -C, -D). Specifically, Figure 5 (a) is a point cloud image of a lifting ear workpiece after through-filtering; after the mainboard plane is extracted, the mainboard extraction effect is shown in the attached figure. Figure 5 (b) shown.

[0112] Step 2: Remove the plane point cloud of the main plate from the actual workpiece point cloud, and perform plane extraction on the remaining point cloud again to obtain the composite plate point cloud.

[0113] The extracted plane point cloud of the main plate is removed from the actual workpiece point cloud to obtain the remaining point cloud. The RANSAC algorithm is used again to extract the plane of the remaining point cloud to obtain the point cloud of the composite plate. Multiple iterations and fitting are performed on the remaining point cloud to determine the composite plate plane equation, thereby extracting the composite plate point cloud.

[0114] Step 3: Project the secondary plate point cloud to the primary plate plane to obtain the projection result.

[0115] After extracting the primary plate plane, remove the plane point cloud from the original point cloud, and perform plane extraction again on the remaining point cloud to obtain the point cloud of the secondary plate. Then, project the secondary plate point cloud onto the primary plate plane for subsequent analysis and calculation. The calculation method of the projection point is as follows:

[0116]

[0117] Where (x0, y0, z0) is the point before projection, (x', y', z') is the point projected onto the primary plate plane, and the projection result is shown in FIG. 8 Figure 5 (c).

[0118] Step 4: Extract the point cloud boundary of the primary plate plane using the Alpha Shapes algorithm.

[0119] The Alpha Shapes algorithm is used to extract the point cloud boundary of the primary plate plane. The core idea of this algorithm is that, in the point set P, draw a circle with radius a through any two points p1 and p2. If there are no other points inside this circle, then p1 and p2 are considered as boundary points. By setting a suitable a value, the point cloud of the primary plate plane is traversed and judged to extract the point cloud boundary of the primary plate plane.

[0120] Step 5: According to the projection result and the point cloud boundary of the primary plate plane, obtain the inner hole boundary points and outer boundary points of the secondary plate projection point cloud.

[0121] According to the projection result and the point cloud boundary of the primary plate plane, obtain the inner hole boundary points and outer boundary points of the secondary plate projection point cloud. The specific method is to traverse each point in the secondary plate projection point cloud and judge whether the point is inside the primary plate plane point cloud boundary. For points inside the boundary, further judge their relative position relationship with the primary plate plane boundary to distinguish the inner hole boundary points and outer boundary points. The extraction effect is shown in FIG. 8 Figure 5 (d), which contains the inner hole boundary points and outer boundary points of the secondary plate projection point cloud.

[0122] Step 6: Cluster the inner hole boundary points and outer boundary points of the secondary plate projection point cloud, and classify based on the clustering result to extract the boundary point cloud data of the ear hole.

[0123] Cluster the inner and outer boundary points of the composite plate's projected point cloud. Using a Euclidean distance clustering algorithm, set an appropriate distance threshold and group points with distances less than the threshold into the same cluster. After clustering, the cluster with the smallest number of points is selected as the boundary point cloud data for the eyelet holes (i.e., the second cluster, which can be adjusted based on actual conditions) based on the actual application scenario and workpiece structure.

[0124] Step 7: Perform spatial circle fitting on the boundary point cloud of the lifting ear hole based on the random sampling consistency algorithm to determine the center of the projected point cloud of the lifting ear hole on the mainboard and the mainboard normal vector.

[0125] A spatial circle fitting is performed on the extracted boundary point cloud of the lifting lug hole based on a random sampling consistency algorithm. A certain number of points are randomly selected from the boundary point cloud of the lifting lug hole to construct a spatial circle model, and the fitting error between this model and other points is calculated. After multiple iterations, the spatial circle model with the smallest fitting error is selected as the boundary circle of the lifting lug hole. Based on the parameters of this spatial circle model, the center position of the projected point cloud of the lifting lug hole on the motherboard is determined. At the same time, since the motherboard normal vector is already determined when the motherboard plane is extracted and does not change during subsequent processing, the final motherboard normal vector is the motherboard normal vector used to determine the center of the lifting lug hole.

[0126] In one embodiment, determining the mainboard plane and the mainboard normal vector based on a random sampling consistency algorithm includes:

[0127] Step 1: Randomly select 3 points from the actual workpiece point cloud and calculate the plane equation determined by the 3 selected points.

[0128] Randomly select three points from the point cloud and calculate the plane equation determined by them. Suppose three non-collinear points P1(x1, y1, z1), P2(x2, y2, z2), and P3(x3, y3, z3) are randomly selected from the point cloud. The normal vector N(A, B, C) of the plane can be calculated by the cross product of the two direction vectors P1P2 and P1P3. Then substitute P1(x1, y1, z1) into the plane equation to obtain the plane parameter D.

[0129] Step 2: According to the plane equation, calculate the distance error between all points in the actual workpiece point cloud and the plane, and determine whether the corresponding point is an interior point based on the error.

[0130] For each point in the actual workpiece point cloud, it is necessary to calculate its distance to the plane determined above. The method of calculating the distance is to substitute the coordinates of the point into the plane equation to obtain an expression for the distance from the point to the plane. The actual distance from the point to the plane can be calculated through this expression. In order to determine whether a point is an inlier, it is necessary to set a reasonable error threshold. This threshold is determined based on actual conditions and experience, and it reflects the requirements for the accuracy of plane fitting. If the calculated distance from the point to the plane is less than the set error threshold, then the point is considered to be an inlier, that is, the point is within the reasonable range of plane fitting; conversely, if the distance is greater than the error threshold, then the point is not an inlier.

[0131] Step 3: Find the plane containing the most internal points, use the found plane as the main board plane, and determine the main board normal vector.

[0132] Since the three randomly selected points may not accurately represent the mainboard plane of the entire point cloud, multiple iterations are required. During each iteration, the above steps of randomly selecting points, calculating the plane equation, calculating the distance error and judging the inliers are repeated. After each iteration, the number of inliers contained in the currently determined plane is recorded. As the number of iterations increases, multiple different planes and the number of inliers they each contain will be obtained. When the set number of iterations is reached, the planes obtained in all iterative processes are evaluated to find the plane with the largest number of inliers. The plane with the most inliers is considered to be the best fitting plane of the mainboard, that is, the mainboard plane to be determined. At the same time, since the corresponding normal vector is obtained each time the plane equation is calculated, the normal vector corresponding to the best fitting plane is the final mainboard normal vector.

[0133] In one embodiment, calculating a target homogeneous change matrix between the actual workpiece point cloud and the template workpiece point cloud based on the center of the projection point cloud of the lifting ear hole on the mainboard and the mainboard normal vector in the actual workpiece point cloud and the center of the projection point cloud of the lifting ear hole on the mainboard and the mainboard normal vector in the template workpiece point cloud includes:

[0134] Step 1: Use the mainboard normal vector in the actual workpiece point cloud to align with the mainboard normal vector in the template workpiece point cloud to determine the rotation relationship between the actual workpiece point cloud and the template workpiece point cloud.

[0135] Use the rotation axis-angle method to calculate the template point cloud normal vector n s Rotate to the actual workpiece point cloud mainboard normal vector n t The calculation of the rotation axis v uses n s With n t The cross product of .

[0136] v=n s ×nt

[0137] If the modulus of the rotation axis v is less than a certain threshold, it means that the two normal vectors are almost aligned and no rotation is required. The rotation matrix is ​​assigned to R = I. If the modulus is greater than a certain threshold, the rotation angle θ around the rotation axis is solved.

[0138] θ=arccos(n s ·n t )

[0139] After having the rotation axis v and the rotation angle, use the following formula to solve the rotation matrix R:

[0140]

[0141] Among them, [v] × is the antisymmetric matrix of v:

[0142]

[0143] With the rotation matrix R, the translation vector can be determined by the correspondence between the center points of the circle:

[0144] t=p t -Rp s

[0145] The homogeneous transformation matrix is:

[0146]

[0147] Step 2: Align the center of the projected point cloud of the lifting ear hole on the main board in the actual workpiece point cloud with the center of the projected point cloud of the lifting ear hole on the main board in the template workpiece point cloud to determine the translation relationship between the actual workpiece point cloud and the template workpiece point cloud.

[0148] After determining the rotation relationship, the center point p of the projection point cloud of the ear hole in the actual workpiece point cloud on the main board is t The center point p of the projection point cloud of the ear hole in the template workpiece point cloud on the main board s , calculate the translation vector t. The calculation formula of the translation vector t is t=p t -Rp s , where R is the rotation matrix calculated in the previous step. Through this calculation, the translation relationship between the actual workpiece point cloud and the template workpiece point cloud is determined.

[0149] Step 3: Based on the rotation relationship and translation relationship, construct the initial homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud.

[0150] According to the rotation matrix R and translation vector t obtained above, the initial homogeneous change matrix Tc of the actual workpiece point cloud and the template workpiece point cloud is constructed. The form of the initial homogeneous change matrix Tc is:

[0151]

[0152] Step 4: Determine the optimal rotation angle based on the rotation relationship and the optimization method of rotation angle traversal.

[0153] The above process is based on the calculation of the normal vector rotation matrix, which can only constrain the two degrees of freedom of rotation. The rotation angle around the normal vector cannot be determined. Figure 5 (e) In order to determine the rotation angle around the normal vector axis, an optimization method based on rotation angle traversal is proposed in this embodiment. Specifically, the template point cloud starts from 0 degrees, takes 10 degrees as a step, and gradually rotates to 360 degrees around the center of the actual workpiece point cloud and the normal vector axis. By calculating the overlap under different rotation angles, the optimal rotation angle corresponding to the highest overlap is determined. Let the center of the workpiece point cloud and the mainboard normal vector be p t and n t , the solution of the homogeneous rigid transformation matrix is ​​as follows:

[0154] θ∈{0°,10°,20°,...,360°};

[0155]

[0156] t=p t -Rp t ;

[0157]

[0158] The method for judging the overlap is to calculate the c2 *T c1 The RMSE (Root Mean Square Error) of the template point cloud and the workpiece point cloud after left multiplication is used to judge:

[0159]

[0160] Find the distances of each point in the modified template point cloud to the N nearest points in the workpiece point cloud, and then average them to obtain the rotation angle that minimizes the RMSE.

[0161] Step 5: According to the optimal rotation angle and the initial homogeneous change matrix, the coarse registration homogeneous change matrix is ​​obtained.

[0162] Get the homogeneous transformation matrix T at this angle c2 , the final coarse registration matrix is: T cf =Tc2 ·T c The point cloud after rough registration is shown in the attached Figure 5 (f) shown.

[0163] Step 6: Use the ICP algorithm to perform fine registration on the coarse registration homogeneous change matrix to obtain the target homogeneous change matrix.

[0164] In addition to the above processing, fine registration is further performed to obtain a more accurate homogeneous transformation matrix. The fine registration method used in the present invention is the ICP (Iterative Closest Point) algorithm, which finds the point cloud of the workpiece and each point p after the rough registration template point cloud. i The nearest point q i , forming a set of matching point pairs, and then calculating the optimal rotation matrix R and translation vector t by singular value decomposition to minimize the error:

[0165]

[0166] And keep applying the change (R, t) until the final iteration condition is reached. The final precise registration matrix T is obtained. f The final homogeneous transformation matrix T from the template point cloud to the actual workpiece point cloud, where T = T f ·T cf .

[0167] Furthermore, assuming that the teaching trajectory of the template point cloud is Path teach {X1, X2, ..., X n}, then the teaching trajectory on the actual workpiece point cloud is Path actual {T·X1, T·X2, ..., T·X n More specifically, X1 = {Loc1, Pos1} consists of the three-dimensional position and posture, and T·X1 = {T·Loc1, T·Pos1}.

[0168] In one embodiment, the weld area point cloud information is obtained according to the adjusted scanning trajectory, and the fillet weld recognition method based on the weld area point cloud information and the straight line intersection points identifies the weld points, including:

[0169] Step 1: Obtain the point cloud information of the weld area according to the adjusted scanning trajectory.

[0170] C. In order to accurately locate the weld points, precise positioning is also required. First, based on the automatically adjusted scanning trajectory, the point cloud information of the weld is obtained. However, since the lug workpiece consists of a main plate and a secondary plate, its line scan imaging presents a stepped distribution, and the weld points are the intersections of straight lines, as shown in the attached figure. Figure 6 As shown in (a).

[0171] Based on the optimized scanning trajectory, the line scanning device is controlled to accurately scan the weld area, and the acquired line scan point cloud data is converted to the robot base coordinate system to obtain complete point cloud information of the weld area. This point cloud information contains the 3D spatial coordinate data of the weld area, providing the basic data for subsequent line extraction and intersection calculation.

[0172] Step 2: According to the point cloud information of the weld area, four straight lines are extracted based on the consistency of random sampling, and the intersection points between any two straight lines are calculated to obtain four intersection points.

[0173] The point cloud information of the weld area is processed using the Random Sampling Consensus (RANSAC) algorithm. The RANSAC algorithm randomly selects point pairs from the point cloud, fits a linear model, and counts the number of points that fit the model. After multiple iterations, the linear model with the largest number of fitting points is selected as the extracted line. In this example, four lines were extracted from the weld area point cloud after RANSAC processing.

[0174] For the four extracted straight lines, calculate the intersection points between each pair of straight lines. Before calculating the intersection point, it is necessary to determine whether the normal vectors of the two straight lines are close to parallel. The specific judgment method is to calculate the angle between the normal vectors of the two straight lines. If the angle is less than the set threshold (for example, the threshold can be set to 5°), it is considered that the normal vectors of the two straight lines are close to parallel, and the intersection calculation of the pair of straight lines is skipped directly. For straight line pairs whose normal vectors are not close to parallel, the coordinates of their intersection points are calculated by solving the system of straight line equations. After the above steps, 4 intersection points will be obtained, as shown in the attached figure. Figure 6 (b) shown.

[0175] Step 3: Eliminate the two intersection points with larger vertical coordinates among the four intersection points.

[0176] Compare the vertical (Z-direction) coordinates of the four intersection points obtained. Remove the two intersection points with larger Z-direction coordinates, and retain the remaining two intersection points with lower Z-direction coordinates. This step aims to improve the accuracy of weld point positioning by removing intersection points that are located too high, possibly due to interference or error.

[0177] Step 4: Find the point closest to the rough positioning weld point among the remaining two intersection points to obtain the weld point.

[0178] Then, among the remaining two points, find the point closest to the rough positioning weld point, calculate the distance D between it and the nearest point, and select the point with the smaller distance to the nearest point as the weld point.

[0179] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0180] Based on the same inventive concept, embodiments of the present application also provide a point cloud recognition device for automatic welding of lug workpiece fillet welds, which is used to implement the aforementioned point cloud recognition method for automatic welding of lug workpiece fillet welds. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the point cloud recognition device for automatic welding of lug workpiece fillet welds provided below can be found in the above-mentioned limitations of the point cloud recognition method for automatic welding of lug workpiece fillet welds, and will not be further elaborated here.

[0181] In one embodiment, Figure 7 As shown, a point cloud recognition device for automatic welding of fillet welds of lifting lug workpieces is provided, comprising:

[0182] The point cloud acquisition module 200 is used to acquire a scene point cloud of an actual workpiece placement scene and extract the actual workpiece point cloud from the scene point cloud;

[0183] A homogeneous transformation module 400 is used to calculate a target homogeneous change matrix between the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm based on the actual workpiece point cloud;

[0184] The scanning trajectory correction module 600 is used to correct the scanning trajectory preset for the actual workpiece based on the target homogeneous change matrix to obtain an adjusted scanning trajectory;

[0185] The weld point recognition module 800 is used to obtain weld area point cloud information according to the adjusted scanning trajectory, and identify weld points based on the weld area point cloud information and the fillet weld recognition method of the straight line intersection.

[0186] In one embodiment, the point cloud acquisition module 200 is also used to scan the actual workpiece; convert each frame of the line scan point cloud into the robot base coordinate system for merging to obtain the scene point cloud of the actual workpiece placement scene; obtain the known position corresponding to the actual workpiece placement workbench; based on the known position, perform direct filtering on irrelevant point clouds in the scene point cloud to extract the actual workpiece point cloud.

[0187] In one embodiment, the homogeneous transformation module 400 is also used to read the template workpiece point cloud corresponding to the actual workpiece point cloud from a preset database; extract the ear hole features in the actual workpiece point cloud, and identify the center of the point cloud projected by the ear hole on the main board and the main board normal vector based on the ear hole features; calculate the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud based on the center of the point cloud projected by the ear hole on the main board in the actual workpiece point cloud and the main board normal vector, and the center of the point cloud projected by the ear hole on the main board in the template workpiece point cloud and the main board normal vector.

[0188] In one embodiment, the homogeneous transformation module 400 is also used to determine the mainboard plane and the mainboard normal vector based on a random sampling consistency algorithm; remove the plane point cloud of the mainboard plane from the actual workpiece point cloud, and perform plane extraction on the remaining point cloud again to obtain a composite plate point cloud; project the composite plate point cloud onto the mainboard plane to obtain a projection result; use the AlphaShapes algorithm to extract the point cloud boundary of the mainboard plane; obtain the inner hole boundary points and outer boundary points of the composite plate projection point cloud based on the projection result and the point cloud boundary of the mainboard plane; cluster the inner hole boundary points and outer boundary points of the composite plate projection point cloud, and classify them based on the clustering results to extract the boundary point cloud data of the lifting ear hole; perform spatial circle fitting on the boundary point cloud of the lifting ear hole based on the random sampling consistency algorithm to determine the center of the projection point cloud of the lifting ear hole on the mainboard, and the mainboard normal vector.

[0189] In one embodiment, the homogeneous transformation module 400 is also used to randomly select 3 points from the actual workpiece point cloud and calculate the plane equation determined by the selected 3 points; according to the plane equation, calculate the distance error from all points in the actual workpiece point cloud to the plane, and judge whether the corresponding point is an inner point based on the error; find the plane containing the most inner points, and use the found plane as the main board plane, and determine the main board normal vector.

[0190] In one embodiment, the homogeneous transformation module 400 is also used to align the mainboard normal vector in the actual workpiece point cloud with the mainboard normal vector in the template workpiece point cloud to determine the rotation relationship between the actual workpiece point cloud and the template workpiece point cloud; align the center of the projected point cloud of the lifting ear hole in the actual workpiece point cloud on the mainboard with the center of the projected point cloud of the lifting ear hole in the template workpiece point cloud on the mainboard to determine the translation relationship between the actual workpiece point cloud and the template workpiece point cloud; construct an initial homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud based on the rotation relationship and the translation relationship; determine the optimal rotation angle based on the rotation relationship and the optimization method of rotation angle traversal; obtain a coarse alignment homogeneous change matrix based on the optimal rotation angle and the initial homogeneous change matrix; and use the ICP algorithm to finely align the coarse alignment homogeneous change matrix to obtain a target homogeneous change matrix.

[0191] In one of the embodiments, the weld point identification module 800 is further configured to obtain weld region point cloud information according to the adjusted scanning track; extract four straight lines based on random sample consensus according to the weld region point cloud information, obtain four intersection points between the straight lines, and remove two intersection points with larger vertical direction coordinates from the four intersection points; and find a point closest to the coarse positioning weld point from the remaining two intersection points to obtain the weld point.

[0192] The modules in the point cloud identification device for automatic welding of lug workpiece fillet welds described above can be implemented wholly or partially by software, hardware, or a combination thereof. The modules described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to the modules.

[0193] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 8 The computer device includes a processor, a memory, and a network interface connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store preset data. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a point cloud identification method for automatic welding of lug workpiece fillet welds.

[0194] Those skilled in the art can understand that Figure 8 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. A specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0195] In one embodiment, a computer device is provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the point cloud identification method for automatic welding of lug workpiece fillet welds described above.

[0196] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the point cloud identification method for automatic welding of lug workpiece fillet welds described above.

[0197] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the above-mentioned point cloud recognition method for automatic welding of fillet welds of lifting lug workpieces.

[0198] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0199] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0200] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A point cloud recognition method for automatic welding of fillet welds of lifting lug workpieces, characterized in that: The method comprises: Acquire a scene point cloud of an actual workpiece placement scene, and extract the actual workpiece point cloud from the scene point cloud; Calculating a target homogeneous change matrix between the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm based on the actual workpiece point cloud; Correcting the preset scanning trajectory of the actual workpiece based on the target homogeneous change matrix to obtain an adjusted scanning trajectory; The weld area point cloud information is acquired according to the adjusted scanning trajectory, and the weld points are identified based on the weld area point cloud information and a fillet weld identification method of straight line intersections.

2. The method according to claim 1, characterized in that The step of obtaining a scene point cloud of an actual workpiece placement scene and extracting the actual workpiece point cloud from the scene point cloud comprises: Scan the actual workpiece; Convert each frame of line scan point cloud to the robot base coordinate system and merge them to obtain the scene point cloud of the actual workpiece placement scene; Obtaining a known position corresponding to the workbench where the actual workpiece is placed; Based on the known position, irrelevant point clouds in the scene point cloud are filtered through to extract the actual workpiece point cloud.

3. The method according to claim 1, characterized in that The step of calculating the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm according to the actual workpiece point cloud comprises: Reading a template workpiece point cloud corresponding to the actual workpiece point cloud from a preset database; Extracting the ear hole feature from the actual workpiece point cloud, and identifying the center of the projected point cloud of the ear hole on the mainboard and the mainboard normal vector based on the ear hole feature; According to the center of the projected point cloud of the lifting ear hole on the main board in the actual workpiece point cloud and the main board normal vector, and the center of the projected point cloud of the lifting ear hole on the main board in the template workpiece point cloud and the main board normal vector, the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud is calculated.

4. The method according to claim 3, characterized in that The step of extracting the lifting ear hole feature from the actual workpiece point cloud and identifying the center of the projection point cloud of the lifting ear hole on the mainboard and the mainboard normal vector based on the lifting ear hole feature includes: Determine the mainboard plane and mainboard normal vector based on random sampling consistency algorithm; Removing the plane point cloud of the main plate from the actual workpiece point cloud, and performing plane extraction again on the remaining point cloud to obtain a composite plate point cloud; Projecting the complex plate point cloud onto the main plate plane to obtain a projection result; Extracting the point cloud boundary of the mainboard plane using the Alpha Shapes algorithm; Obtaining inner hole boundary points and outer boundary points of the composite plate projection point cloud according to the projection result and the point cloud boundary of the main plate plane; Clustering the inner hole boundary points and the outer boundary points of the composite plate projection point cloud, and classifying based on the clustering results to extract the boundary point cloud data of the lifting lug hole; Based on the random sampling consistency algorithm, the boundary point cloud of the ear hole is fitted with a spatial circle to determine the center of the projected point cloud of the ear hole on the mainboard and the mainboard normal vector.

5. The method according to claim 4, characterized in that The determining of the mainboard plane and the mainboard normal vector based on the random sampling consistency algorithm includes: Randomly select three points from the actual workpiece point cloud and calculate the plane equation determined by the three selected points; According to the plane equation, calculating the distance errors between all points in the actual workpiece point cloud and the plane, and judging whether the corresponding points are interior points based on the errors; Find the plane containing the most internal points, use the found plane as the main board plane, and determine the main board normal vector.

6. The method according to claim 3, characterized in that Calculating the target homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud based on the center of the projected point cloud of the lifting ear hole on the main board in the actual workpiece point cloud and the main board normal vector and the center of the projected point cloud of the lifting ear hole on the main board in the template workpiece point cloud and the main board normal vector comprises: Determine the rotational relationship between the actual workpiece point cloud and the template workpiece point cloud by aligning the mainboard normal vector in the actual workpiece point cloud with the mainboard normal vector in the template workpiece point cloud; Determine the translation relationship between the actual workpiece point cloud and the template workpiece point cloud by aligning the center of the projected point cloud of the lifting ear hole on the main board in the actual workpiece point cloud with the center of the projected point cloud of the lifting ear hole on the main board in the template workpiece point cloud; Constructing an initial homogeneous change matrix of the actual workpiece point cloud and the template workpiece point cloud according to the rotation relationship and the translation relationship; Determining the optimal rotation angle based on the rotation relationship and using an optimization method of rotation angle traversal; Obtaining a coarse registration homogeneous change matrix according to the optimal rotation angle and the initial homogeneous change matrix; The coarse registration homogeneous change matrix is ​​finely registered using the ICP algorithm to obtain a target homogeneous change matrix.

7. The method according to claim 1, characterized in that The method for identifying weld points based on the weld area point cloud information obtained according to the adjusted scanning trajectory and the fillet weld identification method based on the weld area point cloud information and the straight line intersection includes: According to the adjusted scanning trajectory, the point cloud information of the weld area is obtained; According to the point cloud information of the weld area, four straight lines are extracted based on the consistency of random sampling, and the intersection points between any two straight lines are calculated to obtain four intersection points; Eliminate the two intersection points with larger vertical coordinates among the four intersection points; Find the point closest to the rough positioning weld point among the remaining two intersection points to obtain the weld point.

8. A point cloud recognition device for automatic welding of fillet welds of lifting lug workpieces, characterized in that: The device comprises: A point cloud acquisition module is used to acquire a scene point cloud of an actual workpiece placement scene and extract the actual workpiece point cloud from the scene point cloud; A homogeneous transformation module is used to calculate a target homogeneous change matrix between the actual workpiece point cloud and the template workpiece point cloud using a registration algorithm according to the actual workpiece point cloud; A scanning trajectory correction module, configured to correct a preset scanning trajectory of the actual workpiece based on the target homogeneous change matrix to obtain an adjusted scanning trajectory; The weld point recognition module is used to obtain weld area point cloud information according to the adjusted scanning trajectory, and identify weld points based on the weld area point cloud information and the fillet weld recognition method of the straight line intersection.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images