An automatic welding label method and device based on three-dimensional point cloud

Abstract

The application discloses a kind of automatic welding label method and device based on three-dimensional point cloud, the method includes: when steel bundle reaches the work station of to-be-welded area, trigger three-dimensional vision sensor to take photograph, collect the three-dimensional point cloud image of steel bundle end face;The point cloud collected is identified and positioned, determine label weldable point, and according to the number of label to be welded and the rule of selecting point, select the label welding point meeting the requirement;The position coordinates of the label welding point meeting the requirement under the robot coordinate system are sent to robot to carry out label welding;After completing label welding action, robot returns to preset safety position, triggers three-dimensional vision sensor to take photograph again, judges whether label is welded successfully;If label welding is successful, welding process ends and waits for the next steel bundle to be in position, otherwise, carry out repair welding operation.The application realizes the automation and intelligentization of steel bundle label welding by machine vision technology, improves the efficiency of steel bundle automatic label welding.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to an automatic welding method and apparatus for steel bundle signs. Background Technology

[0002] After steel is bundled and packaged, steel mills typically weld a number of labels to the ends of each bundle. These labels display basic information about the bundles, such as model number, specifications, and production date. Traditional label welding is done manually, which is inefficient and unsuitable for high-volume production. To improve efficiency, many steel mills now widely use machine-controlled methods to automate label welding.

[0003] The process of automatically welding signs generally includes the following steps: When the steel bundle is transported to the welding position, the system controls a binocular camera to acquire images of the end face of the steel bundle. Through image processing, the first welding point for the sign is selected. Then, based on the three-dimensional coordinates of the first welding point, the system controls a telescopic welding torch to perform welding. After welding is completed, the system controls the binocular camera to acquire images of the end face of the steel bundle again to determine whether the first sign was successfully welded. If welding is unsuccessful, the system controls the binocular camera to acquire images again, processes the images, and re-acquires the first welding point for welding. If welding is successful, the above operation is repeated to select and weld the welding point for the second sign, and so on, until all signs are welded.

[0004] In automated welding, image processing is the most time-consuming step. In the process described above, image acquisition and processing are required before welding each sign to obtain the coordinates of the welding point. Therefore, completing the welding of all signs in a steel bundle will take a considerable amount of time. When the number of signs to be welded is large, the efficiency of this automated welding method is low and cannot meet actual production requirements. Summary of the Invention

[0005] To address the low efficiency of existing steel bundled sign welding methods, this application first provides an automatic sign welding method based on three-dimensional point clouds, characterized by comprising: Collect a 3D point cloud image of the end face of the steel bundle; Point cloud preprocessing includes downsampling, noise removal, and clustering segmentation; For each type of point cloud obtained from clustering and segmentation, perform planar extraction and remove point cloud trailing; Calculate the centroid coordinates of each type of point cloud to obtain a set of centroid coordinate points. Filter out centroid coordinate points whose difference between the maximum and minimum Z coordinates does not meet the requirements or whose number of points extracted from the plane does not meet the requirements. To determine the adhesion of reinforcing bars, the centroid coordinates of a certain type of point cloud that are adhered are screened out; Collision avoidance detection is performed to screen out centroid coordinates of point clouds that pose a collision risk; The welding area is determined by avoiding the edge of the steel bundle, and the centroid coordinate points that the robot cannot reach within the welding area are screened out to obtain the label welding points that meet the requirements. Send the position coordinates of the compliant label welding points in the robot coordinate system to the robot for label welding.

[0006] A further technical solution of the present invention is: Preferably, the acquisition of a three-dimensional point cloud image of the steel bundle end face includes: when the steel bundle arrives at the welding area, triggering a three-dimensional vision sensor to take a picture and acquire a three-dimensional point cloud image of the steel bundle end face.

[0007] Preferably, after the robot completes the label welding, the process further includes: The robot returns to the preset safe position, triggers the 3D vision sensor again to take pictures, and collects the 3D point cloud map of the steel bundle end face again; The success of the signboard welding is determined by the three-dimensional point cloud image of the steel bundle end face collected again. If the sign is successfully welded, the welding process ends and we continue to wait for the next steel bundle to arrive. If the welding of the sign fails, a repair welding operation will be performed.

[0008] Preferably, each type of point cloud obtained by clustering is subjected to plane extraction to remove point cloud tails, including: fitting the plane using the RANSAC algorithm.

[0009] Preferably, the process of determining the adhesion of reinforcing bars and filtering out centroid coordinate points of a type of point cloud that are adhered includes: calculating the median of the number of point clouds based on the number of clusters, calculating the ratio of the number of point clouds in each type corresponding to the remaining centroid coordinate points to the median, and filtering out centroid coordinate points of a type of point cloud whose ratio is not within the normal range.

[0010] Preferably, collision avoidance detection is performed to screen out centroid coordinate points of the point cloud that pose a collision risk, including collision avoidance detection with the welding device: The remaining centroid coordinates after the steel reinforcement adhesion judgment are arranged in descending order of Z coordinate. Take any centroid coordinate point, traverse all remaining centroid coordinate points after the steel bar adhesion judgment, and store the centroid coordinate points that satisfy the following conditions into the first point set. xa-W≤center_set[i].x≤xa+W ya-H≤center_set[i].y≤xa+H Where (xa, ya) represents the coordinates of any chosen centroid point, W and H represent the width and height of the area where the welding device is located, respectively, and center_set[i].x and center_set[i].y represent the x and y coordinates of any remaining centroid point i, respectively; Arrange the centroid coordinates in the first point set in descending order of Z coordinate. Calculate the absolute value of the difference between the Z coordinate of each centroid coordinate point and the maximum value of the Z coordinate in the first point set, and filter out centroid coordinates that do not meet the difference requirements.

[0011] Preferably, the welding area is determined by avoiding the edge of the steel bundle, and the centroid coordinate points within the welding area that the robot cannot reach are eliminated to obtain the required label welding points, including: The remaining centroid coordinates after collision avoidance detection are traversed, and centroid coordinates that meet the following requirements are selected: Xmin+dis_x1≤center_set[i].x≤Xmax-dis_x2 Ymin+dis_y1≤center_set[i].y≤Ymax-dis_y2 Where Xmin, Xmax, Ymin, and Ymax are coordinate values ​​in the robot coordinate system, Xmin and Xmax are the minimum and maximum values ​​of the steel bundle in the X direction, Ymin and Ymax are the minimum and maximum values ​​of the steel bundle in the Y direction, dis_x1 and dis_x2 are the deviations that need to be made in the X direction of the steel bundle for the integrated welding area and the area that the robot cannot reach, and dis_y1 and dis_y2 are the deviations that need to be made in the Y direction of the steel bundle for the integrated welding area and the area that the robot cannot reach; center_set[i].x and center_set[i].y represent the x and y coordinates of any remaining centroid coordinate point i, respectively. The selected centroid coordinates are the weldable points; Based on the number of signs to be welded and the selection rules, select the appropriate welding points for the signs. If one sign needs to be welded, select four welding points in the middle area of ​​the steel bundle end face. If two signs need to be welded, select two welding points each in the left and right areas of the steel bundle end face.

[0012] This application also provides an automatic welding sign device based on three-dimensional point clouds, characterized in that it includes: The acquisition module is used to acquire a 3D point cloud image of the end face of the steel bundle; The preprocessing module performs preprocessing on the point cloud, including downsampling, noise removal, and clustering segmentation. The plane extraction module is used to extract planes from each type of point cloud obtained by clustering and segmentation, and remove point cloud trailing. The point cloud feature labeling module calculates the centroid coordinates of each type of point cloud, obtains a set of centroid coordinate points, and filters out centroid coordinate points of a type of point cloud whose difference between the maximum and minimum Z coordinates does not meet the requirements or whose number of extracted plane points does not meet the requirements. The rebar adhesion judgment module is used to filter out centroid coordinate points of a type of point cloud that are adhered; The collision avoidance detection module is used to filter out centroid coordinates of point clouds that pose a collision risk. The label welding point location calculation module determines the welding area by avoiding the edge of the steel bundle, and filters out the centroid coordinate points in the welding area that the robot cannot reach, thus obtaining the label welding points that meet the requirements. The label welding point location sending module sends the position coordinates of the qualified label welding points in the robot coordinate system to the robot for label welding.

[0013] Preferably, the label welding point position calculation module includes a calibration algorithm module, which is used to realize the conversion from the visual coordinate system to the robot coordinate system using a five-point calibration method.

[0014] This application also provides an automatic welding method for nameplates based on three-dimensional point clouds, characterized by comprising: When the steel bundle arrives at the welding area, the 3D vision sensor is triggered to take a picture and collect a 3D point cloud image of the end face of the steel bundle. The collected point cloud is identified and located to determine the weldable points of the signs. Based on the number of signs to be welded and the point selection rules, the welding points that meet the requirements are selected. If one sign needs to be welded, four welding points are selected in the middle area of ​​the steel bundle end face. If two signs need to be welded, two welding points are selected in the left and right areas of the steel bundle end face. The robot sends the position coordinates of the welding points of the compliant sign in the robot coordinate system, and the robot drives the pre-picked welding nails and the sign to perform the welding of the sign at the coordinate points; After completing the sign welding action, the robot returns to the preset safe position, triggers the 3D vision sensor to take pictures again, and determines whether the sign welding is successful based on the number of signs to be welded and the point cloud image acquired. If the sign is successfully welded, the welding process ends and we continue to wait for the next steel bundle to arrive. If the welding of the sign fails, a repair welding operation will be performed.

[0015] Beneficial effects:

[0016] The automatic welding method for nameplates based on 3D point clouds provided in this application employs a 3D vision processing system. First, it acquires a 3D point cloud image of the steel bundle end face. Then, it automatically identifies and removes point cloud trailing, filters out coordinates of points with adhesion, collision risks, or inaccessible locations by the robot, and automatically obtains the required welding point positions for the nameplates, sending them to the robot for welding. This automatic welding method based on 3D point clouds acquires and processes a 3D point cloud image of the steel bundle end face at once using machine vision technology. Based on the number of nameplates to be welded, it obtains all required welding points at once and sends them to the robot for welding, achieving automation and intelligence in steel bundle nameplate welding and improving the efficiency of automatic steel bundle nameplate welding. Attached Figure Description

[0017] Figure 1 This is a flowchart of the automatic welding sign method based on three-dimensional point clouds in Example 1; Figure 2 This is a schematic diagram of the three-dimensional vision sensor structure in Example 1; Figure 3 This is the preprocessed point cloud image from Example 1; Figure 4 This is a schematic diagram of the point cloud trailing effect in Example 1; Figure 5 This is a schematic diagram of the welding equipment area; Figure 6 This is a schematic diagram of the welding area; Figure 7 A schematic diagram of the area to be selected for welding a sign; Figure 8 A schematic diagram showing the selected areas for welding two signs; Figure 9 This is a schematic diagram of the automatic welding sign device based on three-dimensional point clouds in Example 2; Figure 10 This is a flowchart of the automatic welding sign method based on three-dimensional point clouds in Example 3; Figure 11 This is a flowchart of the automatic welding sign method based on three-dimensional point cloud in Example 4. Detailed Implementation

[0018] To make the technical solution of the present invention easier to understand, the technical solution of the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings. Obviously, the following embodiments are only some embodiments of the present invention and do not constitute a limitation on the technical solution of the present invention. For those skilled in the art, other embodiments can be obtained based on the embodiments of this application without creative effort.

[0019] It should be noted that the order of steps described in the method embodiments below does not constitute a limitation on the technical solution of the present invention. For those skilled in the art, certain step orders can be reasonably adjusted based on the embodiments of this application without creative effort. It should also be noted that the terms "first," "second," etc., used in the device embodiments below are only used to distinguish similar objects and are not used to describe a specific order, nor do they constitute a limitation on the technical solution of the present invention.

[0020] Example 1

[0021] This embodiment provides an automatic welding method for nameplates based on three-dimensional point clouds, such as... Figure 1 As shown, it includes: Step S101: Collect a three-dimensional point cloud image of the end face of the steel bundle.

[0022] This embodiment employs a 3D vision processing system to automatically acquire the locations of the required label welding points on the end face of the steel bundle. The system mainly consists of a 3D vision device, a computer, and a controller. The 3D vision device comprises a 3D vision sensor and a 3D vision sensor bracket, with the sensor fixedly mounted facing the end face of the steel bundle. The 3D vision sensor acquires the 3D point cloud information of the object being measured, thus capturing the point cloud image of the reinforcing bars on the end face of the steel bundle and sending it to the computer for processing.

[0023] Specifically, such as Figure 2 As shown, the 3D vision sensor consists of an optical engine 13, a camera 11, and a 3D vision sensor base plate 12. There are two cameras, which are installed at a certain angle. To ensure the accuracy of the point cloud, the angle between the two cameras is between 12° and 20°.

[0024] Specifically, when the steel bundle arrives at the welding area, a 3D vision sensor is triggered to take a picture and obtain a 3D point cloud map of the end face of the steel bundle.

[0025] Step S102: Preprocess the point cloud, including downsampling, noise removal, and clustering segmentation. The preprocessed 3D point cloud image is shown below. Figure 3 As shown.

[0026] Step S103: Perform planar extraction on each type of point cloud obtained from clustering and segmentation, and remove point cloud tails.

[0027] Because the 3D vision sensor is installed in a fixed position while the steel bundle moves with the chain, and the chain is currently operated manually on site, the consistency of the steel bundle's position is inconsistent each time. Therefore, there is a certain field of view angle between the 3D vision sensor and the steel bundle, resulting in anomalies in the point cloud image acquired by the 3D vision sensor after point cloud preprocessing. These anomalies include not only point clouds showing the rebar end faces but also some trailing phenomena, such as... Figure 4 As shown in the rectangle. If the final selected welding point is this type of abnormal rebar point, its large centroid Z-coordinate will cause the welding torch head to collide, so this abnormal situation needs to be resolved.

[0028] Specifically, to address the point cloud trailing issue, this implementation employs the RANSAC algorithm to fit a plane after clustering and segmentation. Plane extraction is performed on each point cloud class to remove the trailing phenomenon. The RANSAC algorithm mainly consists of obtaining random samples and determining consistency, and the specific algorithm is as follows: (1) Randomly select the minimum number of elements that can form a mathematical model from the input data, and use these elements to calculate the parameters of the corresponding model; (2) Check which elements in all the data can fit the model obtained in the first step. Elements that exceed the error threshold are considered outliers, and elements that are less than the error threshold are considered inliers.

[0029] According to the RANSAC algorithm, this embodiment creates a random sampling consistency object based on the selected fitted point cloud and the planar geometric model, sets a distance threshold (points with a distance of less than 1.2 from the plane are regarded as inliers), and extracts inliers based on the index after performing model estimation, thereby eliminating the point cloud tailing phenomenon.

[0030] Step S104: Calculate the centroid coordinates of each type of point cloud to obtain a set of centroid coordinate points. Perform initial screening of centroid coordinate points, specifically by removing centroid coordinate points whose difference between the maximum and minimum Z coordinate values ​​does not meet the requirements or whose number of points extracted from the plane does not meet the requirements.

[0031] This step involves point cloud feature labeling, starting with calculating the centroid coordinates of each type of point cloud. P c As the initial feature of each type of point cloud, the centroid coordinate set center_set is obtained. The centroid coordinates can be expressed by the following formula (1): (1) Among them, (x) i y i , z i ) represents the coordinates of a point in each type of point cloud, and n represents the total number of points in each type of point cloud.

[0032] After obtaining the centroid coordinates, calculate the difference Z_dis between the maximum and minimum Z coordinates of the plane extracted in step S103. Mark the centroid coordinate points whose difference Z_dis is greater than a set threshold or whose number of points of the extracted plane is less than a set threshold, and filter them out from the centroid coordinate point set center_set.

[0033] Step S105: Determine the adhesion of the reinforcing bars and screen out the centroid coordinates of the point cloud that are adhered.

[0034] Specifically, to ensure the accuracy of the final point selection and welding, the median of the point cloud is first calculated based on the number of clusters (clusters_n). Then, the ratio of the number of points in each cluster (cloud_n) to the median (median) is calculated. Through multiple tests, a range of ratio values ​​(0.4 ≤ ratio ≤ 1.3) is obtained to determine whether the rebars are adhered. If the calculated ratio value is within this range, it indicates that the rebars in this cluster of point clouds are not adhered; otherwise, the rebars in this cluster of point clouds are adhered. Figure 3 As shown in the rectangle. The centroid coordinates of this type of point cloud are filtered out from the centroid coordinate set center_set.

[0035] Here's an example of calculating the median: For the data set 23, 29, 20, 32, 23, 21, 33, 25, the median is calculated as follows: First, sort the data in ascending order. Since this set consists of 8 data points (i.e., n is even), the median is the average of the 2nd and (2+1)th numbers. Therefore, the median for this set is 24. When the number of data points is odd, the median is the (n+1) / 2th number.

[0036] Step S106: Perform collision avoidance detection and screen out the centroid coordinates of a type of point cloud that has a collision risk.

[0037] (1) Collision detection with welding equipment like Figure 5 As shown, since the welding device in the system has certain dimensions (W x H), in order to avoid collisions between the selected welding points and the area of ​​the welding device, it is necessary to pre-determine and filter out coordinate points where collisions may occur. The specific method is as follows: First, sort the coordinates of the remaining centroid coordinates after the rebar adhesion judgment in step S105 by descending order of Z coordinate. Take any centroid coordinate (xa, ya, za) and iterate through all the remaining centroid coordinates after the rebar adhesion judgment in step S105. Store all centroid coordinates within the area (W x H) of the welding device into the first point set local_box, which is the centroid coordinates that satisfy the following conditions: xa-W≤center_set[i].x≤xa+W (2) ya-H≤center_set[i].y≤xa+H (3) Where (xa, ya) represents the coordinates of any chosen centroid coordinate point, W and H represent the width and height of the area where the welding device is located, respectively, and center_set[i] represents any remaining centroid coordinate point i.

[0038] Then, the centroid coordinates within the first point set local_box are sorted in descending order of Z coordinate. The difference dis_z between the absolute value of the Z coordinate of each centroid coordinate and the maximum value of the Z coordinate of the centroid coordinates in the first point set local_box is calculated. Centroid coordinates with a difference dis_z greater than a set threshold are marked and removed from the centroid coordinate set center_set.

[0039] (2) Inspection to prevent sign bending To avoid the selected welding points causing the sign to bend and become easy to fall off after welding, a further anti-bending test can be performed after the anti-collision test with the welding device to screen out the coordinate points that will cause the sign to bend. The specific test method is roughly the same as the anti-collision test of the welding device, except that the size of the test area is different. At this time, the test area is the size of the sign (the specific size of the sign needs to be measured in advance). Replace W and H in formulas (2) and (3) with the width and height of the sign to select the corresponding centroid coordinate points. Update the first point set local_box with the newly selected centroid coordinate point set. Then, similarly, sort the centroid coordinate points in the first point set local_box in descending order of Z coordinate. Calculate the difference dis_z between the absolute value of the Z coordinate of each centroid coordinate point and the maximum value of the Z coordinate of the centroid coordinate points in the first point set local_box. Mark the centroid coordinate points with a difference dis_z greater than the set threshold and remove them from the centroid coordinate point set center_set.

[0040] The above-mentioned anti-bending sign inspection is an optional inspection step, not a necessary one.

[0041] Step S107: Determine the welding area by avoiding the edge of the steel bundle, and screen out the centroid coordinate points that the robot cannot reach within the welding area to obtain the label welding points that meet the requirements.

[0042] Specifically, after step S106, which involves anti-collision detection and filtering out centroid coordinate points from the centroid coordinate point set center_set that collide with the welding device or cause the sign to bend, to prevent the selected welding point from being located at the edge of the steel bundle, which could cause the sign to easily fall off, the following settings are configured: Figure 6The rectangular box in the diagram represents the welding area, where Xmin and Xmax are the minimum and maximum values ​​of the steel bundle in the X direction, Ymin and Ymax are the minimum and maximum values ​​of the steel bundle in the Y direction, dis_x1 and dis_x2 are the deviations made by the combined welding area and the area inaccessible to the robot in the X direction of the steel bundle, and dis_y1 and dis_y2 are the deviations made by the combined welding area and the area inaccessible to the robot in the Y direction of the steel bundle.

[0043] Specifically, the remaining centroid coordinates after collision avoidance detection are traversed, and centroid coordinates that meet the following requirements are selected: Xmin+dis_x1≤center_set[i].x≤Xmax-dis_x2 (4) Ymin+dis_y1≤center_set[i].y≤Ymax-dis_y2 (5) Where center_set[i].x and center_set[i].y represent the x and y coordinates of any remaining centroid point i, respectively.

[0044] The centroid coordinates selected by formulas (4) and (5) are the weldable points; Then, based on the number of signs to be welded and the selection rules, select the appropriate welding points for the signs.

[0045] Specifically, if a sign needs to be welded, then in the middle area of ​​the steel bundle end face (e.g. Figure 7 Four optimal welding points are selected from the area shown in the rectangular box as the final welding points for the sign; if two signs need to be welded, then welding points are selected from the left and right areas of the steel bundle end face (e.g., ...). Figure 8 Two optimal welding points are selected from each of the two rectangular areas shown in the diagram as the final welding points for the signage. The size of the selected areas can be determined empirically. When selecting optimal welding points within the corresponding areas, the corresponding number of welding points closest to the welding device are selected based on the Z-coordinate of the coordinate points within the area. These welding points are the most prominent points on the end face of the steel bundle, which is beneficial for the welding device to perform welding.

[0046] The aforementioned Xmin, Xmax, Ymin, and Ymax are all coordinate values ​​in the robot coordinate system. Therefore, step S107 further includes the step of converting Xmin, Xmax, Ymin, and Ymax from the visual coordinate system to the robot coordinate system. This embodiment uses a five-point calibration method to achieve the coordinate system conversion: (1) Preparation: a. Create the robot coordinate system. After fixing the provided calibration plate and confirming the zero point position of the robot on the calibration plate, the coordinate system can be created with the horizontal direction as the X coordinate and the vertical direction as the Y coordinate, without having to consider the orientation.

[0047] b. Obtain the coordinates of five points in the robot coordinate system. After creating the robot coordinate system, take five points: top, bottom, left, right, and center point on the calibration plate. Manually move between these points to obtain their coordinates in the robot coordinate system, and then save these five coordinates in sequence into the robot.txt file.

[0048] c. Obtain the coordinates of the five corresponding points in the visual coordinate system. Open the 3D vision sensor visualization software, obtain the point cloud image of the calibration board by adjusting the parameters, then open the obtained point cloud image in the calibration software, and use the mouse to pick the coordinates of the five corresponding points in the visual coordinate system in sequence, and save them into the vision.txt file.

[0049] d. Obtain the direction coefficients. Based on the pre-saved robot.txt and vision.txt files and the calibration software, check whether the directions are consistent by comparing them one by one in the X, Y and Z directions. If the directions are consistent, the coefficient is 1; otherwise, the coefficient is -1. The obtained direction coefficients in the three directions are then stored in the dic.txt file.

[0050] (2) Coordinate system transformation: This involves calculating the transformation relationship between two coordinate systems. Since both the visual coordinate system and the robot coordinate system are spatial coordinate systems, it can be transformed into solving for the rotation and translation matrix (rigid transformation or Euclidean transformation) between the two point clouds. Transforming the source point cloud (visual) to the target point cloud (robot) coordinate system can be expressed as the following equation: P t =R·p s +T Among them, P t p s Let R be a pair of corresponding points in the target point cloud and the source point cloud, where R is the rotation matrix and T is the translation matrix.

[0051] Furthermore, the rotation matrix R and translation matrix T can be obtained by solving the 5 pairs of corresponding points in the pre-saved robot.txt and vision.txt files. Solving the rotation and translation matrix between two point clouds is the point cloud registration process. Point cloud registration is divided into coarse registration and static registration. Coarse registration is to find an approximate rotation and translation matrix between two point clouds when the relative positional relationship between the two point clouds is completely unknown. Fine registration is to calculate a more accurate rotation and translation matrix when the initial value of a rotation and translation matrix is ​​known (the initial value is probably already correct).

[0052] Therefore, combining the definitions of fine registration and coarse registration, this embodiment uses coarse registration to solve for the rotation and translation matrix. In coarse registration of point clouds, the RANSAC algorithm can be used to find the optimal transformation matrix to align the two point cloud datasets. The specific solution method is as follows: 1) Randomly selecting a certain number of point pairs is called a sample point pair; 2) Calculate the initial transformation matrix based on the sample point pairs; 3) Calculate the distance between all other point pairs and the transformed point pair, and obtain the set of interior points; 4) Estimate the optimal transformation matrix on the set of interior points; 5) Transform the entire point cloud according to the optimal transformation matrix; 6) Repeat the above steps multiple times and select the transformation matrix with the most interior points as the final result.

[0053] Step S108: Send the position coordinates of the compliant label welding point obtained in step S107 in the robot coordinate system to the robot, and the robot drives the pre-picked welding nails and label to perform label welding at the coordinate points.

[0054] Specifically, if a sign needs to be welded, the robot selects the coordinate point closest to the welding device from the four optimal welding points for welding. If two signs need to be welded, the robot selects the coordinate point closest to the welding device from the two optimal welding points in the left and right areas for welding.

[0055] After completing the sign welding, the robot returns to a preset safe position and triggers the 3D vision sensor to take another picture. Based on the number of signs to be welded and the acquired point cloud image, the robot determines whether the welding was successful. If only one sign needs welding and the point cloud image shows one sign, the welding is successful, and the robot continues to wait for the next steel bundle to arrive; otherwise, it performs a re-welding operation. If two signs need welding and the point cloud image shows both signs, the welding is successful, and the robot continues to wait for the next steel bundle to arrive. If two signs need welding and the point cloud image shows one sign, it needs to determine which side failed to weld and performs a re-welding operation on the failed side. After the re-welding is completed, the 3D vision sensor is triggered again to take another picture to determine whether the welding was successful. During re-welding, the robot selects the coordinate point closest to the welding device from the remaining optimal welding points.

[0056] Example 2

[0057] This embodiment provides an automatic welding sign device based on three-dimensional point clouds, applied to the aforementioned automatic welding sign method, as follows: Figure 9 As shown, it includes: The acquisition module 100 is used to acquire a three-dimensional point cloud map of the end face of the steel bundle; Preprocessing module 110 is used to preprocess the point cloud, including downsampling, noise removal and clustering segmentation; The plane extraction module 120 is used to extract the plane from each type of point cloud obtained by clustering and segmentation, and remove the point cloud trailing. The point cloud feature labeling module 130 is used to calculate the centroid coordinates of each type of point cloud, obtain the centroid coordinate point set, and filter out the centroid coordinate points of a type of point cloud whose difference between the maximum and minimum Z coordinates does not meet the requirements or whose number of points extracted from the plane does not meet the requirements. The rebar adhesion judgment module 140 is used to filter out the centroid coordinate points of a type of point cloud that are adhered. The collision avoidance detection module 150 is used to filter out centroid coordinate points of a type of point cloud that pose a collision risk; The label welding point location calculation module 160 is used to determine the welding area by avoiding the edge of the steel bundle, and to screen out the centroid coordinate points that the robot cannot reach within the welding area to obtain the label welding points that meet the requirements. The label welding point location sending module 170 is used to send the position coordinates of the label welding points that meet the requirements in the robot coordinate system to the robot for label welding.

[0058] The label welding point position calculation module 160 includes a calibration algorithm module, which uses a five-point calibration method to convert the visual coordinate system to the robot coordinate system.

[0059] Example 3

[0060] This embodiment provides an automatic welding method for signs based on three-dimensional point clouds. This embodiment requires welding one sign, such as... Figure 10 As shown, it includes the following steps: (3.1) Determine whether the steel bundle has reached the welding area; (3.2) When the steel bundle arrives at the welding area, the three-dimensional vision sensor is triggered to take a picture and collect a three-dimensional point cloud image of the end face of the steel bundle; (3.3) The point cloud collected is identified and located by the vision processing system to obtain the location of the welding point of the sign that meets the requirements, and its three-dimensional coordinates are sent to the robot. The robot drives the pre-picked welding nails and signs to perform sign welding at the coordinate points. (3.4) After the robot completes the sign welding action, it returns to the preset safe position and triggers the three-dimensional vision sensor to take pictures again; (3.5) Based on the point cloud obtained, determine whether the sign has been successfully welded using a vision processing system; (3.6) If the sign is successfully welded, the welding process ends and the next steel bundle is not yet in place. (3.7) If the welding of the sign fails, a repair welding operation shall be performed.

[0061] In step (3.3) above, the collected point cloud is identified and located using a vision processing system to obtain the required welding point positions for the sign, including the following steps: (3.3.1) Preprocess the point cloud, including downsampling, noisy point processing and clustering segmentation; (3.3.2) Perform planar extraction on each type of point cloud obtained from clustering and segmentation, and remove the point cloud tail; (3.3.3) Calculate the centroid coordinates of each type of point cloud to obtain the centroid coordinate point set. Filter out centroid coordinate points whose difference between the maximum and minimum Z coordinates does not meet the requirements or whose number of points extracted from the plane does not meet the requirements. (3.3.4) Determine the adhesion of the reinforcing bars and screen out the centroid coordinates of the point cloud that are adhered; (3.3.5) Perform collision avoidance detection to screen out the centroid coordinates of a type of point cloud that has a collision risk; (3.3.6) Determine the welding area by avoiding the edge of the steel bundle, and screen out the centroid coordinate points that the robot cannot reach within the welding area to obtain the label welding points that meet the requirements.

[0062] In this embodiment, a sign needs to be welded. Therefore, in step (3.3.6) above, after filtering out the centroid coordinate points that the robot cannot reach within the welding area, four optimal sign welding points are selected in the middle area of ​​the steel bundle end face as the final sign welding points sent to the robot.

[0063] The above steps (3.3.1) to (3.3.6) can be implemented using the specific methods in steps S102 to S07 of Example 1, which will not be repeated here.

[0064] Example 4 This embodiment provides an automatic welding method for signs based on three-dimensional point clouds. This embodiment requires welding two signs, such as... Figure 11 As shown, it includes the following steps: (4.1) Determine whether the steel bundle has reached the welding area; (4.2) When the steel bundle arrives at the welding area, the three-dimensional vision sensor is triggered to take a picture and collect a three-dimensional point cloud image of the end face of the steel bundle; (4.3) The point cloud collected is identified and located by the vision processing system to obtain the location of the welding point of the sign that meets the requirements, and its three-dimensional coordinates are sent to the robot. The robot drives the pre-picked welding nails and signs to perform sign welding at the coordinate points. (4.4) After the robot completes the sign welding action, the robot returns to the preset safe position and triggers the three-dimensional vision sensor to take pictures again; (4.5) Based on the point cloud obtained, determine whether the sign has been successfully welded using a vision processing system; (4.6) If the sign is successfully welded, the welding process ends and the process continues to wait for the next steel bundle to arrive; (4.7) If the welding of the sign fails, a repair welding operation shall be performed.

[0065] In step (4.3) above, the collected point cloud is identified and located using a vision processing system to obtain the location of the welding points of the sign that meet the requirements. This includes the following steps: (4.3.1) Preprocess the point cloud, including downsampling, noisy point processing and clustering segmentation; (4.3.2) Perform planar extraction on each type of point cloud obtained from clustering and segmentation, and remove the point cloud tail; (4.3.3) Calculate the centroid coordinates of each type of point cloud to obtain the centroid coordinate point set. Filter out the centroid coordinate points of a type of point cloud whose difference between the maximum and minimum Z coordinates does not meet the requirements or whose number of points extracted from the plane does not meet the requirements. (4.3.4) Determine the adhesion of the reinforcing bars and screen out the centroid coordinates of the point cloud that are adhered; (4.3.5) Perform collision avoidance detection to screen out the centroid coordinates of a type of point cloud that has a collision risk; (4.3.6) Determine the welding area by avoiding the edge of the steel bundle, screen out the centroid coordinates of the welding area that the robot cannot reach, and obtain the position coordinates of the label welding point in the robot coordinate system.

[0066] In this embodiment, two labels need to be welded. Therefore, in step (4.3.6) above, after filtering out the centroid coordinate points that the robot cannot reach in the welding area, two optimal label welding points are selected in the left and right areas of the steel bundle end face as the final label welding points sent to the robot.

[0067] The above steps (4.3.1) to (4.3.6) can be implemented using the specific methods in steps S102 to S07 of Example 1, which will not be repeated here.

Claims

1. An automatic welding method for nameplates based on three-dimensional point clouds, characterized in that, include: When the steel bundle arrives at the welding area, the 3D vision sensor is triggered to take a picture and collect a 3D point cloud map of the end face of the steel bundle. Point cloud preprocessing includes downsampling, noise removal, and clustering segmentation; For each type of point cloud obtained from clustering and segmentation, perform planar extraction and remove point cloud trailing; Calculate the centroid coordinates of each type of point cloud to obtain a set of centroid coordinate points. Filter out centroid coordinate points whose difference between the maximum and minimum Z coordinates does not meet the requirements or whose number of points extracted from the plane does not meet the requirements. To determine the adhesion of reinforcing bars, the centroid coordinates of a certain type of point cloud that are adhered are screened out; Collision avoidance detection is performed to screen out centroid coordinates of point clouds that pose a collision risk; The welding area is determined by avoiding the edge of the steel bundle, and the centroid coordinate points that the robot cannot reach within the welding area are screened out to obtain the label welding points that meet the requirements. Send the position coordinates of the compliant label welding points in the robot coordinate system to the robot for label welding; After the robot finishes welding the sign, it returns to the preset safety position, triggers the 3D vision sensor to take pictures again, and collects the 3D point cloud map of the steel bundle end face again. The success of the signboard welding is determined by the three-dimensional point cloud image of the steel bundle end face collected again. If the sign is successfully welded, the welding process ends and we continue to wait for the next steel bundle to arrive. If the welding of the sign fails, a repair welding operation will be performed.

2. The automatic welding sign method based on three-dimensional point clouds as described in claim 1, characterized in that, For each type of point cloud obtained from clustering and segmentation, a plane is extracted and the point cloud tail is removed, including fitting the plane using the RANSAC algorithm.

3. The automatic welding sign method based on three-dimensional point clouds as described in claim 1, characterized in that, To determine the adhesion of reinforcing bars, centroid coordinates of point clouds with adhesion are screened out. This includes: calculating the median number of point clouds based on the number of clusters, calculating the ratio of the number of point clouds in each category corresponding to the remaining centroid coordinates to the median, and screening out centroid coordinates of point clouds whose ratios are not within the normal range.

4. The automatic welding sign method based on three-dimensional point clouds as described in claim 1, characterized in that, Collision avoidance detection is performed to screen out centroid coordinate points of the point cloud that pose a collision risk, including collision avoidance detection with welding equipment: The remaining centroid coordinates after the steel reinforcement adhesion judgment are arranged in descending order of Z coordinate. Take any centroid coordinate point, traverse all remaining centroid coordinate points after the steel bar adhesion judgment, and store the centroid coordinate points that satisfy the following conditions into the first point set. xa-W≤center_set[i].x≤xa+W ya-H≤center_set[i].y≤ya+H Where (xa, ya) represents the coordinates of any chosen centroid point, W and H represent the width and height of the area where the welding device is located, respectively, and center_set[i].x and center_set[i].y represent the x and y coordinates of any remaining centroid point i, respectively; Arrange the centroid coordinates in the first point set in descending order of Z coordinate. Calculate the absolute value of the difference between the Z coordinate of each centroid coordinate point and the maximum value of the Z coordinate in the first point set, and filter out centroid coordinates that do not meet the difference requirements.

5. The automatic welding sign method based on three-dimensional point clouds as described in claim 1, characterized in that, Collision avoidance detection is performed to filter out centroid coordinates of point clouds that pose a collision risk, including detection to prevent sign bending. The remaining centroid coordinates after the steel reinforcement adhesion judgment are arranged in descending order of Z coordinate. Take any centroid coordinate point, traverse all remaining centroid coordinate points after the steel bar adhesion judgment, and store the centroid coordinate points that satisfy the following conditions into the first point set. xa-W≤center_set[i].x≤xa+W ya-H≤center_set[i].y≤ya+H Where (xa, ya) represents the coordinates of any chosen centroid point, W and H represent the width and height of the sign, respectively, and center_set[i].x and center_set[i].y represent the x and y coordinates of any remaining centroid point i, respectively; Arrange the centroid coordinates in the first point set in descending order of Z coordinate. Calculate the absolute value of the difference between the Z coordinate of each centroid coordinate point and the maximum value of the Z coordinate in the first point set, and filter out centroid coordinates that do not meet the difference requirements.

6. The automatic welding sign method based on three-dimensional point clouds as described in claim 1, characterized in that, The welding area is determined by avoiding the edge of the steel bundle. Centroid coordinates that are inaccessible to the robot within the welding area are eliminated to obtain the required label welding points, including: The remaining centroid coordinates after collision avoidance detection are traversed, and centroid coordinates that meet the following requirements are selected: Xmin+dis_x1≤center_set[i].x≤Xmax-dis_x2 Ymin+dis_y1≤center_set[i].y≤Ymax-dis_y2 Where Xmin, Xmax, Ymin, and Ymax are coordinate values ​​in the robot coordinate system, Xmin and Xmax are the minimum and maximum values ​​of the steel bundle in the X direction, Ymin and Ymax are the minimum and maximum values ​​of the steel bundle in the Y direction, dis_x1 and dis_x2 are the deviations that need to be made in the X direction of the steel bundle for the integrated welding area and the area that the robot cannot reach, and dis_y1 and dis_y2 are the deviations that need to be made in the Y direction of the steel bundle for the integrated welding area and the area that the robot cannot reach; center_set[i].x and center_set[i].y represent the x and y coordinates of any remaining centroid coordinate point i, respectively. The selected centroid coordinates are the weldable points; Based on the number of signs to be welded and the selection rules, select the appropriate welding points for the signs. If one sign needs to be welded, select four welding points in the middle area of ​​the steel bundle end face. If two signs need to be welded, select two welding points each in the left and right areas of the steel bundle end face.

7. An automatic welding signage device based on three-dimensional point clouds, used to implement the automatic welding signage method based on three-dimensional point clouds as described in any one of claims 1-6, characterized in that, include: The acquisition module is used to acquire a 3D point cloud image of the end face of the steel bundle; The preprocessing module performs preprocessing on the point cloud, including downsampling, noise removal, and clustering segmentation. The plane extraction module is used to extract planes from each type of point cloud obtained by clustering and segmentation, and remove point cloud trailing. The point cloud feature labeling module calculates the centroid coordinates of each type of point cloud, obtains a set of centroid coordinate points, and filters out centroid coordinate points of a type of point cloud whose difference between the maximum and minimum Z coordinates does not meet the requirements or whose number of extracted plane points does not meet the requirements. The rebar adhesion judgment module is used to filter out centroid coordinate points of a type of point cloud that are adhered; The collision avoidance detection module is used to filter out centroid coordinates of point clouds that pose a collision risk. The label welding point location calculation module determines the welding area by avoiding the edge of the steel bundle, and filters out the centroid coordinate points in the welding area that the robot cannot reach, thus obtaining the label welding points that meet the requirements. The label welding point location sending module sends the position coordinates of the qualified label welding points in the robot coordinate system to the robot for label welding.

8. The automatic welding sign device based on three-dimensional point clouds as described in claim 7, characterized in that, The label welding point position calculation module includes a calibration algorithm module, which uses a five-point calibration method to convert the visual coordinate system to the robot coordinate system.

Images