In-situ remanufacturing repair system and method based on three-dimensional point cloud

Abstract

The invention discloses an in-situ remanufacturing repair system and method based on three-dimensional point cloud.The in-situ remanufacturing repair system comprises an industrial personal computer, a binocular structure light camera and a collaborative robot, the binocular structure light camera and the collaborative robot are connected with the industrial personal computer, and the collaborative robot is provided with a welding gun connected with an intelligent welding machine; the method comprises the following steps of: performing hand-eye calibration on a binocular structured light camera by adopting dual quaternion and a singular value decomposition algorithm; a binocular structured light camera is used for shooting the workpiece, and a complete point cloud of the to-be-repaired area is obtained; performing surface reconstruction on the to-be-repaired area by using a Poisson surface reconstruction algorithm; according to the side wall fusion optimization formula, path planning is conducted on the to-be-repaired area model in Cura software, and filling path coordinates of the to-be-repaired area are obtained. According to the method provided by the invention, the defect of incomplete fusion of the side wall of the workpiece after additive remanufacturing repair can be reduced, and meanwhile, the surface flatness of the weld joint adjacent to the side wall is improved.

Description

Background Art

[0002] In the current era background that emphasizes low-carbon development, repairing parts through additive remanufacturing technology can bring huge economic and environmental benefits. The robotic arc additive remanufacturing technology is an important development direction and frontier direction among them. As the remanufacturing repair method with the highest forming efficiency at present, it can controllably stack materials in a specified area to achieve in-situ repair of on-demand forming, and has great potential application value.

[0003] Traditional industrial robots are mainly used for production and manufacturing. In order to have a larger working range, they are usually designed to be relatively large and bulky, and can only be fixedly deployed in a certain place, making it difficult to achieve rapid transfer and transportation. The main goal of in-situ repair is to achieve repair at the service site of the workpiece. Therefore, there will be certain difficulties in using traditional robots for on-site in-situ repair.

[0004] In addition, when using the arc additive remanufacturing technology to repair damaged workpieces, if the path planning method of additive manufacturing is used to plan the path for the repair area, it is easy to cause welding defects such as lack of fusion on the side walls of the parts in the area to be repaired. Summary of the Invention

[0005] The purpose of the present invention is to provide a three-dimensional point cloud-based in-situ remanufacturing repair system and method, which can reduce the lack of fusion welding defects on the side walls of the workpiece after additive remanufacturing repair, and at the same time improve the surface flatness of the weld at the adjacent side walls.

[0006] The present invention provides a three-dimensional point cloud-based in-situ remanufacturing repair system, including: an industrial control computer, a binocular structured light camera and a collaborative robot respectively connected to the industrial control computer; The industrial control computer is used to process point cloud information, reconstruct the three-dimensional model of the area to be repaired of the target workpiece, perform path planning on the reconstructed model, and extract the filling route coordinates after path planning; The collaborative robot is equipped with a welding torch connected to an intelligent welding machine, and is used to perform arc additive repair operations with the welding torch under the control of the industrial control computer; The intelligent welding machine is connected to the welding torch and is used to control the operation of the welding torch and real-time adjust and stabilize the parameters of the welding process; The binocular structured light camera is assembled on the welding torch and is used to obtain the point cloud information of the workpiece surface and transmit the data to the industrial control computer through communication with the upper computer; The robot control cabinet is used to control the collaborative robot equipped with the welding torch to complete the welding operation under the control of the industrial control computer.

[0007] Preferably, the binocular structured light camera is fixed at the target position on the welding torch through the camera bracket, so as to ensure that within the effective shooting distance of the binocular structured light camera, the point cloud of the object surface can be obtained on the premise that the welding torch does not touch external objects.

[0008] The present invention also provides an in-situ remanufacturing and repair method based on three-dimensional point cloud, which is applied to the in-situ remanufacturing and repair system based on three-dimensional point cloud as described in the embodiments of the present invention, and includes: Performing hand-eye calibration on the binocular structured light camera based on the dual quaternion and singular value decomposition algorithm to obtain the pose of the binocular structured light camera in the base coordinate system of the collaborative robot; Taking pictures through the binocular structured light camera to obtain the point cloud data of the surface of the machined target workpiece and performing preprocessing to obtain the complete point cloud data of the area to be repaired after preprocessing; Performing surface reconstruction on the area to be repaired based on the Poisson surface reconstruction algorithm to obtain the surface model of the area to be repaired; Combining the side wall fusion optimization formula, loading the surface model of the area to be repaired and performing path planning on the model of the area to be repaired to obtain the filling repair route coordinates of the area to be repaired.

[0009] Preferably, the target workpiece is a metal workpiece with a target-shaped area to be repaired, the target shape is a regular cylinder, and the bottom surface of the regular cylinder is a quadrilateral with two parallel sides; the area where the regular cylinder is located completely covers the area to be repaired, and only one surface of the regular cylinder does not coincide with the corresponding part of the target workpiece, and the other five surfaces coincide with the corresponding part of the target workpiece.

[0010] Preferably, it further includes: using a registration algorithm to convert the filling repair route coordinates of the area to be repaired generated by Cura software into the camera coordinate system of the binocular structure camera; and converting the repair path coordinates into the TCP coordinate system of the collaborative robot through the hand-eye matrix obtained by hand-eye calibration; Among them, the registration algorithm includes a coarse registration algorithm and a fine registration algorithm. The coarse registration algorithm uses the Super4PCS algorithm. After obtaining the point cloud of the area to be repaired and selecting similar feature points, calculating the transformation matrix between the similar feature points, transforming one point cloud into the coordinate system of another point cloud, and obtaining the optimized registration result through iterative calculation; the fine registration algorithm uses the NICP algorithm, adding constraints on the normal vector and the curvature of the surface where the point cloud is located on the basis of the ICP algorithm, and optimizing the registration result by iteratively finding the nearest point pairs and calculating the transformation matrix.

[0011] Preferably, the step of taking pictures through the binocular structured light camera to obtain the point cloud data of the surface of the machined target workpiece and performing preprocessing to obtain the complete point cloud data of the area to be repaired after preprocessing includes: Preprocess the damaged workpiece, where the preprocessing is to completely remove the damaged area on the damaged workpiece by machining to obtain the processed target workpiece; Place the target workpiece on the welding workbench, and capture all surface information of the workpiece from different angles through a binocular structured light camera. The surface information includes all point clouds of the morphology of the area to be repaired; Based on the RANSAC algorithm, fit the plane equation of the workbench surface for the collected point clouds. Taking the workbench plane equation as a reference, calculate the distances from all points in the collected point clouds to the workbench plane. Remove the point clouds located on and below the workbench surface according to a preset distance threshold, retain the point clouds with the target workpiece as the main body, and perform filtering processing on the point clouds based on a preset filtering algorithm to remove noise points, obtaining the target workpiece point clouds without noise points; Identify and perform the first segmentation on the target workpiece point clouds, dividing them into upper surface point clouds and point clouds other than the upper surface point clouds. Use the RANSAC algorithm to fit the plane equation of the workpiece upper surface and retain the point clouds of the workpiece upper surface; Perform defect identification on the area to be repaired on the upper surface of the target workpiece. According to the plane equation of the upper surface of the target workpiece and the coordinate range of the points in the point clouds, fill points in the point clouds on the upper surface of the target workpiece. If no point clouds on the upper surface of the target workpiece are found around the filled points, the filled points are represented as the points at the defects on the upper surface of the target workpiece and written into a new point cloud, denoted as the point clouds of the defects on the upper surface of the target workpiece; Perform the second segmentation on the workpiece point clouds according to the plane equation of the upper surface of the target workpiece to obtain the point clouds below the upper surface of the target workpiece. Merge the point clouds of the defects on the upper surface of the target workpiece and the point clouds below the upper surface of the target workpiece to obtain the complete point clouds of the area to be repaired.

[0012] Preferably, when performing path planning on the area to be repaired of the target workpiece with a cylindrical target shape, the center line of the weld bead adjacent to the cylinder maintains a preset distance d from the adjacent and parallel side surface of the cylinder offset , that is, the sidewall fusion optimization distance is as follows: d offset =1 / 3*w bead , where the sidewall refers to the part of the four side surfaces of the cylinder in the area to be repaired that coincides with the target workpiece, and w bead represents the width of a single-layer single-pass weld bead, and the lengths of the two parallel sides and the distance between the two parallel sides of the bottom surface of the cylinder in the area to be repaired of the target workpiece are both greater than 2d offset to meet the distance conditions required for the normal application of the algorithm.

[0013] Preferably, according to the parabolic fitting of the weld shape and the principle of equal area, determining the critical distance, that is, the sidewall fusion optimization distance, further includes: If using d overlapDenote the center-to-center distance of bead lap joints as \(k\), and the ratio of the center-to-center distance of bead lap joints to the width of a single-layer single-pass bead as \(k\), then \(d\) overlap =k*w bead , where \(w\) bead is the width of a single-layer single-pass bead; Denote the total number of beads between two parallel sides of the quadrilateral at the bottom of the target-shaped cylinder as \(n\), and the distance between the two parallel sides of the quadrilateral at the bottom as \(d\). The sidewall fusion optimization formula is expressed as \(d = 2*d\) offset +(n - 1)*d overlap , where \(d\) offset is the sidewall fusion optimization distance.

[0014] The present invention also provides an electronic device, including: A memory for storing a processing program; A processor that, when executing the processing program, implements the in-situ remanufacturing repair method based on 3D point cloud as described in the embodiments of the present invention.

[0015] The present invention also provides a readable storage medium, on which a processing program is stored. When the processing program is executed by a processor, it implements the in-situ remanufacturing repair method based on 3D point cloud as described in the embodiments of the present invention.

[0016] In view of the prior art, the present invention has the following beneficial effects: The present invention proposes an in-situ remanufacturing repair system and method based on 3D point cloud. By using a collaborative robot equipped with a welding torch to perform repairs, it can be quickly and conveniently deployed to the working site to achieve high-efficiency in-situ repair.

[0017] The present invention uses a binocular structured light camera to obtain point clouds, which can quickly obtain the surface topography of the damaged part of the workpiece, and its efficiency is higher than that of laser scanning equipment.

[0018] When planning the path, the present invention combines the sidewall fusion optimization formula, which can reduce the unfused weld defects on the sidewalls of the workpiece after additive remanufacturing repair, and at the same time improve the surface flatness of the welds at the adjacent sidewalls.

[0019] When setting parameters in the Cura software, the present invention sets parameters according to the specific-shaped cylinder and the sidewall fusion optimization formula, which can achieve integer-pass weld forming repair while ensuring the center-to-center distance of bead lap joints remains unchanged, reducing the consumption of repair materials. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic structural diagram of the in-situ remanufacturing repair system based on 3D point cloud described in the embodiments of the present invention; Figure 2 It is a schematic flowchart of the in-situ remanufacturing repair method based on 3D point cloud in the embodiments of the present invention; Figure 3 Schematic diagram of the principle of optimized sidewall fusion distance in the embodiments of the present invention; Figure 4 Schematic diagram of optional shapes of the area to be repaired after machining a damaged workpiece in the embodiments of the present invention; Figure 5 Schematic diagram of simulation after the Cura software performs path planning on the area to be repaired in the embodiments of the present invention. Detailed implementation manners

[0021] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0022] As used herein, the term "including" and its variants are open-ended, that is, "including but not limited to". The term "based on" means "at least partially based on". The term "an embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0023] It should be noted that the concepts such as "first", "second", etc. mentioned in the disclosure of this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence relationship of the functions performed by these devices, modules or units.

[0024] It should be noted that the modifications of "one" and "plural" mentioned in the disclosure of this application are illustrative rather than restrictive. Those skilled in the art should understand that unless clearly specified otherwise in the context, it should be understood as "one or more".

[0025] The present invention provides an in-situ remanufacturing repair system based on 3D point cloud, mainly including: an industrial control computer, a collaborative robot, a robot control cabinet, a welding torch, an intelligent welding machine, a binocular structured light camera, and a camera support. Among them, the industrial control computer is used to process point cloud information, reconstruct the 3D model of the area to be repaired of the target workpiece, perform path planning on the reconstructed model, and extract the filling route coordinates after path planning; the collaborative robot is equipped with a welding torch connected to the intelligent welding machine, and is used to perform arc additive repair operations by lapping the welding torch under the control of the industrial control computer. The collaborative robot is the execution mechanism of the system and performs the arc additive repair function by lapping the welding torch; the intelligent welding machine is connected to the welding torch and is used to control the operation of the welding torch, regulate and stabilize various parameters of the welding process in real time, and also provide power supply for the welding torch to ensure the forming quality of the additive remanufacturing repair part; the robot control cabinet is used to control the collaborative robot equipped with the welding torch to complete the welding operation under the control of the industrial control computer. The robot control cabinet supplies power to the collaborative robot and controls the movement process of the collaborative robot; the binocular structured light camera is assembled on the welding torch and is used to obtain the point cloud information of the workpiece surface, and transmits the data to the industrial control computer through communication with the upper computer. The camera support fixes the binocular structured light camera at a suitable position on the welding torch to obtain high-quality point cloud.

[0026] In an embodiment, the binocular structured light camera is fixed at a target position on the welding torch through the camera support, so as to ensure that the point cloud of the object surface can be obtained within the effective shooting distance of the binocular structured light camera on the premise that the welding torch does not touch external objects. Exemplarily, the target position is the side of the welding torch. The binocular structured light camera is fixed on the side of the welding torch through the camera support, so that when the binocular structured light camera shoots the point cloud of the workpiece, it is blocked by the welding torch as little as possible, and the workpiece is within the effective shooting area of the binocular structured light camera.

[0027] Exemplarily, the hand-eye matrix calculation of the eye-in-hand for the binocular structured light camera and the collaborative robot is performed using dual quaternions and the singular value decomposition (SVD) algorithm. After the hand-eye calculation is completed, the pose of the binocular structured light camera in the base coordinate system of the collaborative robot is obtained.

[0028] See Figure 1The following is a schematic structural diagram of an in-situ remanufacturing repair system based on 3D point cloud provided by an embodiment of the present invention. The system in the embodiments of the present application may include: an industrial personal computer, a collaborative robot, a robot control cabinet, a welding torch, an intelligent welding machine, a binocular structured light camera, and a camera bracket; the binocular structured light camera is fixed to the side of the welding torch through the camera bracket; the binocular structured light camera is communicatively connected to the industrial personal computer through a USB interface; the robot control cabinet is communicatively connected to the industrial personal computer through an Ethernet interface; the robot control cabinet is used to control the welding robot to complete the welding operation under the control of the industrial personal computer; wherein, the industrial personal computer is used to: process point cloud information, reconstruct a 3D model of the area to be repaired; perform path planning on the reconstructed model, and extract the filling route coordinates after path planning.

[0029] See Figure 2 As shown, the following is a schematic flowchart of the in-situ remanufacturing repair method based on 3D point cloud provided by an embodiment of the present invention. The process of this method is introduced as follows: Step 201: Perform hand-eye calibration on the binocular structured light camera based on dual quaternions and the singular value decomposition algorithm to obtain the pose of the binocular structured light camera in the base coordinate system of the collaborative robot; in the embodiments of the present application, the hand-eye matrix of the binocular structured light camera and the collaborative robot is calculated using the dual quaternion and the singular value decomposition (SVD) algorithm. After the hand-eye calculation is completed, the pose of the binocular structured light camera in the base coordinate system of the collaborative robot is obtained.

[0030] Step 202: Capture the point cloud data of the surface of the machined target workpiece through the binocular structured light camera and perform preprocessing to obtain the complete point cloud data of the area to be repaired after preprocessing; the target workpiece is a metal workpiece with a target-shaped area to be repaired, the target shape is a regular cylinder, and the bottom surface of the regular cylinder is a quadrilateral with two parallel sides; the area where the regular cylinder is located completely covers the area to be repaired, and only one surface of the regular cylinder does not coincide with the corresponding part of the target workpiece, and the other five surfaces coincide with the corresponding part of the target workpiece.

[0031] The step of capturing the point cloud data of the surface of the machined target workpiece through the binocular structured light camera and performing preprocessing includes: Perform preprocessing on the damaged workpiece. The preprocessing is to completely remove the damaged area on the damaged workpiece through machining to obtain the machined target workpiece; this step performs machining to remove the damage: before repairing the workpiece, it is necessary to first completely remove the damaged part through machining. The area to be repaired after machining should be a regular cylinder with a specific shape, and its bottom surface can be a parallelogram or a trapezoid, and the height should exceed the maximum depth of the damaged area to ensure that the entire damaged part is covered. The regular cylinder requires that only one surface does not coincide with the corresponding part of the workpiece, and the other five surfaces coincide with the corresponding part of the workpiece.

[0032] Place the target workpiece on the welding workbench, and capture all surface information of the workpiece from different angles through a binocular structured light camera. The surface information includes all point clouds of the topography of the area to be repaired. This step involves placing the workpiece and capturing to obtain all surface information of the workpiece located on the workbench.

[0033] Based on the RANSAC algorithm, fit the plane equation of the workbench surface for the collected point clouds. Taking the workbench plane equation as a reference, calculate the distances from all points in the collected point clouds to the workbench plane. Remove the point clouds located on and below the workbench surface according to a preset distance threshold, retain the point clouds with the target workpiece as the main body, and perform filtering processing on the point clouds based on a preset filtering algorithm to remove noise points, obtaining the target workpiece point clouds without noise points. This step performs point cloud data processing: process the collected point cloud data. First, use the RANSAC algorithm to fit the plane equation of the workbench surface, and taking this equation as a reference, set a distance threshold, calculate the distances from all collected points to the workbench plane, and remove the point clouds on and below the workbench surface according to the set distance threshold, thereby retaining the point clouds with the workpiece as the main body.

[0034] Identify and perform the first segmentation on the target workpiece point clouds, dividing them into upper surface point clouds and point clouds other than the upper surface point clouds. Use the RANSAC algorithm to fit the plane equation of the workpiece upper surface and retain the point clouds of the workpiece upper surface. Due to the existence of the area to be repaired, there will be defects in the corresponding position point clouds on the workpiece upper surface. Identify the defects in the area to be repaired on the target workpiece upper surface. According to the plane equation of the target workpiece upper surface and the coordinate range of the points in the point clouds, fill points in the target workpiece upper surface point clouds. If no target workpiece upper surface point clouds are found around the filled points, the filled points are represented as the points at the defective parts of the target workpiece upper surface and written into a new point cloud, denoted as the defective point cloud of the target workpiece upper surface. This step performs point cloud segmentation and defect identification. It can be understood that the workpiece point clouds can be divided into workpiece upper surface point clouds and point clouds below the upper surface. The workpiece upper surface point clouds can use the RANSAC algorithm to fit the plane equation of the workpiece upper surface and retain the workpiece upper surface point clouds. Due to the existence of the area to be repaired, there will be defects in the workpiece upper surface point clouds. This defective part is also the upper surface of the area to be repaired. According to the plane equation of the workpiece upper surface and the coordinate range of the points in the workpiece upper surface point clouds, fill points in the workpiece upper surface point clouds with the workpiece upper surface point cloud density as the step size. If no workpiece upper surface point clouds are found near the filled points, it is regarded as the point at the defective part of the upper surface caused by the existence of the area to be repaired and written into a new point cloud, denoted as the defective point cloud of the workpiece upper surface. Then, segment the workpiece point clouds with the plane where the workpiece upper surface is located to obtain the point clouds below the workpiece upper surface.

[0035] Perform a second segmentation on the point cloud of the workpiece with respect to the plane where the upper surface of the target workpiece is located, to obtain the point cloud below the upper surface of the target workpiece, and merge the defective point cloud on the upper surface of the target workpiece and the point cloud below the upper surface of the target workpiece, to obtain the complete point cloud of the area to be repaired. It can be understood that by merging the defective point cloud on the upper surface of the workpiece and the point cloud of the part below the upper surface of the workpiece, the point cloud of the area to be repaired can be obtained.

[0036] In the embodiment of the present application, before repairing the workpiece, it is necessary to first completely remove the damaged part by machining. The area to be repaired after machining is a regular cylinder with a specific shape, and the bottom surface of the cylinder is a quadrilateral with two parallel sides. Optionally, the shape can be a parallelogram or a trapezoid, as Figure 4 shown; the height of the cylinder exceeds the maximum depth of the damaged area, and the cylinder area should completely cover the damaged part; the regular cylinder requires that only one surface does not coincide with the corresponding part of the workpiece, and the other five surfaces coincide with the corresponding part of the workpiece. Optionally, the machining method can be milling or turning.

[0037] In the embodiment of the present application, after removing the damaged area of the workpiece by machining, the workpiece is placed on the welding workbench, and a binocular structured light camera is used to take pictures at different angles to obtain all the surface information of the workpiece located on the workbench, requiring that the point cloud information of the morphology of the area to be repaired be completely included. For the collected point cloud, the RANSAC algorithm is used to fit the plane equation of the workbench surface. Based on the workbench plane equation, a distance threshold is set, and the distances from all points in the collected point cloud to the workbench plane are calculated. According to the distance threshold, the point cloud on and below the workbench surface is removed, and the point cloud with the workpiece as the main body is retained. Subsequently, the point cloud is filtered to remove noise points. The filtering methods are radius filtering and statistical filtering. Finally, the workpiece point cloud without noise points is obtained. The logic of the radius filtering algorithm is that if the number of other points within a sphere with a certain point in the point cloud as the center and a radius of r is less than a specified threshold, then this point will be removed. The radius filtering algorithm is used to filter out individual outlier noise points. The point cloud is input into the radius filtering calculator of the PCL library to remove the invalid outlier points of the point cloud. The statistical filtering algorithm uses a specific statistical model to analyze and model the collected point cloud data. By calculating the distance and observing whether it conforms to the Gaussian distribution, the points that do not conform to the distribution law are defined as outlier points. This method can remove the clustered outlier points and further improve the quality of the point cloud obtained by the binocular structured light camera. The RANSAC algorithm randomly selects three points from the point cloud as samples, fits a plane model according to the sample points, substitutes the remaining data points in the point cloud into the fitted model, and calculates their distance errors from the model. If the error is less than a certain set threshold, these points are considered inlier points, otherwise they are outlier points. If the number of inlier points reaches the specified requirement, the fitted model is output; if the number of inlier points fails to meet the requirement, three points are selected again as samples, and the above steps are repeated until the preset number of iterations is reached.

[0038] Step 203: Perform surface reconstruction on the area to be repaired based on the Poisson surface reconstruction algorithm to obtain a surface model of the area to be repaired; The point cloud of the area to be repaired is smoothed, and the Poisson surface reconstruction algorithm is used to perform surface reconstruction on the point cloud. The reconstructed model of the area to be repaired is saved as an STL format file. According to the Poisson surface algorithm, a geometric body is divided into the inside and the outside of the geometric body, and the normal vectors of the collected point cloud data of the geometric body can indicate the inside and the outside of the geometric body. By implicitly fitting the indicator function of the geometric body, the information of the discrete points on the surface of the geometric body is transformed into a continuous surface function, thereby reconstructing the surface. The Poisson surface reconstruction algorithm can be used to construct a smooth and continuous three-dimensional surface from discrete point clouds.

[0039] In the embodiment of the present application, when performing path planning on a repair area to be repaired in the shape of a cylinder, a sidewall fusion optimization distance is introduced, where the sidewall refers to the part of the four sides of the cylinder in the repair area to be repaired that coincides with the workpiece, and the sidewall fusion optimization distance refers to the distance that the center line of the weld bead adjacent to the cylinder needs to maintain from the adjacent and parallel sidewall of the cylinder. If this distance is denoted as d offset , and w bead represents the width of a single-layer single-pass weld bead, then d offset = 1 / 3 * w bead . The lengths of the two parallel sides and the distance between the two parallel sides of the bottom surface of the cylinder in the repair area of the machined workpiece need to be greater than 2 times the sidewall fusion optimization distance.

[0040] Figure 3 is a schematic diagram of the principle of the sidewall fusion optimization distance. The main body of the schematic diagram is the cross-section of the weld perpendicular to the weld center line and the cross-section of the workpiece sidewall. The shaded area is the weld area. When the distance between the weld center line and the sidewall is at a certain critical distance, the upper surface of the weld between the right side of the weld center line and the sidewall is a horizontal plane, which can improve the forming repair effect. Using a parabola model to fit the weld shape, according to the equal area of the S1 area and the S2 area, the critical distance can be obtained and defined as the sidewall fusion optimization distance.

[0041] In the embodiment of the present application, if d overlap represents the center distance between weld bead laps, which refers to the distance between the center lines of two parallel and adjacent welds during welding, and k represents the ratio of the center distance between weld bead laps to the width of a single-layer single-pass weld bead, then d overlap = k * w bead . Let n represent the total number of weld beads between the two parallel sides of the bottom quadrilateral of a cylinder with a specific shape, and d represent the distance between the two parallel sides of the bottom quadrilateral, then the sidewall fusion optimization formula can be expressed as d = 2 * d offset + (n - 1) * d overlap . Set the range of the weld bead width used in arc additive repair. According to the sidewall fusion optimization formula, obtain the number of weld beads that ensure the constant weld bead lap distance. If there are multiple integer values in the range of the number of weld beads, then according to the actual situation, select a value and denote it as n integer . Then substitute it into the sidewall fusion optimization formula to obtain the corresponding width of a single-layer single-pass weld bead, denoted as w best .

[0042] Step 204: Combine the sidewall fusion optimization formula, load the surface model of the repair area to be repaired, and perform path planning on the repair area model to obtain the filling repair route coordinates of the repair area to be repaired.

[0043] In the embodiment of the present application, the Cura software is used to load the surface model of the area to be repaired. In the Prepare link, the pose of the model is adjusted to conform to the pose during the later forming repair through the Move and Rotate functions, ensuring that the coordinates generated during path planning conform to the conventional additive manufacturing positions. Optionally, the bottom surface of the adjusted model is parallel to the surface of the printing platform.

[0044] Before setting the parameters of Printsettings in the Cura software, determine the value of k according to the actual situation; let w bead =w best , and calculate the side wall fusion optimization distance d offset and the center distance d overlap of the bead lap. Let w outerwall represent the OuterWallLineWidth parameter in Printsettings of the Cura software, let w innerwall represent the InnerWall(s)LineWidth parameter in Printsettings of the Cura software, let w tbline represent the Top / BottomLineWidth parameter in Printsettings of the Cura software. If d offset <(1 / 2*d overlap ), then w outerwall =2*d offset , w innerwall =1 / 2*d overlap -d offset , w tbline =d overlap ; if d offset =(1 / 2*d overlap ), then w outerwall =w innerwall =w tbline ; if d offset >(1 / 2*d overlap ), then w outerwall =(1 / 2*d overlap -d offset ), w innerwall =2*d offset , w tbline =d overlap . The Top / BottomPattern parameter is set to Lines, and the wiring angle is adjusted so that the Top / BottomLine wiring is parallel to the upper and lower sides of the quadrilateral at the bottom of the specific-shaped column.

[0045] After completing the other parameter settings of Cura software, use Cura software to perform Slice operation on the model to generate a fill path, and use the preview function of the software to simulate and preview the generated route. If the path planning meets the requirements, save the generated route as a GCode format file. Then use the Notepad software that comes with the industrial computer to open the GCode format file to extract the fill route coordinates. If d offset <(1 / 2*d overlap ), the coordinates corresponding to the TYPE:WALL-INNER part are discarded, and only the coordinates of the TYPE:WALL-OUTER and TYPE:SKIN parts are retained; if d offset =(1 / 2*d overlap ), the coordinates of TYPE:WALL-INNER, TYPE:WALL-OUTER, and TYPE:SKIN are retained; if d offset >(1 / 2*d overlap ), the coordinates corresponding to the TYPE:WALL-OUTER part are discarded, and only the coordinates of the TYPE:WALL-INNER and TYPE:SKIN parts are retained. The retained coordinates can be used as the fill route coordinates of the area to be repaired.

[0046] In one embodiment, the in-situ remanufacturing and repair method based on three-dimensional point cloud provided by the embodiment of the present invention also includes: using a registration algorithm to convert the filling repair route coordinates of the area to be repaired generated by the Cura software into the camera coordinate system of the binocular structure camera; using the hand-eye matrix obtained by hand-eye calibration, the repair path coordinates are transferred to the TCP coordinate system of the collaborative robot; wherein, the registration algorithm includes a coarse registration algorithm and a fine registration algorithm, wherein the coarse registration algorithm adopts the Super 4PCS algorithm, obtains the point cloud of the area to be repaired and selects similar feature points, calculates the transformation matrix between similar feature points, transforms one point cloud into the coordinate system of another point cloud, and obtains the optimized registration result through iterative calculation; the fine registration algorithm adopts the NICP algorithm, adds the constraints of the normal vector and the curvature of the surface where the point cloud is located on the basis of the ICP algorithm, and optimizes the registration result by iteratively finding the nearest point pair and calculating the transformation matrix.

[0047] It can be understood that the STL model exported by the Cura software is converted into a PLY format point cloud, and a registration algorithm is used to register it with the model of the area to be repaired in the camera coordinate system, so as to transfer the path coordinates exported by the Cura software to the camera coordinate system. Then, according to the hand-eye matrix, the filling route coordinates of the area to be repaired in the TCP coordinate system of the collaborative robot can be obtained. Optionally, the registration algorithm can be composed of a coarse registration algorithm and a fine registration algorithm. Optionally, the coarse registration algorithm can adopt the Super4PCS algorithm. The Super 4PCS algorithm selects four points with similar features in two point clouds, and then calculates the transformation matrix between these four points to transform one point cloud to the coordinate system of the other point cloud. Through iterative calculation, the Super 4PCS algorithm can gradually optimize the registration result and finally obtain a better registration effect. The fine registration algorithm can adopt the Normal Iterative Closest Point Algorithm (NICP algorithm). The NICP algorithm adds the constraints of the normal vector and the curvature of the surface where the point cloud is located on the basis of the traditional Iterative Closest Point Algorithm (ICP algorithm), which can improve the accuracy and robustness of the registration.

[0048] It should be noted that the division of each module in the embodiments of the present device / system is only a logical function division. In actual implementation, it can be fully or partially integrated into a physical entity, or physically separated. And these modules can all be implemented in the form of software called by a processing element; they can also all be implemented in the form of hardware; or some units can be implemented in the form of software called by a processing element, and some units can be implemented in the form of hardware.

[0049] The implementation principles of the above modules have been described in the foregoing embodiments, so they will not be repeated here.

[0050] Based on the same concept, in some embodiments of the present application, an electronic device is also provided. This electronic device includes a memory and a processor, where the memory is used to store a processing program, and the processor executes the processing program according to instructions. When the processor executes the processing program, the in-situ remanufacturing repair method based on three-dimensional point cloud in the foregoing embodiments is realized.

[0051] In some embodiments of the present application, a readable storage medium is also provided. This readable storage medium can be a non-volatile readable storage medium or a volatile readable storage medium. Instructions are stored in the readable storage medium. When the instructions run on a computer, an electronic device including this readable storage medium executes the foregoing in-situ remanufacturing repair method based on three-dimensional point cloud.

[0052] The embodiments of the present application also provide an in-situ remanufacturing repair device based on three-dimensional point cloud, which may include: a processor and a memory.

[0053] A memory for storing programs. The memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules for implementing the above method), computer instructions, etc. The above computer programs, computer instructions, etc. can be stored in partitions in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0054] The above computer programs, computer instructions, etc. can be stored in partitions in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0055] A processor for executing the computer programs stored in the memory to implement each step in the method involved in the above embodiments.

[0056] Specifically, reference can be made to the relevant descriptions in the previous method embodiments.

[0057] The processor and the memory can be of independent structure or integrated into an integrated structure. When the processor and the memory are of independent structure, the memory and the processor can be coupled and connected through a bus.

[0058] The in-situ remanufacturing repair device based on 3D point cloud in this embodiment can execute the technical solutions in the above method. For the specific implementation process and technical principle, reference can be made to the relevant descriptions in the above method, which will not be elaborated here.

[0059] Those skilled in the art can understand that various aspects of the present invention can be implemented as a system, a method, or a program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "platform" here.

[0060] In addition, an embodiment of the present application further provides a computer-readable storage medium. Computer-executable instructions are stored in the computer-readable storage medium. When at least one processor of the user equipment executes the computer-executable instructions, the user equipment executes the above various possible methods. Among them, the computer-readable medium includes a computer storage medium and a communication medium. The communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium accessible by a general-purpose or special-purpose computer. An exemplary storage medium is coupled to the processor, enabling the processor to read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in the user equipment. Of course, the processor and the storage medium can also exist as discrete components in the communication device.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An in-situ remanufacturing and repair system based on three-dimensional point cloud, characterized in that: include: An industrial computer, a binocular structured light camera and a collaborative robot respectively connected to the industrial computer; The industrial computer is used to process the point cloud information, reconstruct the three-dimensional model of the area to be repaired of the target workpiece, perform path planning on the reconstructed model, and extract the filling route coordinates after the path planning; The collaborative robot is equipped with a welding gun connected to an intelligent welding machine, and is used to load the welding gun to perform arc additive repair operations under the control of an industrial computer; The intelligent welding machine is connected to the welding gun and is used to control the operation of the welding gun and to adjust and stabilize various parameters of the welding process in real time; The binocular structured light camera is mounted on the welding gun and is used to obtain point cloud information on the workpiece surface and transmit the data to the industrial computer through communication with the host computer; The robot control cabinet is used to control the collaborative robot equipped with a welding gun to complete welding operations under the control of an industrial computer.

2. The in-situ remanufacturing and repair system based on three-dimensional point cloud according to claim 1 is characterized in that: The binocular structured light camera is fixed to a target position on the welding gun through the camera bracket to ensure that a point cloud of the object surface can be obtained within an effective shooting distance of the binocular structured light camera under the premise that the welding gun does not touch an external object.

3. An in-situ remanufacturing and repair method based on three-dimensional point cloud, characterized in that: The in-situ remanufacturing and repair system based on three-dimensional point cloud as claimed in any one of claims 1 to 2 comprises: The hand-eye calibration of the binocular structured light camera is performed based on the dual quaternion and singular value decomposition algorithm to obtain the position and posture of the binocular structured light camera in the base coordinate system of the collaborative robot. The point cloud data of the target workpiece surface after machining is obtained by shooting with a binocular structured light camera and preprocessed to obtain the complete point cloud data of the area to be repaired after preprocessing; Reconstructing the surface of the area to be repaired based on a Poisson surface reconstruction algorithm to obtain a surface model of the area to be repaired; Combined with the sidewall fusion optimization formula, the surface model of the area to be repaired is loaded and the path planning is performed on the model of the area to be repaired to obtain the filling repair route coordinates of the area to be repaired.

4. The in-situ remanufacturing and repair method based on three-dimensional point cloud according to claim 3 is characterized in that: The target workpiece is a metal workpiece having a target shape area to be repaired, the target shape is a regular cylinder, and the bottom surface of the regular cylinder is a quadrilateral with two parallel sides; the area where the regular cylinder is located completely covers the area to be repaired, only one face of the regular cylinder does not overlap with the corresponding part of the target workpiece, and the other five faces overlap with the corresponding parts of the target workpiece.

5. The in-situ remanufacturing and repair method based on three-dimensional point cloud according to claim 3 is characterized in that: Also includes: The registration algorithm is used to transform the coordinates of the filling and repairing route of the area to be repaired generated by the Cura software into the camera coordinate system of the binocular structure camera. The hand-eye matrix obtained by hand-eye calibration is used to transform the coordinates of the repairing path into the TCP coordinate system of the collaborative robot. Among them, the registration algorithm includes a coarse registration algorithm and a fine registration algorithm. The coarse registration algorithm adopts the Super 4PCS algorithm. After obtaining the point cloud of the area to be repaired and selecting similar feature points, the transformation matrix between the similar feature points is calculated, and one point cloud is transformed into the coordinate system of another point cloud. The optimized registration result is obtained through iterative calculation; the fine registration algorithm adopts the NICP algorithm. On the basis of the ICP algorithm, the constraints of the normal vector and the curvature of the surface where the point cloud is located are added, and the registration result is optimized by iteratively finding the nearest point pair and calculating the transformation matrix.

6. The in-situ remanufacturing and repair method based on three-dimensional point cloud according to claim 3 is characterized in that: The point cloud data of the target workpiece surface after machining is obtained by shooting with a binocular structured light camera and preprocessed to obtain the complete point cloud data of the area to be repaired after preprocessing, including: Preprocessing the damaged workpiece, wherein the preprocessing is to completely remove the damaged area on the damaged workpiece by machining to obtain a processed target workpiece; The target workpiece is placed on a welding workbench, and all surface information of the workpiece is acquired by photographing from different angles using a binocular structured light camera, wherein the surface information includes all point clouds of the topography of the area to be repaired; The plane equation of the worktable surface is fitted to the collected point cloud based on the RANSAC algorithm. The distance from all points in the collected point cloud to the worktable plane is calculated based on the worktable plane equation. The point cloud located on the worktable surface and below is removed according to a preset distance threshold. The point cloud with the target workpiece as the main body is retained and filtered based on a preset filtering algorithm to remove noise points, so as to obtain a target workpiece point cloud without noise points. Identify and perform the first segmentation of the target workpiece point cloud into the upper surface point cloud and the point cloud other than the upper surface point cloud, use the RANSAC algorithm to fit the plane equation of the upper surface of the workpiece and retain the point cloud of the upper surface of the workpiece; Defect recognition is performed on the area to be repaired on the upper surface of the target workpiece, and points are filled in the point cloud of the upper surface of the target workpiece according to the plane equation of the upper surface of the target workpiece and the coordinate range of the point in the point cloud. If the point cloud of the upper surface of the target workpiece is not found around the filled point, the filled point is represented as a point at the defect of the upper surface of the target workpiece, and a new point cloud is written, which is recorded as the defect point cloud of the upper surface of the target workpiece; The workpiece point cloud is segmented for the second time according to the plane equation of the upper surface of the target workpiece to obtain the point cloud below the upper surface of the target workpiece, and the defective point cloud on the upper surface of the target workpiece and the point cloud below the upper surface of the target workpiece are merged to obtain a complete point cloud of the area to be repaired.

7. The in-situ remanufacturing and repair method based on three-dimensional point cloud according to claim 3 is characterized in that: When planning the path for the target workpiece where the target shape is a column to be repaired, the center line of the weld bead adjacent to the column maintains a preset distance d from the adjacent and parallel side of the column. offset , that is, the side wall fusion optimization distance is as follows: d offset =1 / 3*w bead , wherein the side wall refers to the part where the four sides of the column in the area to be repaired overlap with the target workpiece, w bead represents the width of a single-layer single-pass weld. The length of the two parallel sides of the bottom surface of the column in the area to be repaired and the distance between the two parallel sides of the target workpiece are both greater than 2 times d offset To meet the distance conditions required for the normal application of the algorithm.

8. The in-situ remanufacturing and repair method based on three-dimensional point cloud according to claim 3 is characterized in that: According to the parabolic fitting of the weld shape and the principle of equal area, the critical distance, i.e. the side wall fusion optimization distance, is determined, further including: If d overlap represents the lap center distance of the weld bead, k represents the ratio of the lap center distance of the weld bead to the width of a single layer single bead, then d overlap =k*w bead , w bead is the width of a single-layer single-pass weld; n represents the total number of welds between two parallel sides of the bottom quadrilateral of the target shape cylinder, d represents the distance between the two parallel sides of the bottom quadrilateral, and the side wall fusion optimization formula is expressed as d=2*d offset +(n-1)*d overlap , d offset Optimize distance for sidewall fusion.

9. An electronic device, characterized in that: include: A memory, the memory being used to store a processing program; A processor, wherein when executing the processing program, the processor implements the in-situ remanufacturing and repair method based on three-dimensional point cloud as described in any one of claims 3 to 8.

10. A readable storage medium, characterized in that: The readable storage medium stores a processing program, and when the processing program is executed by the processor, the in-situ remanufacturing and repair method based on three-dimensional point cloud as described in any one of claims 3 to 8 is implemented.

Images