Aircraft coating precision repairing method based on 3D visual data

Abstract

The invention belongs to the technical field of airplane coating repairing, and particularly relates to an airplane coating precision repairing method based on 3D visual data, which comprises the following steps: shooting a workpiece needing coating repairing to obtain an RGB image, and converting the RGB image into a point cloud picture; after the point cloud picture is processed, a 3D model of a workpiece needing coating repairing is obtained, spraying distance parameters are set, a robot track is generated, path simulation is conducted on the track, if simulation is free of interference, an online laser scanner is arranged to run according to the robot track, secondary detection and positioning are conducted on the surface of the workpiece needing coating repairing, and then the workpiece needing coating repairing is obtained. Acquiring spatial position data of a to-be-restored area; the industrial personal computer generates a robot path middle point according to the spatial position data of the area needing to be sprayed and associates the robot path middle point to an I / O control instruction of the robot; spraying parameters are input into the robot, after a robot motion program is generated, the complete robot track is simulated again, and if simulation is free of interference, the complete robot track is imported into a robot controller to execute spraying.

Description

Technical Field

[0001] This invention belongs to the field of aircraft coating repair technology, specifically a precision repair method for aircraft coatings based on 3D visual data. Background Technology

[0002] During aircraft operation, the coating on the aircraft surface may be affected by various factors, such as wind and rain, climate change, and external collisions, leading to damage such as scratches, corrosion, and peeling. To ensure the aircraft's appearance quality and aerodynamic performance, precise coating repair is required. Traditional coating repair methods have certain limitations, such as complex operation, low precision, and low efficiency. Existing painting robots typically require manual teaching or programming to set the painting trajectory, and for irregular aircraft fuselage parts, they mostly use whole-surface atomization spraying. Aircraft parts have problems such as large deformations, irregular coating damage, and difficulty in obtaining accurate models. When using traditional special-purpose machines or painting robots to handle such complex shapes or variable products requiring high precision, problems such as low efficiency, poor precision, and difficulty in robot adjustment arise. Currently, aircraft coating repair is mostly done manually. When there is a need for multiple types and colors of coating spraying or repair combined, it is mostly done by film application. The traditional method of generating spraying trajectory by combining robots with machine vision offline trajectory generation is difficult to adapt to the rapid switching of different batches of products, and has drawbacks such as low spraying accuracy, blurred boundaries, and lengthy and difficult-to-modify robot programs. Summary of the Invention

[0003] The purpose of this invention is to provide a precision repair method for aircraft coatings based on 3D visual data, so as to overcome the drawbacks of low spraying accuracy, blurred boundaries, and lengthy and difficult-to-modify robot programs.

[0004] The technical solution adopted by the present invention to achieve the above objectives is: a method for precision repair of aircraft coatings based on 3D visual data, comprising the following steps:

[0005] 1) The 3D vision inspection device takes pictures of the workpiece that needs coating repair, obtains RGB images, and converts the RGB images into point cloud maps through an industrial control computer;

[0006] 2) After processing the point cloud map with the industrial control computer, obtain the 3D model of the workpiece that needs to be repaired with coating. At the same time, set a fixed spraying distance parameter, generate the robot trajectory, and perform path simulation on the generated robot trajectory. If there is no interference in the simulation, proceed to step 3); otherwise, return to step 1).

[0007] 3) The robot runs along the trajectory obtained in step 2) using a line laser scanner set at the end of the robot tool, and performs secondary detection and positioning on the surface of the workpiece that needs coating repair to obtain spatial location data of the area to be repaired;

[0008] 4) The industrial control computer generates the intermediate points of the robot path based on the obtained spatial location data of the area to be sprayed, and associates them with the robot's I / O control commands;

[0009] 5) Input the spraying parameters into the robot. After generating the robot motion program, perform the simulation method in step 2) again on the complete robot trajectory. If there is no interference in the simulation, import it into the robot controller to execute the spraying.

[0010] The 3D vision inspection device is a depth camera;

[0011] The 3D vision inspection device is installed on the end of the robot tool and connected to the industrial control computer.

[0012] In step 2), the process of obtaining a 3D model of the workpiece requiring coating repair by processing the point cloud image using an industrial control computer via a triangulation algorithm is as follows:

[0013] 1-1) The industrial control computer performs data cleaning operations on the point cloud map, including removing noise points, filling in missing data, and filtering the data, to obtain the cleaned data;

[0014] 1-2) The point cloud data of the workpiece surface obtained by the 3D vision inspection device after cleaning is connected by triangular meshing with a distance parameter greater than the spraying accuracy requirement. That is, the point cloud data is converted into a triangular facet model to make the data easier to process and render.

[0015] In step 2), generating the robot trajectory specifically involves:

[0016] 2-1) Extract the geometric information of the workpiece that needs coating repair from the 3D model;

[0017] 2-2) By fixing the spraying parameters and distance and combining the geometric information of the workpiece to be repaired with the coating, the robot trajectory is initially generated. The robot's TCP point is generated by a straight line perpendicular to the workpiece surface, starting from the center of the robot spray gun tool end.

[0018] 2-3) Maintain a fixed spraying parameter distance, and combine it with the generated robot TCP point to complete the set of spatial points traversed during the process of multiple U-shaped reciprocating spraying actions, i.e., the robot trajectory.

[0019] The geometric information of the workpiece requiring coating repair includes: the surface shape, size, and complexity of the workpiece requiring coating repair;

[0020] In step 2), simulating the generated robot trajectory specifically involves:

[0021] a. Based on the generated motion trajectory data and the obtained 3D model of the workpiece requiring coating repair, a simulation environment is established using MATLAB;

[0022] b. Set the start and end points of the generated trajectory to ensure that the generated trajectory is correctly aligned with the workpiece model and equipment model;

[0023] c. Set collision detection parameters: Set collision detection parameters, including those for equipment boundaries and workpiece boundaries, in the simulation software;

[0024] d. Track the motion path of each robot's TCP point to ensure there is no collision with the workpiece or equipment model;

[0025] e. During the simulation, monitor the collision detection results to check for collisions between the trajectory and the workpiece or equipment model. If a collision or interference is found, record the trajectory point and collision location, and return to step 1) for subsequent analysis and adjustment of the 3D model parameters. Then, re-execute the interference simulation until it is ensured that the generated motion trajectory will not collide or interfere in actual operation, and proceed to step 3).

[0026] Step 3) includes the following steps:

[0027] 3-1) Install a line laser scanner with a precision greater than that required for spraying at the tool end of the six-axis robot;

[0028] 3-2) When the six-axis robot runs on the generated trajectory, the scanner is simultaneously started to perform fine measurements on the workpiece surface and send the data to the industrial control computer;

[0029] 3-3) The industrial control computer forms a new robot spraying path trajectory based on the offset of each spatial point in the robot trajectory after correction. At the same time, it collects the spatial position data of the area to be repaired in the workpiece that needs to be coated, including the position, shape and size information of the repair area.

[0030] Step 4) includes the following steps:

[0031] 4-1) Based on the spatial location data of the area to be repaired in the workpiece requiring coating repair obtained from the secondary inspection and positioning process;

[0032] 4-2) The industrial control computer uses the spatial location data of the area to be repaired in the workpiece to be coated and repaired, combined with the path generated by the path planning algorithm, to generate the intermediate points of the robot's path; the intermediate points can form a complete path that covers the entire repair area; the intermediate points of the path are used to guide the robot's movement path in space.

[0033] 4-3) Path association: The industrial control computer generates a robot motion program in G-code format from the robot trajectory, which includes the intermediate points of the robot path, using robot simulation software;

[0034] In the robot's motion program, corresponding control commands are executed at the midpoints of the path where the spray nozzle or different coating materials need to be switched, based on the actual spraying process.

[0035] The spraying parameters include: spraying distance, spraying angle, and spraying speed.

[0036] The present invention has the following beneficial effects and advantages:

[0037] 1. This invention improves spraying efficiency and precision, reduces manual intervention, and is highly adaptable, capable of quickly handling spraying objects of different shapes and sizes;

[0038] 2. This invention optimizes the spraying trajectory twice using a high-precision line laser scanner, ensuring spraying accuracy and uniformity while avoiding overspray, thereby reducing production costs and improving product quality;

[0039] 3. In the spraying process, the present invention monitors and adjusts the spraying trajectory in real time to adapt to changes in the sprayed object;

[0040] 4. This method performs two trajectory simulations in the workpiece model acquisition stage and the spraying area acquisition stage to ensure the safety of the process. At the same time, the second two-dimensional scan of the workpiece surface ensures the precision of the spraying. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating the automatic generation system for robot spraying trajectories of the present invention. Detailed Implementation

[0042] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0043] like Figure 1 The diagram shown is a flowchart of the robot spraying trajectory automatic generation system of the present invention. The present invention provides a method for precision repair of aircraft coatings based on 3D vision data, which includes the following steps:

[0044] 1) The 3D vision inspection device takes pictures of the workpiece that needs coating repair, obtains RGB images, and converts the RGB images into point cloud maps through an industrial control computer;

[0045] 2) After processing the point cloud map with the industrial control computer, obtain the 3D model of the workpiece that needs to be repaired with coating. At the same time, set a fixed spraying distance parameter, generate the robot trajectory, and perform path simulation on the generated robot trajectory. If there is no interference in the simulation, proceed to step 3); otherwise, return to step 1).

[0046] A. After processing the point cloud image using a triangulation algorithm via an industrial control computer, a 3D model of the workpiece requiring coating repair is obtained, specifically:

[0047] 1-1) The industrial control computer performs data cleaning operations on the point cloud map, including removing noise points, filling in missing data, and filtering the data, to obtain the cleaned data;

[0048] 1-2) The point cloud data of the workpiece surface obtained by the 3D vision inspection device after cleaning is connected by triangular meshing with a distance parameter greater than the spraying accuracy requirement. That is, the point cloud data is converted into a triangular facet model to make the data easier to process and render.

[0049] B. Generate the robot trajectory, specifically:

[0050] 2-1) Extract the geometric information of the workpiece that needs coating repair from the 3D model;

[0051] 2-2) By fixing the spraying parameters and distance and combining the geometric information of the workpiece to be repaired with the coating, the robot trajectory is initially generated. The robot's TCP point is generated by a straight line perpendicular to the workpiece surface, starting from the center of the robot spray gun tool end.

[0052] The geometric information of the workpiece to be repaired by coating includes: the surface shape, size, and complexity of the workpiece to be repaired by coating;

[0053] 2-3) Maintain a fixed spraying parameter distance, and combine it with the generated robot TCP point to complete the set of spatial points traversed during the process of multiple U-shaped reciprocating spraying actions, i.e., the robot trajectory.

[0054] C. Simulate the generated robot trajectory, specifically as follows:

[0055] a. Based on the generated motion trajectory data and the obtained 3D model of the workpiece requiring coating repair, a simulation environment is established using MATLAB;

[0056] b. Set the start and end points of the generated trajectory to ensure that the generated trajectory is correctly aligned with the workpiece model and equipment model;

[0057] c. Set collision detection parameters: Set collision detection parameters, including those for equipment boundaries and workpiece boundaries, in the simulation software;

[0058] d. Track the motion path of each robot's TCP point to ensure there is no collision with the workpiece or equipment model;

[0059] e. During the simulation, monitor the collision detection results to check for collisions between the trajectory and the workpiece or equipment model. If a collision or interference is found, record the trajectory point and collision location, and return to step 1) for subsequent analysis and adjustment of the 3D model parameters. Then, re-execute the interference simulation until it is ensured that the generated motion trajectory will not collide or interfere in actual operation, and proceed to step 3).

[0060] 3) The robot runs along the trajectory obtained in step 2) using a line laser scanner set at the end of the robot tool, and performs secondary detection and positioning on the surface of the workpiece that needs coating repair to obtain spatial location data of the area to be repaired;

[0061] 3-1) Install a line laser scanner with a precision greater than that required for spraying at the tool end of the six-axis robot;

[0062] 3-2) When the six-axis robot runs on the generated trajectory, the scanner is simultaneously started to perform fine measurements on the workpiece surface and send the data to the industrial control computer;

[0063] 3-3) The industrial control computer forms a new robot spraying path trajectory based on the offset of each spatial point in the robot trajectory after correction. At the same time, it collects the spatial position data of the area to be repaired in the workpiece that needs to be coated, including the position, shape and size information of the repair area.

[0064] 4) The industrial control computer generates the intermediate points of the robot path based on the obtained spatial location data of the area to be sprayed, and associates them with the robot's I / O control commands;

[0065] 4-1) Based on the spatial location data of the area to be repaired in the workpiece requiring coating repair obtained from the secondary inspection and positioning process;

[0066] 4-2) The industrial control computer uses the spatial location data of the area to be repaired in the workpiece to be coated and repaired, combined with the path generated by the path planning algorithm, to generate the intermediate points of the robot's path; the intermediate points can form a complete path that covers the entire repair area; the intermediate points of the path are used to guide the robot's movement path in space.

[0067] 4-3) Path association: The industrial control computer generates a robot motion program in G-code format from the robot trajectory, which includes the intermediate points of the robot path, using robot simulation software;

[0068] In the robot's motion program, corresponding control commands are executed at the midpoints of the path where the spray nozzle or different coating materials need to be switched, based on the actual spraying process.

[0069] 5) Input the spraying parameters into the robot. After generating the robot motion program, perform the simulation method in step 2) again on the complete robot trajectory. If there is no interference in the simulation, import it into the robot controller to execute the spraying.

[0070] The spraying parameters include: spraying distance, spraying angle, and spraying speed.

[0071] Example 1:

[0072] This invention provides a precision repair method for aircraft coatings based on 3D vision data, which is implemented using a repair system. The repair system includes: a 3D vision inspection device, a six-axis or higher industrial robot, and an industrial control computer.

[0073] A 3D vision inspection device includes a 3D camera and a light source, and is mounted on a robot tool end with a movable line laser scanner; in this embodiment, a depth camera is selected as the 3D vision inspection device.

[0074] This embodiment uses a six-axis industrial robot;

[0075] Industrial control computers are used to process 3D vision data and communicate with robot controllers;

[0076] During the spraying process, the six-axis industrial robot was replaced with a digital automatic spraying machine, equipped with EcoPaintjet robot-specific multi-nozzle nozzles.

[0077] The specific implementation steps of this invention are as follows:

[0078] 1) Start the 3D vision inspection device to scan the sprayed workpiece and obtain RGB images.

[0079] 2) The vision-based industrial control computer converts the 3D model into a point cloud image, and after triangulation of the point cloud image, obtains the 3D model of the workpiece; the 3D model is then transmitted to the industrial control computer.

[0080] 3) Analyze the shape and size of the object to be sprayed, and generate the robot trajectory using SPRUT CAM software based on a fixed spraying distance parameter;

[0081] 4) Visual simulation of robot trajectory to ensure interference-free spraying.

[0082] 5) The line laser scanner mounted on the robot tool end runs along the trajectory and performs secondary inspection and positioning of the workpiece surface to obtain spatial location data of the area to be pre-sprayed / the area to be repaired;

[0083] 6) The host computer SPRUT CAM software generates the robot path midpoint based on the spatial position of the area to be sprayed obtained in step 3, and associates it with the I / O control of the precision spraying machine;

[0084] 7) Input the spraying distance, angle, and speed parameters to generate the robot motion program. The host system visualizes and simulates the robot coating trajectory. If the simulation passes, it is imported into the robot controller to execute the spraying.

[0085] 8) If the industrial control computer fails to perform a visual simulation of the robot's coating trajectory, the process parameters need to be manually corrected.

[0086] 9) The robot performs automatic spraying tasks according to the spraying trajectory.

[0087] In the spraying process of this embodiment, the spraying trajectory can be monitored and adjusted in real time, and after the spraying is completed, the efficiency and accuracy of the spraying are significantly improved compared with the traditional spraying method.

[0088] In summary, this invention monitors and adjusts the spraying trajectory in real time during the spraying process to adapt to changes in the spraying object. At the same time, two trajectory simulations are performed during the workpiece model acquisition stage and the spraying area acquisition stage to ensure the safety of the process. Meanwhile, the second two-dimensional scan of the workpiece surface ensures the precision of the spraying.

[0089] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0090] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A method for precision repair of aircraft coatings based on 3D visual data, characterized in that, Includes the following steps: 1) The 3D vision inspection device takes pictures of the workpiece that needs coating repair, obtains RGB images, and converts the RGB images into point cloud maps through an industrial control computer; 2) After processing the point cloud map with the industrial control computer, obtain the 3D model of the workpiece that needs to be repaired with coating. At the same time, set a fixed spraying distance parameter, generate the robot trajectory, and perform path simulation on the generated robot trajectory. If there is no interference in the simulation, proceed to step 3); otherwise, return to step 1). 3) The robot runs along the trajectory obtained in step 2) using a line laser scanner set at the end of the robot tool, and performs secondary detection and positioning on the surface of the workpiece that needs coating repair to obtain spatial location data of the area to be repaired; 4) The industrial control computer generates the intermediate points of the robot path based on the obtained spatial location data of the area to be sprayed, and associates them with the robot's I / O control commands; 5) Input the spraying parameters into the robot. After generating the robot motion program, perform the simulation method in step 2) again on the complete robot trajectory. If there is no interference in the simulation, import it into the robot controller to execute the spraying.

2. The method for precision repair of aircraft coatings based on 3D visual data according to claim 1, characterized in that, The 3D vision inspection device is a depth camera; The 3D vision inspection device is installed on the end of the robot tool and connected to the industrial control computer.

3. The method for precision repair of aircraft coatings based on 3D visual data according to claim 1, characterized in that, In step 2), the process of obtaining a 3D model of the workpiece requiring coating repair by processing the point cloud image using an industrial control computer via a triangulation algorithm is as follows: 1-1) The industrial control computer performs data cleaning operations on the point cloud map, including removing noise points, filling in missing data, and filtering the data, to obtain the cleaned data; 1-2) The point cloud data of the workpiece surface obtained by the 3D vision inspection device after cleaning is connected by triangular meshing with a distance parameter greater than the spraying accuracy requirement. That is, the point cloud data is converted into a triangular facet model to make the data easier to process and render.

4. The method for precision repair of aircraft coatings based on 3D visual data according to claim 1, characterized in that, In step 2), generating the robot trajectory specifically involves: 2-1) Extract the geometric information of the workpiece that needs coating repair from the 3D model; 2-2) By fixing the spraying parameters and distance and combining the geometric information of the workpiece to be repaired with the coating, the robot trajectory is initially generated. The robot's TCP point is generated by a straight line perpendicular to the workpiece surface, starting from the center of the robot spray gun tool end. 2-3) Maintain a fixed spraying parameter distance, and combine it with the generated robot TCP point to complete the set of spatial points traversed during the process of multiple U-shaped reciprocating spraying actions, i.e., the robot trajectory.

5. The method for precision repair of aircraft coatings based on 3D visual data according to claim 4, characterized in that, The geometric information of the workpiece requiring coating repair includes: the surface shape, size, and complexity of the workpiece requiring coating repair.

6. The method for precision repair of aircraft coatings based on 3D visual data according to claim 1, characterized in that, In step 2), simulating the generated robot trajectory specifically involves: a. Based on the generated motion trajectory data and the obtained 3D model of the workpiece requiring coating repair, a simulation environment is established using MATLAB; b. Set the start and end points of the generated trajectory to ensure that the generated trajectory is correctly aligned with the workpiece model and the equipment model; c. Set collision detection parameters: Set collision detection parameters, including those for equipment boundaries and workpiece boundaries, in the simulation software; d. Track the motion path of each robot's TCP point to ensure there is no collision with the workpiece or equipment model; e. During the simulation, monitor the collision detection results to check for collisions between the trajectory and the workpiece or equipment model. If a collision or interference is found, record the trajectory point and collision location, and return to step 1) for subsequent analysis and adjustment of the 3D model parameters. Then, re-execute the interference simulation until it is ensured that the generated motion trajectory will not collide or interfere in actual operation, and proceed to step 3).

7. The method for precision repair of aircraft coatings based on 3D visual data according to claim 1, characterized in that, Step 3) includes the following steps: 3-1) Install a line laser scanner with a precision higher than that required for spraying at the tool end of the six-axis robot; 3-2) When the six-axis robot runs on the generated trajectory, the scanner is simultaneously started to perform fine measurements on the workpiece surface and send the data to the industrial control computer; 3-3) The industrial control computer forms a new robot spraying path trajectory based on the offset of each spatial point in the robot trajectory after correction. At the same time, it collects the spatial position data of the area to be repaired in the workpiece that needs to be coated, including the position, shape and size information of the repair area.

8. The method for precision repair of aircraft coatings based on 3D visual data according to claim 1, characterized in that, Step 4) includes the following steps: 4-1) Based on the spatial location data of the area to be repaired in the workpiece requiring coating repair obtained from the secondary inspection and positioning process; 4-2) The industrial control computer uses the spatial location data of the area to be repaired in the workpiece to be coated and repaired, combined with the path generated by the path planning algorithm, to generate the intermediate points of the robot's path; the intermediate points can form a complete path that covers the entire repair area; the intermediate points of the path are used to guide the robot's movement path in space. 4-3) Path association: The industrial control computer generates a robot motion program in G-code format from the robot trajectory, which includes the intermediate points of the robot path, using robot simulation software; In the robot's motion program, corresponding control commands are executed at the midpoints of the path where the spray nozzle or different coating materials need to be switched, based on the actual spraying process.

9. The method for precision repair of aircraft coatings based on 3D visual data according to claim 1, characterized in that, The spraying parameters include: paint type, spraying distance, spraying angle, and spraying speed.

Images