Automobile paint defect detection and polishing method and system

Abstract

The application discloses a kind of automobile paint surface defect detection and polishing method and system, the method includes: offline processing: detection station offline collection point planning is carried out, and the trajectory of multiple detection mechanical arms is planned;Detection station offline collection point calibration is carried out, and the paint surface imaging acquisition component and the 3D positioning camera external parameter of detection station are offline calibrated;Polishing station polishing mechanical arm and polishing station 3D positioning camera hand-eye calibration are carried out;Online processing: paint surface defect detection is carried out, and defect attribute and defect position positioning are obtained;Defect three-dimensional coordinates and normal are transmitted to automatic polishing component by visual positioning mode, and the paint surface defect is polished by automatic polishing component.Processing is carried out.The system is used to execute the above method.The application has the advantages of simple structure, wide application range, high degree of intelligence, high detection and polishing efficiency, etc.

Description

Technical Field

[0001] This invention mainly relates to the field of intelligent technology in automobile body manufacturing, specifically a method and system for detecting and polishing defects in automobile paint. Background Technology

[0002] Painting is a crucial step in automobile body manufacturing. It involves applying paint to the vehicle's surface to enhance its rust and corrosion resistance, as well as its aesthetic appeal. However, automobile body painting is a highly precise and challenging task, as the quality of the paint, the painting environment, and the setting of spraying parameters all affect the final product.

[0003] In the actual manufacturing process, vehicle paint inevitably suffers some dirt or damage, such as dents caused by surface impurities during painting, scratches and stains caused by improper handling during transportation, etc. The existence of these appearance defects will directly affect the manufacturer's brand image and car sales. Therefore, inspecting and repairing vehicle paint defects before leaving the factory is a necessary part of the painting process.

[0004] Because the paint on a car body is made of a special material with high reflectivity, the detection and repair of paint defects is very challenging, especially for intelligent and unmanned paint defect detection.

[0005] Currently, the detection and repair of defects in automotive body paint is still primarily done manually. This manual inspection method, aided by special light sources, involves workers observing and touching defects from multiple angles, recording information such as the type, size, and location of the defects. Then, using hand tools, they repair the paint defects through grinding, polishing, and spraying. While this traditional manual method meets the requirements, it heavily relies on the worker's experience and concentration, demanding a high level of personal skill from the operator. Furthermore, it suffers from low efficiency and inconsistent quality, failing to meet the demands of intelligent and rapid production lines. Moreover, under prolonged periods of intense concentration, the quality of the inspection and repair work steadily declines.

[0006] With the continuous development of visual imaging and robotics technologies, it has become possible for machine vision to guide robots in detecting and repairing paint defects. In recent years, some methods have emerged that use 2D cameras under specially customized light sources to detect paint defects. However, these methods cannot achieve automated polishing because they cannot acquire the three-dimensional information of the defects. Other methods use 3D cameras based on the phase deflection principle to acquire high-precision three-dimensional point clouds of mirror-like vehicle paint surfaces, thereby effectively detecting defects and their three-dimensional coordinate information. However, these methods cannot accurately transmit the location information of the defects to the defect repair robot to achieve automated repair.

[0007] For example, the Chinese patent application "Apparatus and Method for Detecting Defects in Vehicle Paint Surface" (application number 202110117975.2) includes a servo controller, an industrial computer, a vehicle conveying mechanism, several area array cameras, several projectors, and several projection screens. The coordinate system of these components and their orientation relationship with the world coordinate system are pre-calibrated using a calibration plate. The area array cameras, projectors, and projection screens together constitute an image acquisition system for collecting and analyzing surface data of the vehicle under inspection. The industrial computer is connected to the projectors, and the projection screens are positioned around and corresponding to the projectors. The projectors can project four horizontal and four vertical sinusoidal phase-shifted fringe images onto the corresponding projection screens. The area array cameras are distributed around the vehicle under inspection, and each camera takes pictures of the stripes on the projection screen through reflection from the vehicle surface. Within a single field of view of the vehicle, four horizontal and four vertical sinusoidal phase-shifted fringe images, for a total of eight images, can be captured. By designing a servo controller, industrial computer, and vehicle body conveying mechanism to work with an image acquisition system, it is possible to conveniently and efficiently acquire sinusoidal phase-shifted fringe images related to the vehicle body surface, thereby outputting specular reflection maps, diffuse reflection maps, gloss maps, curvature maps, etc. This allows for the detection of dirt, texture, scratches, dents, and unevenness defects on the vehicle body paint surface using traditional image processing algorithms and deep learning algorithms. However, the above technical solution is quite complex in structure, costly, has a narrow scope of application, and insufficient detection accuracy. Most importantly, the above traditional technology can only complete the detection and cannot be automated in conjunction with subsequent repair workstations. Summary of the Invention

[0008] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a method for detecting and polishing automotive paint defects that is simple in structure, has a wide range of applications, a high degree of intelligence, and high efficiency in detection and polishing.

[0009] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0010] A method for detecting and polishing defects in automotive paint, comprising:

[0011] Offline processing: Plan offline data acquisition points for the inspection station and plan the trajectories of multiple inspection robotic arms; calibrate offline data acquisition points for the inspection station and perform offline calibration of the extrinsic parameters of the paint surface imaging acquisition component and the 3D positioning camera for the inspection station; perform hand-eye calibration of the grinding robotic arm and the 3D positioning camera for the grinding station.

[0012] Online processing: Perform paint surface defect detection to obtain defect attributes and defect location; transmit the three-dimensional coordinates and normal of the defect to the automatic sanding component through visual positioning, and the automatic sanding component will sand the paint surface defect.

[0013] As a further improvement to the method of the present invention: the offline data acquisition point planning of the detection station includes:

[0014] Coarse planning of the simulation environment: Import the actual vehicle CAD model into the digital twin simulation system, specify the area of ​​the vehicle to be inspected, and the simulation system automatically plans multiple acquisition points based on the vehicle surface shape and the mirror field of view of the imaging components;

[0015] Fine-tuning of the actual scene: The coarse-planned points in the simulation scene are imported into the actual scene, and the detection robotic arm equipped with an imaging acquisition component collects point cloud data of the vehicle paint surface at each coarse-planned point; All the acquired point cloud data are stitched together based on the extrinsic parameters of the coarse-planned points, where the point extrinsic parameters are calculated by the end pose of the detection robotic arm and the hand-eye calibration results; Point evaluation is performed based on the vehicle surface coverage and imaging quality; Fine-tuning of the points: The point evaluation is performed according to the above steps, and the point fine-tuning and evaluation process is repeated until the actual detection requirements are met.

[0016] As a further improvement to the method of the present invention: the step of offline acquisition point calibration of the detection station includes:

[0017] Step (1): Perform internal parameter calibration on the imaging acquisition component using a checkerboard calibration board;

[0018] Step (2): Keep the large-size calibration plate stationary, and have the detection robotic arm carrying the imaging acquisition component move sequentially to N offline planned points to acquire two-dimensional image data of the large-size calibration plate;

[0019] Step (3): Extract features from the N collected calibration board images and detect the ID and center point information of each calibration block;

[0020] Step (4): Based on the calibration block ID information detected in step (3), perform feature matching on the calibration image of each point with other calibration images, that is, calibration blocks with the same ID are used as a set of matching features.

[0021] Step (5): Based on the camera intrinsic parameters in step (1) and the feature matching information in step (4), the motion recovery structure algorithm is used to simultaneously estimate the three-dimensional coordinates of the feature points of the calibration board and the relative pose of each point.

[0022] Step (6): Based on the estimated 3D coordinates of the feature points in step (5) and the prior size information of the large-size calibration board, estimate the global scale information of the scene;

[0023] Step (7): Based on the relative poses of each point estimated in step (5) and the global scale information estimated in step (6), calculate the pose of each acquisition point relative to the reference point.

[0024] As a further improvement to the method of the present invention: the process of offline calibration of the extrinsic parameters of the paint surface imaging acquisition component and the 3D positioning camera at the inspection station includes:

[0025] Step (10): Adjust the angle of the double-sided calibration plate according to the common field of view of the acquisition component and the 3D positioning camera under the reference point of the detection station;

[0026] Step (20): Move the double-sided calibration plate N times, and the acquisition component and 3D positioning camera simultaneously acquire N images of the calibration plate;

[0027] Step (30): Corner detection of calibration board image;

[0028] Step (40): Calculate the relative pose of the single-sided calibration plate with respect to the acquisition component or the 3D positioning camera;

[0029] Step (50): Estimate the geometry of the double-sided calibration plate based on bundle adjustment and graph optimization;

[0030] Step (60): Calculate the extrinsic parameters of the paint surface imaging acquisition component and the 3D positioning camera.

[0031] As a further improvement to the method of the present invention, it also includes hand-eye calibration of the 3D positioning camera at the grinding station and the grinding robot arm. The process includes: fixing the calibration plate to the end of the grinding robot arm, moving the grinding robot arm multiple times so that the 3D positioning camera at the grinding station can observe the calibration plate, recording the rectangular coordinates of the grinding robot arm and the corresponding calibration plate image, and calibrating the extrinsic parameters of the 3D positioning camera at the grinding station relative to the base system of the grinding robot arm based on the eye-to-hand external calibration algorithm.

[0032] As a further improvement to the method of the present invention: the process of transferring the coordinates of the defect location includes:

[0033] Step S101: Set the coordinate system C of each collection point. i The coordinates of the defects detected are obtained through the external parameters of the offline calibrated points. Transform to reference point coordinate system C r Down;

[0034] Step S102: Convert the defect coordinates in the reference point coordinate system to the external parameters calibrated offline. Transform to the 3D positioning camera coordinate system C of the inspection station d Down;

[0035] Step S103: The defect coordinates in the coordinate system of the 3D positioning camera at the inspection station are converted into the vehicle body correction pose calculated online. Switch to the 3D positioning camera coordinate system C at the polishing station p Down;

[0036] Step S104: The defect coordinates in the 3D positioning camera coordinate system of the grinding station are calibrated offline using hand-eye calibration parameters. Transform to the base coordinate system O of the grinding robot arm p Down.

[0037] As a further improvement to the method of the present invention: during the coordinate transfer process, the transferred defect attributes include one or more of the following: defect size, defect type, spatial location of defect center point, and defect surface normal vector.

[0038] The present invention further provides an automotive paint defect detection and polishing system, comprising:

[0039] A defect detection unit is used to detect and locate defects in the vehicle body paint. It includes two or more detection robotic arms, two or more paint imaging acquisition components, and a defect detection control component. The defect detection control component includes a detection robotic arm controller, an acquisition component controller, and an image processing unit. The detection robotic arm controller is connected to the detection robotic arm, and the acquisition component controller and image processing unit are connected to the paint imaging acquisition components.

[0040] The defect coordinate transfer unit is used to transfer the three-dimensional coordinates of defects detected at the inspection station to the grinding station. It includes two or more 3D positioning cameras for the inspection station, two or more 3D positioning cameras for the grinding station, one set of offline point external parameter calibration device for the paint surface imaging acquisition component, and one set of offline external parameter calibration device for the paint surface imaging acquisition component and the 3D positioning camera of the inspection station.

[0041] The defect polishing unit is used to repair paint defects detected at the inspection station.

[0042] As a further improvement to the system of the present invention: the installation layout and working space of the detection robotic arm correspond one-to-one with the grinding robotic arm, and the installation layout and vehicle body acquisition area of ​​the 3D positioning camera at the detection station correspond one-to-one with the 3D positioning camera at the grinding station.

[0043] As a further improvement to the system of the present invention: In the defect coordinate transfer unit, the offline point external parameter calibration device of the paint surface imaging acquisition component includes a set of two-degree-of-freedom bases and a set of large-size planar calibration plates. The two-degree-of-freedom bases can move up and down and rotate on a single axis, and the large-size calibration plates are composed of thousands of calibration blocks with different IDs.

[0044] As a further improvement to the system of the present invention: in the defect coordinate transfer unit, the paint surface imaging acquisition component and the offline calibration device for the 3D positioning camera of the inspection station include an angle-adjustable bracket and a double-sided calibration plate, which can be adapted to different camera field-of-view layouts.

[0045] Compared with the prior art, the advantages of the present invention are as follows:

[0046] 1. The present invention provides a method and system for detecting and polishing automotive paint defects. It has a simple structure, wide applicability, high level of intelligence, and high detection and polishing efficiency. The present invention adopts a visual positioning scheme, which can greatly improve the accuracy of defect coordinate transmission. The visual calibration accuracy is much higher than the physical positioning accuracy.

[0047] 2. The automotive paint defect detection and polishing method and system of the present invention adopts a distributed strategy in terms of structure, that is, the 3D positioning camera of the detection station and the 3D positioning camera of the polishing station are in one-to-one correspondence, which avoids the errors of global calibration of offline acquisition points in the detection station and global calibration of multiple polishing robotic arms in the polishing station. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the system structure in a specific application example of the present invention.

[0049] Figure 2 This is a flowchart illustrating the method of the present invention in a specific application example.

[0050] Figure 3 This is a schematic diagram illustrating the principle of the method in a specific application example of the present invention.

[0051] Figure 4 This is a schematic diagram of an offline point external parameter calibration device for a paint surface imaging acquisition component in a specific application example.

[0052] Figure 5 This is a flowchart of the offline point extrinsic parameter calibration method for the paint surface imaging acquisition component in a specific application example.

[0053] Figure 6 This is an example diagram of offline point extrinsic parameter calibration for the paint surface imaging acquisition component in a specific application instance.

[0054] Figure 7 This is a schematic diagram of the paint surface imaging acquisition component and the offline calibration device for the extrinsic parameters of the 3D positioning camera at the inspection station in a specific application example.

[0055] Figure 8 This is a flowchart illustrating the offline calibration method for the extrinsic parameters of the paint surface imaging acquisition component and the 3D positioning camera at the inspection station in a specific application example.

[0056] Legend:

[0057] 1. Defect detection station; 2. Defect grinding station; 3. Inspection robotic arm; 4. Paint surface imaging acquisition component; 5. Defect detection control component; 6. 3D positioning camera for inspection station; 7. Vehicle body to be inspected; 8. Grinding robotic arm; 9. Force-controlled grinding head; 10. Grinding control component; 11. 3D positioning camera for grinding station; 12. Two-degree-of-freedom calibration plate base; 13. Large-size calibration plate; 14. Angle-adjustable calibration plate bracket; 15. Double-sided calibration plate. Detailed Implementation

[0058] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] In the description of this application, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0060] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0061] In this application, unless otherwise expressly specified and limited, the terms "assembly," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0062] See Figures 1-8 As shown, this invention is mainly applied to an intelligent system integrating paint defect detection and polishing, dividing the entire work area into a defect detection station 1 and a defect polishing station 2, with the vehicle body 7 to be inspected entering both stations; the system includes:

[0063] The defect detection unit is used to detect and locate defects in the vehicle body paint. It includes two or more detection robotic arms 3, two or more paint imaging acquisition components 4, and a defect detection control component 5. The defect detection control component 5 includes a detection robotic arm controller, an acquisition component controller, and an image processing unit. The detection robotic arm controller is connected to the detection robotic arm 3, and the acquisition component controller and the image processing unit are connected to the paint imaging acquisition components.

[0064] The defect coordinate transfer unit is used to transfer the three-dimensional coordinates of the defects detected at the inspection station to the grinding station. It includes two or more sets of 3D positioning cameras 6 at the inspection station, two or more sets of 3D positioning cameras 11 at the grinding station, one set of offline point external parameter calibration device for paint surface imaging acquisition component, and one set of offline external parameter calibration device for paint surface imaging acquisition component and inspection station 3D positioning camera.

[0065] The defect sanding unit is used to repair paint defects detected at the inspection station. It includes two or more sanding robotic arms 8, two or more force-controlled sanding heads 9, and a sanding control component 10. The sanding control component includes a sanding robotic arm controller and a force-controlled sanding head controller. The sanding robotic arms are connected to the sanding robotic arm controller, and the force-controlled sanding heads are installed at the ends of the sanding robotic arms and connected to the force-controlled sanding head controller.

[0066] In this invention, the installation layout and working space of the detection robotic arm correspond one-to-one with the grinding robotic arm, and the installation layout and vehicle body acquisition area of ​​the 3D positioning camera at the detection station correspond one-to-one with the 3D positioning camera at the grinding station.

[0067] In a specific application example, the defect coordinate transfer unit includes an offline point external parameter calibration device for the paint surface imaging acquisition component, which includes a two-degree-of-freedom base 12 and a large-size planar calibration plate 13. The two-degree-of-freedom base 12 can move up and down and rotate on a single axis, and the large-size calibration plate 13 is composed of thousands of calibration blocks with different IDs.

[0068] Furthermore, in the defect coordinate transfer unit, the paint surface imaging acquisition component and the offline calibration device for the 3D positioning camera at the inspection station include an angle-adjustable bracket 14 and a double-sided calibration plate 15, which can be adapted to different camera field-of-view layouts.

[0069] Furthermore, in specific application examples, the defect attributes transmitted in the coordinate transmission unit of the present invention include defect size, defect type, spatial location of defect center point, and defect surface normal vector.

[0070] like Figure 1 and Figure 2 As shown, the present invention provides a method for detecting and polishing defects in automotive paint, the process of which includes:

[0071] Offline processing: Plan offline data acquisition points for the inspection station and plan the trajectories of multiple inspection robotic arms; calibrate offline data acquisition points for the inspection station and perform offline calibration of the extrinsic parameters of the paint surface imaging acquisition component and the 3D positioning camera for the inspection station; perform hand-eye calibration of the grinding robotic arm and the 3D positioning camera for the grinding station.

[0072] Online processing: First, paint surface defects are detected to obtain defect attributes and defect location; then, the coordinates of the defect location are transmitted to the automatic sanding component, which then sands the paint surface defects.

[0073] In a specific application example, the offline data acquisition point planning for the detection station includes:

[0074] Coarse planning of the simulation environment: Import the actual vehicle CAD model into the digital twin simulation system, specify the area to be inspected on the vehicle body, and the simulation system automatically plans multiple acquisition points based on the vehicle body surface shape and the mirrored field of view of the imaging components. Since there is a certain degree of error between the simulation and reality, fine planning is required in the actual scene.

[0075] Real-world scenario planning; its process may include:

[0076] a) Import the coarsely planned points of the simulation scene into the actual scene, and detect the robotic arm equipped with the imaging acquisition component to collect point cloud data of the car body paint at each coarsely planned point;

[0077] b) All acquired point cloud data are stitched together based on the coarse-planned point extrinsic parameters. The fine-planned point extrinsic parameters for the actual scenario are calculated by detecting the end pose of the robotic arm and the hand-eye calibration results.

[0078] c) Conduct location assessment based on vehicle surface coverage and imaging quality;

[0079] d) Fine-tuning the test sites: This involves evaluating the test sites according to the steps described above, and repeating the fine-tuning and evaluation process until the actual testing requirements are met.

[0080] After the offline data collection points are planned, the trajectories of multiple robotic arms need to be planned to achieve two requirements: first, that the robotic arms do not interfere with each other or with the vehicle body, i.e., that no collisions occur; second, that the trajectories are optimal (with the shortest time) to meet the cycle time requirements. Therefore, in a specific application example, the process of planning the trajectories of the multiple robotic arms can include:

[0081] Coarse planning of the simulation environment: Offline planning points are loaded into the digital twin simulation system. Based on these offline points, the system automatically plans the interference area and sets safety points and path points between interpolation points to complete the trajectory planning of multiple robotic arms. Since there are certain differences between the simulation environment and the actual scene, fine planning for the actual scene is required.

[0082] Fine-tuning for real-world scenarios: Import the simulation environment to coarsely plan the trajectory, and then fine-tune the trajectory based on actual interference and operational conditions.

[0083] After the offline data acquisition points are planned, to ensure the accuracy of the point extrinsic parameters, this invention further employs visual calibration. In a specific application example, the process for calibrating the offline data acquisition points at the detection station may include:

[0084] 1) Divide the planned points within the jurisdiction of each robotic inspection arm into zones, and then, as follows: Figure 4 As shown, the position of the base 12 and the height and angle of the calibration plate 13 are adjusted according to the distribution of the points so that as many points as possible can cover the calibration plate 13.

[0085] 2) Add multiple auxiliary points in the middle of the data collection points to increase the common viewing area of ​​adjacent points and ensure the accuracy of calibration;

[0086] 3) Calibrate the camera intrinsic parameters of the paint surface imaging acquisition component based on the checkerboard calibration board;

[0087] 4) The robotic arm, equipped with a paint surface imaging acquisition component, acquires two-dimensional images of the calibration plate 13 at the acquisition points and auxiliary points;

[0088] 5) Extract features from the calibration board images acquired in step 4) and detect the ID and center point information of each calibration block;

[0089] 6) Based on the calibration block ID information detected in step 5), feature matching is performed on the calibration image of each point and the calibration images of other points, that is, calibration blocks with the same ID are used as a set of matching features.

[0090] 7) Based on the camera intrinsic parameters in step 3) and the feature matching information in step 6), the motion reconstruction structure algorithm is used to simultaneously estimate the three-dimensional coordinates of the feature points on the calibration board and the pose information of each point.

[0091] 8) Based on the estimated 3D coordinates of the feature points in step 7) and the prior size information of the calibration plate 13, estimate the global scale information of the scene;

[0092] 9) Based on the phase pose of each point estimated in step 7) and the global scale information estimated in step 8), calculate the relative pose of each acquisition point relative to the reference point.

[0093] 10) such as Figure 6As shown, if the current calibration board position A cannot be covered by all points, the calibration board needs to be moved to position B and the connecting points can cover both calibration board position A and calibration board position B at the same time. Repeat steps 4)-9) to calculate the extrinsic parameters between the remaining points. Based on the weighted average of multiple connecting points, the extrinsic parameters of the remaining points are converted to the reference point.

[0094] After completing the extrinsic parameter calibration of the offline acquisition points, the coordinates of the defects detected at each point can be transformed to the coordinate system of the reference point. In order to transform the defect coordinates to the coordinate system of the 3D positioning camera at the inspection station, it is necessary to calibrate the extrinsic parameters of the paint surface imaging acquisition component and the 3D positioning camera at the inspection station offline at the reference point. In a specific application example, the process may include the following:

[0095] 1) such as Figure 7 As shown, the angle of the double-sided calibration plate 14 is adjusted according to the common field of view of the 3D positioning camera 6 at the inspection station and the paint surface imaging acquisition component 4 at the reference point.

[0096] 2) Move the double-sided calibration plate 14 N times, and the 3D positioning camera 6 and paint surface imaging acquisition component 4 at the inspection station simultaneously acquire N images of the calibration plate;

[0097] 3) Perform corner detection on the calibration board image acquired in step 2);

[0098] 4) Based on the corner points detected in step 3), calculate the relative pose of the single-sided calibration plate with respect to the acquisition component 4 or the 3D positioning camera 6;

[0099] 5) The geometry of the double-sided calibration plate 14 is estimated based on the bundle adjustment method and graph optimization.

[0100] 6) Estimate the extrinsic parameters of the 3D positioning camera 6 at the detection station and the paint surface imaging acquisition component 4 at the reference point based on the results of steps 4) and 5).

[0101] In a specific application example, the process of eye-to-eye calibration of the 3D positioning camera at the grinding station and the grinding robot arm may include: fixing the calibration plate to the end of the grinding robot arm, moving the grinding robot arm multiple times so that the 3D positioning camera at the grinding station can observe the calibration plate, recording the rectangular coordinates of the grinding robot arm and the corresponding image of the calibration plate, and calibrating the extrinsic parameters of the 3D positioning camera at the grinding station relative to the base coordinate system of the grinding robot arm based on the eye-to-hand external calibration algorithm.

[0102] In a specific application example, when inspecting paint defects, after the vehicle arrives at the inspection station, several (e.g., four) inspection robotic arms take pictures and collect data according to the offline planned collection points. The paint inspection method is used to simultaneously detect and locate defects, and output the attributes of the defects, including defect type, size, three-dimensional coordinates of the center point, and normal of the defect's neighborhood surface.

[0103] In specific application examples, the process for transferring the coordinates of defect locations may include:

[0104] Once the vehicle arrives at the inspection station, the 3D positioning camera at the inspection station simultaneously collects local point clouds of the vehicle body for subsequent vehicle body pose correction.

[0105] After the vehicle body inspection is completed, the coordinate system C of each data collection point will be adjusted. i The coordinates of the defects detected are obtained through the external parameters of the offline calibrated points. Transform to reference point coordinate system C r Down;

[0106] The defect coordinates in the reference point coordinate system are obtained through offline calibrated external parameters. Transform to the 3D positioning camera coordinate system C of the inspection station d Down;

[0107] Once the vehicle body reaches the polishing station, the 3D positioning camera at the polishing station acquires a local point cloud of the vehicle body. This point cloud is then online registered with the point cloud acquired at the inspection station to obtain the vehicle body's alignment pose for correction. Then, the defect coordinates in the coordinate system of the 3D positioning camera at the inspection station are used to correct the vehicle body's posture. Switch to the 3D positioning camera coordinate system C at the polishing station p Down;

[0108] The defect coordinates in the 3D positioning camera coordinate system of the grinding station are calibrated offline using hand-eye calibration parameters. Transform to the base coordinate system O of the grinding robot arm p Down.

[0109] In a specific application example, after the car body arrives at the grinding station and the defect coordinates are transferred, multiple grinding robotic arms begin online task planning, path planning, and trajectory planning, and then work together to grind the defects.

[0110] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting and polishing defects in automotive paint, characterized in that, include: Offline processing: Plan the offline data collection points for the inspection station and plan the trajectories of multiple inspection robotic arms; Offline calibration of data acquisition points at the inspection station was performed, and the external parameters of the paint surface imaging acquisition component and the 3D positioning camera at the inspection station were calibrated offline. Hand-eye calibration of the grinding robot arm and the 3D positioning camera at the grinding station was also performed. Online processing: Detect paint defects to obtain defect attributes and location; transmit the three-dimensional coordinates and normals of the defects to the automatic sanding component via visual positioning, and the automatic sanding component will then sand the paint defects. The offline data acquisition point planning for the testing station includes: Coarse planning of the simulation environment: Import the actual vehicle CAD model into the digital twin simulation system, specify the area of ​​the vehicle to be inspected, and the simulation system automatically plans multiple acquisition points based on the vehicle surface shape and the mirror field of view of the imaging components; Fine-tuning of the actual scene: The coarse-planned points in the simulation scene are imported into the actual scene. The robotic arm equipped with an imaging acquisition component collects point cloud data of the vehicle paint surface at each coarse-planned point. All the acquired point cloud data are stitched together based on the extrinsic parameters of the coarse-planned points. The extrinsic parameters are obtained by calculating the end pose of the robotic arm and hand-eye calibration. The points are evaluated based on the vehicle surface coverage and imaging quality. The points are fine-tuned. The point evaluation is performed according to the above steps. The point fine-tuning and evaluation process is repeated until the actual detection requirements are met. The steps for offline data collection point calibration at the detection station include: Step (1): Perform internal parameter calibration on the imaging acquisition component using a checkerboard calibration board; Step (2): Keeping the large-size calibration plate stationary, the robotic arm carrying the imaging acquisition component moves sequentially to N offline planned points to acquire two-dimensional image data of the large-size calibration plate; Step (3): Extract features from the N collected calibration board images and detect the ID and center point information of each calibration block; Step (4): Based on the calibration block ID information detected in step (3), perform feature matching on the calibration image of each point with other calibration images, that is, calibration blocks with the same ID are used as a set of matching features. Step (5): Based on the camera intrinsic parameters in step (1) and the feature matching information in step (4), the motion recovery structure algorithm is used to simultaneously estimate the three-dimensional coordinates of the feature points of the calibration board and the relative pose of each point. Step (6): Based on the estimated 3D coordinates of the feature points and the prior size information of the calibration board in step (5), estimate the global scale information of the scene; Step (7): Based on the relative poses of each point estimated in step (5) and the global scale information estimated in step (6), calculate the pose of each acquisition point relative to the reference point. The process of offline calibration of the extrinsic parameters of the paint surface imaging acquisition component and the 3D positioning camera at the inspection station includes: Step (10): Adjust the angle of the double-sided calibration plate according to the common field of view of the acquisition component and the 3D positioning camera under the reference point of the detection station; Step (20): Move the double-sided calibration plate N times, and the acquisition component and 3D positioning camera simultaneously acquire N images of the calibration plate; Step (30): Corner detection of calibration board image; Step (40): Calculate the relative pose of the single-sided calibration plate with respect to the acquisition component or the 3D positioning camera; Step (50): Estimate the geometry of the double-sided calibration plate based on bundle adjustment and graph optimization; Step (60): Calculate the extrinsic parameters of the paint surface imaging acquisition component and the 3D positioning camera.

2. The method for detecting and polishing automotive paint defects according to claim 1, characterized in that, It also includes hand-eye calibration of the 3D positioning camera at the grinding station and the grinding robot arm. The process includes: fixing the calibration plate to the end of the grinding robot arm, moving the grinding robot arm multiple times so that the 3D positioning camera at the grinding station can observe the calibration plate, recording the rectangular coordinates of the grinding robot arm and the corresponding image of the calibration plate, and calibrating the external parameters of the 3D positioning camera at the grinding station relative to the coordinate system of the grinding robot arm base based on the eye-to-hand external calibration algorithm.

3. The method for detecting and polishing automotive paint defects according to any one of claims 1 or 2, characterized in that, The process for transferring the coordinates of the defect location includes: Step S101: Set the coordinate system of each collection point The coordinates of the defects detected are obtained through the external parameters of the offline calibrated points. Transform to reference point coordinate system Down; Step S102: Convert the defect coordinates in the reference point coordinate system to the external parameters calibrated offline. Transform to the coordinate system of the 3D positioning camera at the inspection station Down; Step S103: The defect coordinates in the coordinate system of the 3D positioning camera at the inspection station are converted into the vehicle body correction pose calculated online. Switch to the 3D positioning camera coordinate system at the polishing station Down; Step S104: The defect coordinates in the 3D positioning camera coordinate system of the grinding station are calibrated offline using hand-eye calibration parameters. Transform to the coordinate system of the grinding robot arm base Down.

4. The method for detecting and polishing automotive paint defects according to claim 3, characterized in that, During coordinate transfer, the transferred defect attributes include one or more of the following: defect size, defect type, spatial location of defect center point, and defect surface normal vector.

5. A system for detecting and polishing automotive paint defects, employing the method described in any one of claims 1 to 4, characterized in that, include: A defect detection unit is used to detect and locate defects in the vehicle body paint. It includes two or more detection robotic arms, two or more paint imaging acquisition components, and a defect detection control component. The defect detection control component includes a detection robotic arm controller, an acquisition component controller, and an image processing unit. The detection robotic arm controller is connected to the detection robotic arm, and the acquisition component controller and image processing unit are connected to the paint imaging acquisition components. The defect coordinate transfer unit is used to transfer the three-dimensional coordinates of defects detected at the inspection station to the grinding station. It includes two or more 3D positioning cameras for the inspection station, two or more 3D positioning cameras for the grinding station, one set of offline point external parameter calibration device for the paint surface imaging acquisition component, and one set of offline external parameter calibration device for the paint surface imaging acquisition component and the 3D positioning camera of the inspection station. The defect polishing unit is used to repair paint defects detected at the inspection station. In the defect coordinate transfer unit, the offline point external parameter calibration device of the paint surface imaging acquisition component includes a two-degree-of-freedom base and a large-size planar calibration plate. The two-degree-of-freedom base can move up and down and rotate on a single axis, and the large-size calibration plate is composed of thousands of calibration blocks with different IDs.

6. The automotive paint defect detection and polishing system according to claim 5, characterized in that, The installation layout and working space of the inspection robotic arm correspond one-to-one with those of the grinding robotic arm, and the installation layout and vehicle body acquisition area of ​​the 3D positioning camera at the inspection station correspond one-to-one with those of the 3D positioning camera at the grinding station.

7. The automotive paint defect detection and polishing system according to claim 5 or 6, characterized in that, In the defect coordinate transfer unit, the paint surface imaging acquisition component and the offline calibration device for the 3D positioning camera at the inspection station include an angle-adjustable bracket and a double-sided calibration plate, which can be adapted to different camera field-of-view layouts.

Images