A laser cleaning method and laser cleaning robot based on visual guidance

Abstract

The application discloses a kind of laser cleaning method and laser cleaning robot based on visual guidance, the method in the process of cleaning to be cleaned workpiece, obtain the thermal imaging data and laser light intensity signal of to-be-cleaned workpiece, according to the thermal imaging data obtained, it is judged whether the current cleaning area exists the area of temperature exceeding preset range, if there is, according to the thermal imaging data determine the measured temperature value of current cleaning area, according to the laser light intensity signal determine the measured light intensity value of cleaning area, according to preset expected light intensity value and measured light intensity value, calculate light intensity error value, according to preset expected temperature value and measured temperature value, calculate temperature error value, according to light intensity error value and temperature error value get adjusted laser power and light intensity, according to adjusted laser power and light intensity, adjust laser and carry out cleaning. By implementing the present application, laser energy can be accurately and real-time controlled when cleaning to-be-cleaned workpiece.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more particularly to a vision-guided laser cleaning method and a laser cleaning robot. Background Technology

[0002] Currently, existing laser cleaning technologies still require significant manual assistance during the cleaning process, typically employing a "manual teaching-memory reproduction" model. This lack of real-time sensing and adjustment capabilities means that if environmental conditions change, the laser cleaning equipment may not be able to automatically adjust parameters to adapt, leading to unstable cleaning results. Furthermore, existing laser cleaning technologies have limited adaptability to certain complex surfaces, and the equipment may not accurately determine the required cleaning parameters. Consequently, the cleaning robot cannot adjust the laser power and intensity in real-time by sensing the surface condition of the workpiece, resulting in poor cleaning performance. Therefore, improving cleaning efficiency through real-time sensing of the workpiece's surface condition during laser cleaning is a pressing issue that needs to be addressed. Summary of the Invention

[0003] This invention provides a vision-guided laser cleaning method and a laser cleaning robot. The method can accurately and in real time control the laser energy when cleaning the workpiece, thereby automatically adjusting parameters to adapt to changes in environmental conditions and complex surfaces, ensuring consistent cleaning results on the workpiece surface and precise treatment during the cleaning process.

[0004] One embodiment of the present invention provides a vision-guided laser cleaning method, comprising:

[0005] During the cleaning process of the workpiece, thermal imaging data and laser intensity signals of the workpiece are acquired.

[0006] Based on the acquired thermal imaging data, determine whether there are areas in the currently being cleaned where the temperature exceeds the preset range;

[0007] If present, the measured temperature value of the area being cleaned is determined based on the thermal imaging data, and the measured light intensity value of the area being cleaned is determined based on the laser light intensity signal.

[0008] Calculate the light intensity error value based on the preset expected light intensity value and the measured light intensity value; calculate the temperature error value based on the preset expected temperature value and the measured temperature value.

[0009] Based on the light intensity error value and the temperature error value, the adjusted laser power and light intensity are obtained, and the laser is adjusted according to the adjusted laser power and light intensity.

[0010] The adjusted laser is used to clean the area currently being cleaned.

[0011] Furthermore, the vision-guided laser cleaning method further includes:

[0012] Based on the acquired thermal imaging data of the workpiece to be cleaned, it is determined whether there is a secondary cleaning area in the already cleaned area; wherein, the secondary cleaning area includes: an area where the temperature is continuously higher than a preset temperature value for a preset time period and an area where the temperature fluctuation exceeds a preset fluctuation range;

[0013] If present, perform secondary cleaning on the secondary cleaning area.

[0014] Furthermore, the laser cleaning robot includes: a laser cleaning head for performing laser cleaning on the workpiece to be cleaned;

[0015] Before acquiring thermal imaging data and laser intensity signals of the workpiece to be cleaned during the cleaning process, the process includes:

[0016] Obtain a three-dimensional model of the workpiece to be cleaned;

[0017] Based on the three-dimensional model of the workpiece to be cleaned, the surface of the workpiece to be cleaned is divided into several areas to be cleaned.

[0018] Calculate the tangent vector for each region to be cleaned, and calculate the scanning path and the corresponding cleaning path based on the tangent vector information.

[0019] The laser cleaning head is controlled to rotate according to the scanning path and cleaning path, thereby cleaning the workpiece to be cleaned.

[0020] Furthermore, the laser cleaning robot also includes: a 3D scanner and a positioner;

[0021] Before obtaining the three-dimensional model of the workpiece to be cleaned, the process also includes:

[0022] Determine the base coordinate system of the internal space of the laser cleaning robot;

[0023] Based on the known spatial coordinates of the marker points and the base coordinate system, the first coordinate transformation relationship between the laser cleaning robot base and the 3D scanner is confirmed;

[0024] Based on a known plane and the base coordinate system, confirm the second coordinate transformation relationship between the laser cleaning robot base and the laser cleaning head;

[0025] Based on the base coordinate system, confirm the positioner flip axis calibration parameters and rotation axis calibration parameters;

[0026] Based on the base coordinate system, the first coordinate transformation relationship, the second coordinate transformation relationship, the positioner flip axis calibration parameters, and the positioner rotation axis calibration parameters, the coordinate system of the 3D scanner, the coordinate system of the laser cleaning head, and the coordinate system of the positioner are unified with the base coordinate system.

[0027] Furthermore, the 3D scanner includes: a binocular camera;

[0028] The process of obtaining the three-dimensional model of the workpiece to be cleaned includes:

[0029] Based on the pre-modulated structured light pattern, the left image of the workpiece to be cleaned captured by the left eye camera and the right image of the workpiece to be cleaned captured by the right eye camera are obtained in the binocular camera.

[0030] Spatial decoding is performed on the left and right images to obtain the restored left and right images;

[0031] Based on the preset epipolar geometry relationship and stereo vision epipolar constraints, the restored left image and the restored right image are fused to obtain the fused stereo image of the workpiece to be cleaned.

[0032] Based on the camera's calibration parameters, preset system parameters, and the stereoscopic image of the workpiece to be cleaned, the three-dimensional coordinates of the workpiece to be cleaned are calculated, and a three-dimensional model of the workpiece to be cleaned is generated based on the three-dimensional coordinates.

[0033] Further, calculating the tangent vector for each of the regions to be cleaned includes:

[0034] For each region to be cleaned, calculate the curvature tensor of that region;

[0035] Based on the curvature tensor, calculate the eigenvalues ​​corresponding to the curvature tensor and the eigenvectors corresponding to the eigenvalues, and based on the eigenvalues, determine the first principal curvature of the area to be cleaned and the eigenvectors corresponding to the first principal curvature.

[0036] Based on the feature vector, the first principal curvature direction of the area to be cleaned is determined;

[0037] Based on the first principal curvature direction, the tangent vector of the region to be cleaned is obtained.

[0038] Further, the scanning path and the corresponding cleaning path are calculated based on each of the cutting vector information, including:

[0039] Based on the tangent vector information, calculate the rotation angle of the laser cleaning head and the measurement trajectory corresponding to the rotation angle;

[0040] If the measurement trajectory is confirmed to meet the preset conditions, the measurement trajectory is used as the scanning path, and the cleaning path corresponding to the scanning path is determined according to the scanning path.

[0041] The preset conditions include: the measurement trajectory is directly facing the 3D scanner probe, and the overlapping area of ​​the laser beam emitted by the laser cleaning head when it moves along the measurement trajectory is lower than a preset value.

[0042] Another embodiment of the present invention provides a vision-guided laser cleaning robot, comprising:

[0043] The system comprises a sensing module, a laser cleaning head, and a control module; the sensing module includes a PD energy detector and a thermal imaging camera; the control module includes a main controller, a thermal imaging controller, and a light intensity controller.

[0044] The PD energy detector is used to acquire the laser intensity signal of the workpiece to be cleaned and to feed the laser intensity signal back to the intelligent control module.

[0045] The thermal imaging camera is used to acquire thermal imaging data of the workpiece to be cleaned and to feed the thermal imaging data back to the intelligent control module.

[0046] The main controller is used to determine whether there is a region where the temperature exceeds a preset range in the area being cleaned based on the acquired thermal imaging data; if so, it determines the measured temperature value of the area being cleaned based on the thermal imaging data, and determines the measured light intensity value of the area being cleaned based on the laser light intensity signal; it obtains the adjusted laser power and light intensity based on the light intensity error value and the temperature error value, and adjusts the laser based on the adjusted laser power and light intensity.

[0047] The light intensity controller is used to calculate the light intensity error value based on the preset desired light intensity value and the measured light intensity value;

[0048] The thermal imaging controller is used to calculate the temperature error value based on the preset desired temperature value and the measured temperature value;

[0049] The laser cleaning head is used to clean the workpiece to be cleaned according to the adjusted laser.

[0050] Furthermore, the laser cleaning robot also includes: a 3D scanner;

[0051] The 3D scanner is used to scan the workpiece to be cleaned and to construct a 3D model of the workpiece based on the data obtained from the scan.

[0052] Furthermore, the laser cleaning robot also includes: a motion module;

[0053] The motion module includes a robotic arm and a positioner;

[0054] The robotic arm is used to drive the laser cleaning head to move along the cleaning path, so that the laser cleaning head performs laser cleaning according to the cleaning path.

[0055] The positioner is used to rotate until the laser cleaning head can clean according to the cleaning path when the laser cleaning head still cannot clean according to the cleaning path after the robotic arm moves.

[0056] The following benefits can be obtained by implementing the present invention:

[0057] This invention provides a vision-guided laser cleaning method. The method acquires real-time thermal imaging data and laser intensity signals of the area to be cleaned. Based on the acquired thermal imaging data and laser intensity signals, it determines whether there are abnormalities in temperature, laser trajectory, or laser power. When any of these conditions are detected, it calculates the intensity error and temperature error values, and obtains adjusted laser power and intensity based on these values. The laser is then adjusted, and the workpiece is cleaned using the adjusted laser. Therefore, this method can precisely and in real-time control the laser energy when cleaning the workpiece, automatically adjusting parameters to adapt to changes in environmental conditions and complex surfaces, ensuring consistent cleaning results and precise surface treatment during the cleaning process. Attached Figure Description

[0058] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0059] Figure 1 This is a schematic flowchart of a vision-guided laser cleaning method provided in one embodiment of this application;

[0060] Figure 2 This is a schematic diagram of the working system of a vision-guided laser cleaning robot provided in one embodiment of this application;

[0061] Figure 3 This is an internal structural diagram of a laser cleaning head provided in a certain embodiment of this application;

[0062] Figure 4 This is a schematic diagram of a laser cleaning robot provided in one embodiment of this application cleaning a workpiece placed on a plane.

[0063] Figure 5 This is a schematic diagram of a laser cleaning robot provided in one embodiment of this application cleaning a long rod-shaped workpiece.

[0064] Explanation of reference numerals in the attached figures:

[0065] 1. Laser Controller 2. Laser Cleaning Head 3. Laser Output Head 3-1 Collimating Lens 3-2 Beam Splitter 3-3 Focusing Field Lens 3-4 Scanning Galvanometer 3-5 Laser Beam 3-6 Detector Interface 3-7 Beam Expander 3-8 3D Scanner 4. Positioner 5. Sensing Module 6. Robotic Arm 7. Robotic Arm Controller 8. Complex Curved Surface Workpiece to be Cleaned 9. Long Rod Workpiece to be Cleaned 10. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0068] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0069] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0070] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0071] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0072] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0073] See Figure 1 This is a flowchart illustrating a vision-guided laser cleaning method according to an embodiment of the present invention, comprising:

[0074] During the cleaning process of the workpiece, thermal imaging data and laser intensity signals of the workpiece are acquired.

[0075] Based on the acquired thermal imaging data, determine whether there are areas in the currently being cleaned where the temperature exceeds the preset range;

[0076] If present, the measured temperature value of the area being cleaned is determined based on the thermal imaging data, and the measured light intensity value of the area being cleaned is determined based on the laser light intensity signal.

[0077] Calculate the light intensity error value based on the preset expected light intensity value and the measured light intensity value; calculate the temperature error value based on the preset expected temperature value and the measured temperature value.

[0078] Based on the light intensity error value and the temperature error value, the adjusted laser power and light intensity are obtained, and the laser is adjusted according to the adjusted laser power and light intensity.

[0079] The area to be cleaned is cleaned using the adjusted laser.

[0080] Indicatively, the temperature distribution of the area being cleaned is acquired in real time, and it is determined whether there are areas where the temperature exceeds a preset range. The preset temperature range can be adjusted according to the characteristics of different workpieces being cleaned.

[0081] Specifically, the thermal imaging controller and the light intensity controller are disposed in the laser controller 1;

[0082] The laser intensity is adjusted based on the acquired laser intensity signal and thermal imaging data to ensure that the measured intensity I... measured (t) and desired light intensity I desired To minimize the error between them, the adjustment formulas for the laser intensity and power are as follows:

[0083]

[0084] Where P(t) is the output of the light intensity controller in laser controller 1, and e(t) = I desired -I measured (t) is the light intensity error, K p K i and K d These are proportional, integral, and differential gains;

[0085] Specifically, thermal imaging data is used to reflect the temperature distribution of the surface being cleaned, assuming the thermal imaging data is represented as T. measured (t), the desired temperature distribution is T desired The adjustment formula for the thermal imaging data is:

[0086]

[0087] Where Q(t) is the output of the thermal imaging controller in laser controller 1, and e'(t) = T desired -T measured (t) is the temperature error, K' P K' i and K' d , which are the corresponding proportional, integral, and differential gains;

[0088] Next, the outputs of the light intensity and thermal imaging controllers in laser controller 1 are integrated. Specifically, the main controller 2 adds the output of the thermal imaging controller in laser controller 1 to the output of the light intensity controller in laser controller 1 to obtain the final control input for laser power or light intensity.

[0089] P total (T)=P(t)+Q(t);

[0090] In a preferred embodiment, the vision-guided laser cleaning method further includes:

[0091] Based on the acquired thermal imaging data of the workpiece to be cleaned, it is determined whether there is a secondary cleaning area in the already cleaned area; wherein, the secondary cleaning area includes: an area where the temperature is continuously higher than a preset temperature value for a preset time period and an area where the temperature fluctuation exceeds a preset fluctuation range;

[0092] If present, perform secondary cleaning on the secondary cleaning area;

[0093] Indicatively, when a region is found to have a temperature that is consistently higher than a preset temperature value or a temperature fluctuation that exceeds a preset fluctuation range, the region is cleaned.

[0094] Specifically, if there are residues on the workpiece after cleaning, which may cause the temperature in certain areas to remain high, the presence of residues can be determined by checking whether there are abnormal temperature areas or unusual temperature changes in the thermal imaging image after cleaning. The residues can then be removed to ensure consistent cleaning results.

[0095] In a preferred embodiment, the laser cleaning robot includes: a laser cleaning head 3 for performing laser cleaning on the workpiece to be cleaned;

[0096] Before acquiring thermal imaging data and laser intensity signals of the workpiece to be cleaned during the cleaning process, the process includes:

[0097] Obtain a three-dimensional model of the workpiece to be cleaned;

[0098] Based on the three-dimensional model of the workpiece to be cleaned, the surface of the workpiece to be cleaned is divided into several areas to be cleaned.

[0099] Calculate the tangent vector for each region to be cleaned, and calculate the scanning path and the corresponding cleaning path based on the tangent vector information.

[0100] According to the scanning path and cleaning path, the laser cleaning head 3 is controlled to rotate, thereby cleaning the workpiece to be cleaned;

[0101] Indicatively, before starting work, the laser cleaning robot first acquires a three-dimensional model of the workpiece to be cleaned and divides the surface of the three-dimensional model into several areas to be cleaned. Then, it calculates the tangent vector of each area to be cleaned. After obtaining the tangent vectors of all areas to be cleaned on the workpiece, it calculates the scanning path and the corresponding cleaning path based on the tangent vectors.

[0102] In a preferred embodiment, the laser cleaning robot further includes: a 3D scanner 4 and a positioner 5;

[0103] Before obtaining the three-dimensional model of the workpiece to be cleaned, the process also includes:

[0104] Determine the base coordinate system of the internal space of the laser cleaning robot;

[0105] Based on the known spatial coordinates of the marker points and the base coordinate system, the first coordinate transformation relationship between the laser cleaning robot base and the 3D scanner 4 is confirmed.

[0106] Based on a known plane and the base coordinate system, confirm the second coordinate transformation relationship between the laser cleaning robot base and the laser cleaning head 3;

[0107] Based on the base coordinate system, confirm the calibration parameters of the flip axis and the rotation axis of the positioner 5;

[0108] Based on the base coordinate system, the first coordinate transformation relationship, the second coordinate transformation relationship, the flip axis calibration parameters of the positioner 5, and the rotation axis calibration parameters of the positioner 5, the coordinate systems of the 3D scanner 4, the laser cleaning head 3, and the positioner 5 are unified with the base coordinate system.

[0109] To illustrate, in order to ensure that the system can work accurately and reliably, the 3D scanner 4, laser controller 3 and positioner 5 in the laser cleaning robot are uniformly transformed into the base coordinate system of the laser cleaning robot.

[0110] Indicatively, when determining the base coordinate system of the internal space of the laser cleaning robot, it is necessary to first determine the internal and external parameters of the binocular camera, and then construct a projection matrix based on the internal and external parameters. The base coordinate system of the internal space of the laser cleaning robot is then calculated based on the projection matrix.

[0111] Specifically, let the coordinates (x, y) of the corresponding point on the imaging plane of the 3D scanner's 4 probes be... ij ,y ij )

[0112]

[0113] In the formula: c k (x) is the focal length of the binocular camera; H ,y H ) are the coordinates of the main point;

[0114] Because the lens does not produce ideal perspective imaging and has varying degrees of distortion, distortion correction is necessary.

[0115]

[0116] In the formula: dx represents radial asymmetric distortion; dy represents tangential distortion;

[0117] The calibrated intrinsic parameter system is thus obtained. After the camera calibration is completed, the projection is correlated with the camera to obtain the projection matrix, thereby calculating the base coordinate system of the internal space of the laser cleaning robot.

[0118] Indicatively, when calculating the first coordinate transformation relationship between the laser cleaning robot base and the 3D scanner 4, the robot's end pose data and the spatial coordinate data of the 3D scanner 4 probe are obtained based on the known spatial coordinate markers and the base coordinate system. Based on the end pose data and the spatial coordinate data, the first coordinate transformation relationship between the robot end and the 3D scanner 4 is confirmed.

[0119] Specifically, the calculation process for the first coordinate transformation relationship between the robot end effector and the 3D scanner 4 involves using a set of known spatial coordinate markers, moving the laser cleaning robot to multiple positions, and photographing the corresponding markers for calibration. First, the robot is moved to multiple positions, and the pose of its base is represented as follows: This parameter is read from the laser cleaning robot controller; the spatial coordinates of the 3D scanning probe are represented as follows: This parameter determines the 3D coordinate representation of the 3D scanning probe in space by capturing marker points with known spatial coordinates in the image. Based on the formula AX = XB, the coordinate transformation relationship from the robot's end effector to the 3D scanning probe is calculated.

[0120] Indicatively, based on a known plane and the base coordinate system, several three-dimensional point data of the probe of the three-dimensional scanner 4 measuring the known plane are obtained, and the several three-dimensional point data are fitted to confirm the second coordinate transformation relationship between the robot end effector and the laser cleaning head 3.

[0121] Specifically, the second coordinate transformation relationship between the robot end effector and the laser cleaning head 3 is as follows: the laser cleaning robot base is represented as a sector in space, which is represented as a plane in space: ax + by + cz = 1; using the 3D scanner 4 probe as a measuring device, the coordinate transformation relationship is determined for the surface of the laser cleaning beam in space; an arbitrary plane is placed in space, and a set of n 3D points are measured in space, measuring m different positions to obtain the 3D point representation. The plane in space is obtained by fitting calculation, and then it is transformed into the coordinate transformation from robot end end to laser cleaning end end to obtain the second coordinate transformation relationship between robot end end and laser cleaning head 3.

[0122] Schematic, based on the base coordinate system, the first three-dimensional data of the positioner 5 when scanning a known object at a first angle and the second three-dimensional data when scanning the known object at a second angle are obtained; based on the first angle, the second angle, the first three-dimensional data, the second three-dimensional data and the preset linear constraint, the rotation axis calibration parameters of the positioner 5 are confirmed;

[0123] Specifically, the process of determining the rotation axis calibration parameters of the positioner 5 is as follows: placing the obtained object on the positioner 5, acquiring the first angle θ1 and the first three-dimensional data (x,y,z)1 of the positioner 5, acquiring the second angle θ2 and the second three-dimensional data (x,y,z)2 of the positioner 5, and setting the rotation axis spatial representation. The constraint is that the three-dimensional model rotates around a straight line in space: (x,y,z)2=rot[(x,y,z)2,θ1-θ2,(x0,y0,z0,a,b,c)]. The nonlinear optimization constraint is used to iteratively calculate the calibration parameters of the flip axis of the positioner 5.

[0124] In a preferred embodiment, the 3D scanner 4 includes: a binocular camera;

[0125] The process of obtaining the three-dimensional model of the workpiece to be cleaned includes:

[0126] Based on the pre-modulated structured light pattern, the left image of the workpiece to be cleaned captured by the left eye camera and the right image of the workpiece to be cleaned captured by the right eye camera are obtained in the binocular camera.

[0127] Spatial decoding is performed on the left and right images to obtain the restored left and right images;

[0128] Based on the preset epipolar geometry relationship and stereo vision epipolar constraints, the restored left image and the restored right image are fused to obtain the fused stereo image of the workpiece to be cleaned.

[0129] Based on the camera's calibration parameters, preset system parameters, and the stereoscopic image of the workpiece to be cleaned, the three-dimensional coordinates of the workpiece to be cleaned are calculated, and a three-dimensional model of the workpiece to be cleaned is generated based on the three-dimensional coordinates.

[0130] To illustrate, after transforming the 3D scanner 4, laser cleaning head 3, and positioner 5 into the robot coordinate system, the workpiece to be cleaned needs to be reconstructed in 3D.

[0131] Specifically, during the image acquisition stage, the grating projector in the 3D scanner 4 first pre-modulates the structured light pattern, which helps with the subsequent extraction of 3D information.

[0132] Then, the binocular cameras simultaneously photograph the surface of the workpiece to be cleaned, capturing a highly modulated pattern formed by the structured light projector, obtaining the left image of the workpiece to be cleaned captured by the left eye camera and the right image of the workpiece to be cleaned captured by the right eye camera. Then, spatial decoding is performed on the left image of the workpiece to be cleaned captured by the left eye camera and the right image of the workpiece to be cleaned captured by the right eye camera to restore the original phase information. The decoding process usually includes phase folding, hybrid template construction and phase restoration.

[0133] Next, the epipolar geometry of the binocular camera is used to match and calibrate the two images to improve matching accuracy and ensure accurate correspondence under different viewpoints. In addition, the positional relationship of corresponding pixels in the two images is searched by combining the epipolar constraints of stereo vision. Finally, the spatial three-dimensional coordinates of the workpiece to be cleaned are calculated based on the camera calibration parameters and known system parameters to obtain the three-dimensional model of the workpiece to be cleaned.

[0134] In a preferred embodiment, calculating the tangent vector for each of the regions to be cleaned includes:

[0135] For each region to be cleaned, calculate the curvature tensor of that region;

[0136] Based on the curvature tensor, calculate the eigenvalues ​​corresponding to the curvature tensor and the eigenvectors corresponding to the eigenvalues, and based on the eigenvalues, determine the first principal curvature of the area to be cleaned and the eigenvectors corresponding to the first principal curvature.

[0137] Based on the feature vector, the first principal curvature direction of the area to be cleaned is determined;

[0138] Based on the first principal curvature direction, the tangent vector of the region to be cleaned is obtained;

[0139] Specifically, the process of determining the tangent vector of the area to be cleaned is as follows: the selected surface is divided into H×N small regions according to the preset laser beam width H. Then, the regions are parameterized and represented as two-dimensional parameter spaces, where U represents the horizontal direction and V represents the vertical direction. For the surface parameterized as r(u,v), partial derivatives are calculated, where the formula for calculating the partial derivatives is: The curvature tensor is calculated based on the partial derivatives, where the curvature tensor K is expressed as:

[0140]

[0141] Where · represents the dot product of vectors, r uu r uv r vv These are the corresponding partial derivative vectors;

[0142] Next, the eigenvalues ​​and eigenvectors of the curvature tensor K are calculated by solving the eigenvalue problem |K-λI|=0, where I is the identity matrix and the eigenvalues ​​are the eigenvalues. The eigenvalues ​​λ correspond to the principal curvatures, and the eigenvectors correspond to the principal curvature directions. The eigenvector of the largest eigenvalue is taken as the direction of the maximum principal curvature, and the normal vector of the direction of the maximum principal curvature is denoted as n. The tangent vector is defined as t=n ⊥ , specifically, This yields the tangent vector t.

[0143] In a preferred embodiment, calculating the scanning path and the corresponding cleaning path based on each of the cutting vector information includes:

[0144] Based on the tangent vector information, the rotation angle of the laser cleaning head 3 and the measurement trajectory corresponding to the rotation angle are calculated.

[0145] If the measurement trajectory is confirmed to meet the preset conditions, the measurement trajectory is used as the scanning path, and the cleaning path corresponding to the scanning path is determined according to the scanning path.

[0146] The preset conditions include: the measurement trajectory is directly facing the probe of the 3D scanner 4, and the overlapping area of ​​the laser beam emitted by the laser cleaning head 3 when it moves along the measurement trajectory is lower than a preset value.

[0147] Specifically, after obtaining the tangent vector information of each area to be cleaned, the angle of the rotating laser cleaning head 3 or the positioner 5 is calculated based on the tangent vector, as well as the measurement trajectory corresponding to the angle to be rotated. After obtaining the measurement trajectory, it is determined whether the measurement trajectory can meet the following conditions: (1) the measurement trajectory is facing the probe of the three-dimensional scanner 4; (2) whether the overlapping area of ​​the laser beam emitted by the laser cleaning head 3 when it moves along the measurement trajectory is lower than a preset threshold.

[0148] Furthermore, the ability of the laser cleaning head 3 to adaptively adjust the beam width, beam intensity, and robot walking speed as it moves along the measurement trajectory is used as a condition for determining whether the measurement trajectory is a scanning path.

[0149] See Figure 2 This is a schematic diagram of a vision-guided laser cleaning robot system according to an embodiment of the present invention, including: a laser cleaning head 3, a sensing module 6, and a control module; the sensing module 6 includes a PD energy detector and a thermal imaging camera; the control module includes a main controller 2, a thermal imaging controller, and a light intensity controller;

[0150] The PD energy detector is used to acquire the laser intensity signal of the workpiece to be cleaned and to feed the laser intensity signal back to the intelligent control module.

[0151] The thermal imaging camera is used to acquire thermal imaging data of the workpiece to be cleaned and to feed the thermal imaging data back to the intelligent control module.

[0152] The main controller 2 is used to determine whether there is a region where the temperature exceeds a preset range in the area being cleaned based on the acquired thermal imaging data; if so, it determines the measured temperature value of the area being cleaned based on the thermal imaging data, and determines the measured light intensity value of the area being cleaned based on the laser light intensity signal; it obtains the adjusted laser power and light intensity based on the light intensity error value and the temperature error value, and adjusts the laser based on the adjusted laser power and light intensity.

[0153] The light intensity controller is used to calculate the light intensity error value based on the preset desired light intensity value and the measured light intensity value;

[0154] The thermal imaging controller is used to calculate the temperature error value based on the preset desired temperature value and the measured temperature value;

[0155] The laser cleaning head 3 is used to clean the workpiece to be cleaned according to the adjusted laser.

[0156] Specifically, Figure 3 This is an internal structural diagram of the laser cleaning head 3. The internal optical path of the laser cleaning head 3 is divided into a laser transmission optical path and an energy detection optical path by a beam splitter. The laser transmission optical path includes a laser output head 3-1, a collimating lens 3-2, a beam splitter 3-3, a focusing field lens 3-4, a scanning galvanometer 3-5, and a beam expander 3-8. The laser output head 3-1 is connected to a detector interface 3-7, which is a standard QBH interface. The aperture size and surface structure of the collimating lens 3-2, the beam expander 3-8, and the focusing field lens 3-4 are determined according to the output spot characteristics of the laser beam 3-6. The laser beam 3-6 output from the laser source is introduced into the paint removal head after passing through the detector interface 3-7. It is then reflected sequentially by the beam expander 3-8, the collimating lens 3-2, the beam splitter 3-3, and the scanning galvanometer 3-5. Finally, it is focused by the focusing field lens 3-4 and projected onto the surface of the workpiece to be cleaned. The scanning galvanometer 3-5 deflects the laser beam 3-6, thereby performing laser cleaning operations in a certain area.

[0157] Specifically, the PD energy detection module consists of a PD detector, a dichroic mirror and a reflector, a beam splitter and a focusing lens. The PD energy detection module measures and collects the light intensity of the laser in real time and transmits it to the controller.

[0158] Specifically, the thermal imaging camera captures the surface temperature distribution of the laser-cleaned surface in real time, detects the non-uniformity and abnormal temperature distribution of the laser-cleaned surface, and thus promptly identifies problems such as uneven cleaning or the presence of residues.

[0159] In a preferred embodiment, the laser cleaning robot further includes: a 3D scanner 4;

[0160] The 3D scanner 4 is used to scan the workpiece to be cleaned and to construct a 3D model of the workpiece based on the data obtained from the scan.

[0161] In a preferred embodiment, the laser cleaning robot further includes: a motion module;

[0162] The motion module includes a positioner 5 and a robotic arm 7;

[0163] The robotic arm is used to drive the laser cleaning head 3 to move along the cleaning path, so that the laser cleaning head 3 performs laser cleaning according to the cleaning path.

[0164] The positioner 5 is used to rotate until the laser cleaning head 3 can clean according to the cleaning path when the laser cleaning head 3 still cannot clean according to the cleaning path after the robotic arm 7 moves.

[0165] Indicative Figure 2 The robotic arm controller 8 is used to control the movement of the robotic arm 7. As the specific execution unit of the laser cleaning robot, the robotic arm 7 can drive the laser beam to complete the extension, rotation and lifting movements. The positioner 5 is responsible for compensating for the movement posture limitations of the robotic arm 7. When the posture of the robotic arm 7 is limited and cannot meet the full-area scanning or cleaning, the rotation function of the positioner 5 is used to achieve full-surface scanning and cleaning of the workpiece to be cleaned.

[0166] Specifically, the laser cleaning robot can clean complex curved surface workpieces 9. Figure 4 This is a schematic diagram of the laser cleaning robot cleaning a workpiece 9 with a complex curved surface.

[0167] Similarly, the laser cleaning robot can clean long rod-shaped workpieces 10. Figure 5 This is a schematic diagram of the laser cleaning robot cleaning a long rod-shaped workpiece 10.

[0168] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A vision-guided laser cleaning method, applicable to laser cleaning robots, characterized in that, include: Obtain a 3D model of the workpiece to be cleaned; Based on the three-dimensional model of the workpiece to be cleaned, the surface of the workpiece is divided into several areas to be cleaned. For each region to be cleaned, calculate the curvature tensor of that region; Based on the curvature tensor, calculate the eigenvalues ​​corresponding to the curvature tensor and the eigenvectors corresponding to the eigenvalues, and based on the eigenvalues, determine the first principal curvature of the area to be cleaned and the eigenvectors corresponding to the first principal curvature. Based on the feature vector, the first principal curvature direction of the area to be cleaned is determined; Based on the first principal curvature direction, the tangent vector of the area to be cleaned is obtained, and the scanning path and the cleaning path corresponding to the scanning path are calculated based on the information of each tangent vector; wherein, the laser cleaning robot includes a laser cleaning head for performing laser cleaning on the workpiece to be cleaned. According to the scanning path and cleaning path, the laser cleaning head is controlled to rotate, thereby cleaning the workpiece to be cleaned. During the cleaning process of the workpiece, thermal imaging data and laser intensity signals of the workpiece are acquired. Based on the acquired thermal imaging data, determine whether there are areas in the currently being cleaned where the temperature exceeds the preset range; If present, the measured temperature value of the area being cleaned is determined based on the thermal imaging data, and the measured light intensity value of the area being cleaned is determined based on the laser light intensity signal. Based on the preset expected light intensity value and the measured light intensity value, calculate the light intensity error value; based on the preset expected temperature value and the measured temperature value, calculate the temperature error value. Based on the light intensity error value and the temperature error value, the adjusted laser power and light intensity are obtained, and the laser is adjusted according to the adjusted laser power and light intensity. The adjusted laser is used to clean the area currently being cleaned.

2. The vision-guided laser cleaning method as described in claim 1, characterized in that, Also includes: Based on the acquired thermal imaging data of the workpiece to be cleaned, it is determined whether there is a secondary cleaning area in the already cleaned area; wherein, the secondary cleaning area includes: an area where the temperature is continuously higher than a preset temperature value for a preset time period and an area where the temperature fluctuation exceeds a preset fluctuation range; If present, perform secondary cleaning on the secondary cleaning area.

3. The vision-guided laser cleaning method as described in claim 2, characterized in that, The laser cleaning robot also includes: a 3D scanner and a positioner; Before obtaining the three-dimensional model of the workpiece to be cleaned, the process also includes: Determine the base coordinate system of the internal space of the laser cleaning robot; Based on the known spatial coordinates of the marker points and the base coordinate system, the first coordinate transformation relationship between the laser cleaning robot base and the 3D scanner is confirmed; Based on a known plane and the base coordinate system, confirm the second coordinate transformation relationship between the laser cleaning robot base and the laser cleaning head; Based on the base coordinate system, confirm the positioner flip axis calibration parameters and rotation axis calibration parameters; Based on the base coordinate system, the first coordinate transformation relationship, the second coordinate transformation relationship, the positioner flip axis calibration parameters, and the positioner rotation axis calibration parameters, the coordinate system of the 3D scanner, the coordinate system of the laser cleaning head, and the coordinate system of the positioner are unified with the base coordinate system.

4. The vision-guided laser cleaning method as described in claim 3, characterized in that, The 3D scanner includes: a binocular camera; The process of obtaining the three-dimensional model of the workpiece to be cleaned includes: Based on the pre-modulated structured light pattern, the left image of the workpiece to be cleaned captured by the left eye camera and the right image of the workpiece to be cleaned captured by the right eye camera are obtained in the binocular camera. Spatial decoding is performed on the left and right images to obtain the restored left and right images; Based on the preset epipolar geometry relationship and stereo vision epipolar constraints, the restored left image and the restored right image are fused to obtain the fused stereo image of the workpiece to be cleaned. Based on the camera's calibration parameters, preset system parameters, and the stereoscopic image of the workpiece to be cleaned, the three-dimensional coordinates of the workpiece to be cleaned are calculated, and a three-dimensional model of the workpiece to be cleaned is generated based on the three-dimensional coordinates.

5. The vision-guided laser cleaning method as described in claim 4, characterized in that, The scanning path and the corresponding cleaning path are calculated based on each of the cutting vectors, including: Based on the tangent vector information, calculate the rotation angle of the laser cleaning head and the measurement trajectory corresponding to the rotation angle; If the measurement trajectory is confirmed to meet the preset conditions, the measurement trajectory is used as the scanning path, and the cleaning path corresponding to the scanning path is determined according to the scanning path. The preset conditions include: the measurement trajectory is directly facing the 3D scanner probe, and the overlapping area of ​​the laser beam emitted by the laser cleaning head when it moves along the measurement trajectory is lower than a preset value.

6. A vision-guided laser cleaning robot, characterized in that, The laser cleaning robot performs a vision-guided laser cleaning method as described in any one of claims 1 to 5; the laser cleaning robot includes: a sensing module, a laser cleaning head, and a control module; the sensing module includes a PD energy detector and a thermal imaging camera; the control module includes a main controller, a thermal imaging controller, and a light intensity controller; The PD energy detector is used to acquire the laser intensity signal of the workpiece to be cleaned and to feed the laser intensity signal back to the control module; The thermal imaging camera is used to acquire thermal imaging data of the workpiece to be cleaned and to feed the thermal imaging data back to the control module; The main controller is used to determine whether there is a region where the temperature exceeds a preset range in the area being cleaned based on the acquired thermal imaging data; if so, it determines the measured temperature value of the area being cleaned based on the thermal imaging data, and determines the measured light intensity value of the area being cleaned based on the laser light intensity signal; it obtains the adjusted laser power and light intensity based on the light intensity error value and the temperature error value, and adjusts the laser based on the adjusted laser power and light intensity. The light intensity controller is used to calculate the light intensity error value based on the preset desired light intensity value and the measured light intensity value; The thermal imaging controller is used to calculate the temperature error value based on the preset desired temperature value and the measured temperature value; The laser cleaning head is used to clean the workpiece to be cleaned according to the adjusted laser.

7. The vision-guided laser cleaning robot as described in claim 6, characterized in that, The laser cleaning robot also includes: a 3D scanner; The 3D scanner is used to scan the workpiece to be cleaned and to construct a 3D model of the workpiece based on the data obtained from the scan.

8. The vision-guided laser cleaning robot as described in claim 6, characterized in that, The laser cleaning robot also includes: a motion module; The motion module includes a robotic arm and a positioner; The robotic arm is used to drive the laser cleaning head to move along the cleaning path, so that the laser cleaning head performs laser cleaning according to the cleaning path. The positioner is used to rotate until the laser cleaning head can clean according to the cleaning path when the laser cleaning head still cannot clean according to the cleaning path after the robotic arm moves.

Images