Welding seam grinding method, system and device based on laser vision stirring friction

Abstract

The invention relates to the technical field of automatic welding seam grinding, in particular to a welding seam grinding method, system and device based on laser vision stirring friction, and the method comprises the steps that a laser image of a welding seam is obtained, gray threshold segmentation is conducted on the laser image, and a laser stripe image is obtained; extracting sub-pixel feature points of the laser stripe image to obtain a laser stripe center line, and converting a two-dimensional point set of the laser stripe center line into a three-dimensional space to form a laser stripe three-dimensional point cloud; performing feature extraction on the laser stripe three-dimensional point cloud to obtain feature data of a flash area; and based on the feature data of the flash area, a polishing path is planned, and the flash area of the welding seam is polished and machined along the polishing path through a polishing tool. According to the invention, automatic identification and accurate polishing of the weld flash are realized, the polishing efficiency and quality are improved, and manual intervention is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of automated weld grinding, and in particular to a laser vision-based friction stir weld grinding method, system and device. Background Art

[0002] Friction stir welding (FSW) technology, as an advanced solid-phase joining process, is based on the use of high-speed rotating stirring tools (stirring heads) to input friction heat at the joint interface of the workpieces to be welded, causing the material to soften locally to a thermoplastic state. As the stirring head moves along the preset welding track, the softened material is effectively stirred and migrates backward, and then solidifies and combines during the cooling process to form a high-quality welded joint. During this process, the stirring head applies an appropriate amount of axial pressure to ensure that the stirring action penetrates deep into the joint, while controlling the residual stress and deformation to a low level, making it suitable for high-end manufacturing fields such as aviation, aerospace, shipbuilding and automobiles that have strict requirements on welding quality.

[0003] However, if the axial pressure is too large or the welding parameters are improperly configured during the FSW process, the thermoplastic metal may not be fully migrated to the center of the weld and fully integrated with the base material, but may be partially squeezed out around the stirring head, forming a flash defect after cooling. The existence of flash not only affects the appearance of the product, but may also have an adverse effect on the mechanical properties and sealing of the product. Therefore, in the FSW quality control system, the complete removal of flash is regarded as one of the key links, which needs to be achieved through subsequent grinding.

[0004] Currently, manual grinding and robot teaching grinding are the two main methods for removing FSW weld flash. Manual grinding relies on the operator's experience and skills, and uses handheld grinding tools to perform fine work on the weld area, but the dust pollution and noise hazards associated with this process cannot be ignored, and long-term repetitive labor can easily lead to fluctuations in grinding quality and low efficiency, making it difficult to meet the needs of large-scale production. Although robot teaching grinding can improve operation accuracy and efficiency, it has high technical requirements for operators, and still requires personnel to intervene in the grinding environment for programming and monitoring, and fails to completely avoid the potential health threats of harsh working environments.

[0005] In addition, the geometric complexity of the workpiece itself (such as curvature, clamping deformation), the diversity of flash shapes, and the high reflective properties of the aluminum alloy weld surface together constitute a severe challenge to visual inspection and recognition technology, increasing the complexity of the grinding path planning algorithm. In order to achieve high-quality FSW weld flash grinding, it is necessary to ensure the high accuracy of the grinding path planning, and the grinding process needs to be finely controlled to avoid damage to the base material caused by over-grinding and to ensure that the weld transition area is smooth and defect-free. This places extremely high demands on the intelligence and automation level of the entire grinding system. Summary of the invention

[0006] To this end, the technical problem to be solved by the present invention is to overcome the significant limitations of traditional manual and robot teaching polishing methods in the prior art in terms of efficiency, quality control and intelligence level, and the problem that it is difficult to meet high-quality and high-efficiency production needs.

[0007] In order to solve the above technical problems, the present invention provides a method, system and device for grinding friction stir welds based on laser vision, and the method comprises the following steps:

[0008] S1: Acquire a laser image of the weld, perform grayscale threshold segmentation on the laser image, and obtain a laser stripe image;

[0009] S2: extracting sub-pixel feature points of the laser stripe image to obtain the center line of the laser stripe, and converting the two-dimensional point set of the center line of the laser stripe into three-dimensional space to form a three-dimensional point cloud of the laser stripe;

[0010] S3: extracting features from the three-dimensional point cloud of the laser stripes to obtain feature data of the flash area;

[0011] S4: planning a grinding path based on the characteristic data of the flash area, and grinding the flash area of ​​the weld along the grinding path using a grinding tool.

[0012] In one embodiment of the present invention, in S2, the method for obtaining the center line of the laser stripe is:

[0013] S21: De-noising the laser stripe image and enhancing its contrast to obtain a pre-processed laser stripe image;

[0014] S22: Calculating the second-order derivative matrix of each pixel based on the preprocessed laser stripe image;

[0015] S23: extracting feature points of the preprocessed laser stripe image by calculating the determinant and trace of the second-order derivative matrix;

[0016] S24: performing a Taylor series expansion on the feature point using the grayscale value of the pixel point to obtain a Taylor series expansion, and obtaining the coordinates of the sub-pixel feature point by solving an equation whose derivative of the Taylor series expansion is equal to zero;

[0017] S25: Connect the detected sub-pixel feature points to form a laser stripe center line.

[0018] In one embodiment of the present invention, the expression of the second-order derivative matrix H is:

[0019]

[0020] Among them, I xx ,I yy are the second-order derivatives of the pixel in the x and y directions, I xy =I yx is a mixed second-order derivative.

[0021] In one embodiment of the present invention, in S2, a method for forming a three-dimensional point cloud of laser stripes is as follows:

[0022] S211: Obtain the two-dimensional sub-pixel precision coordinates of the center line of the laser stripe, and convert the two-dimensional sub-pixel precision coordinates into three-dimensional coordinates in the camera coordinate system through the camera intrinsic parameter matrix and the laser plane position obtained by hand-eye calibration;

[0023] S212: using the hand-eye conversion matrix, converting the three-dimensional coordinates in the camera coordinate system into the three-dimensional coordinates in the robot tool coordinate system;

[0024] S213: Gather the three-dimensional coordinate points in all robot tool coordinate systems to obtain a three-dimensional point cloud of laser stripes.

[0025] In one embodiment of the present invention, in S211, the method of converting the two-dimensional sub-pixel precision coordinates into three-dimensional coordinates in the camera coordinate system is as follows:

[0026] Converts a two-dimensional sub-pixel precision coordinate (u, v) to a pixel coordinate (u, v, 1) in homogeneous coordinate form T ;

[0027] Using the camera intrinsic matrix, the pixel coordinates (u, v, 1) in homogeneous coordinate form are converted to T Convert to three-dimensional coordinates (x, y, 1) in the camera coordinate system T , the conversion formula is:

[0028] (x,y,1) T =K -1 (u,v,1) T

[0029] Among them, K is the camera intrinsic parameter matrix, and its expression is:

[0030]

[0031] Among them, (f x ,f y ) is the focal length of the camera, f x and f y Corresponding to the focal length of the pixel point in the horizontal and vertical directions of the image, (c x ,c y ) is the optical center of the camera, c x and cy They correspond to the coordinates of the optical center in the horizontal and vertical directions respectively.

[0032] In one embodiment of the present invention, in S212, the method of converting the three-dimensional coordinates in the camera coordinate system into the three-dimensional coordinates in the robot tool coordinate system is as follows:

[0033] According to the camera's depth value z, the three-dimensional coordinates (x, y, 1) in the camera coordinate system T The x- and y-axis coordinates are transformed to obtain the transformed three-dimensional coordinates

[0034] The transformed three-dimensional coordinate X is stretched into the robot tool coordinate system to obtain the three-dimensional coordinate X' in the robot tool coordinate system:

[0035] X'=T RC ·X

[0036] Among them, T RC is the hand-eye conversion matrix, and its expression is:

[0037]

[0038] Among them, R is the rotation matrix and t is the translation vector.

[0039] In one embodiment of the present invention, in S3, the method for obtaining the characteristic data of the flash area is as follows:

[0040] S31: performing voxel filtering on the laser stripe three-dimensional point cloud, and performing cluster segmentation on the point cloud data after voxel filtering to obtain down-sampled point cloud data;

[0041] S32: Based on the downsampled point cloud data, obtain the neighborhood of each sample point, and use the coordinates of all data points in the neighborhood to construct a covariance matrix;

[0042] S33: performing principal component analysis on the covariance matrix to obtain three eigenvectors and corresponding eigenvalues, and taking the eigenvector corresponding to the minimum eigenvalue as the normal vector of the data point;

[0043] S34: Calculating the curvature value of the corresponding sample point based on the normal vectors of all data points in the neighborhood, thereby obtaining the curvature values ​​of all sample points in the downsampled point cloud data;

[0044] S35: Based on the curvature values ​​of all sample points, the sample points whose curvature values ​​are higher than the threshold are used as feature data of the flash feature area.

[0045] In one embodiment of the present invention, in S4, the method for planning the grinding path is as follows:

[0046] S41: De-noising the feature data of the flash area in sections to obtain de-noised feature data, and performing fitting processing on the de-noised feature data to obtain a fitting straight line;

[0047] S42: looping through each point in the denoised feature data, calculating the foot of the perpendicular to the fitting line, and establishing a first KD-Tree structure for all flash area point clouds;

[0048] S43: for each perpendicular foot, use the first KD-Tree structure to search for the flying edge point clouds in its neighborhood, calculate the bounding box volume center and the centroid of these flying edge point clouds, and calculate the weighted center point according to the given weight, and the weighted center point will be used as the flying edge correction center point of the neighborhood;

[0049] S44: constructing a second KD-Tree structure for all parent material point clouds, and for each flash correction center point, searching for point clouds in its neighborhood through the second KD-Tree structure to obtain parent material neighborhood point clouds;

[0050] S45: fitting a base material plane using the base material neighborhood point cloud, projecting the flash correction center point onto the base material plane, the obtained projection point is a point on the grinding path, and the set of all projection points constitutes the grinding path.

[0051] Based on the same inventive concept, the present invention also provides a laser vision-based friction stir weld grinding system, which is used to implement the laser vision-based friction stir weld grinding method, including the following modules:

[0052] A weld image acquisition module, which is used to acquire a laser image of the weld, and perform grayscale threshold segmentation on the laser image to obtain a laser stripe image;

[0053] A point cloud data generation module, the point cloud data generation module is used to extract sub-pixel feature points of the laser stripe image to obtain the center line of the laser stripe, and convert the two-dimensional point set of the center line of the laser stripe into a three-dimensional space to form a three-dimensional point cloud of the laser stripe;

[0054] A flash feature extraction module, which is used to extract features from the three-dimensional point cloud of the laser stripes to obtain feature data of the flash area;

[0055] A weld grinding processing module is used to plan a grinding path based on the characteristic data of the flash area, and grind the flash area of ​​the weld along the grinding path.

[0056] The present invention also provides a laser vision-based friction stir weld grinding device, which includes a grinding tool, a line laser sensor and a control unit. The line laser sensor is installed on the grinding tool. The control unit controls the connection between the grinding tool and the line laser sensor, and the laser vision-based friction stir weld grinding system is stored in the control unit. The control unit obtains a laser image of the weld through the line laser sensor, and uses the grinding tool to grind the flash area of ​​the weld.

[0057] The above technical solution of the present invention has the following advantages compared with the prior art:

[0058] The present invention proposes an innovative method, system and device for automatic grinding of the flash of stir friction welding welds, which integrates technologies such as high-precision visual recognition, three-dimensional reconstruction, point cloud processing, feature extraction and precise positioning planning of the grinding path. The line laser sensor completed by hand-eye calibration captures the weld image, and pre-processes it to eliminate the interference of reflections, and then extracts the center of the laser stripe to achieve three-dimensional reconstruction of the weld. In the three-dimensional point cloud processing stage, voxel filtering and clustering segmentation techniques are used to remove noise, and the flash feature area is accurately located by curvature analysis. Based on these features, the grinding path can be intelligently planned, and three grinding processes are designed to ensure a smooth transition between the weld and the base material. Finally, a hydraulically controlled floating grinding tool is used to quickly and evenly grind the weld surface under constant force or constant position control, effectively remove the flash, and improve the welding quality. In summary, the present invention not only solves the problems of traditional grinding methods in terms of efficiency, accuracy and health risks, but also demonstrates a high level of automation and intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0060] Figure 1 It is a flow chart of a method for grinding a friction stir weld based on laser vision described in the first embodiment of the present invention;

[0061] Figure 2 is a flow chart of the method for obtaining the center line of the laser stripe described in the first embodiment of the present invention;

[0062] Figure 3 is a flow chart of the method for forming a three-dimensional point cloud of laser stripes described in Embodiment 1 of the present invention;

[0063] Figure 4 is a flow chart of the method for obtaining characteristic data of the flash area described in the first embodiment of the present invention;

[0064] Figure 5is a flow chart of the method for planning a grinding path described in the first embodiment of the present invention;

[0065] Figure 6 It is a structural schematic diagram of a laser vision-based friction stir weld grinding system described in the second embodiment of the present invention;

[0066] Figure 7 It is a structural schematic diagram of a laser vision-based friction stir weld grinding device described in Embodiment 3 of the present invention;

[0067] Description of the accompanying drawings: 10, weld image acquisition module; 20, point cloud data generation module; 30, flash feature extraction module; 40, weld grinding processing module;

[0068] 100. grinding tool; 200. line laser sensor; 300. control unit. DETAILED DESCRIPTION

[0069] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0070] Embodiment 1

[0071] Reference Figures 1 to 5 As shown, the present invention provides a method, system and device for grinding friction stir welds based on laser vision, and the method comprises the following steps:

[0072] S1: A laser image of a weld is obtained by a line laser sensor. To reduce the reflection interference at the weld and flash of the metal workpiece, the laser image is segmented by grayscale threshold to obtain a laser stripe image;

[0073] S2: extracting sub-pixel feature points of the laser stripe image to obtain the center line of the laser stripe, and converting the two-dimensional point set of the center line of the laser stripe into three-dimensional space to form a three-dimensional point cloud of the laser stripe;

[0074] S3: extracting features from the three-dimensional point cloud of the laser stripes to obtain feature data of the flash area;

[0075] S4: Based on the characteristic data of the flash area, a grinding path is planned, and the flash area of ​​the weld is ground along the grinding path using a grinding tool, including: first, the grinding plate of the grinding tool is moved parallel to the center of the weld, and after most of the flash is removed, the grinding plate is perpendicular to the weld, and the flash on both sides of the weld are ground respectively to ensure a smooth transition between the weld and the base material.

[0076] Furthermore, in S1, the method for obtaining the laser stripe image is: converting the laser image into a grayscale image, traversing the pixels of the grayscale image, setting the part with a grayscale value less than a threshold to 0, and keeping the grayscale values ​​of the remaining parts unchanged, to obtain the laser stripe image.

[0077] like Figure 2 As shown, in S2, the sub-pixel feature points of the laser stripe image are extracted by Steger algorithm to obtain the center line of the laser stripe:

[0078] S21: To improve the accuracy of subsequent processing, denoising the laser stripe image and enhancing its contrast to obtain a preprocessed laser stripe image;

[0079] S22: Based on the preprocessed laser stripe image, calculate the second-order derivative matrix of each pixel; wherein the expression of the second-order derivative matrix H is:

[0080]

[0081] Among them, I xx ,I yy are the second-order derivatives of the pixel in the x and y directions, I xy =I yx is a mixed second-order derivative.

[0082] S23: by calculating the determinant and trace of the second-order derivative matrix, wherein the determinant reflects the local structural strength of the image at the pixel point, and the trace is related to the local curvature, since the feature points are usually located at the positions where the local maximum value of the determinant and the trace value are appropriate, the feature points of the preprocessed laser stripe image are obtained;

[0083] S24: performing a Taylor series expansion on the feature point using the grayscale value of the pixel point to obtain a Taylor series expansion, where the Taylor series expansion is a function of the grayscale value with respect to the pixel position, and solving an equation whose derivative of the Taylor series expansion is equal to zero by a method such as Newton iteration method to obtain the coordinates of the sub-pixel feature point;

[0084] S25: Connect the detected sub-pixel feature points to form a laser stripe center line. Preferably, after the laser stripe center line is initially acquired, isolated points or short line segments in the laser stripe center line that may be caused by noise may be removed, and the denoised center line may be optimized such as smoothing, and the final laser stripe center line image may be output.

[0085] like Figure 3 As shown, in S2, the method for forming a three-dimensional point cloud of laser stripes is as follows:

[0086] S211: Obtain the two-dimensional sub-pixel precision coordinates of the center line of the laser stripe, and convert the two-dimensional sub-pixel precision coordinates into three-dimensional coordinates in the camera coordinate system through the camera intrinsic parameter matrix and the laser plane position obtained by hand-eye calibration;

[0087] S212: using the hand-eye conversion matrix, converting the three-dimensional coordinates in the camera coordinate system into the three-dimensional coordinates in the robot tool coordinate system;

[0088] S213: Gather the three-dimensional coordinate points in all robot tool coordinate systems to obtain a three-dimensional point cloud of laser stripes.

[0089] Furthermore, in S211, the method of converting the two-dimensional sub-pixel precision coordinates into three-dimensional coordinates in the camera coordinate system is as follows:

[0090] Converts a two-dimensional sub-pixel precision coordinate (u, v) to a pixel coordinate (u, v, 1) in homogeneous coordinate form T ;

[0091] Using the camera intrinsic matrix, the pixel coordinates (u, v, 1) in homogeneous coordinate form are converted to T Convert to three-dimensional coordinates (x, y, 1) in the camera coordinate system T , the conversion formula is:

[0092] (x,y,1) T =K -1 (u,v,1) T

[0093] Among them, K is the camera intrinsic parameter matrix, and its expression is:

[0094]

[0095] Among them, (f x ,f y ) is the focal length of the camera, f x and f y Corresponding to the focal length of the pixel point in the horizontal and vertical directions of the image, (c x ,c y ) is the optical center of the camera, c x and c y They correspond to the coordinates of the optical center in the horizontal and vertical directions respectively.

[0096] Furthermore, in S212, the method of converting the three-dimensional coordinates in the camera coordinate system into the three-dimensional coordinates in the robot tool coordinate system is as follows:

[0097] According to the camera's depth value z, the three-dimensional coordinates (x, y, 1) in the camera coordinate system TThe x- and y-axis coordinates are transformed to obtain the transformed three-dimensional coordinates

[0098] The transformed three-dimensional coordinate X is stretched into the robot tool coordinate system to obtain the three-dimensional coordinate X' in the robot tool coordinate system:

[0099] X'=T RC ·X

[0100] Among them, T RC is the hand-eye conversion matrix, and its expression is:

[0101]

[0102] Among them, R is the rotation matrix and t is the translation vector.

[0103] like Figure 4 As shown, in S3, the method for obtaining the characteristic data of the flash area is as follows:

[0104] S31: voxel filtering is performed on the laser stripe three-dimensional point cloud, while reducing the density of the overall point cloud data to reduce the amount of calculation, key details are retained at the same time, and the point cloud data after voxel filtering is clustered and segmented to eliminate interference caused by mirror reflection generated by laser irradiation on flash edges and smooth areas of welds or interference noise generated by laser irradiation on non-target areas such as workbenches and clamping devices, so as to obtain down-sampled point cloud data;

[0105] S32: Based on the downsampled point cloud data, set a suitable neighborhood size, obtain the neighborhood of each sample point through K-nearest neighbor search (K-NN) or radius search (Radius Search), and use the coordinates of all data points in the neighborhood to construct a covariance matrix;

[0106] S33: Performing PCA principal component analysis on the covariance matrix to obtain three eigenvectors and corresponding eigenvalues, and taking the eigenvector corresponding to the minimum eigenvalue (i.e., the direction with the smallest change) as the normal vector of the data point;

[0107] S34: Based on the normal vectors of all data points in the neighborhood, the curvature value of the corresponding sample point is calculated according to the error value between the normal vectors of all data points in the neighborhood and the normal vectors of the corresponding sample points. The larger the error value, the larger the curvature is generally, so as to obtain the curvature values ​​of all sample points in the downsampled point cloud data;

[0108] S35: Based on the curvature values ​​of all sample points, the sample points whose curvature values ​​are higher than the threshold are used as feature data of the flash feature area.

[0109] like Figure 5As shown, in S4, the method for planning the grinding path is as follows:

[0110] S41: De-noising the feature data of the flash area in sections to eliminate noise points caused by measurement errors or environmental interference, obtaining de-noised feature data, and fitting the de-noised feature data using algorithms such as random sampling consensus (RANSAC) to obtain a fitting straight line;

[0111] S42: looping through each point in the denoised feature data, calculating the foot of the perpendicular to the fitting line, and establishing a first KD-Tree structure for all flash area point clouds;

[0112] S43: for each perpendicular foot, based on the first KD-Tree structure, search is performed with the perpendicular foot as the center, and the point cloud with a distance r<3mm from the perpendicular foot is used as the flying edge point cloud in the neighborhood, and the volume center and centroid of the bounding box of these flying edge point clouds are calculated, and the weighted center point is calculated according to the given weight (the volume center weight is set to 2 / 3, and the centroid weight is set to 1 / 3), and the weighted center point will be used as the flying edge correction center point of the neighborhood;

[0113] S44: constructing a second KD-Tree structure for all parent material point clouds, and for each flash correction center point, searching for point clouds in its neighborhood through the second KD-Tree structure, and taking point clouds with a distance of r<50 mm from the flash correction center point as parent material neighborhood point clouds;

[0114] S45: fitting a base material plane using the base material neighborhood point cloud, projecting the flash correction center point onto the base material plane, the obtained projection point is a point on the grinding path, and the set of all projection points constitutes the grinding path.

[0115] Since the weld of friction stir welding is wide and the flash is unevenly distributed, in order to ensure a smooth transition between the weld and the base material, three grinding paths are designed according to the above scheme to grind the weld flash.

[0116] The center of the weld is roughly ground through the first grinding path. The line connecting the origin of the tool coordinate system of the grinding piece and the projection point of the center of the grinding piece on the cutting plane of the parent material is parallel to the weld. During grinding, the origin of the tool coordinate system of the grinding piece moves along the center of the weld, and there is a certain inclination angle between the grinding piece and the parent material.

[0117] The flash on both sides are finely ground through the second and third grinding paths. The line connecting the origin of the tool coordinate system of the grinding sheet and the projection point of the center of the grinding sheet on the plane of the parent material is perpendicular to the weld. The acute angle formed by the grinding sheet and the plane of the parent material is toward the outside of the weld. During grinding, the origin of the tool coordinate system of the grinding sheet moves along the center of the flash, and the grinding sheet and the parent material have a certain inclination angle.

[0118] Embodiment 2

[0119] Based on the same inventive concept as that of the first embodiment, the present invention also provides a laser vision-based friction stir weld grinding system, which is used to implement the laser vision-based friction stir weld grinding method described in the first embodiment, such as Figure 6 As shown, the system includes the following modules:

[0120] A weld image acquisition module 10, which is used to acquire a laser image of the weld, and perform grayscale threshold segmentation on the laser image to obtain a laser stripe image;

[0121] A point cloud data generation module 20, which is used to extract sub-pixel feature points of the laser stripe image to obtain the center line of the laser stripe, and convert the two-dimensional point set of the center line of the laser stripe into a three-dimensional space to form a three-dimensional point cloud of the laser stripe;

[0122] A flash feature extraction module 30, the flash feature extraction module 30 is used to extract features from the laser stripe three-dimensional point cloud to obtain feature data of the flash area;

[0123] The weld grinding processing module 40 is used to plan a grinding path based on the characteristic data of the flash area, and grind the flash area of ​​the weld along the grinding path.

[0124] A laser vision-based friction stir weld grinding system proposed in this embodiment is used to implement the aforementioned laser vision-based friction stir weld grinding method. Therefore, the specific implementation method of the laser vision-based friction stir weld grinding system can be seen from the aforementioned laser vision-based friction stir weld grinding method embodiment part, for example, the weld image acquisition module 10, the point cloud data generation module 20, the flash feature extraction module 30 and the weld grinding processing module 40 are respectively used to correspond to the implementation of the laser vision-based friction stir weld grinding method in Example 1. Steps S1, S2, S3 and S4, so its specific implementation method can refer to the description of the corresponding various parts of the embodiment. In order to avoid redundancy, it will not be repeated here.

[0125] Embodiment 3

[0126] like Figure 7 As shown, the present invention also provides a laser vision-based friction stir weld grinding device, which includes a grinding tool 100, a line laser sensor 200 and a control unit 300.

[0127] Specifically, the line laser sensor 200 is installed on the grinding tool 100, the control unit 300 controls the connection between the grinding tool 100 and the line laser sensor 200, and the laser vision-based friction stir weld grinding system described in Example 2 is stored in the control unit 300. The control unit 300 obtains the laser image of the weld through the line laser sensor 200, and uses the grinding tool 100 to grind the flash area of ​​the weld.

[0128] The grinding tool 100 is installed on the end of the industrial robot, and the control unit 300 drives the grinding tool 100 to move and operate through the control end of the industrial robot.

[0129] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0130] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0133] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A method for grinding friction stir welds based on laser vision, characterized in that: The following steps are involved: S1: Acquire a laser image of the weld, perform grayscale threshold segmentation on the laser image, and obtain a laser stripe image; S2: extracting sub-pixel feature points of the laser stripe image to obtain the center line of the laser stripe, and converting the two-dimensional point set of the center line of the laser stripe into three-dimensional space to form a three-dimensional point cloud of the laser stripe; S3: extracting features from the three-dimensional point cloud of the laser stripes to obtain feature data of the flash area; S4: planning a grinding path based on the characteristic data of the flash area, and grinding the flash area of ​​the weld along the grinding path.

2. The method for grinding friction stir welds based on laser vision according to claim 1 is characterized in that: In S2, the method for obtaining the center line of the laser stripe is: S21: De-noising the laser stripe image and enhancing its contrast to obtain a pre-processed laser stripe image; S22: Calculating the second-order derivative matrix of each pixel based on the preprocessed laser stripe image; S23: extracting feature points of the preprocessed laser stripe image by calculating the determinant and trace of the second-order derivative matrix; S24: performing a Taylor series expansion on the feature point using the grayscale value of the pixel point to obtain a Taylor series expansion, and obtaining the coordinates of the sub-pixel feature point by solving an equation whose derivative of the Taylor series expansion is equal to zero; S25: Connect the detected sub-pixel feature points to form a laser stripe center line.

3. The method for grinding friction stir welds based on laser vision according to claim 2 is characterized in that: The expression of the second-order derivative matrix H is: Among them, I xx ,I yy are the second-order derivatives of the pixel in the x and y directions, I xy =I yx is a mixed second-order derivative.

4. The method for grinding friction stir welds based on laser vision according to claim 1, characterized in that: In S2, the method for forming the three-dimensional point cloud of laser stripes is as follows: S211: Obtain the two-dimensional sub-pixel precision coordinates of the center line of the laser stripe, and convert the two-dimensional sub-pixel precision coordinates into three-dimensional coordinates in the camera coordinate system through the camera intrinsic parameter matrix and the laser plane position obtained by hand-eye calibration; S212: using the hand-eye conversion matrix, converting the three-dimensional coordinates in the camera coordinate system into the three-dimensional coordinates in the robot tool coordinate system; S213: Gather the three-dimensional coordinate points in all robot tool coordinate systems to obtain a three-dimensional point cloud of laser stripes.

5. The method for grinding friction stir welds based on laser vision according to claim 4 is characterized in that: In S211, the method of converting the two-dimensional sub-pixel precision coordinates into three-dimensional coordinates in the camera coordinate system is as follows: Converts a two-dimensional sub-pixel precision coordinate (u, v) to a pixel coordinate (u, v, 1) in homogeneous coordinate form T ; Using the camera intrinsic matrix, the pixel coordinates (u, v, 1) in homogeneous coordinate form are converted to T Convert to three-dimensional coordinates (x, y, 1) in the camera coordinate system T , the conversion formula is: (x,y,1) T =K -1 ·(u,v,1) T Among them, K is the camera intrinsic parameter matrix, and its expression is: Among them, (f x ,f y ) is the focal length of the camera, f x and f y Corresponding to the focal length of the pixel point in the horizontal and vertical directions of the image, (c x ,c y ) is the optical center of the camera, c x and c y They correspond to the coordinates of the optical center in the horizontal and vertical directions respectively.

6. The method for grinding friction stir welds based on laser vision according to claim 5 is characterized in that: In S212, the method of converting the three-dimensional coordinates in the camera coordinate system into the three-dimensional coordinates in the robot tool coordinate system is as follows: According to the camera's depth value z, the three-dimensional coordinates (x, y, 1) in the camera coordinate system T The x- and y-axis coordinates are transformed to obtain the transformed three-dimensional coordinates The transformed three-dimensional coordinate X is stretched into the robot tool coordinate system to obtain the three-dimensional coordinate X' in the robot tool coordinate system: X‘=T RC ·X Among them, T RC is the hand-eye conversion matrix, and its expression is: Among them, R is the rotation matrix and t is the translation vector.

7. The method for grinding friction stir welds based on laser vision according to claim 1, characterized in that: In S3, the method for obtaining the characteristic data of the flash area is as follows: S31: performing voxel filtering on the laser stripe three-dimensional point cloud, and performing cluster segmentation on the point cloud data after voxel filtering to obtain down-sampled point cloud data; S32: Based on the downsampled point cloud data, obtain the neighborhood of each sample point, and use the coordinates of all data points in the neighborhood to construct a covariance matrix; S33: performing principal component analysis on the covariance matrix to obtain three eigenvectors and corresponding eigenvalues, and taking the eigenvector corresponding to the minimum eigenvalue as the normal vector of the data point; S34: Calculating the curvature value of the corresponding sample point based on the normal vectors of all data points in the neighborhood, thereby obtaining the curvature values ​​of all sample points in the downsampled point cloud data; S35: Based on the curvature values ​​of all sample points, the sample points whose curvature values ​​are higher than the threshold are used as feature data of the flash feature area.

8. The method for grinding friction stir welds based on laser vision according to claim 1, characterized in that: In S4, the method for planning the grinding path is as follows: S41: De-noising the feature data of the flash area in sections to obtain de-noised feature data, and performing fitting processing on the de-noised feature data to obtain a fitting straight line; S42: looping through each point in the denoised feature data, calculating the foot of the perpendicular to the fitting line, and establishing a first KD-Tree structure for all flash area point clouds; S43: for each perpendicular foot, use the first KD-Tree structure to search for the flying edge point clouds in its neighborhood, calculate the bounding box volume center and the centroid of these flying edge point clouds, and calculate the weighted center point according to the given weight, and the weighted center point will be used as the flying edge correction center point of the neighborhood; S44: constructing a second KD-Tree structure for all parent material point clouds, and for each flash correction center point, searching for point clouds in its neighborhood through the second KD-Tree structure to obtain parent material neighborhood point clouds; S45: fitting a base material plane using the base material neighborhood point cloud, projecting the flash correction center point onto the base material plane, the obtained projection point is a point on the grinding path, and the set of all projection points constitutes the grinding path.

9. A laser vision-based friction stir weld grinding system, used to implement the laser vision-based friction stir weld grinding method according to any one of claims 1 to 8, characterized in that: Includes the following modules: A weld image acquisition module, which is used to acquire a laser image of the weld, and perform grayscale threshold segmentation on the laser image to obtain a laser stripe image; A point cloud data generation module, the point cloud data generation module is used to extract sub-pixel feature points of the laser stripe image to obtain the center line of the laser stripe, and convert the two-dimensional point set of the center line of the laser stripe into a three-dimensional space to form a three-dimensional point cloud of the laser stripe; A flash feature extraction module, which is used to extract features from the three-dimensional point cloud of the laser stripes to obtain feature data of the flash area; A weld grinding processing module is used to plan a grinding path based on the characteristic data of the flash area, and use a grinding tool to grind the flash area of ​​the weld along the grinding path.

10. A laser vision based friction stir weld grinding device, characterized in that: It includes a grinding tool, a line laser sensor and a control unit, wherein the line laser sensor is installed on the grinding tool, the control unit controls the connection between the grinding tool and the line laser sensor, and the laser vision-based friction stir weld grinding system as described in claim 9 is stored in the control unit, the control unit obtains a laser image of the weld through the line laser sensor, and uses the grinding tool to grind the flash area of ​​the weld.

Images