Intelligent identification method and system for steel structure fillet weld based on 3D vision

Abstract

The present application relates to the technical field of intelligent welding, in particular to a steel structure fillet weld intelligent identification method and system based on 3D vision, the method comprising: acquiring a point cloud data set of a workpiece, and extracting point cloud data of a weld area therefrom; dividing the weld area into a plurality of weld sub-areas, wherein one weld sub-area corresponds to an independent weld; performing weld coarse skeleton extraction on each weld sub-area, and performing merging processing on weld coarse skeletons with a distance between endpoints less than a threshold value; performing weld fine skeleton point set extraction based on each weld coarse skeleton after merging processing, and fitting a weld path expression based on the weld fine skeleton point set. The present application solves the problems that existing algorithms require operators to repeatedly adjust threshold parameters based on experience, resulting in a lack of adaptive ability, and that when processing workpiece point cloud data, the amount of calculation is large and the speed is slow, which cannot meet the real-time requirements of industrial sites.

Description

Technical Field

[0001] This invention relates to the field of intelligent welding technology, specifically to a method and system for intelligent recognition of fillet welds in steel structures based on 3D vision. Background Technology

[0002] To achieve intelligent upgrades for welding robots, researchers have introduced various sensing technologies into welding systems, primarily including inductive, ultrasonic, acoustic, arc, and optical vision sensing technologies. Among these, optical vision technology has found significant application in the field of intelligent welding due to its remote measurement capabilities, rich information acquisition capacity, and high detection accuracy.

[0003] Current mainstream 3D vision perception technologies include binocular stereo vision, depth cameras, laser triangulation, and projected coded light. Binocular stereo vision technology often encounters difficulties in identifying matching points and decreased measurement accuracy in complex welding environments or when surface textures are indistinct. Depth cameras have limited spatial positioning accuracy, directly affecting the reliability of welding trajectory planning. Laser triangulation solutions often use manual guidance to acquire the workpiece's 3D data and then identify weld features to automatically plan the welding trajectory; however, this manual operation significantly reduces trajectory generation efficiency and increases operating costs.

[0004] While coded structured light sensors can quickly acquire point cloud data of welded workpieces, they still have significant limitations in fillet weld recognition applications. First, existing algorithms are mostly designed for single types of fillet welds, exhibiting poor adaptability to fillet welds of different shapes. Second, point cloud processing algorithms have high computational complexity, especially when processing dense point cloud data common in industrial settings, resulting in long computation times that cannot meet the speed requirements of real-time welding. Finally, most methods still require technicians to manually adjust parameters according to on-site conditions, lacking adaptive capabilities.

[0005] In summary, there is an urgent need for a 3D vision-based intelligent identification method and system for steel structure fillet welds to solve the problems existing in the current technology. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent identification method for steel structure fillet welds based on 3D vision. This method aims to solve the problems of existing algorithms requiring operators to repeatedly adjust threshold parameters based on experience, resulting in a lack of adaptive capability, and the high computational load and slow speed when processing workpiece point cloud data, which cannot meet the real-time requirements of industrial sites. The specific technical solution is as follows:

[0007] A method for intelligent recognition of fillet welds in steel structures based on 3D vision, comprising:

[0008] S1. Obtain the point cloud data set of the workpiece and extract the point cloud data of the weld area from it;

[0009] S2. Divide the weld area into multiple weld sub-regions, with each weld sub-region corresponding to an independent weld.

[0010] S3. Extract the coarse weld skeleton for each weld sub-region, and extract weld skeletons for regions where the distance between endpoints is less than a threshold. The weld seam rough skeleton is merged;

[0011] S4. Based on the merged coarse skeleton of each weld, extract the fine skeleton point set of the weld, and obtain the weld path expression based on the fine skeleton point set of the weld.

[0012] Preferably, step S1 involves acquiring point cloud data of the weld area, specifically:

[0013] Obtain each point cloud data point from the point cloud data set of the workpiece. Local covariance matrix For the local covariance matrix Singular value decomposition yields multiple eigenvalues. The ratio of the smallest eigenvalue to the sum of all eigenvalues ​​is used as the eigenvalue. As point cloud data The evaluation criteria, for all those who meet the requirements Greater than the threshold After performing cluster analysis on the point cloud data, the largest cluster is retained as the point cloud data of the weld area.

[0014] Preferably, the threshold The method for determining it is as follows:

[0015] The point cloud data is divided into proportions Arrange the values ​​in ascending order to construct a sequence, and calculate the first derivative for each ratio value in the sequence. and second derivative The rate of change of slope is obtained. ;in x-axis The vertical axis represents the sequence number of the point cloud data in the sequence. This represents the ratio of point cloud data in the sequence;

[0016] Starting from the beginning of the sequence and searching backwards, the first time the slope change rate is satisfied will be found. Greater than The ratio corresponding to twice the standard deviation is used as the threshold. .

[0017] Preferably, in step S2, the weld area is divided into multiple weld sub-regions, specifically:

[0018] A tree-structured data architecture is used to quickly retrieve each point in the point cloud dataset of the weld area. The nearest neighbor point set is used to perform plane fitting based on the local neighborhood point cloud and calculate the corresponding normal vectors. All normal vectors are then uniformly adjusted to point to the geometric center of the point cloud in the weld area.

[0019] Based on the normal vector similarity criterion, the K-means algorithm is used to divide the point cloud data of the weld area into multiple sub-regions, where each sub-region corresponds to an independent weld.

[0020] To eliminate the influence of measurement noise, a secondary clustering is performed on the point cloud of the segmented weld area. Specifically, the DBSCAN density clustering algorithm is applied to each sub-region to identify and extract the maximum density connected cluster in each cluster as the main cluster.

[0021] Based on the principle of minimum Euclidean distance, all unassigned non-primary cluster points in the weld region are reassigned to the nearest primary cluster to complete the segmentation of the weld region.

[0022] Preferably, the extraction of the coarse weld skeleton for each weld sub-region in step S3 specifically involves:

[0023] Determine the principal direction vector of the point cloud data set of the weld sub-region, and project all point clouds in the weld sub-region onto the principal direction vector;

[0024] Based on the range of projection values ​​of the point cloud onto the principal direction vector, the projection values ​​are divided into... There are 1, 2, 3, 4, 5, 6, 7, 8, 9, 1 It is an integer greater than or equal to 2;

[0025] Calculate the geometric center of each interval, and take the point in each interval that is closest to its geometric center as the skeleton point of the interval. Arrange the obtained skeleton points in the order of the intervals to form the basic skeleton of the weld sub-region.

[0026] After sorting the point cloud in the weld sub-region along the main direction vector, candidate endpoints in the starting and ending regions are selected by 3D edge detection.

[0027] Based on the influence of different candidate endpoints on the change of basic skeleton angle, the optimal starting point is found among the candidate endpoints in the starting region, and the optimal ending point is found among the candidate endpoints in the ending region.

[0028] By taking the optimal starting point as the starting point of the basic skeleton and the optimal ending point as the ending point of the basic skeleton, a rough skeleton for the weld is obtained.

[0029] Preferably, candidate endpoints in the starting and ending regions are selected through 3D edge detection, specifically:

[0030] S311. Use a tree data structure to quickly retrieve each point in the starting or ending region. The nearest neighbor set is used to obtain the local spherical neighborhood of each point;

[0031] S312. Generate uniformly distributed three-dimensional direction vectors within the local spherical neighborhood of each point. ;

[0032] Let the number of directions after excluding poles be . Then the azimuth angle Pitch angle and three-dimensional direction vectors They are represented as follows:

[0033]

[0034]

[0035]

[0036] in, Represents pi;

[0037] S313. Calculate the projection distance of each neighboring point in the local spherical neighborhood of each point on each directional vector. If the projection distance of all neighboring points of a point on a certain directional vector is less than 1 / 2 of the neighborhood radius, then it is determined that there is a gap in that direction; otherwise, it is determined that there is no gap.

[0038] S314. Count the number of missing directions at each point. If the number of missing directions at a certain point accounts for a certain percentage of the total number of directions... The proportion exceeds the threshold If the point is selected as a candidate endpoint, then that point is considered a candidate endpoint.

[0039] Preferably, the optimal endpoint is selected by evaluating the impact of different candidate endpoints on the change in the basic skeleton angle, as follows:

[0040] S321. Add each candidate endpoint in the starting region and the ending region to the basic skeleton as an endpoint. If the candidate endpoint comes from the starting region, it is used as the starting point of the basic skeleton. If the candidate endpoint comes from the ending region, it is used as the ending point of the basic skeleton.

[0041] S322. Calculate the angle between two adjacent segments in each basic skeleton after adding endpoints, and obtain the mean and standard deviation of the angle.

[0042] S323. Select the candidate endpoint with the smallest included angle standard deviation from each candidate endpoint in the starting region as the optimal starting point, and select the candidate endpoint with the smallest included angle standard deviation from each candidate endpoint in the ending region as the optimal ending point.

[0043] S324. Using the optimal starting point as the starting point of the basic skeleton and the optimal ending point as the ending point of the basic skeleton, we obtain the rough skeleton of the weld.

[0044] Preferably, the fine weld skeleton point set is extracted based on the merged weld coarse skeleton, specifically:

[0045] For any weld skeleton after merging, calculate the cumulative arc length parameter of each point. Based on the cumulative arc length parameter of each point, obtain the uniform parameter distribution of each point through normalization. Based on the uniform parameter distribution of each point, establish a continuous curve model to obtain the spline curve.

[0046] Based on spline curve uniform sampling Each sampling point is used to perform fine iteration based on the tubular neighborhood of each sampling point to obtain the point representing the fine skeleton of the weld, thus completing the extraction of the fine skeleton point set of the weld.

[0047] in The integer is greater than or equal to 3. Each sampling point must include both endpoints of the spline curve.

[0048] Preferably, the points representing the fine skeleton of the weld are obtained through refined iteration based on the tubular neighborhood of the sampling points, specifically:

[0049] S411, Let the number of iterations be... The maximum number of iterations is set to 1. ;Sampling points tubular neighborhood As the initial tubular neighborhood From the tubular neighborhood Select the center point :

[0050]

[0051] in, Represents a single point cloud data;

[0052] S412, Calculate the current tubular neighborhood. Geometric centroid of all points inside :

[0053]

[0054] in, Represents tubular neighborhood The total number of point clouds in the cloud;

[0055] S413, Order The geometric centroid obtained in step S412 The current tubular neighborhood is obtained by updating the center. :

[0056]

[0057] in, The point cloud set representing the weld sub-region, Indicates the tubular radius parameter;

[0058] S414, in the current tubular neighborhood Select distance to geometric centroid The nearest actual point is used as the center point. :

[0059]

[0060] S415, if satisfied or Then the center point Add the weld seam fine skeleton point set and end the iteration; otherwise, return to step S412. This is the convergence threshold.

[0061] The present invention also provides a 3D vision-based intelligent recognition system for steel structure fillet welds, including a memory and a processor. The memory stores a computer program, and the processor executes the method described thereon when running the computer program.

[0062] The application of the technical solution of the present invention has the following beneficial effects:

[0063] This invention presents a 3D vision-based intelligent recognition method for fillet welds in steel structures. First, it extracts the point cloud data of the weld region from the point cloud data of the workpiece based on point cloud geometric features. Then, it segments the weld region into multiple weld sub-regions based on normal vector clustering. Next, it extracts a coarse weld skeleton for each individual weld sub-region, and performs threshold-based recognition for welds with endpoints less than a certain distance. The coarse weld skeletons are merged, and then a fine weld skeleton point set is extracted from each merged coarse weld skeleton. The weld path expression is then fitted based on this fine weld skeleton point set. The recognition method of this invention can automatically adapt to various types of fillet welds, where the threshold... It can adaptively adjust according to the ratio sequence of point cloud data without the need for manual parameter adjustment; when extracting the rough skeleton of weld seam sub-regions, it can greatly reduce the amount of point cloud data to be processed by dividing the intervals at equal intervals and setting the start and end regions, as well as by uniformly sampling the spline curves obtained from the rough skeleton of weld seam. This optimizes the processing flow of workpiece point cloud data, improves processing speed while maintaining recognition accuracy, and can meet the real-time requirements of industrial sites.

[0064] The method of this invention can improve the adaptability of welding robots, meet the needs of flexible and intelligent welding, and has good application prospects in the manufacturing fields of steel structures, ships, bridges and other industries.

[0065] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0066] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0067] Figure 1 This is a flowchart of the intelligent recognition method for steel structure fillet welds based on 3D vision according to the present invention. Detailed Implementation

[0068] To facilitate understanding of the present invention, a more complete description is provided below, along with preferred embodiments. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the present invention.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0070] Example 1:

[0071] See Figure 1 This embodiment provides a method for intelligent recognition of steel structure fillet welds based on 3D vision, as detailed below:

[0072] S1. Obtain the point cloud data set of the workpiece and extract the point cloud data of the weld area from it;

[0073] Preferably, each point cloud data point in the point cloud data set of the workpiece is acquired. Local covariance matrix For the local covariance matrix Singular value decomposition yields multiple eigenvalues. The ratio of the smallest eigenvalue to the sum of all eigenvalues ​​is used as the eigenvalue. As an evaluation criterion, for all those that meet the requirements Greater than the threshold After performing cluster analysis on the point cloud data, the largest cluster is retained as the point cloud data of the weld area.

[0074] Specifically, 3D vision technology is used to obtain the point cloud data set of the workpiece. Construct point cloud dataset Each point cloud data Local covariance matrix :

[0075]

[0076] in, For set The total number of point clouds in the cloud. , express Any point in the neighborhood set of points, express The number of point clouds in the neighborhood point set. express The centroid of the neighborhood point set.

[0077] Furthermore, regarding the local covariance matrix... Singular value decomposition is used to obtain eigenvalues. , and ,in The corresponding eigenvectors obtained from the decomposition represent the points. The dominant direction of the point cloud distribution within its local neighborhood (i.e., the neighborhood point set) is then the ratio Represented as:

[0078]

[0079] The ratio corresponding to each point cloud data With set threshold By comparing, the satisfaction rate can be obtained. Greater than the threshold The collection of all point cloud data :

[0080]

[0081] Furthermore, this embodiment provides an adaptive threshold determination method. The specific method is as follows:

[0082] The point cloud data is divided into proportions Arrange the values ​​in ascending order to construct a sequence, and calculate the first derivative for each ratio value in the sequence. and second derivative The rate of change of slope is obtained. ;in x-axis The vertical axis represents the sequence number of the point cloud data in the sequence. This represents the ratio of point cloud data in the sequence;

[0083] Starting from the beginning of the sequence and searching backwards, the first time the slope change rate is satisfied will be found. Greater than The ratio corresponding to twice the standard deviation is used as the threshold. .

[0084] Furthermore, due to the raw measurement data (referring to the point cloud dataset) The dataset contains measurement errors, data holes, and background noise; high-ratio regions may contain isolated noisy point clouds. To further improve data quality, the DBSCAN density clustering algorithm needs to be used to cluster the dataset. Cluster analysis was performed on the point cloud data, and the largest cluster was retained as the point cloud data of the weld area for subsequent data analysis, while isolated points and small-scale point cloud clusters were removed.

[0085] S2. Divide the weld area into multiple weld sub-regions, with each weld sub-region corresponding to an independent weld.

[0086] Preferably, in order to achieve independent identification and processing of each weld seam and avoid mutual interference in the subsequent trajectory fitting process, it is necessary to accurately segment the extracted weld seam region. This embodiment proposes a segmentation method based on normal vector clustering, which utilizes the differences in the normal vector distribution characteristics of different weld seam sub-regions to achieve effective segmentation, specifically:

[0087] First, a KD-tree data structure (i.e., a tree-structured data structure) is used to quickly retrieve each point in the point cloud dataset of the weld area. The nearest neighbor point set is used to perform plane fitting based on the local neighborhood point cloud and calculate the corresponding normal vectors. All normal vectors are then uniformly adjusted to point to the geometric center of the point cloud in the weld area.

[0088] Secondly, based on the normal vector similarity criterion, the K-means algorithm (i.e., K-means algorithm) is used to divide the point cloud data of the weld area into multiple sub-regions, where each sub-region corresponds to an independent weld.

[0089] Subsequently, a second clustering was performed on the point cloud of the segmented weld area to eliminate the influence of measurement noise. Specifically, the DBSCAN density clustering algorithm was applied to each sub-region to identify and extract the maximum density connected cluster in each cluster as the main cluster.

[0090] Finally, based on the principle of minimum Euclidean distance, all unassigned non-major cluster points in the weld region are reassigned to the nearest major cluster to complete the segmentation of the weld region while ensuring the integrity of the weld region.

[0091] S3. Extract the coarse weld skeleton for each weld sub-region, and extract weld skeletons for regions where the distance between endpoints is less than a threshold. The weld seam rough skeleton is merged;

[0092] Preferably, the extraction of the coarse weld skeleton for each weld sub-region specifically involves:

[0093] Determine the principal direction vector of the point cloud data set of the weld sub-region, and project all point clouds in the weld sub-region onto the principal direction vector;

[0094] Based on the range of projection values ​​of the point cloud onto the principal direction vector, the projection values ​​are divided into... There are 1, 2, 3, 4, 5, 6, 7, 8, 9, 1 It is an integer greater than or equal to 2;

[0095] Calculate the geometric center of each interval, and take the point in each interval that is closest to its geometric center as the skeleton point of the interval. Arrange the obtained skeleton points in the order of the intervals to form the basic skeleton of the weld sub-region.

[0096] After sorting the point cloud in the weld sub-region along the main direction vector, candidate endpoints in the starting and ending regions are selected by 3D edge detection.

[0097] Based on the influence of different candidate endpoints on the change of basic skeleton angle, the optimal starting point is found among the candidate endpoints in the starting region, and the optimal ending point is found among the candidate endpoints in the ending region.

[0098] By taking the optimal starting point as the starting point of the basic skeleton and the optimal ending point as the ending point of the basic skeleton, a rough skeleton for the weld is obtained.

[0099] The method for extracting the rough skeleton of the weld in this embodiment will be described in detail below:

[0100] First, principal component analysis is used to determine the main extension direction (i.e., the principal direction vector) of the weld sub-region point cloud dataset. For a 3D point cloud, the principal direction vector can be calculated as follows:

[0101]

[0102] in, This indicates the search for the variable that maximizes the objective function. , Represents the variance function. The point cloud set representing the weld sub-region, Represents the candidate direction vector. This represents the principal direction vector of the weld sub-region.

[0103] Subsequently, all point clouds within the weld sub-region are projected onto the principal direction to obtain a one-dimensional distribution representation; the projection values ​​are calculated as follows:

[0104]

[0105] in, Indicates the center of the weld sub-region The projection value, The centroid of the point cloud set representing the weld sub-region, This represents any point within the weld sub-region.

[0106] Based on the range of projection values ​​of the point cloud in the principal direction, the projection values ​​are divided into... There are 1, 2, 3, 4, 5, 6, 7, 8, 9, 1 Let the minimum and maximum values ​​of the projected values ​​be integers greater than or equal to 2. and Then the boundaries of each interval are defined as:

[0107]

[0108] For each interval Calculate the geometric center of all points within the interval. :

[0109]

[0110] in, Representing an interval The set of indexes for the inner point cloud. Representing an interval The total number of point clouds within, Representing an interval Any point cloud within;

[0111] Select range Inner distance from geometric center The nearest point is used as the skeleton point of this interval to ensure the authenticity and representativeness of the skeleton points. Let point... Distance from geometric center The nearest point, then:

[0112]

[0113] Furthermore, each interval The skeleton points are arranged sequentially to form the basic skeleton of the weld sub-region. Since equal-interval segmentation may cause deviations between the endpoints of the basic skeleton and the actual weld endpoints, affecting the completeness of subsequent trajectory fitting, this embodiment proposes an endpoint correction method based on three-dimensional edge detection and line segment angle variation consistency to further confirm the actual weld endpoints.

[0114] Specifically, since the actual weld endpoints are usually located in the edge region of the point cloud, 3D edge detection of the point cloud is required to screen candidate endpoints that may be actual weld endpoints. In this embodiment, the point clouds of the weld sub-region are sorted along the main direction, and the first W% of the point clouds are selected as the starting region and the last G% as the ending region. Then, candidate endpoints are screened in the starting region and the ending region respectively, where W and G are both greater than 0. In this embodiment, it is preferable that W% and G% are both 5%. By setting the starting region and the ending region, the amount of calculation can be greatly reduced and the calculation efficiency can be improved.

[0115] Furthermore, the method for filtering candidate endpoints in this embodiment is as follows:

[0116] S311. Use the KD-tree data structure to quickly retrieve each point in the starting or ending region. The nearest neighbor set is used to obtain the local spherical neighborhood of each point;

[0117] S312. Generate uniformly distributed three-dimensional direction vectors within the local spherical neighborhood of each point. ;

[0118] Furthermore, let the number of directions after excluding the poles be... Then the azimuth angle Pitch angle and three-dimensional direction vectors They are represented as follows:

[0119]

[0120]

[0121]

[0122] in, It represents pi (π).

[0123] S313. Calculate the projection distance of each neighboring point in the local spherical neighborhood of each point on each directional vector. If the projection distance of all neighboring points of a point on a certain directional vector is less than 1 / 2 of the neighborhood radius, then it is determined that there is a gap in that direction; otherwise, it is determined that there is no gap.

[0124] S314. Count the number of missing directions at each point. If the number of missing directions at a certain point accounts for a certain percentage of the total number of directions... The proportion exceeds the threshold If the value is not specified, then the point is considered an edge point (i.e., a candidate endpoint). In this embodiment, the threshold value is... Based on extensive experiments, a threshold of 40% has been determined to be effective in balancing detection precision and recall.

[0125] Steps S311-S314 can be used to filter out all candidate endpoints (i.e., edge points) in the starting and ending regions. Then, the optimal endpoint is selected by evaluating the impact of different candidate endpoints on the change of the basic skeleton angle, as follows:

[0126] S321. Add each candidate endpoint in the starting region and the ending region to the basic skeleton as an endpoint. If the candidate endpoint comes from the starting region, it is used as the starting point of the basic skeleton. If the candidate endpoint comes from the ending region, it is used as the ending point of the basic skeleton.

[0127] S322. Calculate the angle between two adjacent segments in each basic skeleton after adding endpoints, and obtain the mean and standard deviation of the angle.

[0128] Those skilled in the art will understand that a basic skeleton is composed of multiple skeleton points. Whether the basic skeleton has endpoints added or not, it is essentially a line formed based on multiple points. Therefore, the segment here refers to the line segment between two adjacent points in the basic skeleton.

[0129] S323. Select the candidate endpoint with the smallest included angle standard deviation from each candidate endpoint in the starting region as the optimal starting point, and select the candidate endpoint with the smallest included angle standard deviation from each candidate endpoint in the ending region as the optimal ending point.

[0130] S324. Using the optimal starting point as the starting point of the basic skeleton and the optimal ending point as the ending point of the basic skeleton, we obtain the rough skeleton of the weld.

[0131] Furthermore, let the direction vectors of two adjacent segments in the basic skeleton after adding endpoints be respectively and The included angle between two adjacent segments Represented as:

[0132]

[0133] Specifically, the basic skeleton after adding endpoints refers to the basic skeleton after adding a start point or end point. For example, suppose the starting region has... There are candidate endpoints, and the end region contains... There are 10 candidate endpoints, then there are a total of 10000 endpoints. A basic skeleton after adding endpoints, and then further... In the basic skeleton after adding starting points, the candidate endpoint with the smallest standard deviation of the included angle is selected as the optimal starting point. In the basic skeleton after adding the endpoint, the candidate endpoint corresponding to the smallest standard deviation of the included angle is selected as the optimal endpoint. The optimal starting point and the optimal endpoint are added to the basic skeleton as the starting point and the endpoint, respectively, and finally the rough skeleton of the weld is obtained.

[0134] Preferably, after obtaining the rough weld skeleton for each weld sub-region, in order to ensure the continuity of the weld, in this embodiment, the distance between the endpoints is less than a threshold. The weld skeletons are merged, meaning that if the distance between the endpoints of one weld skeleton and the endpoints of another weld skeleton is less than a threshold value... This involves merging the welds to ensure continuity; in this embodiment, a threshold value is used. It is 2mm.

[0135] S4. Based on the merged coarse skeleton of each weld, extract the fine skeleton point set of the weld respectively, and obtain the weld path expression based on the fine skeleton point set of the weld.

[0136] To further improve the accuracy and continuity of subsequent path fitting, this embodiment proposes a fine skeleton extraction method for point clouds based on tubular region sampling, as follows:

[0137] For any weld skeleton after merging, calculate the cumulative arc length parameter of each point. Based on the cumulative arc length parameter of each point, obtain the uniform parameter distribution of each point through normalization. Based on the uniform parameter distribution of each point, establish a continuous curve model to obtain the spline curve.

[0138] Based on spline curve uniform sampling Each sampling point is used to perform fine iteration based on its tubular neighborhood to obtain points representing the fine skeleton of the weld, thus completing the extraction of the fine skeleton point set of the weld.

[0139] The method for obtaining the fine weld skeleton point set in this embodiment will be described in detail below:

[0140] Specifically, the point set of any coarse skeleton of a weld after merging is represented as: The cumulative arc length parameter at each point is expressed as:

[0141]

[0142]

[0143] in, The first point in the weld rough skeleton point set One point, The first point in the weld rough skeleton point set One point, Representing three-dimensional space, This indicates the number of points in the weld rough skeleton point set. Indicates the first The cumulative arc length parameter at each point, Point of coordinate, Point of coordinate, Point of coordinate.

[0144] The first one is obtained through normalization. Points Uniform parameter distribution:

[0145]

[0146] in, Indicates the first The cumulative arc length parameter at each point Indicates the first Uniform parameter distribution at points This indicates the total arc length of the weld seam's rough skeleton.

[0147] A continuous curve model is established using cubic B-splines, and its independent fitting of each coordinate component is expressed as follows:

[0148]

[0149] in, express exist Components on coordinates, express exist Components on coordinates, express exist Components on the coordinate axis; Indicates the first A B-spline basis function, This represents the total number of B-spline basis functions; , and All are coefficients, solved using the least squares method to ensure that the fitted curve passes through the key control points and maintains good continuity.

[0150] Furthermore, uniform sampling is performed along the spline curve. There are 1 sampling points, among which The integer is greater than or equal to 3. Each sampling point must contain both endpoints of the spline curve. The tubular neighborhood is represented as follows:

[0151]

[0152] in, Indicates sampling point of coordinate, Indicates sampling point of coordinate, Indicates sampling point of coordinate; The point cloud set representing the weld sub-region, Represents a single point cloud data, Indicates sampling point A collection of point cloud data within a tubular neighborhood; This represents the tubular radius parameter, which is adaptively adjusted according to the point cloud density. It is usually taken as 3-5 times the average spacing of the local point cloud to ensure that the sampling area can fully cover the weld width.

[0153] Furthermore, to obtain a more accurate weld centerline, iterative refinement of the sampling points within the tubular neighborhood is required, specifically including the following:

[0154] S411, Let the number of iterations be... The maximum number of iterations is set to 1. ;Sampling points tubular neighborhood As the initial tubular neighborhood From the tubular neighborhood Select the center point :

[0155]

[0156] in, Represents a single point cloud data;

[0157] S412, Calculate the current tubular neighborhood. Geometric centroid of all points inside :

[0158]

[0159] in, Represents tubular neighborhood The total number of point clouds in the cloud;

[0160] S413, Order The geometric centroid obtained in step S412 The current tubular neighborhood is obtained by updating the center. :

[0161]

[0162] S414, in the current tubular neighborhood Select distance to geometric centroid The nearest actual point is used as the center point. :

[0163]

[0164] S415, if satisfied or Then the center point Add the weld seam fine skeleton point set and end the iteration; otherwise, return to step S412. This is the convergence threshold.

[0165] After iteratively refining each sampling point to obtain the weld fine skeleton point set (i.e. weld feature point set), the points in the weld fine skeleton point set are mathematically modeled based on the three-dimensional polynomial least squares fitting method with cumulative arc length parameterization to obtain a continuous and smooth weld path expression.

[0166] At this point, all weld path expressions can be obtained (the number of weld path expressions depends on how many remaining weld skeletons are merged and processed). The obtained weld path expressions can guide the welding robot to perform automatic welding operations. The method in this embodiment can automatically adapt to the identification and path planning of various types of fillet welds without the need for manual parameter adjustment.

[0167] Example 2:

[0168] This embodiment provides a 3D vision-based intelligent recognition system for steel structure fillet welds, including a memory and a processor. The memory stores a computer program, and the processor executes the method in Embodiment 1 when running the computer program.

[0169] Example 3:

[0170] This embodiment provides a storage medium storing a computer program, which, when run, executes the method in Embodiment 1.

[0171] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent recognition of fillet welds in steel structures based on 3D vision, characterized in that, include: S1. Obtain the point cloud data set of the workpiece and extract the point cloud data of the weld area from it; S2. Divide the weld area into multiple weld sub-regions, with each weld sub-region corresponding to an independent weld. S3. Extract the coarse weld skeleton for each weld sub-region, and extract weld skeletons for regions where the distance between endpoints is less than a threshold. The weld seam rough skeleton is merged; S4. Based on the merged coarse skeleton of each weld, extract the fine skeleton point set of the weld respectively, and obtain the weld path expression based on the fine skeleton point set of the weld. Step S1 involves acquiring point cloud data of the weld area, specifically: Obtain each point cloud data point from the point cloud data set of the workpiece. Local covariance matrix For the local covariance matrix Singular value decomposition yields multiple eigenvalues. The ratio of the smallest eigenvalue to the sum of all eigenvalues ​​is used as the eigenvalue. As point cloud data The evaluation criteria, for all those who meet the requirements Greater than the threshold After performing cluster analysis on the point cloud data, the largest cluster is retained as the point cloud data of the weld area; The threshold The method for determining it is as follows: The point cloud data is divided into proportions Arrange the values ​​in ascending order to construct a sequence, and calculate the first derivative for each ratio value in the sequence. and second derivative The rate of change of slope is obtained. ;in x-axis The vertical axis represents the sequence number of the point cloud data in the sequence. This represents the ratio of point cloud data in the sequence; Starting from the beginning of the sequence and searching backwards, the first time the slope change rate is satisfied will be found. Greater than The ratio corresponding to twice the standard deviation is used as the threshold. ; Step S3, specifically the extraction of the rough weld skeleton for each weld sub-region, involves: Determine the principal direction vector of the point cloud data set of the weld sub-region, and project all point clouds in the weld sub-region onto the principal direction vector; Based on the range of projection values ​​of the point cloud onto the principal direction vector, the projection values ​​are divided into... There are 1, 2, 3, 4, 5, 6, 7, 8, 9, 1 It is an integer greater than or equal to 2; Calculate the geometric center of each interval, and take the point in each interval that is closest to its geometric center as the skeleton point of the interval. Arrange the obtained skeleton points in the order of the intervals to form the basic skeleton of the weld sub-region. After sorting the point cloud in the weld sub-region along the main direction vector, candidate endpoints in the starting and ending regions are selected by 3D edge detection. Based on the influence of different candidate endpoints on the change of basic skeleton angle, the optimal starting point is found among the candidate endpoints in the starting region, and the optimal ending point is found among the candidate endpoints in the ending region. By taking the optimal starting point as the starting point of the basic skeleton and the optimal ending point as the ending point of the basic skeleton, a rough skeleton for the weld is obtained.

2. The intelligent recognition method for steel structure fillet welds based on 3D vision according to claim 1, characterized in that, In step S2, the weld area is divided into multiple weld sub-areas, specifically: A tree-structured data architecture is used to quickly retrieve each point in the point cloud dataset of the weld area. The nearest neighbor point set is used to perform plane fitting based on the local neighborhood point cloud and calculate the corresponding normal vectors. All normal vectors are then uniformly adjusted to point to the geometric center of the point cloud in the weld area. Based on the normal vector similarity criterion, the K-means algorithm is used to divide the point cloud data of the weld area into multiple sub-regions, where each sub-region corresponds to an independent weld. To eliminate the influence of measurement noise, a secondary clustering is performed on the point cloud of the segmented weld area. Specifically, the DBSCAN density clustering algorithm is applied to each sub-region to identify and extract the maximum density connected cluster in each cluster as the main cluster. Based on the principle of minimum Euclidean distance, all unassigned non-primary cluster points in the weld region are reassigned to the nearest primary cluster to complete the segmentation of the weld region.

3. The intelligent recognition method for steel structure fillet welds based on 3D vision according to claim 1, characterized in that, Candidate endpoints in the starting and ending regions are selected using 3D edge detection, specifically: S311. Use a tree data structure to quickly retrieve each point in the starting or ending region. The nearest neighbor set is used to obtain the local spherical neighborhood of each point; S312. Generate uniformly distributed three-dimensional direction vectors within the local spherical neighborhood of each point. ; Let the number of directions after excluding poles be . Then the azimuth angle Pitch angle and three-dimensional direction vectors They are represented as follows: in, Represents pi; S313. Calculate the projection distance of each neighboring point in the local spherical neighborhood of each point on each directional vector. If the projection distance of all neighboring points of a point on a certain directional vector is less than 1 / 2 of the neighborhood radius, then it is determined that there is a gap in that direction; otherwise, it is determined that there is no gap. S314. Count the number of missing directions at each point. If the number of missing directions at a certain point accounts for a certain percentage of the total number of directions... The proportion exceeds the threshold If the point is selected as a candidate endpoint, then that point is considered a candidate endpoint.

4. The intelligent recognition method for steel structure fillet welds based on 3D vision according to claim 1, characterized in that, The optimal endpoint is selected by evaluating the impact of different candidate endpoints on the change in the basic skeleton angle, as follows: S321. Add each candidate endpoint in the starting region and the ending region to the basic skeleton as an endpoint. If the candidate endpoint comes from the starting region, it is used as the starting point of the basic skeleton. If the candidate endpoint comes from the ending region, it is used as the ending point of the basic skeleton. S322. Calculate the angle between two adjacent segments in each basic skeleton after adding endpoints, and obtain the mean and standard deviation of the angle. S323. Select the candidate endpoint with the smallest included angle standard deviation from each candidate endpoint in the starting region as the optimal starting point, and select the candidate endpoint with the smallest included angle standard deviation from each candidate endpoint in the ending region as the optimal ending point. S324. Using the optimal starting point as the starting point of the basic skeleton and the optimal ending point as the ending point of the basic skeleton, we obtain the rough skeleton of the weld.

5. The intelligent recognition method for steel structure fillet welds based on 3D vision according to claim 1, characterized in that, Based on the merged coarse weld skeleton, the fine weld skeleton point set is extracted, specifically: For any weld skeleton after merging, calculate the cumulative arc length parameter of each point. Based on the cumulative arc length parameter of each point, obtain the uniform parameter distribution of each point through normalization. Based on the uniform parameter distribution of each point, establish a continuous curve model to obtain the spline curve. Based on spline curve uniform sampling Each sampling point is used to perform fine iteration based on the tubular neighborhood of each sampling point to obtain the point representing the fine skeleton of the weld, thus completing the extraction of the fine skeleton point set of the weld. in The integer is greater than or equal to 3. Each sampling point must include both endpoints of the spline curve.

6. The intelligent recognition method for steel structure fillet welds based on 3D vision according to claim 5, characterized in that, The points representing the fine skeleton of the weld are obtained through refined iteration based on the tubular neighborhood of the sampling points. Specifically: S411, Let the number of iterations be... The maximum number of iterations is set to 1. ;Sampling points tubular neighborhood As the initial tubular neighborhood From the tubular neighborhood Select the center point : in, Represents a single point cloud data; S412, Calculate the current tubular neighborhood. Geometric centroid of all points inside : in, Represents tubular neighborhood The total number of point clouds in the cloud; S413, Order The geometric centroid obtained in step S412 The current tubular neighborhood is obtained by updating the center. : in, The point cloud set representing the weld sub-region, Indicates the tubular radius parameter; S414, in the current tubular neighborhood Select distance to geometric centroid The nearest actual point is used as the center point. : S415, if satisfied or Then the center point Add the weld seam fine skeleton point set and end the iteration; otherwise, return to step S412. This is the convergence threshold.

7. A 3D vision-based intelligent recognition system for steel structure fillet welds, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the method as described in any one of claims 1-6 when running the computer program.

Images