Online Detection and Evaluation Method and System for Weld Appearance and Surface Defects

Abstract

The present invention provides a method and system for on-line detection and evaluation of weld appearance and surface defects, including: scanning along the longitudinal direction of the weld by a line laser to obtain the overall three-dimensional contour height data of the weld appearance; converting the height data into 3D point cloud and 2D grayscale image; extracting the weld area from the 3D point cloud and calculating the overall weld appearance; using a polygon positioning frame to label the surface defects of the weld on the 2D grayscale image; establishing a fully convolutional neural network defect detection model; visualizing, spatially positioning and calculating the geometric dimensions of the defects determined by the model; combining the overall weld appearance with the surface defect detection to comprehensively evaluate the surface quality of the weld; and on-line detecting and evaluating the surface quality of the actually welded weld. The present invention can accurately calculate the overall weld appearance and comprehensively determine the surface defects, overcoming the disadvantages of the traditional weld surface quality detection process being cumbersome, having insufficient adaptability, incomplete information and low detection efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of nondestructive testing of welds. Specifically, it relates to a method and system for on-line detection and evaluation of weld morphology and surface defects. In particular, it relates to a method for on-line detection and evaluation of weld morphology and surface defects based on a fully convolutional neural network and line laser. Background Art

[0002] Welding is one of the most widely used processing and connection methods in modern industrial production. The detection and control of welding quality are very important and determine whether the welding is qualified and meets the usage requirements. Welding quality mainly depends on several aspects such as weld formation, weld microstructure properties, and weld mechanical properties. Among them, weld formation is an important indicator to measure welding quality and is closely related to weld quality. Its detection content mainly includes weld morphology and surface defect detection and weld internal defect detection. Among them, weld internal defect detection is mainly realized by nondestructive testing methods such as X-ray and ultrasonic flaw detection, which are mainly applied to the detection of welds in pressure vessels and important load-bearing structures and already have perfect and strict quality detection standards. Weld morphology and surface defect detection are the most widely used, and basically all industrial welds need to be subjected to morphology and surface defect detection. At present, in the welding of important industries such as nuclear power, chemical industrial containers, high-speed rail manufacturing, and automobile and ship manufacturing, in addition to internal detection requirements, strict weld morphology and surface defect detection are required.

[0003] Traditional weld surface quality detection uses contact measurement methods and uses auxiliary measurement tools such as gauges, calipers, and coordinate measuring machines to measure weld geometric features. It not only has low efficiency and poor stability, but also the measuring tool is easy to directly contact the workpiece surface during the measurement process, causing damage to the workpiece surface. Non-contact measurement technology has emerged. Without contacting the surface of the object to be measured, it obtains data information on the surface of the object through media such as light, electricity, and sound, including ultrasonic detection, eddy current detection, penetrant detection, and visual detection, etc., so as to carry out surface quality detection.

[0004] The line laser scanning method is a relatively advanced non-contact contour detection method in current industrial applications. It can obtain the contour morphology information of an object and realize three-dimensional high-precision measurement of the object. At present, it has been applied to the detection of railway tracks, tire edge wear, etc. Traditional detection methods rely on professional inspectors to evaluate weld quality, and there are problems such as low detection efficiency, insufficient consistency of detection quality, and high professional requirements for judgment. With the continuous improvement of the intelligent level of production and manufacturing, the demand for automated and high-precision weld defect detection technology in the welding field is also increasing. Process optimization, improvement of welding quality, and on-site on-line real-time determination are becoming more and more important.

[0005] In recent years, with the rapid development of computer and digital technologies, computer vision has been widely used in the welding field due to its large amount of information, high precision, and large detection range. Machine vision technology can simulate the visual perception of the human eye, obtain information, extract features, process, and understand from the collected original images, and finally apply them to actual weld defect classification, target detection, and semantic understanding. Weld detection based on machine vision technology meets the needs of digital production and has demonstrated excellent performance in weld quality detection.

[0006] The invention patent with the publication number CN107764205B discloses a three-dimensional detection device and method for the weld appearance of high-frequency resistance welding based on line-structured light scanning. The device includes a line-structured light sensor, a linear displacement transmission system, and a computer. The line-structured light sensor includes a laser, an industrial camera, and a fixed panel; the linear displacement transmission system includes a base, a stepping motor, and a stepping motor controller. The welded object is placed on the base of the transmission system, and line-structured light scanning is used. The industrial camera captures the weld laser stripe image, and through software system processing, the three-dimensional point cloud data of the weld is obtained, and the weld width, height, and other appearance features are analyzed, so as to judge the weld quality.

[0007] The invention patent with the publication number CN104697467B discloses a method for detecting the weld appearance shape and surface defects based on line laser scanning. The actual contour curve of any cross-section of the weld is obtained by using line laser, and the high-dimensional fitting of the actual contour curve is performed to obtain the fitted contour curve. Furthermore, the first derivative curve of the fitted contour curve is obtained, and the addressing range of the weld start point and end point is determined from the first derivative curve. The points on the actual contour curve with the largest difference from the fitted contour curve within the addressing range are taken as the weld start point and end point of the weld cross-section. Scanning longitudinally along the weld, the overall three-dimensional contour image of the weld is obtained. If the difference between any point on the weld contour curve and the fitted contour curve exceeds the preset standard value, it is determined that there is a defect at that weld.

[0008] The invention patent with the publication number CN114354639B discloses a method and system for real-time detection of weld defects based on 3D point cloud. The method includes: setting a normal weld contour template based on historical data; scanning the weld cross-section by line laser to collect the 3D point cloud data of the weld in real time, and recording the initial contour data obtained by the real-time line laser scanning as data; performing inflection point detection based on the DBSCAN density clustering algorithm to determine the current weld contour data; calculating the distance d between the current weld contour data and the normal weld contour template through the DTW algorithm, and thereby judging whether there are defects on the weld surface.

[0009] The invention patent with the publication number CN109900706B discloses a method for detecting weld seams and weld seam defects based on deep learning, which uses the YOLOV3 network to realize the detection of weld seams and / or weld seam defects; the training steps of the network are as follows: using a positioning frame to frame and mark the weld seam in the workpiece image as the training data set; using a positioning frame to frame and mark the defect type of the weld seam defect in the weld seam image as the training data set I; obtaining the coordinates xp, yp of the positioning frame, as well as the width and height dimensions wp, hp; initializing the network; randomly retrieving the input tensor aj for training calculation and outputting the detection result; calculating the error function loss of the prediction result using the detection result; adjusting the weight W and the bias value b in combination with the gradient descent method, and repeating in this way to obtain the trained network.

[0010] The above-mentioned weld seam detection method generally uses single data such as point cloud or image, with a single function, and it is difficult to simultaneously realize the detection and evaluation of weld seam morphology and surface defects; in terms of weld seam morphology detection, single-stripe curve fitting is mostly used to extract weld seams, which is greatly affected by noise; in terms of weld seam defect detection, the defect extraction method based on three-dimensional point cloud mostly uses the outlier detection method to judge the presence or absence of defects, and it is difficult to determine the specific category of defects and the false detection rate is relatively high; while the defect detection method based on two-dimensional image is greatly affected by light and it is difficult to adapt to on-line application in the welding site; at the same time, the target detection model is generally used to obtain the anchor box of the defect, which does not have the ability to extract the defect contour, and it is even more impossible to realize richer defect information such as three-dimensional visualization, spatial positioning and three-dimensional size calculation of the defect. Summary of the Invention

[0011] Aiming at the defects in the prior art, the purpose of the present invention is to provide a method and system for on-line detection and evaluation of weld seam morphology and surface defects.

[0012] According to the method for on-line detection and evaluation of weld seam morphology and surface defects provided by the present invention, it includes:

[0013] Step S1: Scanning along the longitudinal direction of the weld seam with a line laser to obtain the overall three-dimensional contour height data of the weld seam morphology;

[0014] Step S2: Converting the obtained overall three-dimensional contour height data into a 3D point cloud and a 2D grayscale image;

[0015] Step S3: Processing the 3D point cloud, extracting the weld seam area, and calculating the overall morphology of the weld seam;

[0016] Step S4: Using a polygon positioning frame to label the weld seam surface defects in the 2D grayscale image to construct a weld seam surface defect data set;

[0017] Step S5: Establish a fully convolutional neural network defect detection model with the weld 2D grayscale image as the input and pixel-level defect classification as the output, and perform training, validation, and testing;

[0018] Step S6: Visualize, spatially locate, and calculate the geometric dimensions of the defects determined by the defect detection model;

[0019] Step S7: Combine the overall weld morphology with surface defect detection to comprehensively evaluate the surface quality of the weld;

[0020] Step S8: Based on the constructed comprehensive evaluation system for weld morphology and surface defect detection, perform online detection and evaluation of the surface quality of the actually welded welds.

[0021] Preferably, the step S1 includes: performing line laser scanning on the weld in three-dimensional space: taking the longitudinal direction of the weld as the Y-axis direction, the cross-sectional direction as the X-axis direction, and the height as the Z-axis direction, with the line laser parallel to the X-axis, moving along the Y-axis direction at a preset constant speed, and obtaining the height data of the weld cross-section according to the preset acquisition frequency;

[0022] Preferably, the step S2 includes: Step S2.1: Convert the scanned height data into 3D point cloud according to the spatial resolutions of the X-axis and Y-axis; Step S2.2: Use an image processing algorithm combining Sobel and Gaussian enhancement to convert the scanned height data into a 2D grayscale image, and then perform sub-frame division on the 2D grayscale image using an equally spaced cropping method;

[0023] Preferably, the step S3 includes: Step S3.1: Remove the invalid data points in the 3D point cloud; Step S3.2: Identify the points within the weld area of the 3D point cloud; Step S3.3: Obtain the 3D point cloud of the weld area; Step S3.4: Calculate the overall weld morphology based on the points in the weld area;

[0024] Preferably, the step S4 includes: Step S4.1: Use a polygon positioning frame to label different types of surface defects of the weld on the 2D grayscale image to obtain the corresponding mask image; Step S4.2: Rotate, mirror, and add noise to the 2D grayscale image and the mask image for data augmentation to form a weld surface defect dataset; Step S4.3: Divide the obtained dataset into a training set, a validation set, and a test set according to a preset ratio;

[0025] Preferably, the step S5 includes:

[0026] Step S5.1: Establish a fully convolutional neural network defect detection model with the weld 2D grayscale image as the input and pixel-level defect classification as the output;

[0027] Step S5.2: Configure the model training parameters, including setting the learning rate, decay method, optimizer, as well as the batch size batch_size and the number of epochs;

[0028] Step S5.3: Preprocess the 2D grayscale image: First, convert it to a three-channel RGB image, and then crop and scale it to the standard size required by the fully convolutional neural network;

[0029] Step S5.4: Perform transfer learning on the established defect detection model: First, freeze the weight parameters of the feature extraction part, fine-tune the network, and after a preset number of training epochs, then unfreeze the feature extraction part for full-model weight training;

[0030] Step S5.5: Use the validation set to check for overfitting, and determine whether there is an overfitting phenomenon where the loss of the defect detection model for the validation set first decreases and then increases instead. When the loss of the validation set no longer decreases, stop training;

[0031] Step S5.6: Use the test set to test the accuracy and recall rate indicators of the defect detection model, and verify the adaptability and accuracy of the model.

[0032] Preferably, the said Step S6 includes:

[0033] Step S6.1: Extract defects from the network output: The output of the defect detection model is a three-dimensional tensor of [height, width, num_classes], where height, width, and num_classes are the tensor height, width, and depth respectively, and the pixel values in the output three-dimensional tensor are the predicted probability values of the corresponding defect classes; First, scale the three-dimensional defect prediction tensor output by the model back to the original image size, then obtain the mask images of different defects according to the category corresponding to the maximum predicted value, and then perform connected component extraction on the mask images of different category defects respectively to obtain the two-dimensional contours of each defect;

[0034] Step S6.2: Defect visualization, spatial positioning, and size calculation: Reconstruct the two-dimensional topological relationship of the defect based on Delaunay triangulation, and return to the three-dimensional point cloud to obtain the visualization result of the defect; Combine the two-dimensional defect contour with the 3D point cloud for analysis, and obtain the position in the actual weld according to the relative position and frame number of the defect in the sub-frame; Obtain the three-dimensional cuboid bounding box of the defect point cloud, and calculate the length, width, and height of the defect;

[0035] The said Step S7 includes: Combine the weld area obtained in Step S3, determine whether the defect is on the weld surface, exclude misdetected defects, and comprehensively evaluate the weld morphology and whether the defect exceeds the standard according to the corresponding welding quality standards.

[0036] The on-line detection and evaluation system for weld appearance and surface defects provided by the present invention includes:

[0037] Data acquisition module M1: Scanning along the longitudinal direction of the weld by line laser to obtain the overall three-dimensional contour height data of the weld appearance;

[0038] Data conversion module M2: Converting the overall three-dimensional contour height data obtained by scanning into 3D point cloud and 2D grayscale image;

[0039] Appearance calculation module M3: Processing the 3D point cloud, extracting the weld area, and calculating the overall appearance of the weld;

[0040] Data set construction module M4: Using a polygon positioning frame to label the surface defects of the weld on the 2D grayscale image, and constructing a data set of weld surface defects;

[0041] Network construction module M5: Taking the 2D grayscale image of the weld as the input, establishing a fully convolutional neural network defect detection model with pixel-level defect classification as the output, and performing training, verification and testing;

[0042] Defect detection module M6: Visualizing, spatially positioning and calculating the geometric dimensions of the defects determined by the defect detection model;

[0043] Comprehensive quality evaluation module M7: Combining the overall appearance of the weld with the surface defect detection to comprehensively evaluate the surface quality of the weld;

[0044] On-line detection and evaluation module M8: Based on the constructed comprehensive evaluation system for weld appearance and surface defect detection, on-line detecting and evaluating the surface quality of the actually welded weld.

[0045] Preferably, the module M1 includes: Performing line laser scanning on the weld in three-dimensional space: taking the longitudinal direction of the weld as the Y-axis direction, the cross-sectional direction as the X-axis direction, and the height as the Z-axis direction, the line laser is parallel to the X-axis, moving along the Y-axis direction at a preset constant speed, and obtaining the height data of the weld cross-section according to the preset acquisition frequency;

[0046] Preferably, the module M2 includes: Module M2.1: Converting the scanned height data into 3D point cloud according to the spatial resolution of the X-axis and Y-axis; Module M2.2: Using an image processing algorithm combining Sobel and Gaussian enhancement to convert the scanned height data into 2D grayscale image, and then using an equally spaced cropping method to divide the 2D grayscale image into sub-frames;

[0047] Preferably, the module M3 includes: Module M3.1: Removing invalid data points from the 3D point cloud; Module M3.2: Identifying points within the weld area of the 3D point cloud; Module M3.3: Obtaining the 3D point cloud of the weld area; Module M3.4: Calculating the overall morphology of the weld based on the points in the weld area.

[0048] Preferably, the module M4 includes: Module M4.1: Using a polygon positioning frame to label different types of surface defects of the weld on the 2D grayscale image to obtain a corresponding mask image; Module M4.2: Rotating, mirroring, and adding noise to the 2D grayscale image and the mask image for data augmentation to form a weld surface defect data set; Module M4.3: Dividing the obtained data set into a training set, a validation set, and a test set according to a preset ratio.

[0049] Preferably, the module M5 includes:

[0050] Module M5.1: Establishing a fully convolutional neural network defect detection model with the 2D grayscale image of the weld as the input and pixel-level defect classification as the output;

[0051] Module M5.2: Configuring model training parameters, including setting the learning rate, decay method, optimizer, as well as the batch size batch_size and the number of iterations epoch;

[0052] Module M5.3: Preprocessing the 2D grayscale image: First converting it into a three-channel RGB image, and then cropping and scaling it to the standard size required by the fully convolutional neural network;

[0053] Module M5.4: Performing transfer learning on the established defect detection model: First freezing the weight parameters of the feature extraction part, fine-tuning the network, and after a preset number of training iterations, then unfreezing the feature extraction part for full-model weight training;

[0054] Module M5.5: Using the validation set for overfitting inspection to determine whether the loss change of the defect detection model for the validation set shows an overfitting phenomenon of first decreasing and then increasing instead. When the loss of the validation set no longer decreases, stop training;

[0055] Module M5.6: Testing the accuracy and recall rate indicators of the defect detection model with the test set to verify the adaptability and accuracy of the model.

[0056] Preferably, the module M6 includes:

[0057] Module M6.1: Defect extraction from network output: The output of the defect detection model is a three-dimensional tensor of [height, width, num_classes], where height, width, and num_classes are the height, width, and depth of the tensor respectively. The pixel values in the output three-dimensional tensor are the predicted probability values of the corresponding defect classes. First, scale the three-dimensional defect prediction tensor output by the model back to the original image size, then obtain the mask images of different defects according to the class corresponding to the maximum predicted value, and then perform connected component extraction on the mask images of different classes of defects to obtain the two-dimensional contours of each defect.

[0058] Module M6.2: Defect visualization, spatial positioning, and size calculation: Reconstruct the two-dimensional topological relationship of the defect based on Delaunay triangulation and return to the three-dimensional point cloud to obtain the visualization result of the defect; Combine the two-dimensional defect contour with the 3D point cloud for analysis, and obtain the position in the actual weld according to the relative position and frame number of the defect in the sub-frame; Obtain the three-dimensional cuboid bounding box of the defect point cloud, and calculate the length, width, and height of the defect.

[0059] The said module M7 includes: Combining the weld area obtained by module M3, judging whether the defect is on the weld surface, excluding misdetected defects, and comprehensively evaluating the weld morphology and whether the defect exceeds the standard according to the corresponding welding quality standard.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] (1) The traditional passive vision sensing is greatly affected by the detection environment light, etc. The present invention adopts line laser vision detection, which has strong anti-interference ability to the environment light; At the same time, the present invention can simultaneously realize the comprehensive evaluation of weld morphology and defect detection, effectively improving the measurement efficiency;

[0062] (2) The traditional welding defect detection method has a complex process and insufficient adaptability, and it is difficult to adapt to the diverse weld forms and dynamic changes of light on site. The present invention adopts a fully convolutional network of deep learning, makes full use of its efficient feature extraction ability, can realize pixel-level identification of defects, has high accuracy and low missed detection rate, and at the same time has strong adaptability to diverse weld forms on site, and is applicable to on-line detection of welds on site.

[0063] (3) The detection results of traditional weld surface detection methods only contain the distribution information of defects in the image, lacking specific three-dimensional spatial information such as position and size. The present invention uses line laser for three-dimensional measurement, obtains 3D point cloud and 2D grayscale image through data conversion, calculates the weld surface topography parameters based on the point cloud, realizes pixel-level classification of weld surface defects based on the 2D grayscale image, combines the 3D point cloud to obtain the spatial information of defects in the X, Y, and Z dimensions, conducts defect positioning and geometric dimension calculation, and realizes comprehensive and accurate detection and evaluation of the weld surface through one-time on-line scanning of the line laser. Brief Description of the Drawings

[0064] Other features, objectives, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0065] Figure 1 It is a flow chart of an on-line detection and evaluation method for weld topography and surface defects based on a fully convolutional neural network and line laser in an embodiment of the present invention;

[0066] Figure 2 It is a schematic diagram of line laser scanning the weld;

[0067] Figure 3a It is the original weld image, Figure 3b It is the 2D grayscale image;

[0068] Figure 4 It is an intermediate result of the 3D point cloud processing process;

[0069] Figure 5 It is a schematic diagram of calculating the overall topography of the weld;

[0070] Figure 6 It is the annotation result of weld surface defects in the 2D grayscale image;

[0071] Figure 7 It is the adopted fully convolutional neural network structure;

[0072] Figure 8a and Figure 8b It is the convergence situation of the loss function for training the fully convolutional network with the dataset;

[0073] Figures 9a to 9d It is to verify the prediction effect of the fully convolutional neural network with the test set;

[0074] Figure 10 It is the comprehensive detection and evaluation result of weld topography and surface defects. Detailed Embodiment

[0075] The present invention will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several changes and improvements can still be made. These all belong to the protection scope of the present invention.

[0076] Embodiment 1

[0077] An on-line detection and evaluation method for weld morphology and surface defects based on a fully convolutional neural network and line laser is provided in an embodiment of the present invention, as Figure 1 shown below:

[0078] Use a line laser sensor to scan the weld to obtain the overall contour height data of the weld, convert it into a 3D point cloud and a 2D grayscale image, use 3D point cloud processing to extract the complete contour of the weld, and then calculate the overall morphology of the weld; after preprocessing the 2D grayscale image, input it into the fully convolutional neural network weld surface defect detection model to realize the detection of weld surface defects; combine the results of the two to obtain the morphology parameters of the weld and the type, location and geometric size of the defects, and comprehensively evaluate the surface quality of the weld. It includes the following steps:

[0079] Step S1: Scan along the longitudinal direction of the weld through a line laser sensor to obtain the overall three-dimensional contour height data of the weld, as Figure 2 shown;

[0080] The said step S1 includes: performing line laser scanning on the weld in a three-dimensional space. Taking the longitudinal direction of the weld as the Y-axis direction, the cross-sectional direction as the X-axis direction, and the height as the Z-axis direction, the line laser sensor generates a line laser in the X-axis direction, projects and focuses on the weld surface, and moves along the Y-axis direction. In order to ensure the detection accuracy, the origin position of the X-axis is corrected before scanning to align it with the center of the weld. The line laser scans once to generate N points, and the contour information of the weld in the Z-axis is obtained (the length is N, the model of the line laser measuring instrument used in the embodiment is Keyence line laser measuring instrument, N = 800).

[0081] The line laser sensor moves along the Y-axis direction and continuously collects the weld contour data of different cross-sections at a certain scanning frequency; finally, the scanning height data of the overall original contour of the weld is obtained (M×N matrix, M is determined by the specific weld length, N = 800). The acquisition frequency used in the embodiment is 1000hz, and the moving speed is 150mm / s.

[0082] Step S2: Convert the scanned overall three-dimensional contour height data into a 3D point cloud and a 2D grayscale image;

[0083] The said step S2 includes:

[0084] Step S2.1: Convert the scanned height data into 3D point cloud according to the spatial resolutions of the X-axis and Y-axis. First, determine the conversion coefficient of the 3D point cloud. Use x_res to represent the spatial resolution of the X-axis, that is, the actual distance corresponding to adjacent points on the same row of the scanned data, and y_res to represent the spatial resolution of the Y-axis, that is, the actual distance corresponding to adjacent points on the same column; then perform the spatial point cloud conversion. Traverse the data points of the scanned data, and convert each data point into a spatial point in the 3D point cloud through the conversion coefficient. The conversion formula is: X = i * x_res, Y = j * y_res, Z = value; where, X, Y, and Z represent the coordinates of the spatial points in the 3D point cloud, i represents the row of the scanned data, j represents the column of the scanned data, and value represents the height value of the data point (i, j) on the scanned data. In the embodiment, the parameters are x_res = 0.1mm and y_res = 0.15mm.

[0085] Step S2.2: Convert the scanned height data into a 2D grayscale image by using the image processing algorithm of Sobel + Gaussian enhancement. First, use the Sobel operator to calculate the height difference diff_values between each point on each scanned data and its left neighbor; secondly, calculate the maximum value and the minimum value of the height difference diff_values, and linearly scale the range of the corresponding height difference from the minimum value to the maximum value to the range of 0 - 255 as the pixel value of the corresponding grayscale image; then use Gaussian enhancement to improve the image contrast. Specifically, calculate the pixel mean and standard deviation of the grayscale image, calculate the linear scaling coefficient a = 255 / (6 * dev), b = -a * (mean - 3 * dev), and perform linear scaling on each pixel point to obtain the final result of the grayscale image, as shown in Figure 3; use the equal-interval cropping method to divide the 2D grayscale image into sub-frames. In the row direction, divide it into sub-frames of [800, 800] at an interval of 800 rows, and fill the part with less than 800 rows with the grayscale value 128.

[0086] Step S3: Process the 3D point cloud, extract the weld area, as Figure 4 shown, and calculate the overall morphology of the weld.

[0087] The said Step S3 includes:

[0088] Step S3.1: Remove the invalid data points in the 3D point cloud. According to the valid range of the scanned data, set the valid threshold to -200, and directly filter out the values less than this value;

[0089] Step S3.2: Identify the points within the weld area of the 3D point cloud. First, fit the reference plane of the base material; then, traverse all the points in the 3D point cloud, calculate the distance from each point to the reference plane, and if the distance is greater than the set distance threshold, it means that the point belongs to the points within the weld area;

[0090] Step S3.3: Obtain the 3D point cloud of the weld area. First, convert the points within the weld area into a binary image according to the conversion coefficient of the 3D point cloud, where 1 indicates the existence of the point and 0 indicates the non-existence of the point; then remove the redundant noise points through median filtering, use morphological operations to remove the redundant white points and repair the black holes, retain the connected domain with the largest area to obtain the weld area mask; finally, perform 3D point cloud conversion using the weld area mask and the scan data to obtain the point cloud of the weld area;

[0091] Step S3.4: Calculate the overall morphology of the weld based on the point cloud of the weld area. Perform Canny edge extraction according to the weld area mask to obtain the weld edge contour, and calculate the weld edge shape parameters based on this, including the weld length and straightness; calculate the weld shape parameters of each weld cross-section according to the point cloud of the weld area in each row, including the weld thickness, width, concavity and convexity, and the weld leg (as Figure 5 shown), and calculate the statistical results such as the average value and standard deviation of each parameter.

[0092] In step S3.2, use the random sample consensus (RANSAC) algorithm to fit the reference plane of the base material. The process is as follows:

[0093] Step S3.2.1: Randomly select 3 non-collinear target points in the 3D point cloud, find the plane equation of the plane formed by the 3 target points to obtain the plane model;

[0094] Step S3.2.2: Select other points in the 3D point cloud image and calculate the point-plane distance from the selected points to the plane model determined by the plane equation. Compare the point-plane distance with the preset minimum distance threshold. If the point-plane distance is less than the minimum distance threshold, the selected point is an inlier, otherwise it is an outlier, and record the number of all inliers;

[0095] Step S3.2.3: Repeat steps S3.2.1 and S3.2.2. If the number of inliers of the current plane model exceeds the number of inliers of the previous plane model, record the current plane model;

[0096] Step S3.2.4: After reaching the set number of iterations, the plane with the most inliers is the reference plane.

[0097] Step S4: Use a polygon positioning frame to label the surface defects of the weld on the 2D grayscale image and construct a weld surface defect dataset;

[0098] The said step S4 includes:

[0099] Step S4.1: Use a polygon positioning frame to label different types of surface defects of the weld on the 2D grayscale image to obtain the corresponding mask image, as Figure 6 shown;

[0100] Step S4.2: Rotate, mirror, and add noise to the 2D grayscale image and the mask image for data augmentation to form a weld surface defect dataset;

[0101] Step S4.3: Divide the obtained dataset into a training set, a validation set, and a test set according to the ratios of 75%, 10%, and 15%.

[0102] The classification of the weld surface defects includes: porosity, undercut, and overlap; the corresponding class labels during annotation are: background 0, porosity 1, undercut 2, overlap 3, that is, the class parameter num_classes of the dataset = 4.

[0103] Step S5: Establish a fully convolutional neural network defect detection model with the 2D grayscale image as the input and pixel-level defect classification as the output, and perform training, validation, and testing;

[0104] The said Step S5 includes:

[0105] Step S5.1: Establish a fully convolutional neural network defect detection model with the 2D grayscale image as the input and pixel-level defect classification as the output. The input shape of the fully convolutional neural network is [height, width, 3], where height and width are the height and width of the network input image respectively. In the embodiment, height = 512 and width = 512. The structure of the fully convolutional defect detection network is as Figure 7 shown, which can be divided into three parts: The first part is the feature extraction part, which stacks convolutional layers and max-pooling layers to obtain five preliminary effective feature layers; the second part is the feature restoration part, which performs upsampling on the five preliminary effective feature layers obtained by the feature extraction part and conducts feature fusion to obtain a final effective feature layer that fuses all features; the third part is the prediction part, which classifies each feature point using the last obtained effective feature layer.

[0106] The feature extraction part consists of a stack of convolutional layers + max pooling layers, adopting the VGG16 network structure. The input image size is 512*512*3. The specific layers are as follows: conv1: Perform two convolutions with 64 channels of [3,3] to obtain a preliminary effective feature layer of [512,512,64], and then perform 2×2 max pooling to obtain a feature layer of [256,256,64]; conv2: Perform two convolutions with 128 channels of [3,3] to obtain a preliminary effective feature layer of [256,256,128], and then perform 2×2 max pooling to obtain a feature layer of [128,128,128]; conv3: Perform three convolutions with 256 channels of [3,3] to obtain a preliminary effective feature layer of [128,128,256], and then perform 2×2 max pooling to obtain a feature layer of [64,64,256]; conv4: Perform three convolutions with 512 channels of [3,3] to obtain a preliminary effective feature layer of [64,64,512], and then perform 2×2 max pooling to obtain a feature layer of [32,32,512]; conv5: Perform three convolutions with 512 channels of [3,3] to obtain a preliminary effective feature layer of [32,32,512].

[0107] The feature restoration part uses five preliminary effective feature layers for feature fusion, that is, upsampling and stacking the feature layers. When upsampling, directly perform two-fold upsampling and then perform feature fusion. Finally, the obtained feature layer has the same height and width as the input picture.

[0108] In the prediction part, use the feature to obtain the prediction result. Use a 1×1 convolution to adjust the channels and adjust the number of channels of the final layer to num_classes = 4.

[0109] The loss function uses Dice Loss. The calculation formula is as follows:

[0110]

[0111] Among them, |X∩Y| is the true value, that is, TP, and |X| and |Y| are the true value and the predicted value respectively. TP is the true positive (TP), true negative (TN), false positive (FP), and false negative (FN). Among these four parts, the negative example refers to the part of the non-target label, such as the background, etc., and the positive example refers to the part of the target label, that is, the target task.

[0112] Step S5.2: Use the Adam (Adaptive moment estimation) optimizer to update the model weight parameters. The initial value of the learning rate is set to 1e-4, and the learning rate decay method is used, with momentum = 0.9, batch size batch_size = 2, the number of iterations epoch in the freezing stage is 100, and the number of iterations epoch = 100 in the unfreezing stage.

[0113] Step S5.3: Preprocess the 2D grayscale image. First, convert it to a 3-channel RGB image, and crop and scale the original 800×800 image to the standard input size of the fully convolutional neural network, which is 512×512.

[0114] Step S5.4: Perform transfer learning on the established defect detection model. First, freeze the weight parameters of the feature extraction part, including conv1, conv2, conv3, conv4, and conv5, and fine-tune the network. After a certain number of training iterations, then unfreeze the feature extraction part for the weight training of the entire model;

[0115] Step S5.5: Use the validation set to check for overfitting, and determine whether there is an overfitting phenomenon where the loss of the defect detection model on the validation set first decreases and then increases instead. When the loss of the validation set no longer decreases, stop training.

[0116] During the training process, the convergence of the loss function of the model and the changes in the metrics are shown in Figure 8.

[0117] Step S5.6: Use the test set to test the prediction metrics of the defect detection model, including metrics such as the mean intersection over union, pixel accuracy, accuracy, and recall rate. The calculation formulas are as follows:

[0118]

[0119]

[0120]

[0121]

[0122] Among them, assume that there are k + 1 classes including the background, and p ij is the number of pixels belonging to class i but predicted as class j, that is, p ii represents the number of correctly classified class i.

[0123] As shown in Figure 9, it is the semantic segmentation evaluation obtained through the test set.

[0124] Step S6: Visualize the defects determined by the defect detection model, locate them in space, and calculate their geometric dimensions.

[0125] The said Step S6 includes:

[0126] Step S6.1: Extract defects from network data. The output of the defect detection model is a three-dimensional tensor of [height, width, num_classes], where height, width, and num_classes are the height, width, and depth of the tensor respectively, and the pixel values in the output three-dimensional tensor are the predicted probability values of the corresponding defect classes. First, scale the three-dimensional defect prediction tensor output by the model back to the original image size, then obtain the mask images of different defects according to the class corresponding to the maximum predicted value, and then perform connected component extraction on the mask images of different class defects respectively to obtain the two-dimensional contours of each defect;

[0127] Step S6.2: Visualize the defects, locate them in space, and calculate their dimensions. Reconstruct the two-dimensional topological relationship of the defects based on Delaunay triangulation, and return to the three-dimensional point cloud to obtain the visualization result of the defects; Combine the two-dimensional defect contours with the 3D point cloud for analysis, and convert the relative position and frame number of the defects in the sub-frame into their positions in the actual weld; Obtain the three-dimensional cuboid bounding box of the defect point cloud, and calculate the length, width, and height of the defect; The detection results of weld defects include the types, positions, and specific length, width, and height of the defects;

[0128] Step S7: Combine the overall morphology of the weld with the surface defect detection to comprehensively evaluate the surface quality of the weld.

[0129] The said Step S7 includes: Combine the weld area obtained in Step S3 to judge whether the defect is on the weld surface, exclude misdetected defects, and comprehensively evaluate whether the weld morphology and defects exceed the standard according to the corresponding welding quality standards.

[0130] First, exclude misdetected defects according to the basic characteristics of the defects. (1) The position of the defect: The position of the porosity defect is generally inside the weld, the position of the undercut defect is generally near both sides of the weld, and the position of the overlap defect is generally near the middle of the weld; (2) The shape of the defect: Most porosities are circular or elliptical, with regular shapes, undercuts are generally linear, with a certain length, and the shape of the overlap is generally ellipsoidal, with a certain length and width; (3) The size of the defect: The length and width of the porosity are both small, the undercut is relatively slender, the length is generally significantly greater than the width, and the overlap defect is relatively thick, presenting a convex hill shape, with an aspect ratio of about 0.5 - 2. Misdetected defects can be excluded based on these characteristics;

[0131] Then, judge whether the weld morphology and defects exceed the standard according to the detection standard. The main judgment criteria are as follows:

[0132] (a) Whether the weld length exceeds the given standard value;

[0133] (b) Whether the minimum and maximum values and the fluctuation range of the weld thickness exceed the given standard values;

[0134] (c) Whether the minimum and maximum values and the fluctuation range of the weld width exceed the given standard values;

[0135] (d) Whether the straightness of the weld and the deviation ratio on both sides exceed the given standard values;

[0136] (e) Whether the minimum and maximum values and the fluctuation range of the weld concavity and convexity exceed the given standard values;

[0137] (f) Whether the minimum and maximum values and the fluctuation range of the weld leg exceed the given standard values;

[0138] (g) Whether the length, width, and height of the defect exceed the given standard values.

[0139] Figure 10 The comprehensive evaluation results of the obtained weld appearance and surface defects are shown. Among them, the highlighted display indicates the detection results that exceed the tolerance.

[0140] Step S8: According to the constructed comprehensive evaluation system for weld appearance and surface defect detection, online detection and evaluation of the surface quality of the actually welded welds are carried out.

[0141] The said step S8 includes: after welding is completed, use a line laser sensor to scan the weld to obtain the overall contour height data of the weld, convert it into a 3D point cloud and a 2D grayscale image, use 3D point cloud processing to extract the complete contour of the weld, and then calculate the overall appearance of the weld; after preprocessing the 2D grayscale image, input it into the fully convolutional neural network weld surface defect detection model to obtain the pixel-level classification output of the defect; realize defect visualization, calculate the spatial position and three-dimensional dimensions, and combine with the overall appearance of the weld to realize the online comprehensive determination of the weld surface quality.

[0142] The present invention can not only accurately obtain the overall contour appearance of the weld and the pixel-level classification of the defect, but also obtain rich information such as the visualization of the defect, spatial positioning, and the three-dimensional geometric dimensions of the defect; based on laser scanning for data acquisition, it has strong anti-lighting change ability and is suitable for the complex scenarios of the welding site; the defect detection network based on full convolution has fast inference speed, high determination accuracy, requires a small number of samples, and has strong generalization ability, and is suitable for the online real-time determination of various types of welds on site. Therefore, the present invention has great popularization and application value both from the perspectives of economic benefits and social benefits.

[0143] Embodiment 2

[0144] The present invention also provides an on-line detection and evaluation system for weld appearance and surface defects. The on-line detection and evaluation system for weld appearance and surface defects can be implemented by executing the process steps of the on-line detection and evaluation method for weld appearance and surface defects. That is, those skilled in the art can understand the on-line detection and evaluation method for weld appearance and surface defects as a preferred implementation manner of the on-line detection and evaluation system for weld appearance and surface defects.

[0145] The on-line detection and evaluation system for weld appearance and surface defects provided by the present invention includes: a data acquisition module M1: scanning along the longitudinal direction of the weld by line laser to obtain the overall three-dimensional contour height data of the weld appearance; a data conversion module M2: converting the scanned overall three-dimensional contour height data into 3D point cloud and 2D grayscale image; a morphology calculation module M3: processing the 3D point cloud, extracting the weld area, and calculating the overall weld appearance; a data set construction module M4: using a polygon positioning frame to label the surface defects of the weld on the 2D grayscale image and constructing a data set of weld surface defects; a network construction module M5: establishing a fully convolutional neural network defect detection model with the 2D grayscale image of the weld as the input and pixel-level defect classification as the output, and performing training, verification, and testing; a defect detection module M6: visualizing, spatially positioning, and calculating the geometric dimensions of the defects determined by the defect detection model; a comprehensive quality evaluation module M7: combining the overall weld appearance with the surface defect detection to comprehensively evaluate the surface quality of the weld; an on-line detection and evaluation module M8: on-line detecting and evaluating the surface quality of the actually welded weld according to the constructed comprehensive evaluation system for weld appearance and surface defect detection.

[0146] The module M1 includes: performing line laser scanning on the weld in three-dimensional space: taking the longitudinal direction of the weld as the Y-axis direction, the cross-sectional direction as the X-axis direction, and the height as the Z-axis direction, with the line laser parallel to the X-axis, moving along the Y-axis direction at a preset constant speed, and obtaining the height data of the weld cross-section according to the preset acquisition frequency;

[0147] The module M2 includes: module M2.1: converting the scanned height data into 3D point cloud according to the spatial resolutions of the X-axis and Y-axis; module M2.2: converting the scanned height data into 2D grayscale image by using an image processing algorithm combining Sobel and Gaussian enhancement, and then performing sub-frame division on the 2D grayscale image by using an equal-interval cropping method;

[0148] The module M3 includes: module M3.1: removing invalid data points in the 3D point cloud; module M3.2: identifying the points within the weld area of the 3D point cloud; module M3.3: obtaining the 3D point cloud of the weld area; module M3.4: calculating the overall weld appearance according to the points in the weld area;

[0149] The module M4 includes: Module M4.1: Label different types of surface defects of the weld seam on the 2D grayscale image using a polygon positioning frame to obtain a corresponding mask image; Module M4.2: Rotate, mirror, and add noise to the 2D grayscale image and the mask image for data augmentation to construct a weld seam surface defect dataset; Module M4.3: Divide the obtained dataset into a training set, a validation set, and a test set according to a preset ratio.

[0150] The module M5 includes: Module M5.1: Establish a fully convolutional neural network defect detection model with the 2D grayscale image of the weld seam as the input and pixel-level defect classification as the output; Module M5.2: Configure the model training parameters, including setting the learning rate, decay method, optimizer, as well as the batch size batch_size and the number of iterations epoch; Module M5.3: Preprocess the 2D grayscale image: First, convert it into a three-channel RGB image, and then crop and scale it to the standard size required by the fully convolutional neural network; Module M5.4: Perform transfer learning on the established defect detection model: First, freeze the weight parameters of the feature extraction part, fine-tune the network, and after a preset number of training iterations, then unfreeze the feature extraction part for full-model weight training; Module M5.5: Use the validation set for overfitting inspection to determine whether the loss change of the defect detection model for the validation set shows an overfitting phenomenon of first decreasing and then increasing instead. When the loss of the validation set no longer decreases, stop training; Module M5.6: Use the test set to test the accuracy and recall rate indicators of the defect detection model to verify the adaptability and accuracy of the model.

[0151] The module M6 includes: Module M6.1: Extract defects from the network output: The output of the defect detection model is a three-dimensional tensor of [height, width, num_classes], where height, width, and num_classes are the height, width, and depth of the tensor respectively, and the pixel values in the output three-dimensional tensor are the predicted probability values of the corresponding defect classes; First, scale the defect prediction three-dimensional tensor output by the model back to the original image size, then obtain the mask images of different defects according to the category corresponding to the maximum predicted value, and then perform connected component extraction on the mask images of different category defects respectively to obtain the two-dimensional contours of each defect; Module M6.2: Defect visualization, spatial positioning, and size calculation: Reconstruct the two-dimensional topological relationship of the defects based on Delaunay triangulation and return to the three-dimensional point cloud to obtain the visualization result of the defects; Combine the two-dimensional defect contours with the 3D point cloud for analysis, and obtain the position in the actual weld seam based on the relative position and frame number of the defect in the sub-frame; Obtain the three-dimensional cuboid bounding box of the defect point cloud and calculate the length, width, and height of the defect.

[0152] The module M7 includes: combining the weld region obtained by module M3, determining whether the defect is on the weld surface, excluding misdetected defects, and comprehensively evaluating whether the weld appearance and defects exceed the standard according to the corresponding welding quality standards.

[0153] Those skilled in the art know that in addition to implementing the systems, devices, and their respective modules provided by the present invention in the form of pure computer-readable program codes, the method steps can be logically programmed to enable the systems, devices, and their respective modules provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc. to implement the same program. Therefore, the systems, devices, and their respective modules provided by the present invention can be regarded as a kind of hardware component, and the modules included therein for implementing various programs can also be regarded as the structures within the hardware component; the modules for implementing various functions can also be regarded as either software programs for implementing the method or the structures within the hardware component.

[0154] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for online detection and evaluation of weld morphology and surface defects, characterized in that: include: Step S1: Scanning the weld longitudinally by a line laser to obtain the overall three-dimensional profile height data of the weld morphology; Step S2: converting the overall three-dimensional profile height data obtained by scanning into a 3D point cloud and a 2D grayscale image; Step S3: Process the 3D point cloud, extract the weld area, and calculate the overall weld morphology; Step S4: annotating the weld surface defects on the 2D grayscale image using a polygonal positioning frame to construct a weld surface defect data set; Step S5: using the weld 2D grayscale image as input and pixel-level defect classification as output, a fully convolutional neural network defect detection model is established, and training, verification and testing are performed; Step S6: Visualize, spatially locate and calculate geometric dimensions of the defects determined by the defect detection model; Step S7: Combine the overall weld morphology with the surface defect detection to comprehensively evaluate the weld surface quality; Step S8: Based on the constructed weld morphology and surface defect detection and comprehensive evaluation system, the surface quality of the actual weld is detected and evaluated online; The step S3 comprises: step S3.1: removing invalid data points in the 3D point cloud; step S3.2: identifying points in the weld area of ​​the 3D point cloud; step S3.3: obtaining the 3D point cloud of the weld area; step S3.4: calculating the overall shape of the weld based on the point cloud of the weld area; The step S4 comprises: step S4.1: using a polygonal positioning frame to mark different types of surface defects of the weld on the 2D grayscale image to obtain a corresponding mask image; step S4.2: rotating, mirroring, adding noise to the 2D grayscale image and the mask image to perform data augmentation to form a weld surface defect data set; step S4.3: dividing the obtained data set into a training set, a validation set and a test set according to a preset ratio; The step S6 comprises: Step S6.1: Extract defects from the network output: The output of the defect detection model is a three-dimensional tensor of [height, width, num_classes], where height, width and num_classes are the height, width and depth of the tensor respectively, and the pixel value in the output three-dimensional tensor is the predicted probability value of the corresponding defect category; first, the defect prediction three-dimensional tensor output by the model is scaled back to the original image size, and then the mask images of different defects are obtained according to the category corresponding to the maximum predicted value, and then the connected domains of the mask images of defects of different categories are extracted respectively to obtain the two-dimensional contours of each defect; Step S6.2: Defect visualization, spatial positioning and size calculation: Reconstruct the two-dimensional topological relationship of the defect based on Delaunay triangulation, and return to the three-dimensional point cloud to obtain the visualization result of the defect; combine the two-dimensional defect contour with the 3D point cloud for analysis, and obtain the position of the defect in the actual weld according to the relative position of the defect in the subframe and the number of frames; obtain the three-dimensional rectangular bounding box of the defect point cloud, and calculate the length, width and height of the defect; The step S7 includes: combining the weld area obtained in step S3, determining whether the defect is on the weld surface, eliminating falsely detected defects, and comprehensively evaluating the weld morphology and whether the defect exceeds the tolerance according to the corresponding welding quality standard.

2. The on-line detection and evaluation method for weld morphology and surface defects according to claim 1 is characterized in that: The step S1 comprises: performing line laser scanning on the weld in three-dimensional space: the longitudinal direction of the weld is the Y-axis direction, the cross-sectional direction is the X-axis direction, and the height is the Z-axis direction. The line laser is parallel to the X-axis and moves along the Y-axis direction at a preset constant speed, and the height data of the weld cross section is obtained according to a preset acquisition frequency; The step S2 includes: step S2.1: converting the scanning height data into a 3D point cloud according to the spatial resolution of the X-axis and the Y-axis; step S2.2: converting the scanning height data into a 2D grayscale image using the Sobel combined with Gaussian enhancement image processing algorithm, and then dividing the 2D grayscale image into subframes using an equal-interval cropping method.

3. The on-line detection and evaluation method for weld morphology and surface defects according to claim 1 is characterized in that: The step S5 comprises: Step S5.1: Establish a fully convolutional neural network defect detection model with the weld 2D grayscale image as input and pixel-level defect classification as output; Step S5.2: Configure model training parameters, including setting learning rate, decay method, optimizer, batch size batch_size, and number of iterations epoch; Step S5.3: Preprocess the 2D grayscale image: first convert it into a three-channel RGB image, then crop and scale it to the standard size required by the full convolutional neural network; Step S5.4: Perform transfer learning on the established defect detection model: first freeze the weight parameters of the feature extraction part, fine-tune the network, and after a preset number of training iterations, unfreeze the feature extraction part to perform weight training on the entire model; Step S5.5: Use the validation set to perform an overfitting check to determine whether the loss change of the defect detection model for the validation set shows an overfitting phenomenon of first decreasing and then increasing. When the loss of the validation set no longer decreases, stop training; Step S5.6: Use the test set to test the accuracy and recall rate indicators of the defect detection model to verify the adaptability and accuracy of the model.

4. A weld morphology and surface defect online detection and evaluation system, characterized in that: include: Data acquisition module M1: Scans the weld longitudinally through a line laser to obtain the overall three-dimensional profile height data of the weld morphology; Data conversion module M2: converts the overall three-dimensional profile height data obtained by scanning into 3D point cloud and 2D grayscale image; Shape calculation module M3: processes the 3D point cloud, extracts the weld area, and calculates the overall shape of the weld; Dataset construction module M4: annotate the weld surface defects using polygonal positioning frames on the 2D grayscale image to construct a weld surface defect dataset; Network construction module M5: takes the weld 2D grayscale image as input and pixel-level defect classification as output to establish a fully convolutional neural network defect detection model, and conducts training, verification and testing; Defect detection module M6: Defect visualization, spatial positioning and geometric dimension calculation of defects determined by the defect detection model; Comprehensive quality assessment module M7: Combines the overall weld morphology with surface defect detection to conduct a comprehensive assessment of the weld surface quality; Online detection and evaluation module M8: Based on the constructed weld morphology and surface defect detection and comprehensive evaluation system, the actual weld surface quality is detected and evaluated online; The module M3 comprises: module M3.1: removing invalid data points in the 3D point cloud; module M3.2: identifying points in the weld area of ​​the 3D point cloud; module M3.3: obtaining the 3D point cloud of the weld area; module M3.4: calculating the overall shape of the weld based on the point cloud of the weld area; The module M4 includes: module M4.1: using a polygonal positioning frame to mark different types of surface defects of the weld on the 2D grayscale image to obtain a corresponding mask image; module M4.2: rotating, mirroring, adding noise to the 2D grayscale image and the mask image to perform data augmentation to form a weld surface defect data set; module M4.3: dividing the obtained data set into a training set, a validation set and a test set according to a preset ratio; The module M6 comprises: Module M6.1: Extract defects from network output: The output of the defect detection model is a three-dimensional tensor of [height, width, num_classes], where height, width and num_classes are the height, width and depth of the tensor respectively. The pixel value in the output three-dimensional tensor is the predicted probability value of the corresponding defect category. First, the defect prediction three-dimensional tensor output by the model is scaled back to the original image size, and then the mask images of different defects are obtained according to the category corresponding to the maximum predicted value. Then, the connected domains of the mask images of defects of different categories are extracted respectively to obtain the two-dimensional contours of each defect. Module M6.2: Defect visualization, spatial positioning and size calculation: Reconstruct the 2D topological relationship of the defect based on Delaunay triangulation, and return to the 3D point cloud to obtain the visualization result of the defect; combine the 2D defect contour with the 3D point cloud for analysis, and obtain the position of the defect in the actual weld according to the relative position of the defect in the subframe and the number of frames; obtain the 3D rectangular bounding box of the defect point cloud, and calculate the length, width and height of the defect; The module M7 includes: combining the weld area obtained by the module M3, judging whether the defect is on the weld surface, eliminating falsely detected defects, and comprehensively evaluating the weld morphology and whether the defect exceeds the tolerance according to the corresponding welding quality standard.

5. The weld morphology and surface defect online detection and evaluation system according to claim 4 is characterized in that: The module M1 includes: performing line laser scanning on the weld in three-dimensional space: the longitudinal direction of the weld is the Y-axis direction, the cross-sectional direction is the X-axis direction, and the height is the Z-axis direction. The line laser is parallel to the X-axis and moves along the Y-axis direction at a preset constant speed, and the height data of the weld cross section is obtained according to a preset acquisition frequency; The module M2 includes: module M2.1: converting the scanning height data into a 3D point cloud according to the spatial resolution of the X-axis and the Y-axis; module M2.2: converting the scanning height data into a 2D grayscale image using the Sobel combined with Gaussian enhancement image processing algorithm, and then dividing the 2D grayscale image into subframes using an equal-interval cropping method.

6. The weld morphology and surface defect online detection and evaluation system according to claim 4 is characterized in that: The module M5 comprises: Module M5.1: Establish a fully convolutional neural network defect detection model with a 2D grayscale image of the weld as input and pixel-level defect classification as output; Module M5.2: Configure model training parameters, including setting learning rate, decay method, optimizer, batch size batch_size, and number of iterations epoch; Module M5.3: Preprocess the 2D grayscale image: first convert it into a three-channel RGB image, then crop and scale it to the standard size required by the full convolutional neural network; Module M5.4: Transfer learning of the established defect detection model: First, freeze the weight parameters of the feature extraction part, fine-tune the network, and after a preset number of training iterations, unfreeze the feature extraction part to perform weight training of the entire model; Module M5.5: Use the validation set to perform an overfitting check to determine whether the loss change of the defect detection model on the validation set shows an overfitting phenomenon of first decreasing and then increasing. When the loss of the validation set no longer decreases, stop training; Module M5.6: Use the test set to test the accuracy and recall rate indicators of the defect detection model to verify the adaptability and accuracy of the model.

Images