A welding seam defect recognition method and system fusing semantic segmentation and point cloud features

Abstract

This invention discloses a method and system for weld defect identification that integrates semantic segmentation and point cloud features, belonging to the field of automatic welding quality inspection technology. The invention utilizes a deep learning semantic segmentation network to quickly locate the weld ROI region, and compensates and optimizes the segmentation mask through morphological operations; it then combines an improved DLP binocular structured light camera to achieve 3D point cloud reconstruction of the weld region. Based on this, an improved NGDT-RANSAC algorithm based on normal vector constraints is proposed for accurate fitting of the base material plane, and a dual-threshold in-point judgment mechanism is used to achieve fine segmentation of the weld point cloud. The system integrates size detection algorithms for butt welds and fillet welds, respectively using slice projection contour and neighborhood mean iteration methods to extract the size features of the corresponding welds. Furthermore, through defect feature analysis and recognition algorithms, it achieves automatic detection and judgment of various weld defects such as weld beads, spatter, undercut, and burn-through. This invention effectively improves the automation level and detection accuracy of weld defect detection, providing an intelligent solution for non-destructive testing of weld surface quality.

Description

Technical Field

[0001] This invention relates to the field of automatic non-contact inspection technology for welding quality, and in particular to a method and system for weld defect identification that integrates semantic segmentation and point cloud features. Background Technology

[0002] With the rapid development of intelligent manufacturing technology, automated welding technology has been widely used in aerospace, shipbuilding, and automotive industries. However, the development of online welding quality inspection technology has lagged behind. Currently, welding quality inspection mainly relies on manual visual inspection or automated methods based on line laser scanning. Line laser scanning requires relative movement between the sensor and the workpiece, and mechanical vibration during this movement can easily introduce measurement errors, affecting scanning accuracy and inspection reliability. In addition, traditional image processing methods have weak anti-interference capabilities under complex working conditions, making it difficult to achieve high-precision and high-efficiency automatic identification of weld defects.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a method and system for weld defect identification that integrates semantic segmentation and point cloud features, aiming to improve measurement efficiency and anti-interference capability. To achieve the above objective, this invention provides a method for weld defect identification that integrates semantic segmentation and point cloud features, comprising the following steps: The imaging device is controlled to acquire image data of the product. Based on the image data, the positioning of the weld seam area is optimized through semantic segmentation network and mask compensation, and the three-dimensional point cloud data of the product is obtained by binocular parallax stereo matching operation. The imaging device includes a DLP binocular structured light camera. The weld area data and the base material plane area data are determined based on the three-dimensional point cloud data and the preset segmentation algorithm. The weld defect detection results are determined based on the weld area data and the base material plane area data.

[0005] Optionally, the image data is two-dimensional image data and stripe image. The steps of the control imaging device acquiring the image data of the product, optimizing the positioning of the weld seam area based on the image data through a semantic segmentation network and mask compensation, and obtaining the three-dimensional point cloud data of the product through binocular parallax stereo matching calculation include: The DLP binocular structured light camera is controlled to acquire two-dimensional image data and stripe images of the product. The two-dimensional image data is input into a pre-trained semantic segmentation network to determine the initial mask of the weld ROI region. The semantic segmentation network is built based on the YOLOv11-seg architecture. The initial mask is morphologically compensated and optimized to obtain the final mask; The 3D point cloud data is determined based on the final mask and stripe image.

[0006] Optionally, the step of performing morphological compensation optimization on the initial mask to obtain the final mask includes: The center line of the weld seam with a width of one pixel is determined based on the initial mask; Calculate the Euclidean distance from each target pixel to the nearest non-weld boundary, and determine the average width of the weld based on all the Euclidean distances, wherein the target pixel is the pixel on the center line of the weld; A circular structural element is constructed based on the average width of the weld seam, and a morphological expansion operation is performed on the initial mask to obtain the expanded mask. The expanded mask and the workpiece mask are subjected to spatial constraint intersection operation to obtain the final mask.

[0007] Optionally, the step of determining the weld area data and the base material planar area data based on the three-dimensional point cloud data and the preset segmentation algorithm includes: The three-dimensional point cloud data is preprocessed, including voxel downsampling and statistical filtering. Guided sampling is performed based on the normal vector constraints of each point in the preprocessed 3D point cloud data to obtain the fitted parent material plane; The weld area data and the base material plane area data are segmented from the three-dimensional point cloud data based on the base material plane.

[0008] Optionally, the number of the base material planes can be multiple, and the step of segmenting the weld area data and the base material plane area data from the three-dimensional point cloud data based on the base material planes includes: The points that participated in fitting the parent material plane are removed from the three-dimensional point cloud data according to the parent material plane. The rejection points are used as the planar region data of the base material; The remaining points are used as the weld area data, wherein the remaining points are the points other than the discarded points in the three-dimensional point cloud data.

[0009] Optionally, the step of determining the weld defect detection result based on the weld area data and the base material planar area data includes: The weld size parameters are determined based on the geometric characteristics of the weld area data. The weld defect type is determined based on the weld size parameters and preset criteria.

[0010] Optionally, the step of determining the weld defect type based on the weld size parameters and preset criteria includes: The average size is calculated based on the weld size parameters. The difference between the weld size parameters and the average size is compared with a preset weld bead threshold. When the difference is greater than the preset weld bead threshold, the presence of the weld bead defect is determined.

[0011] Furthermore, to achieve the above objectives, the present invention also provides a weld defect recognition device that integrates semantic segmentation and point cloud features. The weld defect recognition device that integrates semantic segmentation and point cloud features includes: a memory, a processor, and a weld defect recognition program that integrates semantic segmentation and point cloud features stored in the memory and executable on the processor. The weld defect recognition program that integrates semantic segmentation and point cloud features is configured to implement the steps of the weld defect recognition method that integrates semantic segmentation and point cloud features as described above.

[0012] Furthermore, to achieve the above objectives, the present invention also provides a weld defect identification system that integrates semantic segmentation and point cloud features, the weld defect identification system integrating semantic segmentation and point cloud features comprising: The point cloud acquisition module is used to control the imaging device to acquire image data of the product, optimize the positioning of the weld seam area through semantic segmentation network and mask compensation based on the image data, and obtain the three-dimensional point cloud data of the product through binocular parallax stereo matching operation. The imaging device includes a DLP binocular structured light camera. The point cloud segmentation module is used to determine weld area data and base material plane area data based on the three-dimensional point cloud data and a preset segmentation algorithm. The defect identification module is used to determine the weld defect detection result based on the weld area data and the base material plane area data.

[0013] Furthermore, to achieve the above objectives, the present invention also provides a storage medium storing a weld defect identification program that integrates semantic segmentation and point cloud features. When the weld defect identification program that integrates semantic segmentation and point cloud features is executed by a processor, it implements the steps of the weld defect identification method that integrates semantic segmentation and point cloud features as described above.

[0014] This invention proposes a weld defect identification method that integrates semantic segmentation and point cloud features. The method acquires image data of a product by controlling an imaging device. Based on the image data, it optimizes the localization of the weld region using a semantic segmentation network and mask compensation, and obtains the three-dimensional point cloud data of the product through binocular parallax stereo matching. The imaging device includes a DLP binocular structured light camera. Weld region data and base material planar region data are determined based on the three-dimensional point cloud data and a preset segmentation algorithm. The weld defect detection result is then determined based on the weld region data and the base material planar region data. Compared to detection methods using laser sensors, this method can statically acquire product information, thereby avoiding motion errors and improving measurement efficiency and anti-interference capabilities. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the structure of a weld defect identification device that integrates semantic segmentation and point cloud features in the hardware operating environment involved in the embodiments of the present invention. Figure 2 This is a flowchart illustrating the first embodiment of a weld defect identification method that integrates semantic segmentation and point cloud features according to the present invention. Figure 3 This is an algorithm execution diagram of a first embodiment of the weld defect identification method that integrates semantic segmentation and point cloud features according to the present invention; Figure 4 This is a flowchart illustrating a second embodiment of a weld defect identification method that integrates semantic segmentation and point cloud features according to the present invention. Figure 5 Network architecture diagram for designing a lightweight transfer learning process for YOLOv11-seg architecture; Figure 6 This is a schematic diagram of a point cloud generation method; Figure 7 This is a flowchart illustrating a third embodiment of the weld defect identification method that integrates semantic segmentation and point cloud features according to the present invention. Figure 8 Schematic diagram of the plane fitting iterative method; Figure 9 A flowchart for using NGDT-RANSAC to perform planar fitting of the parent material region; Figure 10 This is a flowchart illustrating the fourth embodiment of a weld defect identification method that integrates semantic segmentation and point cloud features according to the present invention. Figure 11 A flowchart for separating the weld seam and the base material plane.

[0016] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0018] Reference Figure 1 , Figure 1 This is a schematic diagram of the weld defect identification device that integrates semantic segmentation and point cloud features in the hardware operating environment of the embodiment of the present invention.

[0019] like Figure 1 As shown, the weld defect identification device integrating semantic segmentation and point cloud features may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, an interaction device 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The interaction device 1003 may include a display screen or an input unit such as a keyboard. Optionally, the interaction device 1003 may also be connected to the communication bus via a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0020] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the weld defect identification device that integrates semantic segmentation and point cloud features. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0021] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a weld defect identification program that integrates semantic segmentation and point cloud features.

[0022] exist Figure 1In the weld defect identification device that integrates semantic segmentation and point cloud features, the network interface 1004 is mainly used for data communication with other devices; the interactive device 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the weld defect identification device that integrates semantic segmentation and point cloud features can be set in the weld defect identification device that integrates semantic segmentation and point cloud features. The weld defect identification device that integrates semantic segmentation and point cloud features calls the weld defect identification program that integrates semantic segmentation and point cloud features stored in the memory 1005 through the processor 1001, and executes the weld defect identification method that integrates semantic segmentation and point cloud features provided in the embodiment of the present invention.

[0023] This invention provides a method for weld defect identification that integrates semantic segmentation and point cloud features, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of a weld defect identification method that integrates semantic segmentation and point cloud features according to the present invention.

[0024] In this embodiment, the weld defect identification method that integrates semantic segmentation and point cloud features includes: Step S1: Control the imaging device to acquire image data of the product, optimize the positioning of the weld seam area through semantic segmentation network and mask compensation based on the image data, and obtain the three-dimensional point cloud data of the product through binocular parallax stereo matching operation. The imaging device includes a DLP binocular structured light camera. Optionally, in this embodiment, reference can be made to Figure 3 Specifically, this involves locating the weld seam feature region and acquiring point cloud data. The weld seam feature region refers to the area of ​​the weld seam during the welding process, and the point cloud data refers to 3D point cloud data. Since the image data represents the entire product, before image processing, steps such as cropping can be used to limit the data to a certain range to determine the 3D point cloud data. This step aims to identify the weld seam feature region by building a deep learning semantic segmentation network and acquire the weld seam point cloud data through structured light reconstruction, providing a high-quality data foundation for subsequent defect detection.

[0025] Step S2: Determine the weld area data and the base material plane area data based on the three-dimensional point cloud data and the preset segmentation algorithm; Weld feature region point cloud preprocessing and weld point cloud segmentation. This step aims to reduce the interference of discrete noise points on subsequent operations by preprocessing the acquired point cloud. An improved RANSAC algorithm, namely NGDT-RANSAC, is used to improve the accuracy and efficiency of weld point cloud segmentation.

[0026] Step S3: Determine the weld defect detection result based on the weld area data and the base material plane area data.

[0027] Weld defect identification based on multi-feature criteria. This step integrates dimensional deviations, point cloud distribution features, and geometric morphology to automatically identify and locate typical weld defects such as weld beads, spatter, burn-through, and undercut.

[0028] In this embodiment, image data of the product is acquired by controlling the imaging device. Based on the image data, the weld area is optimized and located through a semantic segmentation network and mask compensation. The three-dimensional point cloud data of the product is obtained by binocular parallax stereo matching. The imaging device includes a DLP binocular structured light camera. The weld area data and the base material plane area data are determined according to the three-dimensional point cloud data and a preset segmentation algorithm. The weld defect detection result is determined according to the weld area data and the base material plane area data. Compared with the detection method using laser sensors, the product information can be collected statically, thereby avoiding motion errors and improving measurement efficiency and anti-interference ability.

[0029] Furthermore, based on the first embodiment, a second embodiment of the weld defect identification method integrating semantic segmentation and point cloud features of the present invention is proposed. In this embodiment, referring to... Figure 4 The image data includes two-dimensional image data and stripe images. The steps of the control imaging device acquiring the image data of the product, optimizing the positioning of the weld seam area based on the image data through semantic segmentation network and mask compensation, and obtaining the three-dimensional point cloud data of the product through binocular parallax stereo matching operation include: Step S11: Control the DLP binocular structured light camera to acquire two-dimensional image data and stripe image of the product, input the two-dimensional image data into a pre-trained semantic segmentation network, and determine the initial mask of the weld ROI region. The semantic segmentation network is built based on the YOLOv11-seg architecture. The core objective of this step, which involves the localization of weld feature regions and the acquisition of point cloud data, is to accurately locate the region of interest (ROI) of the weld, which is the weld area in the entire image or scene that needs to be focused on, analyzed, and processed, and to acquire high-quality 3D point cloud data.

[0030] Step S12: Perform morphological compensation optimization on the initial mask to obtain the final mask; Optionally, this embodiment designs a lightweight transfer learning process based on the pre-trained YOLOv11-seg architecture, and its network architecture diagram is as follows. Figure 5As shown. Compared to mainstream segmentation models such as UNet, YOLOv11-seg significantly improves processing speed while maintaining high segmentation accuracy, making it more suitable for the time-sensitive requirements of industrial vision scenarios. This model seamlessly integrates semantic segmentation tasks through an improved segmentation head, demonstrating high efficiency in real-time detection. Its specific implementation process is as follows: First, multi-scale features are extracted, and the shape and semantic information of the weld are accurately captured through a learnable dynamic convolutional kernel module; then, a minimum loss matching strategy is used to align each convolutional kernel with its corresponding labeled instance, ensuring full utilization of the supervision signal; finally, the entire network is fine-tuned end-to-end on the weld segmentation dataset to further improve segmentation accuracy, and transfer learning is used to meet the task requirements of this invention.

[0031] Step S13: Determine the three-dimensional point cloud data based on the final mask and stripe image.

[0032] The welded workpiece has a simple surface texture. To reliably reconstruct the 3D area of ​​the weld, this invention employs a DLP binocular structured light camera system to acquire depth information of the workpiece surface. This image acquisition system consists of a DLP projector and a pair of cameras. The projector projects phase-coded fringes onto the surface of the object under test, while the left and right cameras simultaneously acquire images to complete phase solving and stereo matching. Traditional methods calculate the phase pixel-by-pixel and perform full-image matching, resulting in high computational cost and generating a large number of non-target point clouds, which require subsequent point cloud removal. To overcome these problems, this invention proposes a dual-region phase constraint based on a single mask, introducing spatial domain constraints in the phase solving stage to suppress the generation of non-target point clouds at the source.

[0033] Specifically, the implementation involves: first, epipolar correction is performed on the dual-camera stripe images and weld seam mask to ensure that corresponding points are collinear; then, the weld seam mask is extracted from the left camera's RGB image and mapped to the coded image coordinate system, defining the left camera's ROI region. Phase is calculated only for pixels in this region to obtain the absolute phase of the ROI in the left image. To avoid matching failure due to mask discrepancies between the two images, only the mask of the left image is used for constraint. Taking into account the consistent ordinate characteristics of corresponding points after epipolar correction, the vertical extreme values ​​of the ROI in the left image are used. Set the calculation area in the right figure as Phase is solved only within this line band, ensuring smooth longitudinal parallax calculation.

[0034] The phase is solved using a three-step phase-shifting method, and the light intensity distribution acquired by the dual cameras is expressed as:

[0035] in: Represents pixels Light intensity on coordinates, This represents the average light intensity. Indicates the modulation amplitude. This represents the phase value to be solved, and α represents the initial phase. Based on the above formula, the three projected phase shifts are 0. , The sinusoidal fringes are obtained by solving the light intensity equations simultaneously to get the enveloping phase. The phase is then expanded using the three-frequency heterodyne principle, and the fringe frequency is selected. , , To ensure absolute phase accuracy, after obtaining the left and right phase images, only the pixels within the ROI region of the left image are processed. In the corresponding row of the right figure Internally searching for corresponding points, calculating phase disparity using the SAD algorithm, and further combining the system baseline length and focal length to restore depth information, the point cloud generation method is as follows: Figure 6 As shown.

[0036] In this embodiment, the two-dimensional image data and stripe image of the product are acquired by controlling the DLP binocular structured light camera. The two-dimensional image data is input into a pre-trained semantic segmentation network to determine the initial mask of the weld ROI region. The initial mask is optimized by morphological compensation to obtain the final mask. The three-dimensional point cloud data is determined based on the final mask and the stripe image, thereby improving the accuracy of the three-dimensional point cloud.

[0037] Furthermore, the step of performing morphological compensation optimization on the initial mask to obtain the final mask includes: The center line of the weld seam with a width of one pixel is determined based on the initial mask; Calculate the Euclidean distance from each target pixel to the nearest non-weld boundary, and determine the average width of the weld based on all the Euclidean distances, wherein the target pixel is the pixel on the center line of the weld; A circular structural element is constructed based on the average width of the weld seam, and a morphological expansion operation is performed on the initial mask to obtain the expanded mask. The expanded mask and the workpiece mask are subjected to spatial constraint intersection operation to obtain the final mask.

[0038] In this embodiment, since RGB images only contain two-dimensional information, the deep learning-based segmentation network has certain errors in identifying the weld edge region. Statistical analysis of the segmentation results reveals that the workpiece region is clearly distinguishable from the background, and the generated workpiece region mask is relatively accurate. However, the weld region and the base material have low contrast and small feature differences at the edges, limiting the segmentation accuracy in this area. To achieve complete weld coarse localization, this invention compensates for missing regions by expanding the mask range and introducing a safety margin. The specific process is as follows: Two initial weld masks are obtained based on image segmentation. With workpiece mask Morphological compensation optimization was employed. Firstly, [the following was done]... Refine the image and extract the center line of the single-pixel wide weld seam. Its mathematical expression is:

[0039] Where Thin(·) represents the skeletonization operator, and k is the number of iterations for convergence of the algorithm to preserve the topological structure. Then, the average weld width W is calculated: for each pixel S on the centerline, its Euclidean distance to the nearest non-weld boundary is calculated, and the average of all distances is taken to obtain the average weld width w. The calculation formula is:

[0040] In the formula, N is the total number of skeleton pixels. Represents pixels Euclidean distance to the nearest non-weld boundary.

[0041] Next, construct a circular structural element with a radius of W, representing the average weld width. For weld mask Perform morphological dilation operation:

[0042] Where ⊕ represents the Minkowski Sum. This expansion operation effectively compensates for errors in the segmentation edges, thereby covering the entire weld area and compensating for segmentation edge errors. To limit the expanded mask from exceeding the workpiece area, the expanded mask needs to be... Spatial constraints are applied within the workpiece area to obtain the final mask image, which is mathematically described as follows:

[0043] This step confines the compensated weld area within the workpiece body, thereby eliminating the ghosting area that may be introduced by the expansion operation.

[0044] In this embodiment, the optimized mask completely covers the actual weld seam, retains a safety margin, reduces the risk of missing point cloud data, and improves system robustness.

[0045] Furthermore, based on the first and second embodiments, a third embodiment of the weld defect identification method integrating semantic segmentation and point cloud features of the present invention is proposed, referring to... Figure 7 The step of determining the weld area data and the base material plane area data based on the three-dimensional point cloud data and the preset segmentation algorithm includes: Step S21, preprocess the three-dimensional point cloud data, the preprocessing including: voxel downsampling and statistical filtering; In this embodiment, the high density of point cloud acquisition leads to a large data volume and prolonged computation time. Furthermore, the point cloud contains discrete noise, necessitating preprocessing to improve data quality. Preprocessing includes point cloud downsampling and filtering for noise reduction. Common downsampling methods include random, uniform, and voxel downsampling. Random downsampling does not consider spatial distribution and is prone to losing important geometric information; uniform downsampling maintains data uniformity but has high computational cost and poor controllability. Voxel downsampling, by dividing the space into cubic lattice cells with side length 'a', maintains shape and uniformity without destroying the structure, making it a more ideal method.

[0046] This invention employs voxel downsampling, and the process is as follows: calculate the point cloud in... , , The extreme values ​​of the directions yield the minimum bounding box side length. , , Set the voxel size 'a' according to the sampling requirements; calculate the voxel mesh size. , , Determine the voxel index H based on the point coordinates.

[0047] The voxels are grouped; the centroid of each point within a voxel is calculated, and the centroid is used to replace all points within that voxel to achieve downsampling.

[0048] Even after downsampling, outliers far from the main object still exist, requiring further filtering. Commonly used radius filtering is fast, but requires manual setting of the radius and threshold. Statistical filtering identifies outliers based on the mean and standard deviation of the distances between a point and its K nearest neighbors, offering more accurate calculations. This invention employs statistical filtering, calculating distance thresholds using the mean, standard deviation, and standard deviation coefficient to effectively remove noise, improve point cloud purity, and lay the foundation for subsequent segmentation.

[0049] Step S22: Guided sampling is performed based on the normal vector constraints of each point in the preprocessed 3D point cloud data to obtain the fitted parent material plane; The surface of a welded workpiece can be divided into the base material plane, the joint between plates, and the irregular weld area. Due to the complex and unpredictable morphology of the weld, this invention employs a strategy of "first extracting the base material plane, then reverse-engineering the weld area" to achieve weld point cloud segmentation. RANSAC, with its strong noise resistance and high tolerance for defects, is chosen as the multi-plane fitting framework. To meet the high-precision requirements of welding scenarios, this invention proposes an improved algorithm based on normal vector constraints, NGDT-RANSAC. By optimizing the initial point selection and interior point determination, the fitting stability and accuracy are improved. A schematic diagram of the algorithm can be found in [reference needed]. Figure 8 .

[0050] First, a normal vector similarity constraint is introduced during the sampling phase to improve the effectiveness of the initial plane. Specifically, a point is randomly selected. The k-nearest neighbor is searched based on a KD-tree, and the local normal vector is obtained by PCA on the neighboring points. Randomly select except Other points As a result, the normal vector of the candidate point is calculated, and the angle between the normal vector of the candidate point and the normal vector of the initial point is obtained using the dot product formula. :

[0051] Set the threshold for the angle between normal vectors When the candidate point satisfies ≤ The points satisfy the constraint of consistent normal vector direction. Ultimately, by A three-point sample set is formed by adding two candidate points that satisfy the constraints and using them to initialize the planar model. This strategy leverages local geometric features of the point cloud to significantly reduce invalid sampling and improve model convergence efficiency.

[0052] After constructing the initial model, interior points are determined using both distance and normal vector constraints. The spatial plane equation is expressed as:

[0053] In the formula: A, B, and C are the three coordinate components of the plane unit normal vector, satisfying A² + B² + C² = 1, and D is the distance from the origin to the plane. This planar model is then used to validate the remaining points in the point cloud data. For each point... Calculate its distance to the fitted plane:

[0054] Set the distance threshold to ,when When, the point is directly determined as an exterior point. When the value is less than or equal to δ, the determination is further made by examining the direction of the normal vector. Specifically, this involves calculating the normal vector. With the normal vector of the fitted plane Angle between:

[0055] Set the threshold for the angle between the normal vectors to... ,like ≤ If the normal vector of a point is aligned with the plane, it is considered an interior point. This dual-threshold rule effectively avoids the edge point misjudgment problem caused by traditional RANSAC relying solely on distance thresholds, significantly improving the segmentation accuracy of plate seam regions.

[0056] To select the optimal plane, this invention constructs a comprehensive scoring function:

[0057] Where N is the number of interior points. Indicates the first The distance from each inlier to the fitted plane. Let the angle between its normal vector and the plane normal vector be denoted as . This serves as a penalty factor to balance the impact of the number of inliers and the error term. The scoring system considers both inlier size and fitting accuracy. During iteration, a scoring buffer of size K is maintained, and the coefficient of variation of its scoring sequence is calculated. When η < ε for K consecutive iterations, the model is considered to have converged and the iteration is terminated. This adaptive termination strategy can dynamically determine the stable state of the algorithm, prevent oversampling, and improve overall efficiency.

[0058] Step S23: Segment the weld area data and the base material plane area data from the three-dimensional point cloud data according to the base material plane.

[0059] Furthermore, the number of the base material planes can be multiple, and the step of segmenting the weld area data and the base material plane area data from the three-dimensional point cloud data based on the base material planes includes: The points that participated in fitting the parent material plane are removed from the three-dimensional point cloud data according to the parent material plane. The rejection points are used as the planar region data of the base material; The remaining points are used as the weld area data, wherein the remaining points are the points other than the discarded points in the three-dimensional point cloud data.

[0060] In this embodiment, NGDT-RANSAC is used to complete the planar fitting of the base material region, and it has the ability to fit one or two non-coplanar base materials. For example... Figure 9 As shown in the flowchart, the process is as follows: First, fit the first plane to the original point cloud, and calculate the proportion r of the remaining unfitted point cloud. If r > v (the decision coefficient), it indicates that another base material plane still exists, and the already fitted plane point cloud needs to be removed, and the remaining point cloud needs to be fitted again. If r ≤ v, it is considered that all base material planes have been fitted. Finally, all the removed areas are the base material planes, and the remaining point cloud is the weld area, providing high-precision input for subsequent defect detection.

[0061] Furthermore, based on any of the above embodiments, a fourth embodiment of the weld defect identification method integrating semantic segmentation and point cloud features of the present invention is proposed, referring to... Figure 10 The step of determining the weld defect detection result based on the weld area data and the base material planar area data includes: Step S31: Determine the weld size parameters based on the geometric characteristics of the weld area data; In this embodiment, after the weld seam point cloud preprocessing is completed, the feature region is first sliced ​​at equal intervals along the Y-axis. The spatial pose and boundary range of the point cloud are determined using the OBB minimum bounding box, and the point cloud is projected onto a plane with the longest side of the bounding box as the slicing direction for contour extraction.

[0062] The dimensions of the butt weld are obtained by using the slope iteration method to determine the highest point and weld toe point: 1) Traverse the contour point cloud and take the point with the largest Z value as the highest point h; 2) Divide the contour into left and right parts based on the X coordinate of h; 3) Calculate the slope change of adjacent points, and the inflection point is the point where an extreme value appears, thus obtaining the left and right weld toes. , ;4) with , Fit a straight line, and calculate the perpendicular distance from h to that line as the complementary height. , and The distance is the weld width .

[0063] The fillet weld dimensions are based on two planes obtained from the division of the base material, and the point cloud at their intersection is extracted as the weld toe region. The weld leg length is calculated using trigonometric methods, taking the average of the distances from the weld toe point to the intersection line of the two planes, as the final weld leg length. The weld throat length is the median distance from the point cloud of the weld region to the intersection line, such as... Figure 11 The diagram shown is the algorithm flowchart.

[0064] Step S32: Determine the weld defect type based on the weld size parameters and preset criteria.

[0065] Weld beads are abnormal accumulations caused by overflowing molten metal, with a height significantly greater than normal welds and a depth not exceeding 3mm. This invention identifies weld beads based on dimensional deviations: butt welds use reinforcement height parameters, and fillet welds use weld throat parameters; the average dimensions of all cross-sections are calculated as a normal baseline; a deviation threshold is set, determined according to process standards. When the difference between a cross-sectional dimension and the average value exceeds the threshold, it is identified as a preliminary weld bead screening area, and its three-dimensional range is recorded.

[0066] Spatter defects are caused by unstable electric arcs, improper welding parameters, or material problems, and manifest as molten metal particles adhering to the weld surface, forming protrusions. According to standards, no more than one spatter with a diameter greater than 1 mm should exist within the visible area of ​​the weld. Therefore, the key to identification lies in detecting discrete protruding point clouds and determining whether their size exceeds the limit. This invention identifies spatter defects based on point cloud clustering and geometric analysis.

[0067] In step two, after extracting the parent material plane using RANSAC, some unclassified point clouds may represent spatter point sets. First, Euclidean clustering is used to divide these discrete points into multiple clusters, grouping points with a distance less than a threshold into one cluster. To avoid noise interference, a minimum point count threshold is set, and clusters with too few points are removed. If the point set of the spatter region is represented as... Calculate the centroid of this region using the following formula. :

[0068] In the formula, n is the number of points in the region point cloud; the distance from the centroid of the calculated region point set to the centroid is calculated, and the maximum distance is selected. The minimum circumcircle diameter is calculated using the following formula. :

[0069] The system retains areas with diameters greater than a set threshold, and the remaining areas are identified as splash defects. Clusters with diameters greater than 1 mm are identified as splash defects, enabling the counting, size measurement, and location positioning of splashes.

[0070] Weld penetration defects, caused by excessive penetration depth, result in voids on the back side, a serious quality issue requiring product scrapping. The key to identifying these defects lies in detecting void areas with missing point clouds. This invention employs an interval judgment method, slicing the weld along the Y-axis at equal intervals to generate multiple contour segments parallel to the XOZ plane. The X-coordinate distribution of each contour is analyzed. In a normal weld, the coordinates should continuously cover the weld width; if weld penetration exists, the void area will lack point clouds, and the X-coordinates will show obvious discontinuities. The coordinate difference between the middle weld area of ​​each contour is calculated. If the interval difference is greater than half the weld width, a potential weld penetration is determined at that location, and the contour number is recorded. With interval difference After traversing all contours, the anomaly numbers are counted and the weld penetration area is determined. The weld penetration width is determined by the difference between the maximum and minimum interval values. = The weld penetration length is calculated based on the abnormal profile number range and the slice interval. =(Serial number span × sampling interval). This method can achieve accurate detection, location, and dimensional quantification of weld penetration defects, providing a basis for weld quality judgment.

[0071] Undercut defects are caused by improper welding operations, leading to excessive erosion of the base metal at the weld edge, forming grooves that affect structural strength. The standard for undercut defects is a depth less than 1 / 10 of the weld material thickness and not exceeding 1 mm. Therefore, the key to identification lies in detecting whether abnormal concavity appears at the weld edge. This invention uses Z-axis difference analysis to detect undercut. After completing the base material plane segmentation in step two, the average Z-axis value of all points on the base material plane is calculated. As a height reference, the weld contour point cloud is extracted and arranged sequentially, and calculations are performed for each contour point. The difference is then compared with the bite edge judgment threshold. If the difference is greater than the threshold, that point is taken as the initial key point for bite edge judgment. Then, its left and right adjacent points are detected. , If the height difference still exceeds the threshold, the presence of an edge-cutting defect in the area is confirmed, and its range and depth are recorded. If adjacent points do not meet the conditions, they are considered noise and removed. After traversing all contour points, the location and depth measurement of the edge-cutting area can be completed. By combining benchmark comparison, neighborhood verification, and noise removal, the accuracy and stability of edge-cutting identification can be improved, ensuring that the detection results meet the evaluation criteria.

[0072] Furthermore, the step of determining the weld defect type based on the weld size parameters and preset criteria includes: The average size is calculated based on the weld size parameters. The difference between the weld size parameters and the average size is compared with a preset weld bead threshold. When the difference is greater than the preset weld bead threshold, the presence of the weld bead defect is determined.

[0073] Weld beads are abnormal accumulations caused by overflowing molten metal, with a height significantly greater than normal welds and a depth not exceeding 3mm. This invention identifies weld beads based on dimensional deviations: butt welds use reinforcement height parameters, and fillet welds use weld throat parameters; the average dimensions of all cross-sections are calculated as a normal baseline; a deviation threshold is set, determined according to process standards. When the difference between a cross-sectional dimension and the average value exceeds the threshold, it is identified as a preliminary weld bead screening area, and its three-dimensional range is recorded.

[0074] Furthermore, this invention also proposes a weld defect recognition system that integrates semantic segmentation and point cloud features. The weld defect recognition system that integrates semantic segmentation and point cloud features includes: The point cloud acquisition module is used to control the imaging device to acquire image data of the product, optimize the positioning of the weld seam area through semantic segmentation network and mask compensation based on the image data, and obtain the three-dimensional point cloud data of the product through binocular parallax stereo matching operation. The imaging device includes a DLP binocular structured light camera. The point cloud segmentation module is used to determine weld area data and base material plane area data based on the three-dimensional point cloud data and a preset segmentation algorithm. The defect identification module is used to determine the weld defect detection result based on the weld area data and the base material plane area data.

[0075] Furthermore, this invention also proposes a weld defect recognition device that integrates semantic segmentation and point cloud features. The weld defect recognition device that integrates semantic segmentation and point cloud features includes: a memory, a processor, and a weld defect recognition program that integrates semantic segmentation and point cloud features stored in the memory and executable on the processor. The weld defect recognition program that integrates semantic segmentation and point cloud features is configured to implement the steps of the weld defect recognition method that integrates semantic segmentation and point cloud features as described above.

[0076] Furthermore, this embodiment of the invention also proposes a storage medium storing a weld defect identification program that integrates semantic segmentation and point cloud features. When the weld defect identification program that integrates semantic segmentation and point cloud features is executed by a processor, it implements the steps of the weld defect identification method that integrates semantic segmentation and point cloud features as described above.

[0077] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0078] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0079] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0080] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for weld defect identification that integrates semantic segmentation and point cloud features, characterized in that, The weld defect identification method that integrates semantic segmentation and point cloud features includes the following steps: The imaging device is controlled to acquire image data of the product. Based on the image data, the positioning of the weld seam area is optimized through semantic segmentation network and mask compensation, and the three-dimensional point cloud data of the product is obtained by binocular parallax stereo matching operation. The imaging device includes a DLP binocular structured light camera. The weld area data and the base material plane area data are determined based on the three-dimensional point cloud data and the preset segmentation algorithm. The weld defect detection results are determined based on the weld area data and the base material plane area data.

2. The weld defect identification method integrating semantic segmentation and point cloud features as described in claim 1, characterized in that, The image data includes two-dimensional image data and stripe images. The steps of the control imaging device acquiring the image data of the product, optimizing the positioning of the weld seam area based on the image data through semantic segmentation network and mask compensation, and obtaining the three-dimensional point cloud data of the product through binocular parallax stereo matching operation include: The DLP binocular structured light camera is controlled to acquire two-dimensional image data and stripe images of the product. The two-dimensional image data is input into a pre-trained semantic segmentation network to determine the initial mask of the weld ROI region. The semantic segmentation network is built based on the YOLOv11-seg architecture. The initial mask is morphologically compensated and optimized to obtain the final mask; The 3D point cloud data is determined based on the final mask and stripe image.

3. The weld defect identification method integrating semantic segmentation and point cloud features as described in claim 2, characterized in that, The step of performing morphological compensation optimization on the initial mask to obtain the final mask includes: The center line of the weld seam with a width of one pixel is determined based on the initial mask; Calculate the Euclidean distance from each target pixel to the nearest non-weld boundary, and determine the average width of the weld based on all the Euclidean distances, wherein the target pixel is the pixel on the center line of the weld; A circular structural element is constructed based on the average width of the weld seam, and a morphological expansion operation is performed on the initial mask to obtain the expanded mask. The expanded mask and the workpiece mask are subjected to spatial constraint intersection operation to obtain the final mask.

4. The weld defect identification method integrating semantic segmentation and point cloud features as described in claim 1, characterized in that, The step of determining the weld area data and the base material planar area data based on the three-dimensional point cloud data and the preset segmentation algorithm includes: The three-dimensional point cloud data is preprocessed, including voxel downsampling and statistical filtering. Guided sampling is performed based on the normal vector constraints of each point in the preprocessed 3D point cloud data to obtain the fitted parent material plane; The weld area data and the base material plane area data are segmented from the three-dimensional point cloud data based on the base material plane.

5. The weld defect identification method integrating semantic segmentation and point cloud features as described in claim 4, characterized in that, The number of base material planes can be multiple, and the step of segmenting the weld area data and the base material plane area data from the three-dimensional point cloud data based on the base material planes includes: The points that participated in fitting the parent material plane are removed from the three-dimensional point cloud data according to the parent material plane. The rejection points are used as the planar region data of the base material; The remaining points are used as the weld area data, wherein the remaining points are the points other than the discarded points in the three-dimensional point cloud data.

6. The weld defect identification method integrating semantic segmentation and point cloud features as described in any one of claims 1 to 5, characterized in that, The step of determining the weld defect detection result based on the weld area data and the base material planar area data includes: The weld area data is sliced ​​at equal intervals, and the geometric features of each slice contour are extracted to determine the weld size parameters. The weld defect type is determined based on the weld size parameters and preset criteria.

7. The weld defect identification method integrating semantic segmentation and point cloud features as described in claim 6, characterized in that, The step of determining the weld defect type based on the weld size parameters and preset criteria includes: The average size is calculated based on the weld size parameters. The difference between the weld size parameters and the average size is compared with a preset weld bead threshold. When the difference is greater than the preset weld bead threshold, the presence of the weld bead defect is determined.

8. A weld defect recognition system integrating semantic segmentation and point cloud features, characterized in that, The weld defect recognition system that integrates semantic segmentation and point cloud features includes: The point cloud acquisition module is used to control the imaging device to acquire image data of the product, optimize the positioning of the weld seam area through semantic segmentation network and mask compensation based on the image data, and obtain the three-dimensional point cloud data of the product through binocular parallax stereo matching operation. The imaging device includes a DLP binocular structured light camera. The point cloud segmentation module is used to determine weld area data and base material plane area data based on the three-dimensional point cloud data and a preset segmentation algorithm. The defect identification module is used to determine the weld defect detection result based on the weld area data and the base material plane area data.

9. A weld defect identification device integrating semantic segmentation and point cloud features, characterized in that, The weld defect identification device integrating semantic segmentation and point cloud features includes: a memory, a processor, and a weld defect identification program integrating semantic segmentation and point cloud features stored in the memory and executable on the processor. The weld defect identification program integrating semantic segmentation and point cloud features is configured to implement the steps of the weld defect identification method integrating semantic segmentation and point cloud features as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a weld defect identification program that integrates semantic segmentation and point cloud features. When the weld defect identification program that integrates semantic segmentation and point cloud features is executed by the processor, it implements the steps of the weld defect identification method that integrates semantic segmentation and point cloud features as described in any one of claims 1 to 7.

Images