A structured light and line laser fusion weld seam thickness grading three-dimensional positioning method and system

Abstract

The present application relates to the technical field of weld positioning, and discloses a weld thickness grading three-dimensional positioning method and system combining structured light and linear laser, which has the technical scheme as follows: S1, a structured light camera is used to take pictures of a workpiece to obtain workpiece images; S2, based on the workpiece images, a channel space attention mechanism and a Center Net convolutional neural network are used to perform coarse weld position identification to determine the weld position; S3, based on the identified weld position, a linear laser camera is used to scan the weld position to obtain three-dimensional point cloud data of the weld; and S4, based on the three-dimensional point cloud data, a Marr-Hildreth edge detection algorithm is used to identify the three-dimensional coordinates of the weld, so that the three-dimensional data of the workpiece weld can be quickly and accurately positioned.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of weld positioning, more particularly, it relates to a weld thickness grading three-dimensional positioning method and system fusing structured light and line laser. BACKGROUND

[0002] In the shield machine cutter welding project, the cutter as the core component of the shield machine equipment, its weld trajectory often presents the spatial distribution characteristics of large curvature and multiple degrees of freedom, and the welding work range can reach dozens of square meters, which puts forward very high requirements for the positioning and identification system of the robot.

[0003] The traditional single sensor identification scheme faces significant challenges in such large-scale and high-dynamic operation scenarios:

[0004] Taking a structured light camera as an example, although the three-dimensional point cloud of the welding area can be quickly obtained by face array projection, the measurement accuracy decreases sharply with the increase of the working distance, especially when there are complex rust, oil stains or high light reflection areas on the cutter surface, the point cloud missing rate can reach more than 10%, which leads to the failure of weld feature extraction;

[0005] Taking a line laser camera as an example, although it has micron-level measurement accuracy, its line-by-line scanning mode needs to cooperate with the high-frequency motion of the mechanical arm to cover the entire cutter surface, and the time-consuming of single full-size scanning is as long as tens of minutes, which is difficult to meet the real-time demand of the welding process.

[0006] More seriously, the shield machine cutter will produce millimeter-level deformation displacement during the welding process, and the single sensor system cannot simultaneously meet the dual needs of global deformation compensation and local weld tracking.

[0007] The current traditional method for weld identification is:

[0008] 1. One method is to directly use a line laser camera to extract the scanning features of the weld. First, the spatial position relationship model of the laser plane and the camera needs to be established, which is usually obtained by calibrating the camera intrinsic parameters and the laser plane equation. When the laser stripe is irradiated to the object surface, the three-dimensional coordinates corresponding to each pixel point in the stripe image captured by the camera can be solved by triangulation. The key identification methods include light stripe center line extraction and sub-pixel level positioning technology. After obtaining the center coordinates of the stripe, the two-dimensional pixel points are mapped to three-dimensional point cloud data by combining the calibration parameters and the triangulation model.

[0009] Although this method can improve the local feature resolution, it has the problems of narrow single imaging field of view, low global reconstruction efficiency, and difficulty in adapting to large-size workpiece positioning.

[0010] 2. Another weld seam feature extraction method is to use a plane structured light camera. First, a plane structured light vision sensor is used to scan the workpiece to obtain three-dimensional point cloud data, and the minimum oriented bounding box of the point cloud is calculated to determine the main direction of the weld seam. Along the direction, an equal-interval cutting plane is generated to implement point cloud slicing. By establishing a local coordinate system, the three-dimensional sliced point cloud is projected onto a two-dimensional plane, and a grid processing is performed to form a binary image, solving the complexity of three-dimensional data processing.

[0011] This method has obvious limitations only relying on the plane structured light: the plane structured light is easily disturbed by metal reflection in dynamic scanning, resulting in large point cloud noise, and the reconstruction accuracy is insufficient for complex geometric profile features such as steep groove edges and small curvature weld seams, and the reconstruction speed is slow when high-resolution scanning is performed.

[0012] In summary, the existing weld seam recognition methods have many limitations in practical application, such as insufficient accuracy, great influence of environment, and point cloud loss, etc., which cannot meet the high precision and high stability requirements of welding robots in industrial applications. Therefore, it is particularly necessary to explore and use other more advanced weld seam recognition and positioning methods to improve welding efficiency and precision, overcome the shortcomings of existing technologies, and ensure the stability and accurate operation of robots in complex working conditions. SUMMARY

[0013] The purpose of the present application is to provide a structured light and line laser fusion weld seam roughness grading three-dimensional positioning method and system, which can quickly and accurately position the three-dimensional data of the workpiece weld seam.

[0014] The above technical purpose of the present application is realized by the following technical scheme: a structured light and line laser fusion weld seam roughness grading three-dimensional positioning method, comprising the following steps:

[0015] S1, capturing the workpiece by a structured light camera to obtain a workpiece image;

[0016] S2, based on the workpiece image, performing weld seam position rough identification by a channel space attention mechanism and a Center Net convolutional neural network to determine the weld seam position;

[0017] S3, based on the identified weld seam position, scanning the weld seam position by a line laser camera to obtain three-dimensional point cloud data of the weld seam;

[0018] S4, based on the three-dimensional point cloud data, identifying the three-dimensional coordinates of the weld seam by an edge detection algorithm.

[0019] As a preferred technical scheme of the present application, S1 comprises: capturing the workpiece by a structured light camera to obtain an RGB image, and projecting coded fringes on the workpiece and capturing deformed fringe images;

[0020] S2 comprises:

[0021] S21, based on the deformed fringe image, a depth map is calculated, the depth map is aligned and fused with the RGB image, and a four-channel RGB-D tensor is generated;

[0022] S22, the four-channel RGB-D tensor is input into a Center Net convolutional neural network fused with a channel space attention mechanism for processing, and weld three-dimensional information is output.

[0023] As a preferred technical solution of the present application, in S21, the wrapped phase of the deformed fringe image is calculated by the phase shift method, and the depth map is calculated according to the calibration parameters of the camera;

[0024] S22 comprises:

[0025] S221, through the feature encoding network of the Center Net convolutional neural network, the four-channel RGB-D tensor is feature extracted to generate a multi-scale feature map;

[0026] S222, through the channel attention mechanism, the multi-size feature map is processed by channel weight to obtain a channel enhanced feature;

[0027] S223, through the spatial attention mechanism, the channel enhanced feature is processed by spatial weight to obtain a channel space enhanced feature;

[0028] S224, the channel space enhanced feature is applied with Gaussian kernel diffusion to obtain a weld center probability heat map; at the same time, the weld size, center point offset and depth correction are parallelly regressed to output the weld three-dimensional information.

[0029] As a preferred technical solution of the present application, S222 comprises:

[0030] S2221, the multi-size feature map is processed by global maximum pooling and average pooling to obtain a maximum pooling channel vector and an average pooling channel vector;

[0031] S2222, the maximum pooling channel vector and the average pooling channel vector are input into a shared MLP to generate a channel attention weight;

[0032] S2223, the channel attention weight is applied to the multi-size feature map to obtain a channel enhanced feature.

[0033] S223 comprises:

[0034] S2231, the channel enhanced feature is processed by maximum pooling and average pooling along the channel to obtain a maximum pooling space feature map and an average pooling space feature map;

[0035] S2232: Concatenate the max pooling spatial feature map and the average pooling spatial feature map, and generate spatial attention weights through a large convolutional kernel;

[0036] S2233. Apply spatial attention weights to channel enhancement features to obtain channel spatial enhancement features.

[0037] As a preferred embodiment of the present invention, S4 includes:

[0038] S41. Preprocessing operations on 3D point cloud data, including sequential pass-through filtering, dimension transformation, coordinate mapping, and region segmentation and repair, to obtain the target depth grayscale image and dimension mapping relationship;

[0039] S42. Using the Marr-Hildreth edge detection algorithm, Gaussian filtering, Laplacian operation, and zero-crossing detection are performed sequentially on the target depth grayscale image to obtain the two-dimensional coordinates of the weld edge.

[0040] S43. Based on the dimension mapping relationship, obtain the three-dimensional coordinates corresponding to the two-dimensional coordinate data of the weld edge.

[0041] As a preferred embodiment of the present invention, S41 includes:

[0042] S411. Based on the preset weld seam buffer expansion amount, perform pass-through filtering on the three-dimensional point cloud data to retain the point cloud data of the key areas of the weld seam.

[0043] S412. Project the retained 3D point cloud data onto the laser scanning plane and quantize it along the depth direction to generate a depth-mapped grayscale image.

[0044] S413. Based on 3D point cloud data and depth-mapped grayscale images, establish dimensional mapping relationships and construct a bidirectional lookup table;

[0045] S414. Separate the bevel region from the parent material background in the depth-mapped grayscale image to generate a binary mask image. Then perform adaptive threshold segmentation. After segmentation, perform morphological restoration on the binary mask image to obtain the target depth grayscale image.

[0046] A three-dimensional positioning system for weld seam coarseness and fineness classification based on the fusion of structured light and line laser, comprising:

[0047] Structured light cameras are used to photograph workpieces and acquire images of them.

[0048] The coarse identification module is used to perform coarse identification of weld seam location and determine the weld seam location by using the channel spatial attention mechanism and the Center Net convolutional neural network;

[0049] A line laser camera is used to scan the weld seam location and obtain three-dimensional point cloud data of the weld seam;

[0050] The fine recognition module is used to identify the three-dimensional coordinates of the weld seam using the Marr-Hildreth edge detection algorithm;

[0051] The time-division triggering control module controls the application timing of the structured light camera and the line laser camera through hardware synchronization signals.

[0052] As a preferred embodiment of the present invention, the structured light camera and the line laser camera are also respectively equipped with filtering devices, which eliminate crosstalk between light sources of different wavelengths through spectral isolation technology.

[0053] In summary, the present invention has the following beneficial effects:

[0054] By deeply collaborating with a structured light camera and a line laser camera, the core contradiction of balancing efficiency and accuracy in large-scale welding scenarios is resolved. The structured light camera, with its wide field of view, quickly locks onto the weld area, significantly reducing the scanning range of the line laser and avoiding the redundant time consumption of traditional full-frame line laser scanning. The line laser, guided by coarse positioning, focuses on the target area, capturing the microscopic morphological features of the weld through high-precision line-by-line scanning. The two cameras achieve seamless data integration through a dynamic triggering mechanism, overcoming the physical limitations of a single sensor in terms of measurement range and accuracy, and eliminating multi-source data bias through spatial registration technology. This forms a closed-loop collaborative system of global coarse positioning and local fine detection, providing highly robust spatial perception capabilities for welding complex curved surfaces.

[0055] An innovative framework fusing deep learning and classical image processing was designed, significantly improving reliability in harsh industrial environments. Based on a neural network architecture using multimodal data (RGB-D), an intelligent feature selection mechanism enhances the perception of key weld features, effectively suppressing the impact of interference factors such as metal reflection and oil smudges on recognition accuracy. Combined with an improved edge detection algorithm, it can still accurately extract bevel geometric parameters even under abnormal conditions such as missing point clouds and deformation displacement. This hierarchical processing mechanism retains the physical interpretability of traditional algorithms while incorporating the adaptive advantages of deep learning, maintaining stable detection performance in complex scenarios such as dynamic thermal deformation and strong light interference.

[0056] Through the collaborative optimization of hardware architecture and software algorithms, a complete welding positioning solution has been constructed. The rigid connection and vibration damping design of the dual sensors effectively suppress mechanical vibration interference, while the time-division triggering control module eliminates optical crosstalk, ensuring spatiotemporal consistency of multi-source data acquisition. Innovative 3D calibration technology achieves precise mapping across sensor coordinate systems, supporting rapid model changeover and process switching. A dynamic compensation mechanism can track welding thermal deformation in real time, and combined with path planning algorithms, automatically generate the optimal welding trajectory, forming a closed-loop control throughout the entire process of "perception-decision-execution." This significantly improves the consistency and automation level of the welding process, providing reliable technical support for large-scale equipment manufacturing. Attached Figure Description

[0057] Figure 1 This is a flowchart of the method of the present invention;

[0058] Figure 2 This is a preliminary flowchart of the coarse identification process of the present invention;

[0059] Figure 3 This is a schematic diagram illustrating the application of the channel and spatial attention mechanism of the present invention;

[0060] Figure 4 This is a detailed flowchart of the coarse identification process of the present invention;

[0061] Figure 5 This is a flowchart of the line laser scanning preprocessing of the present invention;

[0062] Figure 6 This is a detailed identification flowchart of the present invention. Detailed Implementation

[0063] The present invention will be further described in detail below with reference to the accompanying drawings.

[0064] like Figure 1 As shown, this invention provides a three-dimensional positioning method for weld seam coarseness classification by fusing structured light and line laser, comprising the following steps:

[0065] Preparation step S0: Select the equipment to be used, ensuring that the selected equipment can meet the dual requirements of global perception efficiency and local detection accuracy.

[0066] The structured light camera is selected to be highly resistant to ambient light interference and robust to stripe decoding, ensuring stable output of basic point clouds even in industrial scenarios such as metal reflection and dust. The line laser camera is selected to be equipped with high-speed scanning and high signal-to-noise ratio, which can penetrate the interference of welding spatter and accurately capture key features such as bevel root gap and blunt edge tilt angle.

[0067] The specifications of the selected structured light camera are shown in Table 1, and the specifications of the line laser camera are shown in Table 2.

[0068] Table 1. Main Specifications of Structured Light Cameras

[0069] ;

[0070] Table 2. Main Specifications of Line Laser Cameras

[0071] ;

[0072] In terms of spatial layout, the structured light camera is installed at an angle at the end of the robotic arm to form a panoramic coverage of the workpiece, while the line laser camera needs to be installed vertically or laterally according to the geometric features of the target area. The installation is carried out by adjusting the bracket with multiple degrees of freedom to ensure that the fields of view of the two are effectively complementary in space, which avoids optical obstruction and retains the necessary overlapping area for data registration.

[0073] To suppress vibration interference caused by the robotic arm's movement, a lightweight composite material mounting base was used, combined with a multi-level damping structure, to mount the structured light camera and line laser camera, reducing the impact of high-frequency vibration on point cloud quality. A time-division triggering control module was also designed to coordinate the exposure sequence of the two cameras via hardware synchronization signals, avoiding cross-interference from active light sources. When using two cameras, differentiated filtering devices were configured for each camera, and spectral isolation technology was used to eliminate crosstalk between different wavelengths of light sources. A mapping relationship was established in the overlapping field of view, providing a spatial alignment basis for subsequent multi-source point cloud fusion.

[0074] After the structured light camera and line laser camera are installed, a unified measurement benchmark is established through joint calibration. The calibration process utilizes three-dimensional calibration targets with stable geometric features and planar targets to collect multiple sets of spatial feature points within the fields of view of both the structured light camera and the line laser camera. A nonlinear optimization algorithm is used to solve the overdetermined equation system AX=XB composed of the measurement data. Finally, the hand-eye calibration coordinate transformation matrix between the structured light camera and the line laser camera is solved to ensure the consistency and accuracy of the data in the global coordinate system.

[0075] S1, such as Figure 2 and 4 As shown, a structured light camera is used to photograph the workpiece and obtain an image of the workpiece.

[0076] Specifically, a structured light camera is used to photograph the workpiece and acquire RGB images. These RGB images can be directly obtained through the API interfaces provided by the structured light camera manufacturer. The structured light camera provides complete RGB image acquisition capabilities through its standardized SDK development interface, which supports calls from multiple programming languages.

[0077] The structured light camera also projects a specific coded stripe optical pattern onto the workpiece and quickly captures deformed stripe images, which can be used to scan the workpiece surface over a large area, quickly obtain global three-dimensional information of the welding area, and quickly screen out potential areas of the weld under strong arc light and dust interference, so as to define the target range for fine identification in subsequent S3 and S4.

[0078] It is important to note that the components of the structured light camera—the camera and the projector—are two separate hardware units. The structured light camera of this invention employs a highly integrated hardware architecture, with its core consisting of a camera module and a projector module. Specifically, the camera module refers to the CMOS / CCD image sensor assembly integrated within the structured light camera. This assembly includes an optical lens, an infrared filter, and an image acquisition circuit, responsible for capturing the image of deformable fringes modulated by the workpiece surface. The projector module is a DLP micro-projection system integrated within the structured light camera, composed of a DMD digital micromirror array, a collimating lens group, and an infrared laser light source, specifically designed to project precisely coded sinusoidal phase-shifted fringes onto the workpiece surface.

[0079] RGB images and deformed stripe images are two completely different types of image data, acquired through independent physical channels. An RGB image is a visible light image containing red, green, and blue color information. Its acquisition principle is as follows: the RGB sensor integrated within the structured light camera captures visible spectral information from ambient light through a Bayer color filter array. Each pixel consists of three sub-pixels (R, G, and B), outputting a 24-bit true-color image. This image directly reflects the color, texture, and other appearance features of the workpiece surface. Deformed stripe images, on the other hand, are grayscale images acquired through an infrared sensor in the structured light camera. The acquisition process involves: a built-in projector emitting a coded stripe pattern in the near-infrared band; after modulation by the workpiece surface morphology, the camera's infrared sensor captures the stripe deformation, outputting a single-channel 16-bit grayscale image specifically for phase calculation and 3D reconstruction.

[0080] S2, such as Figure 2 As shown, based on the workpiece image, the weld position is coarsely identified through the channel spatial attention mechanism and the Center Net convolutional neural network, thus effectively distinguishing the weld from background noise, reducing the false detection rate, and ensuring the reliability of coarse positioning.

[0081] Specifically, S2 includes:

[0082] S21. Based on the deformed fringe image, the wrapping phase of the deformed fringe image is calculated using the phase-shifting method, with the following formula: ,in This represents the number of period groups (or phase shift steps) of sinusoidal phase-shifting fringes used in structured light 3D measurement. Specifically... This represents the total number of sinusoidal grating patterns with a fixed phase difference that are sequentially projected by the built-in projection module of the structured light camera during a single 3D measurement. To wrap the phase, the range is The absolute phase is recovered using a phase unrolling algorithm. Based on the calibration parameters, namely the extrinsic parameters R and t of the camera and projector, and the intrinsic parameter matrices Kcam and Kproj, the depth map D(x, y) is calculated using triangulation. Where B is the baseline distance between the structured light camera and the projector. It's the camera's focal length. It is the wavelength of the stripes. It is a calibration constant.

[0083] The depth map is aligned and fused with the RGB image to generate a four-channel RGB-D tensor, expressed as follows: , where H×W is the image resolution, the depth channel D is normalized to the range [0, 1], and R represents the set of real numbers, which is used here as a label for the tensor space.

[0084] S22. The four-channel RGB-D tensor is input into a Center Net convolutional neural network that incorporates a channel spatial attention mechanism for processing, and the output is the three-dimensional information of the weld. S22 includes:

[0085] S221. Using the feature encoding network of the Center Net convolutional neural network, features are extracted from the four-channel RGB-D tensor to generate multi-scale feature maps. Center Net adopts an encoder-decoder architecture and uses the Res Net backbone feature encoding network. The input four-channel RGB-D tensor is encoded by Res Net-18 to generate multi-scale feature maps. Each Res Block contains two convolutions (Conv+BN+ReLU) and skip connections. The output feature map size is downsampled to H / 16×W / 16×512 step by step. Through the cascaded residual blocks, the network gradually abstracts high-level semantic information while preserving spatial details, providing multi-level feature support for subsequent heatmap prediction and geometric regression.

[0086] S222, such as Figure 3 As shown, channel attention mechanism is used to process multi-size feature maps by channel weights to obtain channel-enhanced features; S222 includes:

[0087] S2221. Perform global max pooling and average pooling on the multi-size feature maps to obtain max pooling channel vectors and average pooling channel vectors.

[0088] Channel weights are generated by aggregating spatial information to highlight important feature channels. Global max pooling and average pooling are performed on the feature map F to generate two channel description vectors:

[0089] , ;

[0090] S2222: Input the max-pooling channel vector and the average-pooling channel vector into the shared MLP, and generate channel attention weights through non-linear mapping: , where σ is the Sigmoid activation function, and the MLP contains one hidden layer (number of neurons C / 16) and ReLU activation;

[0091] S2223. Apply the channel attention weights to the multi-size feature map to obtain channel-enhanced features;

[0092] After spatial information compression and weight generation, channel feature enhancement begins by multiplying the channel weights by the original features channel by channel to obtain the enhanced channel features F′. ,in This indicates channel-by-channel multiplication. This operation enhances channel features related to the weld, such as edges and regions with abrupt changes in depth, while suppressing noise from irrelevant channels.

[0093] Channel weights essentially characterize the importance coefficient of each feature channel in the weld inspection task. For example, in an RGB-D four-channel input, the depth channel (D) and the red channel (R) typically receive higher weights because they contain information about weld geometry deformation and oxidation color difference, while the blue channel (B), which is affected by welding spatter, is dynamically suppressed.

[0094] S223. By using a spatial attention mechanism, spatial weighting is applied to the channel enhancement features to obtain the channel spatial enhancement features;

[0095] S223 includes:

[0096] S2231. Perform max pooling and average pooling along the channel to enhance the channel features, and obtain the max pooling spatial feature map and the average pooling spatial feature map.

[0097] The spatial attention module focuses on key spatial locations within the weld area, performing max pooling and average pooling on the channel enhancement feature F′ along the channel dimension to generate two spatial feature maps: , ;

[0098] S2232: Concatenate the max pooling spatial feature map and the average pooling spatial feature map, and generate spatial attention weights through a large convolutional kernel;

[0099] The two-path features are concatenated and then used to generate spatial attention weights through a convolutional layer. : ,in For a 7×7 convolution kernel, [;] represents channel concatenation, and σ is the Sigmoid activation function.

[0100] S2233. Apply spatial attention weights to channel enhancement features to obtain channel spatial enhancement features.

[0101] Through the channel information aggregation and spatial weight generation described above, the spatial weights are multiplied pixel-by-pixel with the channel enhancement features to obtain the final enhancement features. , ,in This indicates channel-by-channel multiplication. This operation enhances channel features related to the weld (such as edges and abrupt depth changes) and suppresses noise from irrelevant channels.

[0102] After Center Net convolutional neural network analyzes the approximate location of the weld seam, S222 and S223 perform further coarse localization. Features processed by the residual block are then weighted using the channel and spatial attention mechanism CBAM, readjusting the weights of features in both the spatial and channel dimensions. These weights are then added to and fused with the features from the decoding layer, allowing the network to fully utilize effective features and improve detection accuracy. In the channel and spatial attention mechanisms, the CBAM module significantly enhances the network's ability to perceive key features of the weld area through a hierarchical feature selection mechanism.

[0103] The channel attention mechanism first compresses global information along the channel dimension of the feature map output by the residual block, then uses max pooling to capture the most significant feature responses in the weld area, while simultaneously obtaining channel-level statistical features through average pooling. This dual-channel pooling strategy effectively avoids information loss caused by a single pooling operation. Max pooling is sensitive to local extrema, such as high-contrast areas like metallic reflections and bevel edges, while average pooling reflects the overall activation intensity distribution of the channels.

[0104] Building upon channel-enhanced features, the spatial attention mechanism further focuses on the spatial structural features of the weld. This module compresses multi-channel feature maps into two spatial response maps through dual-pooling operations along the channel dimension: the max-pooling response map highlights local extreme features such as the weld centerline and bevel inflection points, while the average-pooling response map reflects the overall spatial distribution trend of the welding area. The concatenation of these two response maps constructs a multi-granularity feature representation in the spatial dimension. Subsequently, a large-size convolutional kernel (7×7) integrates the spatial context information. This wide receptive field design effectively correlates the weld center point with its surrounding bevel structure, avoiding false activations caused by local noise. The final generated spatial attention weight map exhibits a distinct banded high-response region, whose shape closely matches the actual weld orientation, validating the module's accurate capture of spatial geometric features.

[0105] Channel attention and spatial attention employ a cascaded rather than parallel structure, a design that follows a "channel-first, spatial-last" feature optimization logic. Channel-dimensional feature filtering provides a high-quality feature base for spatial attention, eliminating redundancy, while spatial-dimensional weight adjustments further enhance the detailed representation of the target region based on channel optimization. When working together, the channel attention module can be viewed as a "global filter" for feature channels, while the spatial attention module acts as a "local focuser" for spatial location.

[0106] S224. Gaussian kernel diffusion is applied to enhance the channel space features to obtain a probability heatmap of the weld center; simultaneously, weld size, center point offset, and depth correction are performed in parallel to output three-dimensional weld information. Specifically:

[0107] Gaussian kernel diffusion: Heatmap of predicted weld center point in network output layer The model is a Gaussian distribution, and the center point of the labeled weld (x c y c Generate a Gaussian nuclear diffusion heatmap, expressed as: ,in and These are the center coordinates after downsampling. For adaptive standard deviation.

[0108] Next, we defined the loss function, using an improved FocalLoss to address the class imbalance problem. The loss function is as follows: ;

[0109] Where α=2 and β=4 are mainly used to suppress the gradient contribution of simple samples, and N is the number of positive samples. The above heatmap branch explicitly models the weld center location in the form of probability density, providing a strong supervision signal for the target region for subsequent geometric regression.

[0110] Weld size regression: To accurately describe the weld morphology, the network regresses the weld size (w, h) and center point offset (Δx, Δy) in parallel. First, size regression is defined, and the output weld width and height are calculated using L1 loss. N represents the total number of weld feature points processed in the current batch. k is the feature point index variable, and the traversal range is k∈[1,N]. This represents the actual weld width at the k-th measurement point. Let be the weld width predicted by the network at point k. This represents the actual weld height at the k-th measurement point. Let be the weld height at point k predicted by the network.

[0111] Center point offset: Define offset compensation to correct the quantization error caused by downsampling. The L1 loss is: N represents the total number of weld feature points processed in the current batch. K is the feature point index variable, and the traversal range is k∈[1,N]. This represents the x-axis offset of the k-th point predicted by the network. Let x be the true x-coordinate of the k-th measurement point. This represents the y-axis offset of the k-th point predicted by the network. Let be the true y-coordinate of the k-th measurement point.

[0112] The aforementioned dimensional regression and offset compensation together optimize the positioning accuracy of the weld boundary box, making up for the shortcomings of single thermal image prediction in terms of spatial resolution.

[0113] Depth Correction: Since depth measurements from structured light cameras are susceptible to interference from metallic reflections, the network outputs an additional depth correction value Δz. Huber loss is used to enhance robustness.

[0114] N represents the total number of weld feature points processed in the current batch. K is the feature point index variable, and the traversal range is k∈[1,N]. This represents the true depth offset of the k-th sampling point. This is the depth correction value for the k-th sampling point predicted by the network.

[0115] The deep regression branch corrects system errors through end-to-end learning and incorporates the complementarity of RGB and depth information into network training, thereby improving the reliability of 3D localization.

[0116] Based on the above analysis, a weighted sum of the total loss function is defined and optimized during training: ;

[0117] The parameters are set to typical weights: λheat=1, λsize=0.1, λoffset=1, λdepth=0.5. By dynamically balancing the multi-task loss, the network achieves synergistic optimization among heatmap confidence, geometric accuracy, and depth correction, avoiding overfitting to a single task.

[0118] S3. Based on the identified weld position, the weld position is scanned using a line laser camera to obtain the three-dimensional point cloud data of the weld.

[0119] S4, such as Figure 6 As shown, the three-dimensional coordinates of the weld are identified based on three-dimensional point cloud data using the Marr-Hildreth edge detection algorithm.

[0120] After the initial coarse identification using a structured light camera following the steps described above, the first part of the fine identification section for weld seam classification and positioning is line laser scanning and data preprocessing. At this point, the robot, having undergone coarse positioning and previous hand-eye calibration, is moved to the initial position for fine identification. Guided by the prior weld seam position provided by the coarse identification, the line laser camera initiates a high-precision scan. Its working principle is based on optical triangulation: the laser projects a linear spot onto the workpiece surface, and the camera calculates the three-dimensional coordinates by the displacement of the spot on the imaging plane. During the scanning process, the robotic arm moves at a constant speed along the weld seam to ensure sufficient point cloud density.

[0121] S4 includes:

[0122] S41, such as Figure 5 As shown, the preprocessing operations for 3D point cloud data include sequential pass-through filtering, dimension transformation, coordinate mapping, and region segmentation and repair, resulting in a target depth grayscale image and dimension mapping relationship.

[0123] S41 includes:

[0124] S411. Based on the preset weld seam buffer expansion amount, perform pass-through filtering on the three-dimensional point cloud data to retain the point cloud data of the key areas of the weld seam.

[0125] 1. To eliminate environmental noise interference, a direct-pass filter is first performed, using the following formula:

[0126] ,

[0127] Where x min =X weld −Δ buffer x max =X weld +Δ buffer , where Δ buffer=20mm (buffer expansion to accommodate potential welding heat deformation) After filtering, the point cloud amount is reduced by about 60%, while retaining key areas such as the bevel root and blunt edge. This greatly reduces the amount of data processing.

[0128] S412. Project the retained 3D point cloud data onto the laser scanning plane and quantize it along the depth direction to generate a depth-mapped grayscale image. Specifically, project the 3D point cloud onto the laser scanning plane (XY plane) and quantize it along the Z-axis to generate a grayscale image, facilitating the intervention of traditional image processing algorithms. This process needs to resolve the contradiction between depth resolution (Z-axis quantization accuracy) and noise suppression.

[0129] In this formula, zmed(u, v) is the median Z-coordinate of all points within the raster (u, v), suppressing the influence of outliers (such as splashes). μz and σz are the mean and standard deviation of the Z-axis of the filtered point cloud. The above method compresses 3D geometric information into a 2D grayscale space through depth mapping, similar to converting a 3D topographic map into a contour map, providing standardized input for subsequent image processing. The median filtering and 3σ truncation in the formula essentially suppress random noise while preserving the main shape of the slope. This is equivalent to performing noise reduction and enhancement processing on the 3D data.

[0130] S413. Based on 3D point cloud data and depth-mapped grayscale images, establish dimensional mapping relationships and construct a bidirectional lookup table;

[0131] Specifically, this involves constructing a 3D-2D coordinate index to back-locate subsequent image processing results (such as edge coordinates) to 3D space. This requires establishing a bidirectional mapping relationship and building a bidirectional lookup table to achieve cross-dimensional data closure. The dimensional mapping relationship is as follows: ;

[0132] Extended base coordinates x base y base To reserve a boundary buffer and prevent index out-of-bounds errors in the boundary point cloud due to rounding errors, the design of the round function and extended base coordinates ensures the numerical stability of the mapping process and avoids index misalignment caused by floating-point errors.

[0133] S414. Separate the bevel region from the parent material background in the depth-mapped grayscale image to generate a binary mask image. Then perform adaptive threshold segmentation. After segmentation, perform morphological restoration on the binary mask image to obtain the target depth grayscale image.

[0134] Specifically, in welding scenarios, interference from material reflection and oxidation color difference necessitates adaptive thresholding and morphological restoration. An improved large-law method is employed for threshold segmentation. Adaptive threshold segmentation solves the classification problem of gray-level overlapping regions, and its improved Otsu algorithm finds a balance between maximizing inter-class variance and ensuring reasonable target region size. Where T is the optimal segmentation threshold, and w1 and w2 are the optimized weight coefficients. For the between-class variance, parameters ω1=0.7 and ω2=0.3 are defined, and for balancing the between-class variance... Constraints on the proportion of the foreground. The foreground pixel proportion is constrained to around 30% (an empirical value) to avoid oversegmentation. Its physical meaning is to maximize inter-class differences while limiting the proportion of the bevel region to conform to actual working conditions. After threshold segmentation, morphological processing begins. The binary image undergoes dilation followed by erosion to fill small holes and smooth edges to eliminate isolated spots caused by missing point clouds or noise, ensuring the connectivity of the bevel contour. The formula is: B3 is a 3×3 cross-shaped core to eliminate small holes, and B5 is a 5×5 rectangular core to smooth the bevel edge burrs.

[0135] S42. Using the Marr-Hildreth edge detection algorithm, Gaussian filtering, Laplacian operation, and zero-crossing detection are performed sequentially on the target depth grayscale image to obtain the two-dimensional coordinates of the weld edge.

[0136] Gaussian filtering: As a core step in preprocessing, Gaussian filtering is physically significant in that it establishes a balance between spatial smoothness and edge preservation.

[0137] The Gaussian kernel function essentially reshapes the depth-mapped grayscale image generated by laser scanning in the frequency domain using a spatially weighted averaging strategy. The exponentially decaying weight distribution characteristic causes the contribution of neighboring pixels to the center pixel to decrease in a bell-shaped curve. This mechanism suppresses high-frequency noise (such as salt-and-pepper noise caused by spatter particles) while maintaining the gradient continuity of the bevel edge. Especially for the common metal oxide layer reflection interference in welding scenarios, the smoothing effect of Gaussian filtering can effectively bridge local brightness abrupt changes, forming a macroscopically consistent bevel morphology. The formula is: Where σ represents the standard deviation of the Gaussian kernel, controlling the smoothing intensity. An excessively large σ leads to blurred edges, while a too small σ results in residual noise. A convolution operation is then performed, involving a weighted sum of the kernel and local image regions, mathematically expressed as: Where I(x, y) is the original grayscale image, and I′(x, y) is the smoothed image. The Gaussian filtering method effectively filters out high-frequency noise by relying on the exponential decay characteristic of the Gaussian kernel, while also ensuring a centrally symmetrical distribution of weights, avoiding the edge blurring effect of mean filtering.

[0138] Laplacian Operation: The design of the Laplacian operator reflects the sensitivity of second-order differentials to geometric features. Its discretized convolution kernel, through the symmetrical arrangement of negative weights at the center and positive weights in the neighborhood, essentially represents a quantized response to image curvature. When applied to an image after Gaussian filtering, the operator generates strong response values ​​with opposite signs on both sides of the bevel edge—a positive peak in the upper region due to a sharp decrease in depth, and a negative peak in the root region due to a sudden increase in depth. This bidirectional response characteristic allows the algorithm to distinguish the geometric orientation of the bevel, providing a directional basis for subsequent slope analysis. Especially in the weld heat-affected zone, the changing trend of the Laplacian response amplitude can indirectly reflect the degree of thermal deformation of the material, providing feature-level input for adaptive path planning.

[0139] The Laplacian operator is a second-order differential operator, highly sensitive to abrupt changes in grayscale (such as bevel edges), and its response intensity is positively correlated with the edge curvature. Edge regions are located by calculating the second derivative of the image through convolution.

[0140] The formula is: Where E(x, y): edge intensity map, positive values ​​indicate gray-level protrusions (such as the upper edge of a bevel), and negative values ​​indicate indentations (such as the root of a bevel). Physical meaning: the larger the absolute value of the Laplace response, the higher the edge intensity.

[0141] Zero-crossing detection: After Gaussian filtering and Laplace kernel operation, zero-crossing detection is performed. Its engineering value lies in transforming abstract mathematical responses into operable physical boundaries. Through a neighborhood sign comparison strategy, the algorithm can capture sub-pixel-level transition features of bevel edges. Essentially, it locates edge crossing points in continuous space by reversing the sign of the discrete differential response. This detection mechanism has unique adaptability to edge blurring caused by welding deformation: when thermal deformation makes it difficult to determine precise boundaries using traditional thresholding methods, the spatial distribution of zero-crossing points can still maintain a stable topological structure. The gradient threshold constraint introduced during detection achieves an adaptive balance between weak edge preservation and noise suppression by dynamically adjusting the sensitivity threshold. A zero-crossing point is defined as the pixel position in the edge intensity map E(x, y) where the sign changes, representing the extreme point of the second derivative of grayscale (corresponding to the peak of the first derivative, i.e., the true edge). The detection algorithm first performs neighborhood sign comparison: for each pixel (x, y), it checks whether there is a response value with the opposite sign in its 8-neighborhood. Next, a gradient threshold constraint is applied: a valid edge is determined only when the difference between a positive and negative response pair exceeds the threshold Tedge. The specific mathematical conditions are as follows:

[0142] The parameter is set as follows To dynamically adapt to image contrast, non-maximum suppression (NMS) is introduced to preserve local maximum response points and eliminate duplicate edges. The above detection outputs a binary edge map B(x, y) ∈ {0, 1}, where B(x, y) = 1 represents an edge pixel.

[0143] S43. Based on the dimension mapping relationship, obtain the three-dimensional coordinates corresponding to the two-dimensional coordinate data of the weld edge.

[0144] After obtaining the two-dimensional coordinate information of the weld edge through zero-cross detection, the system achieves accurate mapping from image space to three-dimensional physical space through a pre-built 3D-2D coordinate index, i.e., dimensional mapping relationship.

[0145] In S42 and S43, after the aforementioned laser scanning and data preprocessing steps, the Marr-Hildreth edge detection method is used to obtain the two-dimensional coordinate information of the weld. Finally, three-dimensional coordinate data is returned after 3D-2D coordinate indexing. The Marr-Hildreth method achieves a balance between noise resistance and edge positioning accuracy by combining the Gaussian-Laplacian operator (LoG) with zero-crossing detection. Its core lies in capturing regions of abrupt grayscale changes (bevel edges) using a second-order differential operator and mapping the two-dimensional image coordinates to three-dimensional space.

[0146] After completing the above steps, the host computer automatically generates multi-layer, multi-pass welding paths based on the feature parameters, plans the welding torch movement trajectory and process parameters, and forms a fully closed-loop control link of "detection-positioning-planning". This process seamlessly connects the image processing results to the physical welding space through cross-dimensional data fusion and spatial mapping technology.

[0147] Corresponding to the above method steps, the present invention also provides a weld coarseness-grading three-dimensional positioning system based on the fusion of structured light and line laser, comprising:

[0148] Structured light cameras are used to photograph workpieces and acquire images of them.

[0149] The coarse identification module integrates a channel spatial attention mechanism and a Center Net convolutional neural network to perform coarse identification of weld seam locations and determine the weld seam location.

[0150] A line laser camera is used to scan the weld seam location and obtain three-dimensional point cloud data of the weld seam;

[0151] The fine recognition module is used to identify the three-dimensional coordinates of the weld seam using the Marr-Hildreth edge detection algorithm.

[0152] Both the structured light camera and the line laser camera are vision modules. The structured light camera is mainly responsible for coarse weld seam identification, while the line laser scanning camera is mainly responsible for fine weld seam identification.

[0153] The coarse-grained structured light camera uses the Yinniu R-132 structured light camera to acquire RGB-D images of the welding site. The Center Net neural network can quickly and accurately locate the approximate position of the weld.

[0154] The fine-grained identification section utilizes the DeepVision SR-7900 line laser camera to obtain point cloud images. After coarse identification determines the approximate weld area, the line laser camera, as a high-precision measurement module, intervenes to perform local fine-grained scanning of the target area. Line laser technology, based on the principle of triangulation, generates sub-millimeter-precision 3D point cloud data through laser line scanning, accurately analyzing the geometric features of the weld, such as bevel angle, depth, and gap width. Its high-resolution characteristics can capture subtle changes at the weld edge, and combined with edge detection algorithms to dynamically optimize data quality, it maps depth information to a grayscale image according to a linear relationship. Marr-Hildreth edge detection is used to find and select the weld edge, finally obtaining the weld data information. Even under complex interference such as metal spatter and oxide layer coverage, stable measurement accuracy is maintained. The collaborative working mechanism of the two achieves seamless integration through data fusion technology. The global coordinate information provided by the structured light camera is combined with the local high-precision data from the line laser camera to form a complete 3D model of the welding scene, providing accurate input for robot path planning.

[0155] The structured light camera and line laser camera of this invention can meet the technical requirements of coarse-to-fine two-level data processing: the coarse-grained point cloud output by the structured light camera provides a spatial reference for the initial positioning of the weld, while the high-precision point cloud from the line laser camera performs fine reconstruction of the bevel details. The collaborative work of the two constructs a hierarchical data input channel for multi-level recognition algorithms. Through spatial layout optimization and collaborative control strategies, the structured light camera and line laser camera achieve a synergistic effect with limited resources, providing reliable hardware support for weld feature extraction under complex working conditions.

[0156] The technical solution of this invention is particularly suitable for welding complex workpieces such as tunnel boring machine cutterheads, where the various types of bevels (such as X-type, Y-type, and K-type) place higher demands on the adaptability and robustness of the recognition system. Through a phased processing strategy, both the efficiency of large-scale scanning and high-precision analysis of key areas are ensured, significantly improving the level of welding automation. Furthermore, the system's built-in dynamic anti-interference mechanism can adaptively adjust the light source intensity and data processing algorithm, effectively addressing common challenges in industrial settings such as ambient light fluctuations and smoke obstruction. Ultimately, this coarse-fine collaborative recognition mode not only reduces the need for manual intervention but also significantly improves the consistency of welding quality and process controllability, providing reliable technical support for the intelligent upgrading of high-end equipment manufacturing.

[0157] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A three-dimensional positioning method for weld seam coarseness and fineness classification by fusing structured light and line laser, characterized in that: Includes the following steps: S1. Using a structured light camera, a panoramic view of the workpiece is formed, the workpiece is photographed, RGB images are obtained, coded stripes are projected onto the workpiece, and deformed stripe images are captured. S2. Based on the deformed stripe image, the wrapping phase of the deformed stripe image is calculated by the phase shift method, and the depth map is calculated according to the camera calibration parameters. The depth map is aligned and fused with the RGB image to generate a four-channel RGB-D tensor. The four-channel RGB-D tensor is input into the Center Net convolutional neural network with channel spatial attention mechanism for processing, and the output is the three-dimensional information of the weld, i.e. the weld position. S3. Based on the identified weld position, the weld position is scanned by a line laser camera to obtain the three-dimensional point cloud data of the weld. The line laser camera selects the installation pose according to the geometric features of the target area and complements the field of view of the structured light camera. The structured light camera and the line laser camera coordinate the exposure sequence through time-division trigger control, and the two light sources do not interfere with each other. S4. Based on 3D point cloud data, the 3D coordinates of the weld are identified through an edge detection algorithm.

2. The three-dimensional positioning method for weld coarseness and fineness classification by fusion of structured light and line laser as described in claim 1, characterized in that: S22 includes: S221. Using the feature encoding network of the Center Net convolutional neural network, feature extraction is performed on the four-channel RGB-D tensor to generate a multi-scale feature map. S222. Channel weighting is performed on multi-size feature maps through a channel attention mechanism to obtain channel-enhanced features; S223. By using a spatial attention mechanism, spatial weighting is applied to the channel enhancement features to obtain the channel spatial enhancement features; S224. Gaussian kernel diffusion is applied to enhance the channel space features to obtain a probability heatmap of the weld center. At the same time, the weld size, center point offset, and depth are corrected in parallel to output the three-dimensional information of the weld.

3. The three-dimensional positioning method for weld coarseness and fineness classification by fusion of structured light and line laser as described in claim 2, characterized in that: S222 includes: S2221. Perform global max pooling and average pooling on the multi-size feature maps to obtain max pooling channel vectors and average pooling channel vectors. S2222: Input the max pooling channel vector and the average pooling channel vector into the shared MLP to generate channel attention weights; S2223. Apply the channel attention weights to the multi-size feature map to obtain channel-enhanced features; S223 includes: S2231. Perform max pooling and average pooling along the channel to enhance the channel features, and obtain the max pooling spatial feature map and the average pooling spatial feature map. S2232: Concatenate the max pooling spatial feature map and the average pooling spatial feature map, and generate spatial attention weights through a large convolutional kernel; S2233. Apply spatial attention weights to channel enhancement features to obtain channel spatial enhancement features.

4. The three-dimensional positioning method for weld coarseness and fineness classification by fusion of structured light and line laser as described in claim 1, characterized in that: S4 include: S41. Preprocessing operations on 3D point cloud data, including sequential pass-through filtering, dimension transformation, coordinate mapping, and region segmentation and repair, to obtain the target depth grayscale image and dimension mapping relationship; S42. Using the Marr-Hildreth edge detection algorithm, Gaussian filtering, Laplacian operation, and zero-crossing detection are performed sequentially on the target depth grayscale image to obtain the two-dimensional coordinates of the weld edge. S43. Based on the dimension mapping relationship, obtain the three-dimensional coordinates corresponding to the two-dimensional coordinate data of the weld edge.

5. The three-dimensional positioning method for weld coarseness and fineness classification by fusion of structured light and line laser as described in claim 4, characterized in that: S4 1 includes: S411. Based on the preset weld seam buffer expansion amount, perform pass-through filtering on the three-dimensional point cloud data to retain the point cloud data of the key areas of the weld seam. S412. Project the retained 3D point cloud data onto the laser scanning plane and quantize it along the depth direction to generate a depth-mapped grayscale image. S413. Based on 3D point cloud data and depth-mapped grayscale images, establish dimensional mapping relationships and construct a bidirectional lookup table; S414. Separate the bevel region from the parent material background in the depth-mapped grayscale image to generate a binary mask image. Then perform adaptive threshold segmentation. After segmentation, perform morphological restoration on the binary mask image to obtain the target depth grayscale image.

6. A three-dimensional positioning system for weld seam coarseness and fineness classification by fusing structured light and line laser, characterized in that: include: A structured light camera is used to create a panoramic coverage of the workpiece, capture images of the workpiece, obtain RGB images, project coded stripes onto the workpiece, and capture images of deformed stripes. The coarse recognition module is used to calculate the wrapping phase of the deformed stripe image based on the deformed stripe image using the phase shift method, and calculate the depth map according to the camera calibration parameters. The depth map is aligned and fused with the RGB image to generate a four-channel RGB-D tensor. The four-channel RGB-D tensor is input into the Center Net convolutional neural network with channel spatial attention mechanism for processing, and the output is the three-dimensional information of the weld, i.e. the weld position. A line laser camera is used to scan the weld position and obtain three-dimensional point cloud data of the weld. The line laser camera selects the installation pose according to the geometric features of the target area and complements the field of view of the structured light camera. The fine recognition module is used to identify the three-dimensional coordinates of the weld seam using the Marr-Hildreth edge detection algorithm; The time-division triggering control module controls the application timing of the structured light camera and the line laser camera through hardware synchronization signals, ensuring that the two light sources do not interfere with each other.

7. A three-dimensional positioning system for weld coarseness and fineness classification based on the fusion of structured light and line laser as described in claim 6, characterized in that: The structured light camera and the line laser camera are also equipped with filtering devices, which use spectral isolation technology to eliminate crosstalk between light sources of different wavelengths.

Images