A method and system for tracking the formation of a gasketed joint of a steel fluid conveying product based on 3D point cloud and deep learning

Abstract

The application discloses a kind of based on 3D point cloud and deep learning's steel fluid conveying product's slit connection forming tracking method and system, including the 3D depth image of real-time acquisition steel fluid conveying product surface forming slit area;Real forming slit is extracted in complex background by deep learning quickly, and high-precision mask chart is output;Based on mask chart, obtain effective 3D forming slit surface feature as ROI region;The three-dimensional profile of the ROI region is geometrically fitted by visual algorithm, and the real three-dimensional space center coordinates of the current forming slit section are calculated;The real three-dimensional space center coordinates are subtracted from reference center coordinates, and the deviation value is obtained;If deviation absolute value < set value, it is determined that the pipe body is normal mechanical shaking;If deviation absolute value ≥ set value, compensation and rectification are carried out.

Description

Technical Field

[0001] This invention relates to an automatic tracking method and system for the gap connection forming process of steel fluid conveying products. Background Technology

[0002] Steel fluid conveying products are widely used in oil and gas transportation and large-scale water supply and drainage projects due to their high pressure resistance and flexible size selection. In the manufacturing process of steel fluid conveying products, the joint forming process is a key step that determines the final structural strength and sealing performance. Ensuring that the welding torch is always precisely aligned with the center line of the formed joint is a prerequisite for ensuring welding quality and avoiding serious welding defects such as porosity, slag inclusions, and incomplete penetration.

[0003] However, in existing steel fluid conveying product production lines, complex spatial displacement deviations inevitably occur during the dynamic process of continuous rotational forming and forward welding of steel fluid conveying products. On the one hand, due to the unevenness of the raw material edges, the gaps after forming are not absolutely uniform. On the other hand, during long-distance transmission and rotation, the steel body itself will generate unpredictable mechanical vibrations and jumps. These factors cause the spatial position of the forming gaps to exhibit irregular dynamic drift. However, existing gap tracking technologies in the steel fluid conveying product industry are all based on traditional visual detection algorithms without the participation of deep learning models. It is difficult to determine whether it is a forming gap or an indentation based solely on the height difference of the steel plate surface. The latest data is simply taken as the offset, resulting in low accuracy. Summary of the Invention

[0004] This invention aims to solve the key technical problem of high-precision, interference-resistant, real-time automatic tracking of the center line of forming gaps under harsh working conditions by fusing 3D laser contour scanning, deep learning, and traditional machine vision algorithms. It proposes a method and system for tracking the forming gaps of steel fluid transport products based on 3D point clouds and deep learning. This overcomes the limitation of easily confused indentation / forming gap identification, achieving accurate identification of forming gaps through intelligent indentation filtering; improves the system's adaptability to complex environments with high dust and high temperatures, reducing manual intervention; is compatible with various forming gaps and raw material steel plates from various brands, meeting the production needs of multiple production lines; and constructs an independently developed vision-software-hardware collaborative control system, reducing the industry's dependence on external suppliers.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a method for tracking the gap connection forming of steel fluid transport products based on 3D point cloud and deep learning, comprising the following steps: Step S1: Real-time acquisition of 3D depth images of the forming gap area on the surface of the steel fluid conveying product; Step S2 uses deep learning to quickly extract real molding gaps from complex backgrounds and outputs a high-precision mask image. Step S3: Based on the mask image, obtain effective 3D molding gap surface features as ROI regions; use a visual algorithm to geometrically fit the three-dimensional contour of the ROI region to calculate the true three-dimensional spatial center coordinates of the current molding gap cross section; subtract the true three-dimensional spatial center coordinates from the reference center coordinates to obtain the deviation value. If the absolute value of the deviation is less than the set value in step S4, it is determined to be normal mechanical vibration of the pipe body; if the absolute value of the deviation is greater than or equal to the set value, compensation and correction are performed.

[0006] As a further technical solution, after acquiring the images in step S1, the images need to be stitched together, and the process is as follows: 1.1 Multi-frame cumulative stitching The XYZ three-channel point cloud and grayscale image acquired in each frame are vertically stitched along the row direction; the total number of rows after stitching does not exceed the preset row number threshold. If the threshold is exceeded, the oldest row is discarded, only the newest row is kept, and the row difference between the old and new data is output for subsequent algorithms to correct the timestamp. 1.2 Motion Coordinate Compensation Based on the difference between the current point's row index r in the cumulative image and the middle row, multiply by a fixed step size parameter to recalculate the point's Y coordinate value; the Y coordinate of the middle row is set to zero, and the coordinate increases / decreases linearly when it is far from the middle row, thereby eliminating the influence of raw material movement on the Y coordinate and achieving accurate alignment between point cloud data and physical space; 1.3 Parameters are configurable The threshold for the number of stitched rows and the Y-axis step size are both dynamically read through an XML configuration file, supporting flexible adjustments for different linear speeds and precision requirements. As a further technical solution, in step S2, an optimized YOLOv8 instance segmentation model is used to quickly and robustly identify the real molding gap region and output a high-precision mask image as the region of interest for subsequent precise calculation.

[0007] As a further technical solution, the YOLOv8 instance segmentation model is as follows: Using CSPDarknet as the backbone network, richer gradient flow information is obtained by introducing the C2f module; and a bidirectional fusion structure of feature pyramid and path aggregation network is adopted in the neck, which greatly enhances the feature extraction and multi-scale perception capabilities of the slender, irregular small target of the forming gap on the surface of steel fluid conveying products. The model's prediction head adopts a decoupled structure, processing classification, bounding box regression, and mask segmentation tasks independently. Its total loss function combines classification loss, bounding box loss, and mask loss. In response to the unique morphology of the external welds on steel fluid conveying products, we abandoned the generalized annotation of the entire drawing and only performed high-precision polygonal local annotation on the effective area of ​​the actual formed gap on the surface of the steel fluid conveying products, deliberately excluding flux accumulation areas and simple spatter points as background. An improved YOLOv8 target detection model was trained based on a labeled molding gap dataset to obtain a molding gap recognition mask.

[0008] As a further technical solution, step 3 includes the following steps: 3.1 Multi-channel data processing and local region cropping (ROI extraction): Perform channel separation on the image in step S1 to extract independent spatial coordinate image matrices in the X, Y, and Z dimensions; Using the molding gap recognition mask as a mask, threshold segmentation is performed on the Z-axis (height) image to remove invalid background scatter points that exceed the reasonable height range, thereby obtaining effective 3D molding gap surface features as ROI regions; The extracted ROI region is bounded by a minimum bounding rectangle, and then a rectangle closing operation is performed to eliminate local contour breaks caused by flux obstruction or light interference, thus obtaining a continuous and complete forming gap region.

[0009] 3.2 Identify genuine and counterfeit molding gaps and eliminate interference in continuous and complete molding gap areas: 3.3 Precise calculation using a bottom-up sliding window; As a further technical solution, 3.2 is specifically as follows: To address the issue of indentations caused by the manufacturing process of steel fluid conveying products being misidentified as forming gaps, the algorithm employs a row traversal strategy.

[0010] Line-by-line scanning and connected component analysis: Within the minimum bounding rectangle of the forming gap region, generate rectangular bars of pixels with a preset height line by line, find their intersection with the closed operation region of the forming gap, and separate the connected components of each line. Expansion and intersection with effective data: Perform rectangular expansion on each connected component, then find the intersection with the effective height region, retaining only the portion of Z value within the effective range; Gray-scale feature extraction: For the preserved regions, extract their mean gray-scale value on the X-coordinate image to obtain the mean X-coordinate value of the connected regions in that row; Deviation comparison and indentation removal: Calculate the absolute difference between the mean and the center of the image captured by the camera. If the absolute difference is greater than the first preset threshold, the area is determined to be an indentation and discarded; otherwise, it is retained, and the connected regions with the smallest difference are selected. Real Region Merging: Perform a union operation on all the remaining connected regions to form the final real-formed gap object; As a further technical solution, 3.3 is specifically as follows: Extract the effective analysis area at the bottom of the molding gap: Define the boundaries of the analysis region; Sliding window parameter settings allow the window to slide upwards layer by layer from the bottom boundary; Perform window processing; The output is the clustering result based on the central value.

[0011] As a further technical solution, the method for extracting the effective analysis area at the bottom of the molding gap is as follows: For the forming gap of steel fluid conveying products, take the area extending upward from the bottom of the image within the second preset threshold range, generate a rectangle, and intersect it with the actual forming gap object to obtain the area used for analysis; if the area of ​​this region is less than the preset height, the deviation is returned directly as 0. As a further technical solution, the method for determining the boundary of the analysis region is: to calculate the minimum bounding rectangle for the intersection region, and obtain the upper boundary, lower boundary, left boundary, and right boundary; As a further technical solution, the window processing flow is as follows: For each window, a rectangular region is generated, and its intersection with the actual formed gap object is calculated. Connected components in the intersection are separated, and the region with the largest area is selected. The center of the area of ​​the largest region is calculated, and if the area is less than a third preset threshold range, the current window is skipped. The largest region is rectangularly dilated, and then image difference is performed between it and the original region to obtain the region's outer contour. A rectangular opening operation is performed on the difference result to eliminate isolated noise points, and then its minimum bounding rectangle is calculated to obtain the left and right boundary ranges. The opening operation result is separated into connected components, and the left and right regions are filtered based on column coordinates. The left and right regions are then unioned separately, and their intersection with the effective height region is calculated to ensure data validity. Extract the center of the area of ​​the left and right sides. If either area is less than 1, skip it; otherwise, extract the mean grayscale value of the left and right sides on the X-coordinate image to obtain the mean X-coordinate value of the left boundary and the mean X-coordinate value of the right boundary. Take the arithmetic mean of the left and right means as the instantaneous center coordinate of the window.

[0012] As a further technical solution, the center value clustering output process is as follows: store the instantaneous center coordinates calculated by all windows into a tuple. If the number of effective windows is greater than 1, calculate the arithmetic mean of all instantaneous centers as the true three-dimensional spatial center coordinates of the current forming gap section; otherwise, output 0.

[0013] As a further technical solution, step S3 includes a maximum offset threshold protection mechanism. If the calculation result changes abnormally, zero deviation will be forced to be output to avoid the danger of catastrophic overshoot such as machine collision in the actuator.

[0014] Secondly, the present invention also provides a gap connection forming tracking system for steel fluid conveying products based on 3D point cloud and deep learning, as follows: The first module is configured to acquire 3D depth images of the forming gap area on the surface of the steel fluid conveying product in real time; The second module is configured to quickly extract real molding gaps in complex backgrounds through deep learning and output a high-precision mask image. The third module is configured to acquire effective 3D molding gap surface features based on a mask image, which are used as the ROI region; perform geometric fitting on the three-dimensional contour of the ROI region using a visual algorithm to calculate the true three-dimensional spatial center coordinates of the current molding gap section; and calculate the deviation value by subtracting the true three-dimensional spatial center coordinates from the reference center coordinates. The fourth module is configured to determine normal mechanical vibration of the pipe body if the absolute value of the deviation is less than the set value; and to perform compensation and correction if the absolute value of the deviation is greater than or equal to the set value.

[0015] As a further technical solution, the first module is a 3D laser contour sensor with a dustproof and spatterproof protective cover, installed in front of the welding torch. It projects laser contour lines onto the surface of the rotating steel fluid conveying product in real time and acquires 3D depth images of the forming gap area. The physical installation distance L between the 3D laser contour sensor and the submerged arc welding torch is calibrated. ,in The total delay compensation time set for the system.

[0016] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed, it implements the gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning.

[0017] Fourthly, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein, when the program instructions are executed by a computer, the computer executes the gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning.

[0018] Compared with existing technologies, the automatic tracking method and system for forming gaps in steel fluid conveying products proposed in this invention have the following significant advantages: This invention introduces a visual algorithm combined with a deep learning model to improve the accuracy of forming gap recognition. Trial operation tests showed no deviation. Instead of simply using the latest data as the offset, it utilizes a sliding window to collect all forming gap position data from the last movement to the latest time, calculating the offset row by row, and averaging the multiple data sets as the final offset value sent to the PLC. This greatly reduces the likelihood of offset calculation errors caused by missing or incorrect data sets. Furthermore, the introduced dead-zone filtering strategy (for offset judgment) effectively eliminates false offsets caused by steel body vibration, avoids high-frequency ineffective oscillations of the slide table, extends the service life of the mechanical actuator, and ensures long-term stable operation of the system under harsh conditions. Attached Figure Description Figure 1 A schematic diagram of the overall structure of the present invention; Figure 2 C2f structure diagram of the present invention; Figure 3 The FPN+PAN structure diagram of this invention; Figure 4 The present invention includes a non-bevel / V-bevel structure diagram; (a) shows a non-bevel structure; (b) shows a V-bevel structure. Figure 5 A schematic diagram of the molding gap markings in this invention; Figure 6 Example of a mask pattern for this invention (white represents the molding gap); Figure 7 Schematic diagram of selected ROI region; Figure 8 Schematic diagram of molding gaps and indentations; Figure 9 Calculation process; Figure 10 Overall process diagram; Detailed Implementation It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless otherwise expressly indicated by the invention, the singular form is also intended to include the plural form. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. This invention aims to solve the key technical problem of high-precision, interference-resistant, real-time automatic tracking of the center line of forming gaps under harsh working conditions by fusing 3D laser contour scanning, deep learning, and traditional machine vision algorithms. It proposes a method and system for tracking the forming gaps in steel fluid transport products based on 3D point clouds and deep learning. By combining improved deep learning models (such as YOLO-v8) and 3D point cloud processing algorithms, it achieves rapid identification and positioning of the forming gap, thereby guiding the motion module to perform precise movement according to intelligent dynamic deviation compensation. This helps reduce carbon emissions and promote green manufacturing; eliminates leakage hazards and ensures product sealing integrity; protects the occupational health of operators and extends product service life. Specifically, this invention is an automatic tracking algorithm for forming gaps based on visual perception and automatic operation, applicable to the pipe manufacturing scenario of steel fluid transport products, achieving the following objectives: overcoming the limitation of easily confused indentation / forming gap identification, achieving accurate identification of forming gaps by intelligent filtering of indentations; improving the system's adaptability to complex environments with high dust and high temperatures, and reducing manual intervention; It is compatible with various forming gaps and raw material steel plates from various brands, meeting the production needs of various production lines.

[0020] The present invention proposes a method for tracking the gap connection forming of steel fluid transport products based on 3D point cloud and deep learning, comprising the following steps: Step S1: Real-time acquisition of 3D depth images of the forming gap area on the surface of the steel fluid conveying product; details are as follows. A 3D laser contour sensor is used: A 4096-point line structured light 3D laser contour sensor is used as the vision acquisition element with a horizontal field of view of 5cm. By projecting laser lines onto the surface of the steel pipe and receiving the reflected light, high-precision three-dimensional contour point cloud data and depth images of the forming gap area are obtained.

[0021] Front-mounted protective installation structure: Addressing the harsh working conditions of welding processes, including significant flux spillage and potential spatter, the sensor employs a front-mounted design (installed at a predetermined position in front of the welding torch's travel direction). Simultaneously, the sensor is encased in a specially designed dustproof and spatter-proof protective cover. The bottom of the cover features a high-transmittance protective lens, effectively isolating the optical lens from dust and welding spatter contamination, ensuring the sensor's imaging stability during long-term operation.

[0022] The image capture and stitching process is as follows: 1.1 Multi-frame cumulative stitching The `AppendAndCrop` function vertically stitches together the XYZ three-channel point cloud (xyzMat) and grayscale image (grayMat) captured in each frame along the row direction (`cv::vconcat`). The total number of rows after stitching does not exceed the preset row threshold `ImageBatchRow` (default 500 rows). If the limit is exceeded, the oldest row is discarded, and only the newest `ImageBatchRow` row is retained. The row difference between the old and new data is output via `rowDiff` for subsequent algorithms to correct the timestamps.

[0023] 1.2 Motion Coordinate Compensation As the steel pipe continues to rotate and move forward, the actual Y-direction (direction of travel) coordinate increases linearly with frame accumulation. The code calls the ModifyYChannel function, which recalculates the Y-coordinate value of the current point by multiplying the difference delta between the current point's row index r and the middle row mid in the accumulated image by a fixed step parameter YStep: yVal = delta * YStep. The Y-coordinate of the middle row is set to zero, while the coordinate increases / decreases linearly away from the middle row, thus eliminating the influence of the raw material's movement on the Y-direction coordinate and achieving accurate alignment between the point cloud data and physical space.

[0024] 1.3 Parameters are configurable The imageBatchRow threshold for the number of stitched rows and the Y-axis step size YStep are both dynamically read through the XML configuration file (CameraDriver.xml), supporting flexible adjustments for different line speeds and precision requirements.

[0025] Step S2: Process the image. 2.1 Deep Learning Feature Extraction and Target Localization In complex welding conditions, to eliminate interference from flux coverage, strong arc light reflection, and uneven steel edges, this system utilizes an optimized YOLOv8 instance segmentation model to quickly and robustly identify the actual forming gap region and output a high-precision mask as the region of interest (ROI) for subsequent precise calculation by visual geometric algorithms. (1) Backbone network and multi-scale fusion: The model uses CSPDarknet as the backbone network and introduces the C2f (C2 with better gradient Flow) module to obtain richer gradient flow information; and adopts a bidirectional fusion structure of feature pyramid and path aggregation network (FPN+PAN) in the neck, which greatly enhances the feature extraction and multi-scale perception capabilities of the slender, irregular small target of the forming gap on the surface of the steel fluid conveying product.

[0026] (2) Decoupled Head and Loss Function Design: The model's prediction head adopts a decoupled structure, processing classification, bounding box regression, and mask segmentation tasks independently. Its total loss function integrates classification loss, bounding box loss, and mask loss, as shown in the following formula:

[0027] Among them, bounding box loss Using a combination of CIoU (Complete Cross-Union Ratio) and DFL (Distributed Focal Point Loss), mask loss The cross-entropy function is used for calculation. This mechanism ensures that the pixel-level boundary of the forming gap can still be accurately defined even under conditions of splashing and blurred edges.

[0028] (3) Customized data annotation strategy: In view of the special morphology of the external weld of steel fluid conveying products, the generalized annotation of the whole picture is abandoned. Only the effective area of ​​the actual forming gap on the surface of the steel fluid conveying products (such as no bevel / V-shaped bevel) is annotated with high precision polygonal local annotation. The flux accumulation area and simple splash points are deliberately excluded as background.

[0029] 2.2 Precision Measurement and Deviation Calculation Using Machine Vision After obtaining the 2D forming gap mask output by deep learning, the system uses a machine vision algorithm library to perform cross-dimensional fusion of the 2D mask image and high-precision 3D point cloud depth data, and performs high-precision geometric measurement and fitting to accurately calculate the lateral and height compensation data that the welding torch needs to perform. The specific execution flow of this algorithm is as follows: (1) Multi-channel data processing and local region cropping (ROI extraction): The Decompose3 function is used to perform channel separation on the raw data acquired by the 3D laser contour sensor, and to extract independent spatial coordinate image matrices in the X, Y, and Z dimensions.

[0030] Using the Mask image as a mask, the Threshold function is used to perform threshold segmentation on the Z-axis (height) image to remove invalid background scatter points (such as extremely deep holes or excessively high splashes) that exceed the reasonable height range, thereby obtaining effective 3D molding gap surface features as ROI regions.

[0031] The effective region is bounded by a minimum bounding rectangle and morphological closing is performed to eliminate local contour breaks caused by light occlusion.

[0032] The extracted molding gap area is bounded by a minimum bounding rectangle (SmallestRectangle1), followed by a rectangle closing operation (ClosingRectangle1) to eliminate local contour breaks caused by flux obstruction or light interference, thus obtaining a continuous and complete molding gap area.

[0033] (2) Identification of genuine and counterfeit molding gaps and removal of interference: To address the issue of indentations caused by the manufacturing process of steel fluid conveying products being misidentified as forming gaps, the algorithm employs a row traversal strategy.

[0034] Line-by-line scanning and connected component analysis: Within the minimum bounding rectangle of the forming gap region, generate rectangular bars (GenRectangle1) with a height of 1 pixel line by line, find their intersection with the closed operation region of the forming gap, and separate the connected components of each line.

[0035] Dilation and intersection with valid data: Dilation is performed on each connected component, and then the intersection with the valid height region is calculated, retaining only the part with Z value within the valid range.

[0036] Gray-scale feature extraction: For the preserved regions, extract their mean gray-scale values ​​(GrayFeatures, feature type "mean") on the X-coordinate image to obtain the mean X-coordinate of the connected regions in that row.

[0037] Deviation Comparison and Indentation Removal: Calculate the absolute difference between the mean and the center of the image captured by the camera (BaseCenterX). If the absolute difference is greater than a first preset threshold, the region is determined to be an indentation and discarded; otherwise, it is retained, and the connected region with the smallest difference is selected (using TupleSortIndex).

[0038] Real region merging: Perform a union operation on all the remaining connected regions to form the final real-formed gap object (MaskObject).

[0039] (3) Precise calculation using a bottom-up sliding window: To overcome the slight local distortions caused by uneven raw materials at the edges of the molding gaps, the algorithm does not use a simple averaging method across the entire image. Instead, it innovatively introduces a bottom-up sliding window fitting strategy. By setting specific window height, sliding step size, and number of calculations, features are extracted by sliding upwards from the bottom of the molding gap image.

[0040] Extracting the effective analysis area at the bottom of the forming gap: For forming gaps in steel fluid conveying products, the area extending upwards from the bottom of the image within a second preset threshold range is selected, and a rectangle (GenRectangle1) is generated. This rectangle intersects with the actual forming gap object to obtain the area used for analysis. If the area of ​​this region is smaller than the preset range, a deviation of 0 is returned directly.

[0041] Determine the boundaries of the analysis region: Calculate the minimum bounding rectangle (SmallestRectangle1) for the intersection region to obtain the upper boundary Row1, lower boundary Row2, left boundary Column1, and right boundary Column2.

[0042] Sliding window parameter settings: Set the window height, sliding step size, and number of windows. Slide the window layer by layer from the bottom boundary upwards.

[0043] Window handling process: For each window, a rectangular region (GenRectangle1) is generated, and its intersection with the actual formed gap object is calculated. Connected components in the intersection are separated (Connection), and the region with the largest area is selected. The area center (AreaCenter) of this largest region is calculated; if the area is less than a third preset threshold range, the current window is skipped.

[0044] The largest region is dilated using a rectangle (DilationRectangle), and then image difference is performed between it and the original region to obtain the region's outer contour. A rectangle opening operation (OpeningRectangle) is performed on the difference result to eliminate isolated noise points, and then its smallest bounding rectangle (SmallestRectangle) is calculated to obtain the left and right boundary ranges. The opening operation result is separated into connected components (Connection), and the left and right regions are filtered based on column coordinates. The left and right regions are then unioned (Union1), and their intersection with the valid height region (RegionValid) is calculated to ensure data validity.

[0045] Extract the area center (AreaCenter) of the left and right regions. If either area is smaller than a preset range, skip it; otherwise, extract the mean grayscale values ​​(GrayFeatures) of the left and right regions on the X-coordinate image to obtain the mean X-coordinate of the left and right boundaries. Take the arithmetic mean of the left and right means as the instantaneous center coordinate of the window.

[0046] Centroid clustering output: Store the instantaneous center coordinates calculated from all windows into a tuple. If the number of valid windows is greater than 1, calculate the arithmetic mean of all instantaneous centers and use it as the true 3D spatial center coordinates of the current forming gap section. Otherwise, output 0.

[0047] Biased clustering output and overshoot protection: Offset Calculation: The absolute center coordinates obtained from all sliding windows are averaged to obtain the true three-dimensional spatial center coordinates of the forming gap under the cross-section of the steel pipe at the current moment. The difference between this true coordinate and the reference center coordinate is used to obtain the actual lateral compensation offset (Offset) required by the welding torch.

[0048] Maximum offset protection: The algorithm has a maximum offset threshold (MaxOffsetValue, default 15mm) protection mechanism. Once the calculation result changes abnormally (exceeding the allowable range), it will force the output to be zero offset to avoid the danger of catastrophic overshoot such as machine collision in the actuator.

[0049] Step S3, Main Control Filter Anti-jitter filtering strategy (core control logic): The calculated forming gap deviation data is sent to the PLC control system in real time. To address the inherent mechanical jitter generated during the rotational forming and conveying process of steel fluid conveying products, a dead-band filtering algorithm is introduced within the PLC. The system sets an offset threshold: when the absolute value of the input deviation is less than the set value, the system determines it as high-frequency mechanical jitter of the tube body, filters out the data, and keeps the welding torch position unchanged; when the absolute value of the deviation is greater than or equal to the set value, the system determines it as a genuine forming gap offset and triggers a correction action.

[0050] Specifically, the automatic tracking method for forming gaps in steel fluid transport products according to the present invention, in the actual submerged arc welding production process of steel fluid transport products, has the following specific workflow and implementation steps: Step S1: Device initialization and parameter calibration Start the steel fluid conveying product transfer and rotary forming mechanism, and activate the submerged arc welding machine and the automatic tracking equipment for the forming gap. Based on the current linear velocity (V) of the steel pipe, determine the physical installation distance (L) between the 3D laser profile sensor and the submerged arc welding torch. To meet the requirements of forward-looking control, the distance L must satisfy the formula... ,in The total delay compensation time set for the system (in this embodiment) (Approximately 2 seconds).

[0051] Step S2: Front-end visual acquisition and feature recognition A 3D laser contour sensor, mounted in front of the welding torch and equipped with a dustproof and spatterproof protective cover, projects a laser contour line onto the surface of the rotating steel fluid conveyor product in real time and acquires 3D depth images of the forming gap area. The acquired image data is transmitted in real time at high speed to a GPU industrial control computer via a gigabit network cable.

[0052] Step S3: Algorithm Solution and Time Consumption After receiving the image, the GPU-based industrial control computer sequentially invokes its built-in deep learning model and machine vision algorithm. First, deep learning quickly extracts the actual Region of Interest (ROI) of the forming seam from complex backgrounds (such as flux and reflections). Then, the vision algorithm geometrically fits the 3D contour of this region, calculating the precise 3D spatial coordinates of the centerline of the forming seam at the current cross-section, and determining the lateral and height deviation values. This data acquisition and algorithm processing process takes approximately 1 second.

[0053] Step S4: Dead-time filtering and signal transmission The industrial control computer sends the calculated deviation data to the PLC control system. The PLC executes the anti-jitter dead-zone filtering strategy in real time: if the received absolute value of the deviation is <0.2mm, it is determined to be normal mechanical vibration of the tube body, and the PLC does not issue an action command; if the absolute value of the deviation is ≥0.2mm, the PLC converts the deviation value into a drive pulse or analog signal and sends it to the stepper motor driver of the electric slide.

[0054] Step S5: Delay Compensation and Precise Correction After receiving the motion command, the cross-shaped electric slide table activates the two-dimensional motion module to move the submerged arc welding torch towards the target position. The response and movement process of this mechanical actuator takes approximately 1 second.

[0055] Thus, from the moment the sensor acquires the image (step S2) to the completion of the slide movement (step S5), the system takes approximately 2 seconds in total. Due to the pre-matching of the physical spacing, the previously detected deviation in the forming gap is now directly below the submerged arc welding torch as the raw material moves forward. The welding torch precisely lands on the center line of the forming gap, completing a "proactive" delay compensation tracking action. Steps S2 to S5 are executed at high frequency during the welding process, achieving fully automated continuous dynamic tracking.

[0056] The specific experimental procedure is as follows: (I) Hardware System Setup 1. Vision Unit: A 3D line laser contour sensor with 4096 dots is selected and installed 15cm directly above the molding gap to ensure that the field of view covers the full width of the molding gap (typical value 50mm). 2. Data processing unit: The industrial control computer is equipped with a GPU to support CUDA-accelerated deep learning inference.

[0057] (II) Algorithm Training and Debugging 1. Dataset Construction: Collect over 3000 images of formed seams containing different steel strips and different shearing methods, label instance masks using the LabelMe tool, and divide the dataset into training, testing, and validation sets in a 7:2:1 ratio; 2. Model Optimization: Using mixed precision training technology, the model was fine-tuned based on the YOLO-V8 pre-trained model. After 300 epochs, the mAP@0.5 on the test set reached 99.0%. (III) Industrial Site Verification Testing was conducted in a production scenario for steel fluid transport products in a casting industry: 1. Test conditions: 10mm thick steel plate, forward speed of steel fluid conveying product is 1.7m / min, ambient temperature is 20±5℃, and dust concentration is 30mg / m³; 2. Performance Indicators: Recognition rate: 100% (tested on 191 steel fluid conveying products, each 17m long). Recognition efficiency: 1 second / time, continuous operation for 24 hours without failure; Furthermore, steel fluid conveying products made from different types of steel can be directly used after molding; Furthermore, for different bevel types and pipe diameters, quick adaptation can be achieved through the following methods: 1. Model transfer learning: Based on the existing YOLO-V8 model, add 100-200 new target molding gap category labeled data, and train for 50 epochs to achieve industrial-grade accuracy; 2. If the pipe diameter is greater than 12mm, fine-tune the number of image stitching rows and the maximum over-limit threshold to accommodate the forming gaps of large-diameter steel fluid conveying products.

[0058] This invention proposes a "look-ahead control" strategy based on fixed delay compensation. By installing a 3D laser sensor in the front and utilizing the total system time—combining the algorithm computation time (approximately 1 second) and the mechanical slide execution time (approximately 1 second)—the advancing speed of the steel pipe and the physical distance are precisely matched. This achieves a look-ahead closed loop in both space and time, ensuring that the forming gap is precisely below the welding torch at the instant the correction action is completed. This completely eliminates the hysteresis error of traditional tracking systems and ensures extremely high alignment accuracy.

[0059] This invention, through the design of a front protective cover and lens, perfectly overcomes the contamination of optical sensors by a large amount of flux falling and splashing during welding. At the same time, the introduction of a 0.2mm dead zone filtering strategy at the PLC main control terminal effectively eliminates the false offset caused by steel body vibration, avoids high-frequency ineffective oscillation of the slide table, extends the service life of the mechanical actuator, and ensures the long-term stable operation of the system under harsh working conditions.

[0060] This invention introduces a visual algorithm + deep learning model mode to improve the accuracy of forming gap recognition. After trial operation and testing, there was no deviation phenomenon. Moreover, instead of taking the latest data as the offset, it uses a sliding window to take all forming gap position data from the last movement to the latest time, calculates the offset row by row, and takes the average of multiple groups as the final offset value to send to the PLC. This greatly avoids the situation where the offset calculation is incorrect due to the loss or error of a certain group of data.

[0061] Example 2 This embodiment provides a gap connection forming tracking system for steel fluid conveying products based on 3D point cloud and deep learning, as follows: The first module is configured to acquire 3D depth images of the forming gap area on the surface of the steel fluid conveying product in real time; The second module is configured to quickly extract real molding gaps in complex backgrounds through deep learning and output a high-precision mask image. The third module is configured to acquire effective 3D molding gap surface features based on a mask image, which are used as the ROI region; perform geometric fitting on the three-dimensional contour of the ROI region using a visual algorithm to calculate the true three-dimensional spatial center coordinates of the current molding gap section; and calculate the deviation value by subtracting the true three-dimensional spatial center coordinates from the reference center coordinates. The fourth module is configured to determine normal mechanical vibration of the pipe body if the absolute value of the deviation is less than the set value; and to perform compensation and correction if the absolute value of the deviation is greater than or equal to the set value.

[0062] As a further technical solution, the first module is a 3D laser contour sensor with a dustproof and splashproof protective cover set in front of the welding torch. It projects laser contour lines onto the surface of the rotating steel fluid conveying product in real time and collects 3D depth images of the forming gap area.

[0063] Example 3 Embodiment 3 of the present invention provides a computer-readable storage medium having a computer program stored thereon for calculating... When the machine program is executed, it implements the gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning as described in any embodiment of the present invention.

[0064] Example 4 Embodiment 4 of the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer executes the gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning as described in any embodiment of the present invention.

[0065] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0066] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0067] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0068] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drive, portable hard drive, read-only memory (ROM). Various media that can store program code, such as only memory, random access memory (RAM), magnetic disks or optical disks.

[0069] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0070] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for tracking the gap connection forming of steel fluid transport products based on 3D point cloud and deep learning, characterized in that, Includes the following steps: Step S1: Real-time acquisition of 3D depth images of the forming gap area on the surface of the steel fluid conveying product; Step S2 uses deep learning to quickly extract real molding gaps from complex backgrounds and outputs a high-precision mask image. Step S3: Based on the mask image, obtain effective 3D molding gap surface features as ROI regions; use a visual algorithm to geometrically fit the three-dimensional contour of the ROI region to calculate the true three-dimensional spatial center coordinates of the current molding gap cross section; subtract the true three-dimensional spatial center coordinates from the reference center coordinates to obtain the deviation value. If the absolute value of the deviation is less than the set value in step S4, it is determined to be normal mechanical vibration of the pipe body; if the absolute value of the deviation is greater than or equal to the set value, compensation and correction are performed.

2. The gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning as described in claim 1, characterized in that, In step S1, the acquired images undergo multi-frame accumulation and stitching, as well as motion coordinate compensation. The process is as follows: 1.1 Multi-frame cumulative stitching The XYZ three-channel point cloud and grayscale image acquired in each frame are vertically stitched along the row direction; the total number of rows after stitching does not exceed the preset row number threshold. If the threshold is exceeded, the oldest row is discarded, only the newest row is kept, and the row difference between the old and new data is output for subsequent algorithms to correct the timestamp. 1.2 Motion Coordinate Compensation Based on the difference between the current point's row index r in the cumulative image and the middle row, multiplied by a fixed step size parameter, the Y coordinate value of the point is recalculated; the Y coordinate of the middle row is set to zero, and the coordinate increases / decreases linearly when it is far from the middle row, thereby eliminating the influence of raw material movement on the Y-direction coordinate and achieving accurate alignment between point cloud data and physical space.

3. The gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning as described in claim 1, characterized in that, The specific steps of S2 are as follows: Using CSPDarknet as the backbone network, richer gradient flow information is obtained by introducing the C2f module; and a bidirectional fusion structure of feature pyramid and path aggregation network is adopted in the neck, which greatly enhances the feature extraction and multi-scale perception capabilities of the slender, irregular small target of the forming gap on the surface of steel fluid conveying products. The model's prediction head adopts a decoupled structure, processing classification, bounding box regression, and mask segmentation tasks independently. Its total loss function combines classification loss, bounding box loss, and mask loss. In response to the unique morphology of the external welds on steel fluid conveying products, we abandoned the generalized annotation of the entire drawing and only performed high-precision polygonal local annotation on the effective area of ​​the actual formed gap on the surface of the steel fluid conveying products, deliberately excluding flux accumulation areas and simple spatter points as background. An improved YOLOv8 target detection model was trained based on a labeled molding gap dataset to obtain a molding gap recognition mask.

4. The gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning as described in claim 1, characterized in that, Step 3 includes the following steps: 3.1 Perform channel separation on the image in step S1 to extract independent spatial coordinate image matrices in the three dimensions of X, Y, and Z; Using the molding gap recognition mask as a mask, threshold segmentation is performed on the Z-axis image to remove invalid background scatter points that exceed the reasonable height range, and obtain effective 3D molding gap surface features as ROI regions. The extracted ROI regions are bounded by the minimum bounding rectangle and then a rectangle closing operation is performed to eliminate local contour breaks caused by flux obstruction or light interference, and obtain continuous and complete molding gap regions. 3.2 Identify genuine and counterfeit molding gaps and eliminate interference in continuous and complete molding gap areas: 3.3 Precise calculation using a bottom-up sliding window.

5. The gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning as described in claim 4, characterized in that, The specific details of 3.2 are as follows: To address the issue of indentations caused by the manufacturing process of steel fluid conveying products being misidentified as forming gaps, the algorithm employs a row traversal strategy. Line-by-line scanning and connected component analysis: Within the minimum bounding rectangle of the forming gap region, generate rectangular analysis bars of preset height line by line, find their intersection with the closed operation region of the forming gap, and separate the connected components of each line; Expansion and intersection with effective data: Perform rectangular expansion on each connected component, then find the intersection with the effective height region, retaining only the portion of Z value within the effective range; Gray-scale feature extraction: For the preserved regions, extract their mean gray-scale value on the X-coordinate image to obtain the mean X-coordinate value of the connected regions in that row; Deviation comparison and indentation removal: Calculate the absolute difference between the mean and the center of the image captured by the camera; if the absolute difference is greater than the first preset threshold, the area is determined to be an indentation and discarded. Otherwise, retain them and filter out the connected regions with the smallest difference; Real Region Merging: Perform a union operation on all remaining connected regions to form the final real-formed gap object.

6. The gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning as described in claim 4, characterized in that, The specific details of 3.3 are as follows: Extract the effective analysis area at the bottom of the molding gap: Define the boundaries of the analysis region; Sliding window parameter settings allow the window to slide upwards layer by layer from the bottom boundary; For each window, a rectangular region is generated, and its intersection with the actual formed gap object is calculated. Connected components within the intersection are separated, and the region with the largest area is selected. Calculate the center of the area of ​​the largest region; if the area is smaller than a set number of pixels, skip the current window. The largest region is rectangularly dilated, and then image difference is performed with the original region to obtain the region's outer contour. A rectangular opening operation is performed on the difference result to eliminate isolated noise points, and then its minimum bounding rectangle is calculated to obtain the left and right boundary ranges. The opening operation result is separated into connected components, and the left and right regions are filtered based on column coordinates. The left and right regions are then unioned and intersected with the effective height region to ensure data validity. The area centers of the left and right regions are extracted; if either area is smaller than a first preset area threshold, it is skipped; otherwise, the grayscale mean of the left and right regions on the X-coordinate image is extracted to obtain the mean X-coordinate of the left and right boundaries. The arithmetic mean of the left and right means is taken as the instantaneous center coordinate of the window. Store the instantaneous center coordinates calculated from all windows into a tuple. If the number of valid windows is greater than 1, calculate the arithmetic mean of all instantaneous centers and use it as the true three-dimensional spatial center coordinates of the current forming gap section.

7. A gap connection forming tracking system for steel fluid conveying products based on 3D point cloud and deep learning, characterized in that, as follows: The first module is configured to acquire 3D depth images of the forming gap area on the surface of the steel fluid conveying product in real time; The second module is configured to quickly extract real molding gaps in complex backgrounds through deep learning and output a high-precision mask image. The third module is configured to acquire effective 3D molding gap surface features based on a mask image, which are used as the ROI region; perform geometric fitting on the three-dimensional contour of the ROI region using a visual algorithm to calculate the true three-dimensional spatial center coordinates of the current molding gap section; and calculate the deviation value by subtracting the true three-dimensional spatial center coordinates from the reference center coordinates. The fourth module is configured to determine normal mechanical vibration of the pipe body if the absolute value of the deviation is less than the set value; and to perform compensation and correction if the absolute value of the deviation is greater than or equal to the set value.

8. The gap connection forming tracking system for steel fluid transport products based on 3D point cloud and deep learning as described in claim 1, characterized in that, The first module is a 3D laser contour sensor with a dustproof and spatterproof protective cover, positioned in front of the welding torch. It projects a laser contour line onto the surface of the rotating steel fluid conveyor product in real time and acquires 3D depth images of the forming gap area. The physical installation distance L between the 3D laser contour sensor and the submerged arc welding torch is... ,in The total delay compensation time set for the system.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning as described in any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, wherein when the program instructions are executed by a computer, the computer performs the gap connection forming tracking method for steel fluid transport products based on 3D point cloud and deep learning as described in any one of claims 1-7.

Images