Automatic cruise and weld seam recognition tracking collection inner welding machine, control method and equipment

Abstract

The application discloses an internal welding machine, a control method and equipment for automatic cruising and weld seam identification and tracking collection, and the internal welding machine comprises a vehicle body, a taper head, a posture adjusting mechanism, a walking mechanism, a laser radar, a holder with an imaging device, an angle sensor and a controller. The controller comprises: a pipeline center point extraction module, which iteratively solves the center point set based on the geometric constraint that the pipeline wall normal intersects at the pipeline center; a cruising speed planning module, which constructs a B-spline curve according to the center point set to calculate the pipeline curvature and plan the speed; a multi-modal adaptive speed regulation module, which performs gating fusion on point cloud, image and angle data, and outputs environment confidence and safe passing confidence to generate the expected speed; a weld seam identification and tracking collection module, which identifies the weld seam through a visual segmentation network and controls the imaging device to track and collect the weld seam, and after identification, controls the internal welding machine to stop at an effective imaging position for image collection. The application realizes the full automation of the autonomous cruising, weld seam searching and image collection of the internal welding machine in the pipeline.

Description

Technical Field

[0001] This invention relates to the field of intelligent pipeline welding, and in particular to an internal welding machine, control method, and equipment for automatic cruise and weld seam identification, tracking, and acquisition. Background Technology

[0002] Currently, the quality assessment of weld formation after welding in oil and gas pipelines ranging from 508mm to 1422mm mainly relies on non-destructive testing methods such as AUT (ultrasonic testing) and DR (radiological testing). AUT involves scanning the weld seam against the outside of the pipe, requiring manual setup of the acquisition equipment. DR uses a crawler inside the pipe carrying a radiation source, with the operator placing magnetic / radioactive positioning sources on the outside of the pipe for each weld seam to locate the crawler. Neither of these methods involves self-guided weld seam search within the main pipe or visual framing of the optimal stopping point (such as vision-based weld seam location and image acquisition), thus requiring reliance on optical weld inspection.

[0003] Optical weld inspection primarily covers the geometric and surface quality indicators of welds, such as undercut, weld beads, surface porosity, arc craters, lap joints, weld width, and surface spatter. These are aspects that AUT and DR equipment cannot accurately measure. Currently, weld seam image acquisition equipment equipped with cameras mainly relies on manual remote control for seam location. On one hand, operators depend on image acquisition and real-time image signal transmission equipment to manually stop the camera based on the view inside the pipe. This is easily affected by factors such as lighting inside the pipe, the field of view, and signal delay, making it easy to miss the optimal visual framing position. On the other hand, semi-automatic pipe circumferential weld image acquisition equipment mainly relies on ultrasonic ranging sensors to determine the stop position based on weld height. The pan-tilt camera rotates 360 degrees according to a fixed program to acquire weld images. However, during rotation, it cannot detect whether the weld has deviated from the image acquisition field of view, easily leading to lost weld images. Furthermore, the stopping position determined by the ranging sensor is easily interfered with by internal pipe threads, debris, etc.

[0004] In summary, the assessment of weld surface quality in confined spaces within pipes urgently requires an unmanned, highly intelligent sensing system capable of autonomously navigating within the pipe to locate welds, automatically adjusting its posture to adapt to the pipe's curvature, and consistently aligning with the weld during image acquisition. However, existing technologies suffer from the following technical shortcomings: 1. Existing point cloud processing methods for pipe inner walls rely on the symmetrical distribution of point clouds to calculate the geometric center. When a single frame of point cloud can only collect part of the pipe wall due to the limitation of the field of view, the fitted axis deviates significantly from the actual pipe centerline and cannot accurately guide the welding machine through the bend section. 2. The cruise speed planning of the internal welding machine relies on manually setting a fixed threshold, which cannot be adaptively and smoothly adjusted according to multiple factors such as pipe curvature, slope, and machine posture, and there is a risk of the cone head scraping the pipe wall and the equipment tipping over. 3. The single-frame semantic segmentation model is used for weld seam recognition inside the pipe. Due to the darkness and light inside the pipe and walking vibration, it is prone to frame skipping and weld seam mask flickering. Traditional video segmentation schemes (such as 3D CNN or optical flow method) have too high computational complexity and are difficult to implement real-time inference on edge computing devices. Summary of the Invention

[0005] The purpose of this invention is to overcome the technical problems existing in the prior art and to provide an internal welding machine, control method and equipment for automatic navigation and weld seam identification and tracking. It adds optical imaging and digital image processing technology to the existing multi-degree-of-freedom internal welding robot to form a control system that relies on the internal welding machine's flexible pipe bending and passage ability and visual weld seam positioning.

[0006] The objective of this invention is achieved through the following technical solution: A first aspect of the present invention provides an internal welding machine for automatic navigation and weld seam identification and tracking, comprising: Internal welding of the locomotive body; The cone-shaped head is installed on the inner welding machine body; An attitude adjustment mechanism, mounted on the inner welding machine body, is used to adjust the position and posture of the cone head relative to the center line of the pipe. A traveling mechanism, mounted on the inner welding machine body, is used to drive the inner welding machine body to move along the pipeline axis inside the pipeline; A lidar, installed at the front end of the cone, is used to scan the inner wall of the pipe to obtain three-dimensional point cloud data of the inner wall of the pipe; A gimbal is mounted on the front end of the cone head, and an imaging device is provided on it; the imaging device is used to acquire images inside the tube, and the gimbal is used to adjust the deflection of the imaging device. An angle sensor, installed on the inner welding machine body, is used to collect the roll angle and pitch angle of the inner welding machine body; The controller is communicatively connected to the lidar, gimbal, angle sensor, attitude adjustment mechanism, and walking mechanism, respectively. The controller includes: The pipeline center point extraction module is configured to iteratively solve for the pipeline center point set based on the three-dimensional point cloud data acquired by the lidar and the geometric constraint that the pipe wall normal intersects at the pipeline center. The internal welding machine cruise speed planning module is configured to construct a B-spline curve based on the set of pipe center points to calculate the pipe curvature and output the pipe pose information; control the cone to remain at the center of the pipe based on the pipe pose information, and plan the internal welding machine cruise speed based on the changing trend of the pipe curvature. The multimodal adaptive speed control module is configured to perform multimodal gating fusion on the three-dimensional point cloud data, the image inside the pipe, and the data collected by the angle sensor, and output the confidence level of the environment inside the pipe and the confidence level of safe passage; and generate the expected cruising speed of the inner welding machine based on the pipe curvature, the confidence level of the environment inside the pipe, and the confidence level of safe passage, so as to control the inner welding machine to travel along the pipe. The weld seam recognition and tracking acquisition module is configured to identify weld seams through a visual segmentation network during the movement of the inner welding machine along the pipeline, and control the imaging device to track and acquire weld seams according to their positions. When a weld seam is identified, the inner welding machine is controlled to decelerate and stop at a preset effective imaging position. The pan-tilt unit controls the imaging device to perform 360-degree circumferential weld seam image acquisition, and the weld seam is always kept in the center of the image during the 360-degree circumferential weld seam image acquisition process.

[0007] Preferably, the pipeline center point extraction module iteratively solves for the pipeline center point set using the following formula: in, In any iteration step Below, a single center point From current coordinates The new coordinates to be transferred to in the next iteration step. Let represent the optimization objective function for the radial centering attraction and the constraint of the pipe wall normal. Represents the local spatial orientation anisotropy factor. Indicates the center point of other pipelines The sum of the vectors of the repulsive forces generated.

[0008] Preferably, the internal welding machine cruise speed planning module includes: The sequence sorting unit is configured to arrange the set of pipeline center points into an ordered sequence from near to far along the pipeline extension direction using a nearest neighbor algorithm; B-spline curve construction unit, configured to construct B-spline curve parametric equations based on the ordered sequence; The pipe curvature calculation unit is configured to calculate the curvature of each point on the axis of the pipe ahead based on the B-spline curve, and output the pipe curvature change trend from near to far. The cone head posture control unit is configured to control the posture adjustment mechanism to keep the cone head aligned with the center line of the pipe according to the pipe posture information; The internal welding machine cruise speed calculation unit is configured to plan the internal welding machine cruise speed according to the trend of the pipe curvature change from near to far, and the internal welding machine cruise speed is proportional to the magnitude of the pipe curvature.

[0009] Preferably, the multimodal adaptive speed control module is configured with a multimodal adaptive speed control model, the multimodal adaptive speed control model comprising: The parallel feature extraction layer is configured to contain three parallel feature processing channels, which respectively convert the 3D point cloud data, the in-tube image and the data collected by the angle sensor into feature vectors with consistent feature dimensions. The gated fusion layer is configured to dynamically generate attention weights for each feature processing channel through a fully connected layer, and to weight and synthesize the three feature vectors output by the parallel feature extraction layer according to the attention weights, so as to output a unified latent feature representation. A multi-task regression layer is configured to map the unified latent feature representation to in-pipe environment confidence and safe passage confidence.

[0010] Preferably, the desired cruising speed of the internal welding machine is calculated using the following formula: in, Indicates the confidence level of the environment within the pipe. Indicates the confidence level of safe passage. Indicates the maximum permissible driving speed. Indicates the maximum centripetal acceleration. Indicates the curvature of the pipe.

[0011] Preferably, the visual segmentation network includes: The multi-frame feature extraction layer is configured to use a multi-scale hierarchical feature extraction network as the backbone network to extract multi-scale features of the current image frame and the cached historical image frames in the pipeline, respectively. The spatiotemporal difference gated fusion layer is configured to calculate the absolute difference between the features of the current image frame in the pipeline and the features of the historical image frames in the pipeline in the highest layer feature map of the backbone network, generate spatial gate weights based on the absolute difference, and fuse the features of the current image frame in the pipeline and the features of the historical image frames in the pipeline based on the spatial gate weights to obtain fused features. The hybrid feature decoding layer is configured to feed the fused features and the low-level spatial features of the current in-pipe image frame into the multilayer perceptron decoder for fusion, output the weld seam segmentation result of the current in-pipe image frame, output the weld seam semantic probability map through category projection and resolution interpolation, and convert the weld seam semantic probability map into a weld seam mask matrix.

[0012] Preferably, controlling the imaging device to track the weld seam based on the weld seam position specifically includes: The pixel coordinates that are true in the weld mask matrix are fitted with a univariate linear equation using least squares: in, The angle of inclination of the weld centerline in the image coordinate system. The intercept; Based on the tilt angle and intercept, calculate the control values ​​of each joint of the gimbal, and control the gimbal to keep the weld seam in the center of the image. When the tilt angle of each joint of the gimbal reaches the preset optimal imaging point, the walking mechanism is controlled to stop walking; and the gimbal is controlled to perform 360-degree rotation imaging starting from the position where the weld seam image is captured.

[0013] Preferably, the loss function used by the visual segmentation network is as follows: in, It is a combination loss function of traditional binary crossover loss and dice loss. It is the current frame image with the actual weld mask. It is the semantic probability map of the weld seam in the current image frame within the pipe. It is the semantic probability map of the weld seam in the previous image frame within the pipe. It is a time-series balanced weight. It is the time-series smoothing loss function.

[0014] A second aspect of the present invention provides a control method for the internal welding machine described in the first aspect, comprising the following steps: S1. The laser radar scans the inner wall of the pipe to obtain three-dimensional point cloud data of the inner wall of the pipe, the imaging device collects images inside the pipe, and the angle sensor collects the roll angle and pitch angle of the inner welding machine body. S2. Based on the three-dimensional point cloud data, the set of pipe center points is iteratively solved by the geometric constraint that the pipe wall normals intersect at the pipe center; S3. Construct a B-spline curve based on the set of pipe center points to calculate the pipe curvature and output the pipe pose information; control the cone to remain at the center of the pipe based on the pipe pose information, and plan the inner welding machine's cruising speed based on the changing trend of the pipe curvature; S4. Perform multimodal gating fusion on the three-dimensional point cloud data, the image inside the pipe, and the data collected by the angle sensor to output the confidence level of the environment inside the pipe and the confidence level of safe passage; and generate the expected speed of the inner welding machine's cruise based on the pipe curvature, the confidence level of the environment inside the pipe, and the confidence level of safe passage to control the inner welding machine to travel along the pipe. S5. During the process of the inner welding machine traveling along the pipeline, the weld seam is identified by the visual segmentation network, and the imaging device is controlled to track and collect the weld seam according to the position of the weld seam; wherein, when the weld seam is identified, the inner welding machine is controlled to decelerate and stop at the preset effective imaging position, and the pan-tilt unit controls the imaging device to perform 360-degree circumferential weld seam image acquisition, and the weld seam is always kept in the center of the image during the 360-degree circumferential weld seam image acquisition process.

[0015] A third aspect of the present invention provides an electronic device including a memory and a processor, wherein the memory stores computer instructions executable on the processor, and the processor executes relevant steps in the control method described in the second aspect when executing the computer instructions.

[0016] It should be further noted that the technical features corresponding to the above options can be combined or substituted to form new technical solutions if there is no conflict.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. The accuracy of extracting the center point of the bend in the pipe is significantly improved, ensuring safe passage. The pipe center point extraction module introduces a pipe wall normal constraint term as a geometric prior into the iterative update formula. When pipe curvature causes only a portion of the pipe wall to be scanned in a single frame point cloud, the traditional local gravity center method would offset the fitted axis towards the visible side of the pipe wall. However, the normal constraint term in this invention utilizes the geometric law that the normal to the cylindrical pipe wall must pass through the center of the circle, forcing the center point to revert to the true axis determined by the intersection of the normals. This mechanism significantly improves the accuracy of axis extraction for curved pipe sections, providing a reliable basis for cone head pose control and effectively avoiding the risk of the cone head scraping the pipe wall.

[0018] 2. The internal welding machine's cruise speed is planned in advance, balancing efficiency and safety. The cruise speed planning module arranges the center point set into an ordered sequence and constructs a B-spline curve. The curvature calculation unit outputs the curvature change trend of the pipe ahead from near to far. The internal welding machine's cruise speed calculation unit plans the speed accordingly, and the speed is proportional to the pipe curvature. The existing solution cannot make predictive motion planning because it cannot perceive the overall situation of the pipe ahead. If the speed is too high when the straight pipe enters the large curvature bend, the attitude adjustment mechanism will not be able to respond in time, resulting in cone head scraping.

[0019] 3. Multimodal adaptive speed regulation enables smooth end-to-end speed planning, enhancing environmental adaptability. The multimodal adaptive speed control model plans the automatic cruise speed of the inner welding machine. Compared with the rigid constraints of manual configuration with complex coupling, this invention achieves smooth speed planning of the inner welding machine from inside to outside the pipe through mathematical continuous mapping. Multimodal gating fusion also overcomes the performance degradation problem of single sensors under specific working conditions (such as camera overexposure after exiting the pipe, excessive noise from laser multipath effect). This allows the invention to decelerate in advance before entering a curve and automatically accelerate after exiting the curve; it can also determine the automatic cruise speed based on factors such as being inside the pipe, near the pipe opening, exiting the pipe opening, going up or down slopes, and the horizontal attitude of the machine body, which not only improves the walking efficiency but also ensures the safety of construction personnel and equipment.

[0020] 4. Strong spatiotemporal continuity in weld seam identification, effectively suppressing mask flicker. The visual segmentation network employs a multi-frame feature extraction layer to extract multi-scale features from the current frame and historical frames respectively. A spatiotemporal difference-gated fusion layer calculates the absolute difference between the features of the current frame and historical frames in the highest-level feature map of the backbone network to quantify local image mutations (lighting changes, weld reflections, etc.), thereby generating spatial gating weights and fusing temporal features. Traditional single-frame segmentation networks lose weld recognition information in dark environments with changing light or when weld reflections occur at specific angles, leading to severe flickering or incomplete shapes in the predicted mask. This invention, through spatiotemporal difference-gated fusion, can perceive and identify regions affected by instantaneous interference, dynamically reducing the weight of the current frame and actively retrieving features from undisturbed historical frames to supplement it, thus maintaining the integrity and consistency of the weld mask temporally. Compared to 3D-CNN or cross-frame global attention schemes, the computational complexity of this invention is linearly related to image resolution, resulting in low computational overhead and suitability for real-time deployment on edge computing devices.

[0021] 5. Automatic weld seam tracking and complete circumferential weld seam acquisition enable full-process automation. The weld position calculation unit uses least-squares fitting to derive the weld centerline equation from the true pixel coordinates of the weld mask. Based on the tilt angle and intercept, it calculates the gimbal joint control values ​​to keep the weld centered in the image. When the tilt angle of each gimbal joint reaches the preset optimal imaging point, it controls the walking mechanism to stop and performs a 360-degree rotation imaging starting from the weld position. The entire process requires no manual intervention, achieving full automation from weld search, positioning, and tracking to complete acquisition of the circumferential weld, overcoming the limitations of existing technologies that rely on manual remote control or ranging sensors for stopping.

[0022] 6. Temporal smoothing constraint training improves the temporal consistency of segmentation. The visual segmentation network loss function adds temporal smoothing loss to the traditional segmentation loss, forcing the network to learn temporally consistent mask outputs. This makes the tilt angle and intercept of the weld centerline parameters change smoothly in the video stream instead of high-frequency jitter, providing a high-quality data foundation for PTZ tracking and further improving the stability and integrity of circumferential weld image acquisition. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the internal welding machine structure shown in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the working principle of the internal welding machine according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the control method according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the laser radar extraction of the center point of the pipeline in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating weld seam identification and tracking according to an embodiment of the present invention.

[0024] In the diagram: 1-Imaging device; 2-Gimbal; 3-LiDAR; 4-Cone; 5-Attitude adjustment mechanism; 6-Walking mechanism; 7-Inner welding machine body. Detailed Implementation

[0025] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It should be noted that the defects in the solutions in the prior art are all the results of the inventors' practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the embodiments of this application in the following text should be the inventors' contributions to this application in the process of invention and creation, and should not be understood as technical content known to those skilled in the art.

[0027] In view of the technical problems pointed out in the background art, the present invention provides the following embodiments: In one exemplary embodiment, an internal welding machine is provided that enables automatic navigation and weld seam identification and tracking data acquisition, such as... Figure 1 As shown, it includes: 7. Internal welding machine body; Cone 4 is mounted on the inner welding machine body 7; The attitude adjustment mechanism 5 is installed on the inner welding machine body 7 and is used to adjust the position of the cone 4 relative to the center line of the pipe. The traveling mechanism 6 is mounted on the inner welding machine carriage 7 and is used to drive the inner welding machine carriage 7 to move along the pipeline axis inside the pipeline. The lidar 3 is installed at the front end of the cone 4 and is used to scan the inner wall of the pipe to obtain three-dimensional point cloud data of the inner wall of the pipe. A gimbal 2 is mounted on the front end of the cone head 4, and an imaging device 1 is mounted on it; the imaging device 1 is used to acquire images inside the tube, and the gimbal 2 is used to adjust the deflection of the imaging device 1. An angle sensor (not shown in the figure) is installed on the inner welding machine body 7 and is used to collect the roll angle and pitch angle of the inner welding machine body 7. The controller (not shown in the figure) is communicatively connected to the lidar 3, gimbal 2, angle sensor, attitude adjustment mechanism 5, and walking mechanism 6, respectively. The controller includes: The pipeline center point extraction module is configured to iteratively solve the pipeline center point set based on the three-dimensional point cloud data acquired by the lidar 3 and the geometric constraint that the pipe wall normal intersects at the pipeline center. The internal welding machine cruise speed planning module is configured to construct a B-spline curve based on the set of pipe center points to calculate the pipe curvature and output the pipe pose information; control the cone to remain at the center of the pipe based on the pipe pose information, and plan the internal welding machine cruise speed based on the changing trend of the pipe curvature. The multimodal adaptive speed control module is configured to perform multimodal gating fusion on the three-dimensional point cloud data, the image inside the pipe, and the data collected by the angle sensor, and output the confidence level of the environment inside the pipe and the confidence level of safe passage; and generate the expected cruising speed of the inner welding machine based on the pipe curvature, the confidence level of the environment inside the pipe, and the confidence level of safe passage, so as to control the inner welding machine to travel along the pipe. The weld seam recognition and tracking acquisition module is configured to identify weld seams through a visual segmentation network during the process of the inner welding machine traveling along the pipeline, and control the imaging device to track and acquire weld seams according to the weld seam position; wherein, when a weld seam is identified, the inner welding machine is controlled to decelerate and stop at a preset effective imaging position, and the pan-tilt unit 2 controls the imaging device to perform weld seam image acquisition.

[0028] In this embodiment, the imaging device 1 employs a gimbal camera and may be equipped with an auxiliary illumination source to adapt to the dark environment inside the tube. The gimbal 2 has at least two degrees of freedom, enabling pitch and yaw movements. An angle sensor is mounted on the bottom of the inner welding machine body 7 and may be a MEMS inertial measurement unit (IMU).

[0029] Specifically, such as Figure 2As shown, the internal welding machine acquires high-density 3D point cloud data by scanning the inner wall of the curved pipe using a lidar 3 mounted in front of the cone head 4. The pipe axis is extracted from the 3D point cloud data to control the welding machine's attitude as it changes with the pipe's curvature. Multimodal gating fusion is performed using the 3D point cloud data acquired by the lidar, the pipe image data acquired by the gimbal camera, and the slope and fuselage attitude angle data acquired by the angle sensor. This results in the output of pipe environment confidence and safe passage confidence indices, which are used to generate the internal welding machine's desired cruise speed. This desired cruise speed is applicable to the following scenarios: horizontal and vertical slopes, entering and exiting curved pipes, exiting pipes, and when the fuselage is tilted. The walking mechanism 6 controls the internal welding machine's movement based on this desired cruise speed.

[0030] During the process of controlling the internal welding machine to move through, the gimbal camera collects images inside the pipe in real time. The gimbal camera identifies whether there is a weld seam through a visual segmentation network. When no weld seam is found, the gimbal 2 controls the angle of the gimbal camera to always face forward, waiting for the weld seam to appear.

[0031] As the internal welding machine moves forward, weld seams appear in the field of view. The gimbal camera begins tracking the weld seam closest to the machine. The internal welding machine continues to decelerate, and gimbal 2 constantly adjusts the camera lens angle to keep the weld seam centered in the image. As the internal welding machine approaches the weld seam, the image becomes increasingly clear. When the controller detects that the gimbal camera's deflection angle equals the preset imaging angle, it determines that the internal welding machine has stopped at the preset effective imaging position. It then begins 360-degree tracking of the weld seam and acquiring weld seam images.

[0032] It should be noted that in this embodiment, the weld is a circumferential weld, and a single weld can have multiple preset imaging angles. These preset imaging angles are multiple discrete imaging positions set by the gimbal 2 during its rotation around the pipeline axis to acquire weld images from different circumferential orientations. Simultaneously, during the 360-degree circumferential weld image acquisition process, based on weld image feedback, the gimbal 2 and the inner welding machine are controlled to always keep the weld centered in the image.

[0033] Furthermore, when using forward-looking lidar for single-frame scanning, the field of view is limited by the curvature of the pipe and the obstruction of the front, so only part of the inner wall of the pipe can usually be scanned. General geometric center point extraction algorithms rely heavily on the symmetrical distribution of 3D point cloud data and only calculate local gravitational center points. When half of the point cloud is missing, the gravitational center point will be offset towards the visible side of the pipe wall in the radial direction of the pipe, causing the fitted axis to deviate significantly from the central axis of the real pipe.

[0034] To address this, this embodiment configures a bend center point extraction model in the pipeline center point extraction module, using the geometric constraint that the pipe wall normals intersect at the pipeline center to solve the aforementioned problem of missing point clouds on one side, significantly improving the accuracy of obtaining the pipeline axis in a single frame point cloud. Simultaneously, it avoids the enormous computational burden and engineering implementation difficulties associated with real-time mapping using SLAM solutions. The specific implementation is as follows: 3D point cloud data is represented as input data as follows: , This represents the number of 3D points output by the LiDAR 3 in a single frame. The center point of the pipeline is represented as output data as follows: , This represents the number of pre-extracted pipe center points. The bend center point extraction model is a regularized spatial manifold dynamic contraction system, as shown below: This model describes the process at any iteration step. Below, a single center point From current coordinates Towards the next new coordinate The transfer, of which: The objective function for optimizing radial centering attraction and pipe wall normal constraint; This is a convex quadratic optimization problem constrained by radial centering attraction and the pipe wall normal, requiring the optimal spatial location point. It is necessary to "approach the local data center" "and minimizing the distance to the pipe wall normal" To achieve the optimal balance between them.

[0035] It is the repulsive force between the center points, which prevents the center point set from sticking together through spatial repulsion and through the local spatial anisotropy factor. Control measures are implemented to ensure that the set of center points spreads smoothly only along the axis.

[0036] : a spatial location point To the local data gravitational center Distance error.

[0037] : is the local gravitational center of the point cloud. ,in , .

[0038] : is the neighborhood radius, which is the spherical radius of the interaction between the local gravitational force and the repulsive force at the center point of the point cloud.

[0039] : for the first Each center point For the center of the ball, The set of point clouds within a sphere of radius .

[0040] : Point cloud density weights are used to eliminate point cloud density differences caused by uneven scanning resolution of lidar at different ranges, preventing the center point from being pulled off course by the gravity of high-density point clouds. .

[0041] : This refers to the pipe wall normal constraint term, specifically the spatial location point. Normal to its neighboring point cloud The sum of distance errors is expressed as .in , representing a perpendicular to the normal. The normal plane is such that any spatial vector multiplied by this operator will be projected onto the normal plane. This allows for the calculation of the perpendicular distance from a point to a line. It is a 3x3 identity matrix.

[0042] Regularization parameter, serving as the distribution coefficient between two error terms. This determines the algorithm's tendency to trust "observed data" and "geometric priors." When At that time, completely trust the data. At that time, the ability to correct for unilateral defects is strongest when the normals of the cylinders intersect with a high degree of trust.

[0043] Therefore: With the center of the circle, The center points of other pipes within a sphere of radius [missing information] The sum of the repulsive force vectors generated , , .

[0044] It is a local spatial anisotropy factor, acting as a repulsive force. The dynamic weights control the degree of repulsive force release. First, the three-dimensional weighted covariance matrix of the local neighborhood center points is calculated. ,right Perform eigenvalue decomposition to obtain descending eigenvalues. , Geometric meaning: When the center point exhibits a three-dimensional discrete clustered state, , Approaching 0.33, the repulsion term is suppressed to allow the center point to shrink towards the extreme point. As the center point gradually flattens into a one-dimensional curve, Dominant Approaching 1.0, the exclusion term is almost fully effective to cause the center point to slide uniformly along the pipe axis.

[0045] Furthermore, before the internal welding machine begins its patrol along the pipeline, it must first determine whether the current patrol environment is safe and establish a safe patrol speed. If there are obstructions within the pipeline or the internal welding machine has already exited the pipe opening, the internal welding machine will automatically decelerate and stop, and report the patrol results. If traveling on inclines or declines, or if the horizontal deflection angle of the internal welding machine is too large, the machine's travel speed must be reduced accordingly compared to its travel speed when the machine is level with the horizontal pipe section.

[0046] To address this, this embodiment configures a multimodal adaptive speed control model in the multimodal adaptive speed control module. This model performs multimodal gating fusion of LiDAR data, visual image data, and angle sensor data in the latent feature space. It replaces the manually configured, complex, coupled, and rigid constraints with end-to-end in-pipe environment confidence and safe passage confidence. Through mathematically continuous mapping, it achieves smooth speed planning for the welding machine from inside to outside the pipe. The multimodal adaptive speed control model includes: The parallel feature extraction layer is configured to contain three parallel feature processing channels, which respectively convert the three-dimensional point cloud data (3D), the in-tube image (2D), and the data collected by the angle sensor (1D) into feature vectors with consistent feature dimensions; The gated fusion layer is configured to dynamically generate attention weights for each feature processing channel through a fully connected layer, and to perform weighted synthesis of the three feature vectors output by the parallel feature extraction layer based on the attention weights, thereby outputting a unified latent feature representation of the environment in which the welding machine is located. A multi-task regression layer is configured to map the unified latent feature representation to pipeline environment confidence and safe passage confidence. Specifically, the model is divided into two independent channels at the end, mapping the unified latent feature representation to two control parameters. The safe passage confidence measures the overall pipeline slope, machine horizontal angle, and other risks, serving as a scaling factor for the internal welding machine's travel speed. The pipeline environment confidence measures information about the overall pipeline environment (including whether there are foreign objects ahead, whether the machine is inside the pipeline, whether it is near the pipe opening, and whether it is exiting the pipe opening).

[0047] In the training phase of the multimodal adaptive speed regulation model, in order to ensure the stable output of the gated fusion layer in the network under the condition of single sensor degradation (such as camera tube overexposure, excessive laser multipath noise), noise is added to the original sampled images, point clouds, and angle data during the forward propagation process during training. This includes adding random missing values ​​to the point cloud, adding overexposed bright spots to the image, and adding random noise to the angle data.

[0048] Furthermore, when using a camera to photograph the environment inside the pipe, the limited lighting conditions inside the pipe and the unavoidable vibrations during the movement of the welding machine result in unstable single-frame image features. Traditional single-frame semantic segmentation models are prone to frame skipping and missed detections, as well as flickering of the output weld mask (inconsistent weld positions in time and space). Existing video segmentation schemes mostly employ spatiotemporal 3D convolution (3D CNN) or optical flow methods, which have high computational complexity and are difficult to implement low-latency real-time inference on edge computing devices in the welding machine scenario. Using a global cross-attention mechanism between multiple video frames introduces... The computational complexity ( (This refers to the number of pixels in a single frame image). At the same time, the global attention mechanism can easily introduce noise from irrelevant areas in the weld background, damaging the structure of the weld edge.

[0049] To address this, this embodiment configures a weld seam image recognition model, namely a visual segmentation network, in the weld seam recognition and tracking module. The visual segmentation network includes: The multi-frame feature extraction layer is configured to use a multi-scale hierarchical feature extraction network (such as SegFormer-MiT-B1) as the backbone network to extract multi-scale features of the current in-pipe image frame and the cached historical in-pipe image frames, ensuring that the surface micro-texture and macro-spatial distribution information of the weld can be effectively extracted by the network.

[0050] The spatiotemporal difference-gated fusion layer is configured to introduce a spatiotemporal difference-gated fusion mechanism into the highest-level feature map (containing abstract information) of the backbone network. It calculates the absolute difference between the features of the current image frame in the pipe and the features of the historical image frames in the pipe, and uses this as a measure of local image mutations (illumination changes, weld reflections). It generates spatial gating weights based on the absolute difference, and fuses the features of the current image frame in the pipe and the features of the historical image frames in the pipe based on the spatial gating weights to obtain fused features. The hybrid feature decoding layer is configured to feed the fused features and the low-level spatial features of the current in-pipe image frame into the multilayer perceptron decoder for fusion, output the weld seam segmentation result of the current in-pipe image frame, output the weld seam semantic probability map through category projection and resolution interpolation, and convert the weld seam semantic probability map into a weld seam mask matrix.

[0051] The visual segmentation network employs temporal smoothing constraints during training. During the training phase, the mean squared error based on the prediction probability maps of adjacent frames is added to the loss function to constrain the temporal continuity of the segmentation mask. The total loss function... Defined as: , in, It is a combination loss function of traditional binary crossover loss and dice loss. It is the current frame image with the actual weld mask. It is the predicted probability map output by the model. It is a time-series balancing weight. It is a temporal smoothing loss function, which uses the mean square error between the prediction of the current frame and the prediction of the previous frame. These represent the image height and width resolutions, respectively.

[0052] In another exemplary embodiment, based on the above-described design concept of the internal welding machine, a control method for the internal welding machine is provided, such as... Figure 3 As shown, it includes the following steps: S1. The laser radar scans the inner wall of the pipe to obtain three-dimensional point cloud data of the inner wall of the pipe, the imaging device collects images inside the pipe, and the angle sensor collects the roll angle and pitch angle of the inner welding machine body. S2. Based on the three-dimensional point cloud data, the set of pipe center points is iteratively solved by the geometric constraint that the pipe wall normals intersect at the pipe center; S3. Construct a B-spline curve based on the set of pipe center points to calculate the pipe curvature and output the pipe pose information; control the cone to remain at the center of the pipe based on the pipe pose information, and plan the inner welding machine's cruising speed based on the changing trend of the pipe curvature; S4. Perform multimodal gating fusion on the three-dimensional point cloud data, the image inside the pipe, and the data collected by the angle sensor to output the confidence level of the environment inside the pipe and the confidence level of safe passage; and generate the expected speed of the inner welding machine's cruise based on the pipe curvature, the confidence level of the environment inside the pipe, and the confidence level of safe passage to control the inner welding machine to travel along the pipe. S5. During the process of the inner welding machine traveling along the pipeline, the weld seam is identified by the visual segmentation network, and the imaging device is controlled to track and collect the weld seam according to the position of the weld seam; wherein, when the weld seam is identified, the inner welding machine is controlled to decelerate and stop at the preset effective imaging position, and the pan-tilt unit controls the imaging device to perform 360-degree circumferential weld seam image acquisition, and the weld seam is always kept in the center of the image during the 360-degree circumferential weld seam image acquisition process.

[0053] For example, step S1 specifically includes: S11. Raw Point Cloud Downsampling: To maintain real-time performance in high-density 3D point cloud scanning (e.g., millions of points), a maximum limit is provided for the number of 3D point clouds (e.g., less than 12,000 points). If the number of 3D point clouds is too large, random uniform sampling is used to reduce the data size.

[0054] S12. Calculate the initial neighborhood radius :make It can adaptively adjust according to the size and sparsity of the point cloud, ensuring that there is a stable number of statistical points in the local area. It is the diagonal length of the directed bounding box of the point cloud data.

[0055] S13. Center point The initial values ​​are obtained by using iterative voxel downsampling to avoid clustering in high-density point cloud regions.

[0056] S14. Use KD-tree to build a point cloud index to improve the search speed of local nearest neighbors.

[0057] S15. Calculate the point cloud density weights mentioned above. .

[0058] S16. Calculate the unit normal vector of the point cloud surface. For each A local covariance matrix is ​​constructed within its neighborhood. Through eigenvalue decomposition, the eigenvector corresponding to its smallest eigenvalue is used as the unit normal vector. This yields the normal vector set for all point clouds. .

[0059] For example, step S2, based on the above-mentioned model for extracting the center point of the bend, specifically includes: S21. Outer loop: Preset the number of outer loop iterations (e.g., 5 rounds). Round-robin calculation to update neighborhood radius .

[0060] S22. Inner Loop: Presets the number of inner loops (e.g., 15 rounds), in the... Round iteration, for each center point Perform the following operations: a. Neighborhood search: Using a KD-tree, search for the point set of the surface point cloud within a sphere of radius h, to obtain the above... .

[0061] b. Calculate the gravitational center of the aforementioned local point cloud. and solve .

[0062] c. Calculate the above repulsive force vector .

[0063] d. Calculate the anisotropy factor of the above local structures. .

[0064] e. Update center point position .

[0065] S23. Convergence Determination: Calculate the average single-step displacement of all center points : like Less than the set iteration precision (e.g.) If the iteration converges within the current neighborhood radius, the inner loop is terminated early, and the next outer loop is initiated. Otherwise, let... Continue the inner loop.

[0066] For example, step S3 specifically includes: S31. Sort the above set of pipeline center points. The above pipeline center point outputs a random set of points through iterative loops. This set is then arranged into a sequence of points from nearest to farthest using the Nearest Neighbor algorithm, such as... Figure 4 As shown.

[0067] S32. Constructing Spline Curves Solving the parametric equations of the B-spline curve using the center point set of the pipeline ,in To normalize the pipe arc length, It is the number of control points for the B-spline curve, determined by the desired smoothness of the pipeline curve. These are the control points of the B-spline curve. It is in the control points B-spline basis functions of degree.

[0068] S33. Control the position of the inner welding machine cone 4. Take a point on the B-spline curve near cone 4 as the coordinates of the current pipe position center point (e.g., take...). The coordinates are The direction vector of the current pipe position is ( After determining the location and orientation of the pipe where the inner welding machine is located, the attitude adjustment mechanism 5 on the inner welding machine is controlled to ensure that the cone 4 always coincides with the center line of the pipe section, thus avoiding collision with the pipe wall during the process.

[0069] S34. Planned Welding Machine Cruise Speed The above-mentioned B-spline curve parametric equations When the value is 3, the curve is second-order continuous, and the curvature at each point on the curve can be calculated. The planned speed of the welding machine cruising in the pipeline is proportional to the curvature of the pipeline. The curvature at each point on the pipeline axis is: Based on the curvature of the pipeline ahead, the cruise speed of the welding machine can be planned from near to far. In addition to taking into account the pipeline curvature parameter, the cruise speed also takes into account factors such as whether the equipment is inside the pipeline, approaching the pipeline opening, whether there are obstructions, going up or down slopes, and the tilt of the machine body.

[0070] Existing pipe-walking solutions for internal welding machines lack the ability to perceive the overall condition of the pipe ahead, thus hindering predictive motion planning. When moving from a straight pipe to a large-curvature bend, the attitude adjustment mechanism 5 has limited response speed at high speeds, leading to a risk of scraping the cone head 4 and severely impacting equipment safety. By extracting the axis from the point cloud image of the visible pipe ahead, the machine can determine the pipe's position, orientation, and curvature in areas yet to be reached. This allows for pre-deceleration before entering the bend, autonomous acceleration after exiting the bend, autonomous adjustment of the cone head 4's posture to adapt to changes in pipe curvature, and timely stopping when obstructed by foreign objects inside the pipe. This ensures equipment safety and improves walking efficiency.

[0071] For example, step S4, based on the multi-sensor fusion multimodal adaptive speed control model described above, explains how the inner welding machine determines the target speed and stops safely during automatic cruising inside the pipe, specifically including: Step S41. Multimodal data acquisition and preprocessing The original 3D point cloud data is acquired using the lidar. A 3D region of interest (ROI) bounding box is used to filter out cluttered point clouds at distant points. Then, voxel filtering and farthest-point sampling methods are used to uniformly downsample the point cloud within the bounding box, resulting in a sparse point cloud representation containing a fixed number of N feature points. Where N is the number of feature point clouds, which is 2048, and C is the number of channels, including three-dimensional coordinates and reflection intensity.

[0072] The image is obtained by sampling a 2D RGB tube image using the gimbal camera, scaling it to a fixed resolution using bilinear interpolation, and then normalizing it. The preprocessed image has a width and height of 1000 x ... .

[0073] The roll angle and pitch angle of the current internal welding machine are obtained through the angle sensor and normalized to obtain... ,in For roll angle, For the extreme roll angle threshold, For pitch angle, This is the threshold for the extreme pitch angle.

[0074] Step S42. Parallel feature extraction of three branches The sampled point cloud The input is fed into the point cloud feature extraction branch, which uses PointNet as a 3D geometric encoder. This encoder uses a one-dimensional convolution operator to extract high-dimensional local spatial geometric features point by point, upscaling each point cloud feature to 128 dimensions. To eliminate the interference of the point cloud input order on feature extraction and to aggregate global geometric information, a max pooling layer is used. Transforming a 3D feature matrix into a 1D global geometric feature vector .

[0075] Normalized image The input is fed into the image feature extraction branch, which uses MobileNetV3-Small as the visual encoder. This encoder utilizes multi-scale local convolution operators and attention mechanisms to compress the image resolution to 7x7, but increases the number of pixel features to 576. To eliminate the influence of pixel location and aggregate global contextual semantics, a global average pooling layer is used to transform the 576x7x7 dimensional feature tensor into a one-dimensional global visual feature vector. .

[0076] Since the angle sensor data has a low dimensionality, a multilayer perceptron with two-layer linear transformation, LayerNorm, and GELU activation functions is used to map it into a one-dimensional attitude feature vector. .

[0077] Step S43. Three-modal interactive attention gating fusion The three feature vectors mentioned above are concatenated into a joint feature matrix. The gated weight vectors assigned to the three modes are obtained by passing through a fully connected layer and normalizing the data. The three modal feature vectors are multiplied channel-by-channel by the gate weight vector and then summed to output the deep fusion feature vector. : Step S44. Dual-task return output fuse feature vectors The input is fed into a multilayer perceptron, which can optionally have one hidden layer and one dual-head output layer. A Softmax activation function is appended after the hidden layer, serving as the output header for the in-system environment confidence, outputting a scalar value. .

[0078] A Softmax activation function is appended after the hidden layer, serving as the safe passage confidence output header, and outputting a scalar value. .

[0079] Step S45. Output cruise speed Based on the confidence level of the internal environment output by the dual-task regression layer Confidence level of secure communication Combined with the curvature of the pipe The maximum centripetal acceleration is set. and the maximum permissible driving speed. The overall cruise target speed is obtained. : When the internal welding machine is completely inside the pipe and there are no foreign objects... Approaching a maximum value of 1, the expected cruise speed is determined by the pipe curvature and the confidence level of secure communication. The greater the pipe curvature, the lower the expected speed for automatic cruise.

[0080] When the lidar and gimbal camera collect data near the pipe opening... The value decreases as the distance from the pipe opening decreases, and when the vehicle exits the pipe opening... Approaching 0, the speed of the inner welding machine is reduced to zero and then stopped.

[0081] When the internal welding machine enters the ramps... It will decrease as the slope angle increases. When the slope angle exceeds the above-mentioned limit pitch angle threshold... hour, When the speed approaches zero, the internal welding machine automatically cruises to the desired speed, stops, sends an abnormal alarm, and requires manual intervention.

[0082] When the internal welding machine tilts its horizontal posture due to excessive speed while turning a bend, As the horizontal attitude angle of the machine body (the aforementioned roll angle) increases and decreases, the travel speed of the inner welding machine decreases, which is beneficial for the inner welding machine to return to the horizontal attitude of the machine body.

[0083] For example, step S5, based on the above weld image recognition model, describes a control method for acquiring weld images to track weld positions, specifically including: Step S51. Timing-stable weld identification S511. Historical Frame Feature Aggregation and Spatiotemporal Feature Difference Calculation High-level features output by a multi-scale hierarchical feature extraction network (such as SegFormer-MiT-B1) are used as features for the current frame. The cached historical frame feature list is (Preferably, two consecutive frames can be selected as historical frames). First, the historical features are stitched together along the channel dimension: The aggregated historical reference features are obtained through the temporal aggregation layer. .

[0084] To quantify whether the current frame is affected by abrupt changes in lighting or weld reflections at the current moment, the pixel-level absolute difference between the current features and historical features is calculated to obtain the spatiotemporal difference features: .

[0085] S512. Gating Weight Calculation and Gating Weighted Fusion The current feature is concatenated with spatiotemporal difference features, and a single-channel gating weight is generated through an attention mechanism. The gating weights represent the degree of confidence in the features of the current frame. .

[0086] Using gating weights The fusion ratio between the current frame features and historical frame features is adjusted pixel by pixel in space to obtain the fused features: ,in This indicates element-wise multiplication. It is a matrix of all ones. Finally, a cross-channel convolution is used to output the final features: Traditional single-frame segmentation networks suffer from weld seam recognition information loss in dark environments with varying lighting conditions or at specific weld seam angles. Furthermore, the gimbal camera experiences jitter during tracking based on weld seam centerline parameters, leading to severe flickering or incomplete weld seam shapes in the predicted weld seam mask within the video stream. This step introduces a spatiotemporal difference-gated fusion mechanism to calculate spatiotemporal feature differences (as described above). It can sense and identify these regions affected by momentary interference, and dynamically reduce the weight of the current frame (as mentioned above). It actively retrieves features from undisturbed historical frames to supplement the data, thereby maintaining the integrity and consistency of the weld mask in terms of timing.

[0087] S513. Feature Decoding and Weld Seam Mask Output The output of the spatiotemporal difference gating fusion module The features, along with the remaining features output by the multi-scale hierarchical feature extraction network, are fed into the multilayer perceptron decoder for fusion, outputting the segmentation result of the current frame. Then, the weld semantic probability map is output through category projection and resolution interpolation. , Each pixel value represents the probability that the pixel belongs to the weld seam region. Set the weld seam segmentation threshold. , probability graph Convert to a binary weld mask, and obtain the weld mask matrix after selecting the largest connected component. .

[0088] This step uses high-level features from a multi-scale hierarchical feature extraction network as input for spatiotemporal difference-gated fusion. It primarily considers that lower-level features extracted are low-level geometric features such as edges and textures, which are easily affected by transient signals such as lighting changes and weld reflections. At higher levels of the network structure, strong semantic features, similar to "whether it is a weld," are output. Therefore, during spatiotemporal difference fusion, low computational cost can be used to correct transient noise at lower levels through global semantics. Furthermore, compared to video segmentation schemes based on 3D-CNN and cross-frame global attention, this application utilizes depthwise separable convolution and pixel-wise differencing to achieve temporal fusion. Its computational complexity is linearly related to image resolution, resulting in low computational overhead and suitability for deployment on welding equipment within edge computing environments.

[0089] S514. Determination of Weld Existence calculate The proportion of pixels with true matrix element values ​​out of the total number of pixels is used to set a weld presence threshold (preferably 1% of the total number of pixels). If the number of true pixels is greater than the threshold, the gimbal tracking function is activated; otherwise, if no weld is detected, the gimbal camera continues to face forward, and the internal welding machine continues to execute the control logic of the above-mentioned LiDAR cruise.

[0090] S52. Weld Seam Location Tracking and Image Acquisition S521. Calculate weld position and perform gimbal tracking. Will The values ​​in the middle are the pixel coordinates of the true value. A univariate linear equation is fitted using least squares: ,in It represents the inclination angle and intercept of the weld centerline in the image coordinate system. (The rest of the text appears to be a series of seemingly unrelated phrases and sentences, possibly from different sources.) Based on the specific kinematic parameters of the gimbal, the control values ​​of each joint of the gimbal can be calculated. Continuing with step S521, the gimbal is controlled to keep the weld seam image stably centered while the internal welding machine moves. When the tilt angle of each joint of the gimbal approaches the preset optimal imaging point, the internal welding machine is controlled to decelerate and stop.

[0091] S522. Image Acquisition of Circumferential Welds After the internal welding machine stops at the optimal imaging point, the gimbal begins to rotate 360 ​​degrees from the position where the weld image has been captured. During the rotation and acquisition of the weld image, step S521 continues. Figure 5 As shown, the weld image is kept centered in the field of view at all times.

[0092] S523. Weld Quality Judgment The collected images of circumferential welds were stitched together in a panoramic manner, and a model of initial screening using an image segmentation network followed by manual verification was used to determine the types of weld defects and their areas.

[0093] In another exemplary embodiment, based on the same inventive concept as the method embodiment, an electronic device is provided, including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and the processor executes the control method provided in the embodiment of the present invention when executing the computer instructions.

[0094] The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.

[0095] The embodiments of the subject matter and functional operation described in this specification can be implemented in: tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing device or for controlling the operation of a data processing device. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing device.

[0096] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0097] Suitable processors for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0098] It should be understood that each block in a flowchart or block diagram can represent a module, segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0099] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.

Claims

1. An internal welding machine with automatic cruise and weld seam identification and tracking data acquisition, characterized in that, include: Internal welding of the locomotive body; The cone-shaped head is installed on the inner welding machine body; An attitude adjustment mechanism, mounted on the inner welding machine body, is used to adjust the position and posture of the cone head relative to the center line of the pipe. A traveling mechanism, mounted on the inner welding machine body, is used to drive the inner welding machine body to move along the pipeline axis inside the pipeline; A lidar, installed at the front end of the cone, is used to scan the inner wall of the pipe to obtain three-dimensional point cloud data of the inner wall of the pipe; A gimbal is mounted on the front end of the cone head, and an imaging device is provided on it; the imaging device is used to acquire images inside the tube, and the gimbal is used to adjust the deflection of the imaging device. An angle sensor, installed on the inner welding machine body, is used to collect the roll angle and pitch angle of the inner welding machine body; The controller is communicatively connected to the lidar, gimbal, angle sensor, attitude adjustment mechanism, and walking mechanism, respectively. The controller includes: The pipeline center point extraction module is configured to iteratively solve for the pipeline center point set based on the three-dimensional point cloud data acquired by the lidar and the geometric constraint that the pipe wall normal intersects at the pipeline center. The internal welding machine cruise speed planning module is configured to construct a B-spline curve based on the set of pipe center points to calculate the pipe curvature and output the pipe pose information; control the cone to remain at the center of the pipe based on the pipe pose information, and plan the internal welding machine cruise speed based on the changing trend of the pipe curvature. The multimodal adaptive speed control module is configured to perform multimodal gating fusion on the three-dimensional point cloud data, the image inside the pipe, and the data collected by the angle sensor, and output the confidence level of the environment inside the pipe and the confidence level of safe passage; and generate the expected cruising speed of the inner welding machine based on the pipe curvature, the confidence level of the environment inside the pipe, and the confidence level of safe passage, so as to control the inner welding machine to travel along the pipe. The weld seam recognition and tracking acquisition module is configured to identify weld seams through a visual segmentation network during the movement of the inner welding machine along the pipeline, and control the imaging device to track and acquire weld seams according to their positions. When a weld seam is identified, the inner welding machine is controlled to decelerate and stop at a preset effective imaging position. The pan-tilt unit controls the imaging device to perform 360-degree circumferential weld seam image acquisition, and the weld seam is always kept in the center of the image during the 360-degree circumferential weld seam image acquisition process.

2. The internal welding machine according to claim 1, characterized in that, The pipeline center point extraction module iteratively solves for the pipeline center point set using the following formula: in, In any iteration step Below, a single center point From current coordinates The new coordinates to be transferred to in the next iteration step. Let represent the optimization objective function for the radial centering attraction and the constraint of the pipe wall normal. Represents the local spatial orientation anisotropy factor. Indicates the center point of other pipelines The sum of the vectors of the repulsive forces generated.

3. The internal welding machine according to claim 1, characterized in that, The internal welding machine cruise speed planning module includes: The sequence sorting unit is configured to arrange the set of pipeline center points into an ordered sequence from near to far along the pipeline extension direction using a nearest neighbor algorithm; B-spline curve construction units are configured to construct B-spline curve parametric equations based on the ordered sequence; The pipe curvature calculation unit is configured to calculate the curvature of each point on the axis of the pipe ahead based on the B-spline curve, and output the pipe curvature change trend from near to far. The cone head posture control unit is configured to control the posture adjustment mechanism to keep the cone head aligned with the center line of the pipe according to the pipe posture information; The internal welding machine cruise speed calculation unit is configured to plan the internal welding machine cruise speed according to the trend of the pipe curvature change from near to far, and the internal welding machine cruise speed is proportional to the magnitude of the pipe curvature.

4. The internal welding machine according to claim 1, characterized in that, The multimodal adaptive speed control module is configured with a multimodal adaptive speed control model, which includes: The parallel feature extraction layer is configured to contain three parallel feature processing channels, which respectively convert the 3D point cloud data, the in-tube image and the data collected by the angle sensor into feature vectors with consistent feature dimensions. The gated fusion layer is configured to dynamically generate attention weights for each feature processing channel through a fully connected layer, and to weight and synthesize the three feature vectors output by the parallel feature extraction layer according to the attention weights, so as to output a unified latent feature representation. A multi-task regression layer is configured to map the unified latent feature representation to in-pipe environment confidence and safe passage confidence.

5. The internal welding machine according to claim 4, characterized in that, The desired cruising speed of the internal welding machine is calculated using the following formula: in, Indicates the confidence level of the environment within the pipe. Indicates the confidence level of safe passage. Indicates the maximum permissible driving speed. Indicates the maximum centripetal acceleration. Indicates the curvature of the pipe.

6. The internal welding machine according to claim 1, characterized in that, The visual segmentation network includes: The multi-frame feature extraction layer is configured to use a multi-scale hierarchical feature extraction network as the backbone network to extract multi-scale features of the current image frame and the cached historical image frames in the pipeline, respectively. The spatiotemporal difference gated fusion layer is configured to calculate the absolute difference between the features of the current image frame in the pipeline and the features of the historical image frames in the pipeline in the highest layer feature map of the backbone network, generate spatial gate weights based on the absolute difference, and fuse the features of the current image frame in the pipeline and the features of the historical image frames in the pipeline based on the spatial gate weights to obtain fused features. The hybrid feature decoding layer is configured to feed the fused features and the low-level spatial features of the current in-pipe image frame into the multilayer perceptron decoder for fusion, output the weld seam segmentation result of the current in-pipe image frame, output the weld seam semantic probability map through category projection and resolution interpolation, and convert the weld seam semantic probability map into a weld seam mask matrix.

7. The internal welding machine according to claim 6, characterized in that, The step of controlling the imaging device to track the weld seam based on the weld seam position specifically includes: The pixel coordinates that are true in the weld mask matrix are fitted with a univariate linear equation using least squares: in, The angle of inclination of the weld centerline in the image coordinate system. The intercept; Based on the tilt angle and intercept, calculate the control values ​​of each joint of the gimbal, and control the gimbal to keep the weld seam in the center of the image. When the tilt angle of each joint of the gimbal reaches the preset optimal imaging point, the walking mechanism is controlled to stop walking; and the gimbal is controlled to perform 360-degree rotation imaging starting from the position where the weld seam image is captured.

8. The internal welding machine according to claim 6, characterized in that, The loss function used by the visual segmentation network is as follows: in, It is a combination loss function of traditional binary crossover loss and dice loss. It is the current frame image with the actual weld mask. It is the semantic probability map of the weld seam in the current image frame within the pipe. It is the semantic probability map of the weld seam in the previous image frame within the pipe. It is a time-series balanced weight. It is the time-series smoothing loss function.

9. A control method for the internal welding machine according to any one of claims 1-8, characterized in that, Includes the following steps: S1. The laser radar scans the inner wall of the pipe to obtain three-dimensional point cloud data of the inner wall of the pipe, the imaging device collects images inside the pipe, and the angle sensor collects the roll angle and pitch angle of the inner welding machine body. S2. Based on the three-dimensional point cloud data, the set of pipe center points is iteratively solved by the geometric constraint that the pipe wall normals intersect at the pipe center; S3. Construct a B-spline curve based on the set of pipe center points to calculate the pipe curvature and output the pipe pose information; control the cone to remain at the center of the pipe based on the pipe pose information, and plan the inner welding machine's cruising speed based on the changing trend of the pipe curvature; S4. Perform multimodal gating fusion on the three-dimensional point cloud data, the image inside the pipe, and the data collected by the angle sensor to output the confidence level of the environment inside the pipe and the confidence level of safe passage; and generate the expected speed of the inner welding machine's cruise based on the pipe curvature, the confidence level of the environment inside the pipe, and the confidence level of safe passage to control the inner welding machine to travel along the pipe. S5. During the process of the inner welding machine traveling along the pipeline, the weld seam is identified by the visual segmentation network, and the imaging device is controlled to track and collect the weld seam according to the position of the weld seam; wherein, when the weld seam is identified, the inner welding machine is controlled to decelerate and stop at the preset effective imaging position, and the pan-tilt unit controls the imaging device to perform 360-degree circumferential weld seam image acquisition, and the weld seam is always kept in the center of the image during the 360-degree circumferential weld seam image acquisition process.

10. An electronic device comprising a memory and a processor, wherein the memory stores computer instructions executable by the processor, characterized in that, When the processor executes computer instructions, it performs the relevant steps in the control method of claim 9.

Images