Magnetic attraction robot inspection method and system based on weld visual tracking

Abstract

The invention relates to the technical field of industrial visual intelligent detection, in particular to a magnetic attraction robot inspection method and system based on weld visual tracking. The specific implementation process comprises the following steps: performing illumination equalization processing on continuous frame original images, decomposing and filtering texture layer data by using total variation regularization, and retaining structural layer data; predicting a region of interest by using Kalman filtering, and inputting the region of interest into a double-flow feature fusion semantic segmentation network; extracting welding seam features by utilizing a spatial detail branch and a context semantic branch, and generating a welding seam mask through an attention gating mechanism; performing skeleton vectorization on the mask image, calculating a curvature change rate and a perspective gradient rate, and constructing a mapping model; and generating a visual tracking control vector in combination with the transverse position deviation and the angle deviation. According to the method, by means of total variation regularization decomposition and perspective gradient analysis, the welding seam features can be extracted under the complex industrial texture background, and robot inspection robustness and tracking precision are improved.

Description

Technical Field

[0001] This invention relates to the field of industrial vision intelligent inspection technology, specifically to a magnetic suction robot inspection method and system based on weld seam visual tracking. Background Technology

[0002] In the field of visual inspection of industrial surface defects, image acquisition and recognition of specific linear texture targets (e.g., structural features such as welds) on the surface of large structural components are essential. Existing technologies typically employ mobile visual acquisition terminals (e.g., magnetic robots equipped with camera units) to cruise and capture images on the target surface to obtain continuous video stream data. They also utilize photoelectric signals to perform threshold judgments on the geometric protrusions of the target, assisting the acquisition terminal in visual acquisition along the target's extension direction.

[0003] However, existing technologies have inherent limitations in practical applications. In real industrial data acquisition scenarios, target surfaces are filled with non-uniform light reflection, complex background texture interference (e.g., paint texture, oil noise, and rust spots), and high-frequency optical noise. Existing technologies struggle to effectively segment and extract key visual features of target objects from low signal-to-noise ratio image data, resulting in the inability to accurately construct the target's central skeleton and orientation vector. When the topological structure of the visual scene undergoes abrupt changes (e.g., from a planar surface to angled or multifaceted geometric changes), it is impossible to build an effective 3D semantic understanding model of the scene based on 2D image data. The lack of predictive and perceptual capabilities for changes in path geometric attributes leads to target loss or misclassification by visual tracking algorithms.

[0004] To address this, a magnetic suction robot inspection method and system based on weld seam visual tracking is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a magnetic suction robot inspection method and system based on weld seam visual tracking, which performs visual tracking of the weld seam inspection of the magnetic suction robot.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A magnetic suction robot inspection method based on weld seam vision tracking includes: The continuous frame original images acquired by the magnetic suction robot are subjected to illumination equalization processing, and the texture layer data containing rust, oil stains and paint noise in the continuous frame original images are decomposed and filtered using total variational regularization, while retaining the structural layer data containing weld geometry information. The region of interest (ROI) in the current frame is predicted using the Kalman filter algorithm based on the motion vectors of historical frame images. The structural layer data within the ROI is then input into a dual-stream feature fusion semantic segmentation network. Weld seam features are extracted using the spatial detail branch and contextual semantic branch of the dual-stream feature fusion semantic segmentation network, and non-weld seam features are suppressed using an attention gating mechanism to output a mask image of the weld seam region. The mask image is vectorized and extracted to calculate the curvature change rate of the skeleton pixels and the perspective gradient rate of the weld mask width along the skeleton direction. Based on the curvature change rate and perspective gradient rate, a mapping model between the relative position and posture deviation of the local weld and the magnetic robot is constructed. Combining the lateral position deviation and angular deviation of the skeleton centerline, a visual tracking control vector for the navigation magnetic robot inspection is generated.

[0007] Preferably, the specific implementation process of performing illumination equalization processing on the continuous frame original images and decomposing and filtering the texture layer data using total variational regularization to retain the structure layer data includes: converting the acquired continuous frame original images to a luminance-chrominance color space and extracting the luminance component; processing the luminance component using a contrast-limited adaptive histogram equalization algorithm to generate an enhanced luminance component and reconstructing it into an illumination-equalized image; constructing an energy minimization model containing a total variational regularization term; solving the energy minimization model using an primal-dual iterative algorithm; retaining the edge geometric discontinuity of the weld by minimizing the total variational function; classifying rust, oil stains, and paint noise into the texture layer data and filtering them; and outputting structure layer data containing only weld geometric information.

[0008] Preferably, the specific implementation process of predicting the region of interest (ROI) of the current frame using the Kalman filter algorithm based on the motion vectors of historical frame images, and inputting the structural layer data of the ROI into the dual-stream feature fusion semantic segmentation network includes: using the geometric center of the weld in the previous historical frame image as the observation input, correcting the prediction bias using the state update equation of the Kalman filter, and calculating the prior position estimate of the weld in the current frame based on the state transition matrix; using the prior position estimate as the anchor point, generating a rectangular window covering the potential displacement range of the weld as the ROI; using the spatial coordinates of the ROI to perform matrix clipping on the structural layer data, and inputting it into the dual-stream feature fusion semantic segmentation network.

[0009] Preferably, the specific implementation process of extracting weld seam features using the spatial detail branch and contextual semantic branch in the dual-stream feature fusion semantic segmentation network, and suppressing non-weld seam features through an attention gating mechanism to output a mask image of the weld seam region includes: feeding the structural layer data as an input tensor to the spatial detail branch, extracting spatial feature maps that preserve edge details while maintaining spatial resolution through stacked span convolutional layers; transmitting the input tensor in parallel to the contextual semantic branch, reducing the feature map size by using continuous downsampling convolution and downsampling strategies to expand the effective receptive field, obtaining semantic contextual features that encode the weld seam topology, concatenating the spatial feature map with the upsampled semantic contextual features to generate multi-scale fusion features; the attention gating mechanism calculates the spatial attention weight matrix using the semantic contextual features as a guiding signal, and performs element-wise multiplication with the multi-scale fusion features to suppress non-weld seam background regions and enhance the weld seam region, and obtains a mask image of the weld seam region after binarization.

[0010] Preferably, the specific implementation process of extracting the skeleton vectorization of the mask image and calculating the curvature change rate of the skeleton pixels and the perspective gradient rate of the weld mask width along the skeleton direction includes: extracting the single-pixel connected domain skeleton reflecting the center direction of the weld using a morphological thinning algorithm; traversing the single-pixel connected domain skeleton using an eight-neighbor chain code tracking strategy to convert discrete pixel coordinates into an ordered skeleton vector sequence; calculating the tangent vector and normal vector of each node on the skeleton vector sequence based on the differential geometry principle to generate the curvature change rate characterizing the degree of weld bending; searching the mask edges to both sides along the normal vector direction of each node in the skeleton vector sequence, calculating the Euclidean distance between the intersection of the normal and the mask contour, and constructing a weld width distribution sequence; performing gradient calculation on the weld width distribution sequence along the skeleton extension direction, analyzing the trend of width data changing with spatial depth, and generating perspective gradient rate data characterizing the visual perspective effect.

[0011] Preferably, the specific implementation process of constructing a mapping model between the relative position and attitude deviation of the local weld and the magnetic robot based on the curvature change rate and perspective gradient rate includes: by analyzing the attenuation trend of the width data in the longitudinal direction of the image, the spatial angle between the normal vector of the weld surface and the optical axis of the camera is solved, and the pitch attitude component of the magnetic robot relative to the current local weld cutting plane is calculated; a distortion correction matrix is ​​generated using the spatial angle, and a reprojection transformation is performed on the coordinate sequence of the skeleton pixels to map the image coordinates of the oblique view to the plane coordinates of the normal view, restoring the true topological shape of the weld in the local plane; the angle between the tangent direction vector of the true topological shape and the vertical axis of the image center is calculated, and the yaw angle deviation of the magnetic robot is analyzed; the normal distance from the principal point of the image to the center line of the reprojected skeleton is measured as the lateral position deviation; and the pitch attitude component, yaw angle deviation, and lateral position deviation are associated to establish a mapping model.

[0012] Preferably, the specific implementation process for generating the visual tracking control vector for the navigation magnetic robot's inspection, combining the lateral position deviation and angular deviation of the skeleton centerline, includes: inputting the lateral position deviation and angular deviation as error feedback signals to a multivariable proportional-integral-derivative controller to calculate the lateral correction component and the rotational correction component; based on a fuzzy logic weighted algorithm, dynamically allocating weight coefficients according to the deviation amplitude, fusing the lateral correction component and the rotational correction component to synthesize a target angular velocity command; according to the differential drive kinematics model of the magnetic robot, performing inverse kinematics calculation on the target angular velocity command and the reference inspection linear velocity to convert it into the target rotational speed value of the left and right drive wheels; performing trapezoidal acceleration / deceleration smoothing processing and amplitude limiting on the target rotational speed value to generate a visual tracking control vector and navigate the inspection motion of the magnetic robot.

[0013] A magnetic suction robot inspection system based on weld seam vision tracking includes: The background noise filtering module performs illumination equalization processing on the continuous frame original images collected by the magnetic robot, and uses total variational regularization to decompose and filter the texture layer data containing rust, oil stains and paint noise in the continuous frame original images, while retaining the structural layer data containing weld geometry information. The weld seam feature recognition module uses the motion vectors of historical frame images and the Kalman filter algorithm to predict the region of interest in the current frame. The structural layer data of the region of interest is input into the dual-stream feature fusion semantic segmentation network. The spatial detail branch and the context semantic branch in the dual-stream feature fusion semantic segmentation network are used to extract weld seam features, and non-weld seam features are suppressed by an attention gating mechanism to output a mask image of the weld seam region. The visual tracking inspection module extracts the skeleton vector from the mask image, calculates the curvature change rate of the skeleton pixels and the perspective gradient rate of the weld mask width along the skeleton direction; based on the curvature change rate and perspective gradient rate, it constructs a mapping model between the relative position and posture deviation of the local weld and the magnetic robot; and combines the lateral position deviation and angular deviation of the skeleton centerline to generate the visual tracking control vector for the navigation magnetic robot inspection.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention utilizes illumination equalization and total variational regularization techniques to decompose the original image into a texture layer containing rust, oil stains, and paint noise, and a structure layer containing geometric information. By filtering the texture layer and retaining the structure layer, interference from non-uniform illumination and complex background textures is effectively removed, significantly improving the visual imaging quality and signal-to-noise ratio of weld seams in harsh industrial environments.

[0015] 2. This invention employs a dual-stream feature fusion semantic segmentation network (spatial detail branch + contextual semantic branch) combined with an attention gating mechanism, enabling simultaneous extraction of weld detail features and semantic features, effectively suppressing background interference. The Kalman filter algorithm is used to predict the region of interest in the next frame, improving the algorithm's processing speed and endowing the system with the ability to perceive path changes, effectively preventing target loss or classification errors during visual tracking.

[0016] 3. This invention proposes a mapping model based on the rate of change of skeleton curvature and the rate of perspective gradient. By analyzing the perspective change trend of weld width, the three-dimensional attitude information (such as pitch angle) and yaw angle deviation of the robot relative to the weld plane can be deduced from the two-dimensional image. This enables the robot to adapt to abrupt changes in weld topology (such as turning angle from the plane) and achieve more precise navigation and inspection control. Attached Figure Description

[0017] Figure 1 This is a flowchart of a magnetic suction robot inspection method based on weld seam visual tracking proposed in this invention;

[0018] Figure 2 This is a structural diagram of a magnetic suction robot inspection system based on weld seam visual tracking proposed in this invention;

[0019] Figure 3 This is a schematic diagram of the weld seam inspection process of the magnetic suction robot proposed in this invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It must be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to constitute any limitation on the scope of protection of this invention. Therefore, all equivalent changes or modifications conceived by those skilled in the art based on the content disclosed in this invention without inventive effort should fall within the scope of protection claimed by this invention.

[0021] Reference Figures 1 to 3 This invention proposes a magnetic suction robot inspection method and system based on weld seam visual tracking, the technical solution of which is as follows: Example

[0022] Reference Figure 1 This embodiment proposes a magnetic robot inspection method based on weld seam visual tracking, including: The continuous frame original images acquired by the magnetic suction robot are subjected to illumination equalization processing, and the texture layer data containing rust, oil stains and paint noise in the continuous frame original images are decomposed and filtered using total variational regularization, while retaining the structural layer data containing weld geometry information. The region of interest (ROI) in the current frame is predicted using the Kalman filter algorithm based on the motion vectors of historical frame images. The structural layer data within the ROI is then input into a dual-stream feature fusion semantic segmentation network. Weld seam features are extracted using the spatial detail branch and contextual semantic branch of the dual-stream feature fusion semantic segmentation network, and non-weld seam features are suppressed using an attention gating mechanism to output a mask image of the weld seam region. The mask image is vectorized and extracted to calculate the curvature change rate of the skeleton pixels and the perspective gradient rate of the weld mask width along the skeleton direction. Based on the curvature change rate and perspective gradient rate, a mapping model between the relative position and posture deviation of the local weld and the magnetic robot is constructed. Combining the lateral position deviation and angular deviation of the skeleton centerline, a visual tracking control vector for the navigation magnetic robot inspection is generated.

[0023] Furthermore, the specific implementation process of performing illumination equalization on the continuous frame original images and decomposing and filtering the texture layer data using total variational regularization to retain the structure layer data includes: converting the acquired continuous frame original images to a luminance-chrominance color space and extracting the luminance component; processing the luminance component using a contrast-limited adaptive histogram equalization algorithm to generate an enhanced luminance component and reconstruct an illumination-equalized image; constructing an energy minimization model containing a total variational regularization term; solving the energy minimization model using an primal-dual iterative algorithm; preserving the edge geometric discontinuity of the weld by minimizing the total variational function; classifying rust, oil stains, and paint noise into the texture layer data and filtering them; and outputting structure layer data containing only weld geometric information.

[0024] Specifically, the system receives continuous frame images in RGB format captured by a high-frame-rate industrial camera mounted on a magnetic robot. Since luminance and chromaticity information are highly coupled in the RGB color space, direct processing can easily lead to color distortion. Therefore, the continuous frame images are converted to the YCbCr luminance-chromaticity color space using a linear transformation matrix. In the YCbCr space, the image is separated into independent luminance components (Y channel) and chromaticity components (Cb and Cr channels). Addressing the common phenomenon of coexistence of localized reflections and shadows in weld inspection scenarios, the luminance component is selected as the processing target, and enhanced using the Limiting Contrast Adaptive Histogram Equalization (CLAHE) algorithm. During this process, the luminance component image is divided into several non-overlapping rectangular sub-blocks, for example, setting the sub-block size to an 8×8 pixel grid, and calculating the local histogram of each sub-block. To prevent excessive noise amplification in homogeneous regions (large areas of smooth metal surfaces), a clipping threshold is set between 2.0 and 3.0. Histogram portions exceeding this threshold are cropped and evenly distributed to the remaining parts of the histogram. Then, bilinear interpolation is used to eliminate blockiness at sub-block boundaries, generating an enhanced luminance component. Finally, the enhanced luminance component is recombinated with the original chromaticity components Cb and Cr, and inversely transformed back to RGB space to reconstruct an illumination-equalized image.

[0025] To remove interfering texture details from the illumination-equalized image while preserving the geometric topology of the weld, an energy minimization model incorporating a total variational regularization term is constructed. This energy minimization model is based on the TV-L2 model, assuming the illumination-equalized image consists of a structure layer containing the object's geometric contours and a texture layer containing repeating patterns and noise. In constructing the energy minimization model, the objective function is defined as a weighted sum of a data fidelity term and a total variational regularization term. The data fidelity term constrains the approximation of the solved structure layer image to the original input image in terms of pixel values, typically represented using the L2 norm. The total variational regularization term uses the L1 norm of the image gradient to measure the smoothness of the image; its core characteristic is that it allows for edge transitions while penalizing high-frequency oscillations.

[0026] In the solution process, an primal-dual iterative algorithm is introduced to solve the energy minimization model. The specific numerical calculation process is as follows: Initialize the original variables (corresponding to the structural layer image) and dual variables (corresponding to the dual vector of the gradient field), and set the regularization weight parameter (e.g., a value of 0.15) to balance the degree of denoising and the edge preservation ability. In each iteration step, first fix the dual variables, and update the original variables along the gradient descent direction of the energy function with respect to the original variables. This step achieves denoising and smoothing of the image. Using the updated original variables, update the dual variables through the gradient ascent direction, and project and truncate the dual variables to satisfy the convex set constraint condition. The primal-dual iterative algorithm alternately executes the above update steps, for example, setting the number of iterations to 50 to 100, until the energy function converges.

[0027] By minimizing the total variation function to suppress small-scale oscillatory changes in the image, in actual weld images, rust, oil stains, and paint noise appear as high-frequency, fine-grained texture features, which are classified as texture layer data and stripped during the total variation minimization process. Meanwhile, weld edges and bevel contours, which exhibit step changes in pixel values ​​and belong to the geometric structure information of the image, are retained in the structure layer data. The final output is structure layer data containing only weld geometry information. This data is in grayscale matrix form, eliminating color and fine texture interference, and highlighting the weld's skeleton orientation and edge boundaries.

[0028] This embodiment effectively solves the problem of uneven lighting on large cylindrical or curved structural components under point light source illumination through CLAHE processing in YCbCr space, avoiding overexposure in highlight areas and loss of detail in shadow areas. Utilizing total variational regularization decomposition technology, it smooths out complex industrial noise such as rust and oil stains while maintaining the sharpness of weld edges, preventing subsequent skeleton extraction breakage or displacement caused by edge blurring. This improves the robustness and accuracy of the magnetic robot in weld path recognition.

[0029] Furthermore, the specific implementation process of predicting the region of interest (ROI) of the current frame using the Kalman filter algorithm based on the motion vectors of historical frame images, and inputting the structural layer data within the ROI into the dual-stream feature fusion semantic segmentation network includes: using the geometric center of the weld in the previous historical frame image as the observation input, correcting the prediction bias using the state update equation of the Kalman filter, and calculating the prior position estimate of the weld in the current frame based on the state transition matrix; using the prior position estimate as the anchor point, generating a rectangular window covering the potential displacement range of the weld as the ROI; using the spatial coordinates of the ROI to perform matrix clipping on the structural layer data, and inputting it into the dual-stream feature fusion semantic segmentation network.

[0030] Specifically, since the magnetic robot's motion is continuous but subject to random disturbances caused by mechanical vibrations at the microscopic level when crawling on a large metal wall, a four-dimensional state vector is defined, containing the abscissa, ordinate, lateral velocity, and longitudinal velocity of the weld center. When processing video stream data, the weld image output from the previous historical frame (time t-1) is extracted. By calculating the zeroth and first moments, the geometric center coordinates of the weld in the previous historical frame are obtained, and these coordinates are fed as the observation input to the Kalman filter. Then, using the state update equation of the Kalman filter, combined with the current observation and the predicted value from the previous time, the optimal estimation deviation is calculated, thereby correcting the system's internal error covariance matrix. Based on a preset state transition matrix describing a uniform or uniformly accelerated motion model, the corrected state vector is mapped to the current time (time t), thus calculating the prior position estimate of the weld in the current frame. For example, in actual working conditions, if the magnetic robot inspects the weld seam in a straight line at a speed of 20 millimeters per second and the image acquisition frame rate is 30 frames per second, the filter can accurately predict the specific directional displacement of the weld seam center in the image coordinate system by about 5 to 10 pixels based on the historical motion vector.

[0031] Using the coordinates of the previously estimated position as anchor points, a region of interest (ROI) is constructed. To cover sudden displacements of the robot due to uneven road surfaces or minor bending changes in the weld itself, single-point coordinates are not used directly. Instead, a rectangular window covering the potential displacement range of the weld is generated as the ROI. Specifically, assuming the original image resolution is 2448×2048 pixels, a fixed-size rectangular window of 512×512 pixels is set, or a long strip window with a width three times the weld width and a height of 512 pixels is dynamically set based on historical weld width data. The center of this rectangular window is the previously estimated position. If the window edge exceeds the image boundary, boundary filling or shrinking is automatically performed to ensure the validity of the window coordinates.

[0032] The generated structural layer data is matrix-cropped using the spatial coordinates of the region of interest. Since the structural layer data is a grayscale matrix with texture noise removed, the corresponding sub-matrices are indexed and extracted from the original high-resolution structural layer matrix based on the coordinates of the four vertices of the rectangular window. This process is equivalent to "focusing" the line of sight in a panoramic image, retaining only the geometric information of the weld and its adjacent background, while discarding large background areas far from the weld (which usually contain useless rivets, scratches, and other interference). The smaller structural layer sub-image patches obtained after cropping are fed as input tensors into a two-stream feature fusion semantic segmentation network for subsequent processing.

[0033] This embodiment introduces a Kalman filter algorithm to dynamically predict and lock the region of interest, reducing the computational load on the subsequent deep neural network. When linear scratches resembling weld seams exist in the image background, they are automatically filtered out because they are not within the predicted motion trajectory window. This effectively solves the problem of high misidentification rates in complex industrial environments and significantly improves the stability and accuracy of the magnetic robot inspection.

[0034] Furthermore, the specific implementation process of extracting weld seam features using the spatial detail branch and contextual semantic branch in the dual-stream feature fusion semantic segmentation network, and suppressing non-weld seam features through an attention gating mechanism to output a mask image of the weld seam region includes: feeding the structural layer data as an input tensor to the spatial detail branch, extracting spatial feature maps that preserve edge details while maintaining spatial resolution through stacked span convolutional layers; transmitting the input tensor in parallel to the contextual semantic branch, reducing the feature map size by using continuous downsampling convolution and downsampling strategies to expand the effective receptive field, obtaining semantic contextual features encoding the weld seam topology, concatenating the spatial feature map with the upsampled semantic contextual features to generate multi-scale fusion features; the attention gating mechanism calculates the spatial attention weight matrix using the semantic contextual features as a guiding signal, and performs element-wise multiplication with the multi-scale fusion features to suppress non-weld seam background regions and enhance the weld seam region, and obtains the mask image of the weld seam region after binarization.

[0035] Specifically, the input tensor is a single-channel 512×512 pixel matrix. The dual-stream feature fusion semantic segmentation network splits it into two parallel processing paths. The first path is the spatial detail branch, designed to preserve high-frequency geometric information of the weld edges. In this branch, the input tensor is processed by stacking small-span convolutional layers, specifically using convolution operations with a kernel size of 3×3 and a stride of 1 or 2, without using layers that would lose spatial information, such as max pooling. This shallow and high-resolution network structure (e.g., outputting a 128×128 feature map after three convolutional layers) can extract spatial feature maps that preserve edge details while maintaining a wide spatial resolution, ensuring that the weld boundaries are clearly distinguishable at the pixel level and do not become blurred as the network depth increases.

[0036] The input tensor is transmitted in parallel to the context semantic branch, which adopts a structure similar to the first three stages of ResNet-34. The first layer uses a 7*7 convolution with a stride of 2, followed by a max pooling layer. Subsequently, three sets of residual modules are used for continuous downsampling, compressing the feature map size from 512*512 to 32*32 and expanding the number of channels to 256. Continuous downsampling convolution and a fast downsampling strategy are employed to reduce the feature map size and expand the effective receptive field, allowing each pixel on this branch to "see" a larger area of ​​the original image, thereby obtaining semantic context features encoding the weld topology. These features contain abstract information such as the weld's orientation, continuity, and relative relationship with the surrounding background. Since the spatial feature map and semantic context features have inconsistent sizes, a bilinear interpolation algorithm is used to upsample the low-resolution semantic context features to align their size with the high-resolution spatial feature map (e.g., adjusting both to 128×128). The spatial feature map is concatenated with the upsampled semantic context features along the channel dimension to generate a multi-scale fusion feature containing rich edge information and global semantic information.

[0037] To further eliminate background interference, an attention gating mechanism is introduced. In this mechanism, deep semantic context features are used as guiding signals, mapped to the attention space through a 1×1 convolutional layer, and a spatial attention weight matrix with values ​​between 0 and 1 is calculated using a sigmoid activation function. This weight matrix exhibits high response values ​​(close to 1) in areas where weld seams exist, and low response values ​​(close to 0) in areas with oil stains, scratches, or background. The spatial attention weight matrix is ​​then element-wise multiplied with the multi-scale fused features. This process is equivalent to adding a "filter" to the feature map, adaptively suppressing the feature responses of non-weld seam background areas and enhancing the feature representation of the weld seam area.

[0038] Finally, the attention-weighted feature map is compressed to 1 channel using a 1×1 convolutional layer, and the output is mapped to a probability range of 0 to 1 using the Sigmoid function. A binarization threshold of 0.5 is set, with pixels having a probability greater than 0.5 marked as 255 (white, representing the weld), and pixels with a probability less than or equal to 0.5 marked as 0 (black, representing the background). This binarization process yields a mask image containing only the weld region.

[0039] The dual-stream feature fusion semantic segmentation network designed in this embodiment uses spatial detail branching to solve the problems of blurred weld edges and inaccurate positioning, ensuring the lateral control accuracy of visual tracking. By combining contextual semantic branching with attention gating mechanism, it effectively distinguishes linear interference similar to weld morphology (such as long strip scratches or seams) using global information, solving the problem of false detection in complex texture backgrounds and improving the environmental adaptability of the magnetic robot.

[0040] Furthermore, the specific implementation process of extracting the skeleton vectorization of the mask image and calculating the curvature change rate of the skeleton pixels and the perspective gradient rate of the weld mask width along the skeleton direction includes: extracting the single-pixel connected domain skeleton reflecting the center direction of the weld using a morphological thinning algorithm; traversing the single-pixel connected domain skeleton using an eight-neighbor chain code tracking strategy to convert the discrete pixel coordinates into an ordered skeleton vector sequence; calculating the tangent vector and normal vector of each node on the skeleton vector sequence based on the differential geometry principle to generate the curvature change rate characterizing the degree of weld bending; searching the mask edges to both sides along the normal vector direction of each node in the skeleton vector sequence, calculating the Euclidean distance between the intersection of the normal and the mask contour, and constructing a weld width distribution sequence; performing gradient calculation on the weld width distribution sequence along the skeleton extension direction, analyzing the trend of width data changing with spatial depth, and generating perspective gradient rate data characterizing the visual perspective effect.

[0041] Specifically, the process receives a binary mask image output from the previous step, where white areas represent weld seams and black areas represent the background. To eliminate interference from the weld seam width, the center path of the weld seam is located, and a morphological thinning algorithm is used to extract a single-pixel connected component skeleton reflecting the direction of the weld seam center. During this process, an iterative erosion operation is employed to peel away the edge pixels of the binary mask layer by layer while maintaining the connectivity of the weld seam topology, until the weld seam region of a certain width is shrunk to a center line only one pixel wide. For example, for a weld seam mask with a width between 30 and 50 pixels, after approximately 15 to 25 iterations of thinning, a continuous white skeleton line with a single pixel width is output.

[0042] To perform mathematical analysis on the skeleton line, discrete raster pixels must be converted into ordered vector data. An eight-neighbor chain code tracking strategy is employed to traverse the single-pixel connected component skeleton. Specifically, the eight-neighbor chain code tracking strategy operates as follows: The bottom of the image is scanned to determine the starting point of the skeleton and designated as the current node; the positions of the eight neighboring pixels around the current node (i.e., top, bottom, left, right, and four diagonal directions) are searched in a clockwise or counterclockwise direction to find the next non-zero pixel as the successor node, and the movement direction code is recorded. This search process is recursively executed until the end of the skeleton, converting the discrete pixel coordinates into an ordered skeleton vector sequence. The coordinates of this sequence are then subjected to a five-point smoothing filter to eliminate jagged quantization noise introduced by image digitization.

[0043] Based on the principle of differential geometry, the tangent and normal vectors of each node in the skeleton vector sequence are calculated to generate the rate of curvature change, which characterizes the degree of weld bending. For each node in the skeleton vector sequence, its first derivative is calculated using the central difference method to obtain the tangent direction vector, and then the derivative of the tangent vector with respect to the arc length is calculated to obtain the curvature. Specifically, the change in tangential angle between adjacent nodes is calculated, which reflects the degree of local bending of the weld in the plane. If the rate of curvature change of a certain segment of the skeleton sequence exceeds a preset threshold (e.g., the radius of curvature is less than 200 pixels), it indicates that there is a sharp turn or bend in the weld ahead, which provides a direct basis for the robot to adjust the yaw angle. In order to extract the three-dimensional spatial depth information from the two-dimensional image, the width and perspective gradient rate are calculated. The mask edge is searched on both sides along the normal vector direction of each node in the skeleton vector sequence (i.e., the direction perpendicular to the tangent). Specifically, starting from the skeleton node, the pixel value of the mask image is detected pixel by pixel in the positive and negative directions of the normal. Once a pixel value changes from 255 to 0, it is determined to be a weld edge point. Calculate the Euclidean distance between the two edge points to construct a weld width distribution sequence that corresponds one-to-one with the skeleton nodes.

[0044] The gradient of the weld width distribution sequence along the skeleton extension direction is calculated to analyze the trend of width data changing with spatial depth, generating perspective gradient rate data characterizing the visual perspective effect. If the camera optical axis of the magnetic robot is perpendicular to the weld surface, the weld width in the image should remain basically constant; if the robot tilts, due to the perspective projection principle of "nearer objects appear larger and farther objects appear smaller," the weld width in the image will show a linear decrease or increase. For example, if the weld width at the bottom of the image is detected to be 50 pixels, while the weld width at the top of the image linearly decreases to 30 pixels, and excluding the possibility of narrowing due to the process of the weld itself, the linear regression slope of this width sequence is calculated to obtain the perspective gradient rate. This data is statistically related to the angle between the camera optical axis and the weld plane.

[0045] This embodiment improves upon the problem of monocular vision's inability to perceive 3D posture in the absence of a depth sensor by jointly analyzing skeleton vectorization and perspective gradient rate. The perspective variation law (perspective gradient rate) of the weld width is used to extract the robot's pitch angle information; the analysis based on the curvature change rate endows the robot with the ability to predict the weld path, thereby enabling smoother and more precise posture adjustment and path tracking.

[0046] Furthermore, based on the curvature change rate and perspective gradient rate, the specific implementation process of constructing a mapping model between the relative position and attitude deviation of the local weld and the magnetic robot includes: by analyzing the attenuation trend of the width data in the longitudinal direction of the image, the spatial angle between the normal vector of the weld surface and the optical axis of the camera is solved, and the pitch attitude component of the magnetic robot relative to the current local weld cutting plane is calculated; a distortion correction matrix is ​​generated using the spatial angle, and a reprojection transformation is performed on the coordinate sequence of the skeleton pixels to map the image coordinates of the oblique view to the plane coordinates of the normal view, restoring the true topological shape of the weld in the local plane; the angle between the tangent direction vector of the true topological shape and the vertical axis of the image center is calculated, and the yaw angle deviation of the magnetic robot is analyzed; the normal distance from the principal point of the image to the center line of the reprojected skeleton is measured as the lateral position deviation; the pitch attitude component, yaw angle deviation, and lateral position deviation are associated to establish a mapping model.

[0047] Specifically, from a vertical shooting perspective, the width of the weld seam in the image should remain consistent. However, due to the principle of perspective projection, when the magnetic robot tilts relative to the weld seam plane, the weld seam width in the image will nonlinearly decrease or expand with the longitudinal coordinate. Based on the prior assumption that the weld seam width is relatively uniform over short distances, by analyzing the perspective gradient change (i.e., the degree of near-wide and far-narrow) of the width data obtained in the image's longitudinal direction, combined with the camera's intrinsic parameter matrix (including focal length, pixel size, etc.), and based on a pinhole camera imaging model, let the actual width of the weld seam be... The camera focal length is The vertical distance between the camera's optical center and the weld plane is The vertical coordinate in the image coordinate system is weld projection width at the location By sampling multiple points (y, y) on the skeleton line The data is fitted using nonlinear least squares to determine the pitch attitude components of the magnetic robot relative to the current local weld cut plane. .

[0048] A distortion correction matrix is ​​generated using the spatial angle to eliminate the misleading effect of perspective distortion on path direction judgment. This matrix is ​​typically a 3×3 homography matrix, used to describe the mapping relationship from the image coordinate system to the world coordinate system (or orthophoto plane). This matrix is ​​applied to perform a reprojection transformation on the curvature change rate and coordinate sequence of the skeleton pixels. This process is equivalent to correcting the camera's viewpoint from a "tilted view" to a "vertical top view" at the algorithmic level, thereby restoring the true topological morphology of the weld in the local plane. In specific test data, camera calibration parameters are used to eliminate nonlinear distortions (such as barrel or pincushion distortion) caused by the optical lens. For the trapezoidal or linear convergence shape of straight welds in the image caused by the tilted viewpoint, after the reprojection transformation based on the homography matrix, the skeleton lines are restored to a parallel or straight shape in the world coordinate system, thereby eliminating the direction judgment interference introduced by perspective projection.

[0049] Within the corrected space that restores the true topology, the yaw angle deviation of the magnetic robot is analyzed. The tangent vector of the true topology (i.e., the reprojected skeleton line) at the current position is calculated, and the angle between this vector and the vertical axis of the image center (representing the robot's forward direction of travel) is calculated. For example, if the corrected skeleton tangent vector points to the upper right of the image, forming a positive 5-degree angle with the vertical axis, it indicates that the robot's current heading is biased towards the left side of the weld seam and needs to be adjusted to the right. This angle is the resolved yaw angle deviation of the magnetic robot. To determine the lateral control error, the normal distance from the principal point of the image to the center line of the reprojected skeleton is measured as the lateral position deviation. The principal point of the image usually corresponds to the optical center projection point of the camera, representing the physical center of the robot. The vertical Euclidean distance from this principal point to the corrected skeleton line is calculated, and this pixel distance is converted to a physical distance (e.g., millimeters) according to the camera calibration parameters. For example, if the calculated distance from the principal point to the left of the skeleton line is 3.5 millimeters, this value is the lateral position deviation, indicating that the robot needs to translate 3.5 millimeters to the left to center.

[0050] A mapping model is established by associating the pitch attitude components, yaw angle deviation, and lateral position deviation. This model is a multi-input multi-output logic module that encapsulates the three independent geometric deviations into a state error vector. This vector defines the three-dimensional spatial difference between the robot's current state point and the ideal inspection path.

[0051] This embodiment utilizes reprojection transformation technology to mitigate the distortion effect of perspective distortion on path geometric features (such as curvature and angle), enhancing the realism and accuracy of yaw angle and lateral deviation calculations. This enables the magnetic robot to sense and correct its own pitch tilt, significantly improving its adaptability and tracking accuracy on complex curved paths during wall-climbing inspection operations.

[0052] Furthermore, the specific implementation process for generating the visual tracking control vector for the navigation magnetic robot's inspection, combining the lateral position deviation and angular deviation of the skeleton centerline, includes: inputting the lateral position deviation and angular deviation as error feedback signals to a multivariable proportional-integral-derivative controller to calculate the lateral correction component and rotational correction component; based on a fuzzy logic weighted algorithm, dynamically allocating weight coefficients according to the deviation amplitude, fusing the lateral correction component and rotational correction component to synthesize a target angular velocity command; according to the differential drive kinematics model of the magnetic robot, performing inverse kinematics calculation on the target angular velocity command and the reference inspection linear velocity to convert it into the target rotational speed value of the left and right drive wheels; performing trapezoidal acceleration / deceleration smoothing and amplitude limiting on the target rotational speed value to generate a visual tracking control vector and navigate the inspection motion of the magnetic robot.

[0053] Specifically, the lateral position deviation (i.e., the distance between the robot's center and the weld seam, in millimeters) and angular deviation (i.e., the angle between the robot's heading and the weld seam tangent, in radians) obtained from the previous steps are input as error feedback signals to a multivariable proportional-integral-derivative (PID) controller. This controller contains two independent control loops: a lateral control loop and an angle control loop. In the lateral control loop, the proportional, integral, and derivative terms of the lateral position deviation are calculated. The proportional term responds to the current error amplitude, the integral term eliminates steady-state errors caused by mechanical asymmetry, and the derivative term predicts the error change trend to suppress overshoot. For example, setting the lateral proportional coefficient to 1.5, the integral coefficient to 0.05, and the derivative coefficient to 0.2, the lateral correction component for correcting distance errors is calculated. In the angle control loop, using the PID parameter set for the angle deviation, the rotational correction component for correcting heading errors is calculated. Physically, these two components represent the linear velocity adjustment and angular velocity adjustment required to eliminate the deviation, respectively.

[0054] Establish a fuzzy inference rule base and define lateral deviation. and angle deviation The fuzzy subset is {NB, NS, ZO, PS, PB}, and a triangular membership function is used. Weight adjustment coefficients. The fuzzy rule table is designed as follows: when When it is PB, regardless of Why assign a larger weighting coefficient to the horizontal correction component? (For example: 0.8); when is ZO and When the value is PB, reduce the weight of the rotation correction component. (For example: 0.2), prioritize ensuring the robot quickly returns to the vicinity of the weld; when Smaller but When the error is large (e.g., distance error less than 2 mm, but yaw angle greater than 5 degrees), the weight allocation is dynamically adjusted, emphasizing rotational correction. Based on the dynamically generated weight coefficients, the lateral correction component and the rotational correction component are weighted and fused to synthesize the final target angular velocity command. This command represents the overall rotational rate that the robot needs to generate, simultaneously considering the requirements of approaching the weld and the alignment direction.

[0055] Based on the differential drive kinematic model of the magnetic robot, the target angular velocity command and the reference inspection linear velocity are calculated using inverse kinematics. The magnetic robot employs a dual-wheel differential drive structure, reading the preset reference inspection linear velocity (e.g., 0.1 m / s) and the robot's physical wheelbase parameters (e.g., the center distance between the left and right wheels is 300 mm). According to the principles of rigid body kinematics, the target linear velocity of the left drive wheel is equal to the reference linear velocity minus the product of the target angular velocity and half the wheelbase, while the target linear velocity of the right drive wheel is equal to the reference linear velocity plus the product of the target angular velocity and half the wheelbase. Using the radius parameter of the drive wheels (e.g., 50 mm), the above linear velocity values ​​are converted into target rotational speed values ​​(in revolutions per minute) for the left and right drive wheels.

[0056] To prevent motor overload or robot mechanical vibration caused by sudden changes in control commands, trapezoidal acceleration / deceleration smoothing and amplitude limiting are applied to the target speed value. A maximum acceleration threshold (e.g., 0.5 m / s²) and a maximum speed limit are set. When the calculated target speed undergoes a step change, a transition interpolation point is inserted on the time axis to generate a trapezoidal or S-shaped speed change curve, ensuring a smooth transition of the actual issued speed command to the target value. If the speed of any wheel exceeds the motor's physical limit in the calculation results, to prevent unexpected changes in the robot's motion curvature (turning radius) due to unilateral speed cutoff, the system executes a proportional scaling strategy. That is, the scaling factor is calculated as "motor physical limit / currently calculated maximum speed," and the speeds of both left and right wheels are multiplied by this factor simultaneously. This ensures that the motor is not overloaded, maintains the originally planned motion trajectory direction, and only reduces the overall inspection line speed. The combination of the left and right wheel speed data after the above processing generates the final visual tracking control vector, which is sent to the motor driver via the underlying CAN bus to guide the inspection motion of the magnetic robot.

[0057] This embodiment combines multivariable PID control with fuzzy logic, resulting in rapid response when deviations are large and smooth fine-tuning when deviations are small, effectively preventing the magnetic robot from oscillating on both sides of the weld. Through trapezoidal acceleration / deceleration planning and kinematic calculation, the control commands are ensured to conform to the robot's physical inertial characteristics, reducing mechanical impact and enabling the camera mounted on the robot to obtain a more stable video stream, thus improving the visual recognition accuracy of the weld.

[0058] Example 2: This embodiment provides a magnetic suction robot inspection system based on weld seam vision tracking for application in weld seam inspection using a magnetic suction robot. (Refer to...) Figure 2 The system includes a background noise filtering module, a weld seam feature recognition module, and a visual tracking inspection module.

[0059] The background noise filtering module performs illumination equalization processing on the continuous frame original images collected by the magnetic robot, and uses total variational regularization to decompose and filter the texture layer data containing rust, oil stains and paint noise in the continuous frame original images, while retaining the structural layer data containing weld geometry information. The weld seam feature recognition module uses the motion vectors of historical frame images and the Kalman filter algorithm to predict the region of interest in the current frame. The structural layer data of the region of interest is input into the dual-stream feature fusion semantic segmentation network. The spatial detail branch and the context semantic branch in the dual-stream feature fusion semantic segmentation network are used to extract weld seam features, and non-weld seam features are suppressed by an attention gating mechanism to output a mask image of the weld seam region. The visual tracking inspection module extracts the skeleton vector from the mask image, calculates the curvature change rate of the skeleton pixels and the perspective gradient rate of the weld mask width along the skeleton direction; based on the curvature change rate and perspective gradient rate, it constructs a mapping model between the relative position and posture deviation of the local weld and the magnetic robot; and combines the lateral position deviation and angular deviation of the skeleton centerline to generate the visual tracking control vector for the navigation magnetic robot inspection.

[0060] Furthermore, the background noise filtering module receives continuous frame images captured in real time by an industrial camera mounted on a magnetic robot via a high-speed data interface (such as GigE or USB 3.0). Since large metal structural components (such as spherical tanks and ship hulls) often exhibit uneven lighting and strong reflections due to curvature variations, the module does not process directly in the RGB color space. Instead, it first converts the continuous frame images to the YCbCr luminance-chrominance color space through a nonlinear transformation. In this space, the module separates the Y component, which contains only luminance information, and applies the Limiting Contrast Adaptive Histogram Equalization (CLAHE) algorithm to it. Specific parameter settings include, for example, dividing the image into an 8×8 pixel non-overlapping grid, setting a clipping threshold of 2.5 to prevent excessive amplification of noise in flat areas, calculating the local histogram of each grid and performing bilinear interpolation to generate an enhanced luminance component with uniform illumination distribution, and then merging it with the unmodified chrominance components (Cb, Cr) and inversely transforming it back to the RGB space to generate an evenly illuminated image. To extract the simple weld geometry from complex surface textures, the module utilizes total variational regularization to perform structural texture decomposition on the illumination-equalized image. Internally, the module constructs an energy minimization model based on variational methods, assuming that the input image is a linear superposition of a "structure layer" representing the object's geometric contours and a "texture layer" representing subtle repeating patterns and random noise. During computation, the module employs a primal-dual hybrid gradient algorithm to iteratively solve this energy minimization problem. The specific data processing flow is as follows: initialize the structure layer matrix and the dual variable matrix, and set the regularization weight parameter (e.g., a value of 0.15). In each iteration, the L1 norm of the image gradient is calculated as a regularization term. This regularization term has anisotropic diffusion characteristics, capable of smoothing small changes within the image while preserving the step characteristics of the image edges. After 50 to 100 rapid iterations and convergences, the pixel changes in the original image that exhibit high-frequency oscillations and lack geometric continuity, i.e., data containing rust, oil stains, and paint noise, are classified as texture layer data and stripped and filtered; while regions that exhibit significant pixel value jumps and spatial continuity, i.e., data containing geometric information such as weld edges and bevel shapes, are retained in the structure layer data.

[0061] Furthermore, the weld feature recognition module reads the weld image output from the previous historical frame image processing, calculates its geometric moments to determine the center position coordinates, and combines this with the historical motion vectors of the magnetic robot (including velocity and acceleration directions) to perform state estimation of the weld center position in the current frame using a Kalman filter algorithm. For example, at a sampling frequency of 30 frames per second, if the robot crawls uniformly upwards and to the right, the Kalman filter will predict, based on the state update equation of the previous moment, that the weld center will undergo a specific displacement in the image coordinate system at the current moment. Using this predicted position estimate as an anchor point, a rectangular window (e.g., a fixed-size window of 512×512 pixels) covering the potential displacement range of the weld is dynamically generated as the region of interest. The spatial coordinates of the region of interest are used to perform matrix cropping on the input structural layer data. The module feeds the cropped structural layer data as an input tensor to the built-in dual-stream feature fusion semantic segmentation network. This network runs in parallel on two independent branch paths: a spatial detail branch and a contextual semantic branch. In the spatial detail branch, the module uses stacked small-span (stride of 1) convolutional layers to process the input tensor, aiming to extract high-frequency edge and texture details without reducing the spatial resolution of the feature map, ensuring pixel-level localization accuracy of the weld edge. In parallel, in the contextual semantic branch, the module employs a continuous downsampling convolution strategy to rapidly reduce the size of the feature map (compressing 512×512 to 32×32), thereby significantly expanding the receptive field. This operation enables the network to capture the global topological structure of the image, i.e., "semantic context features," thus recognizing the overall extension trend of the weld rather than local noise. To fuse these two types of features and suppress background interference, the module upsamples the deep semantic context features, aligning their size with the shallow spatial feature map, and concatenates them along the channel dimension to generate multi-scale fused features. An attention gating mechanism is introduced, using the semantic context features as a guiding signal to calculate a spatial attention weight matrix, where each value represents the probability weight of a corresponding pixel belonging to the weld. The weight matrix is ​​then multiplied element-wise with the multi-scale fused features. In the preferred embodiment for processing backgrounds with severe corrosion, although the rust exhibits strong edge features in the spatial detail branch, it is judged as background in the contextual semantic branch due to a lack of long-distance continuity. Therefore, during the multiplication process, its corresponding weight value is reduced (tending towards 0), while the features of the weld area are enhanced (tending towards 1). The module performs convolutional compression and sigmoid activation on the attention-weighted feature map, outputting a probability distribution map, and performs binarization using a preset threshold (e.g., 0.5). The final output is a mask image containing only the white weld area and the black background area.

[0062] Furthermore, the visual tracking and inspection module processes the mask image using a morphological thinning algorithm, shrinking the weld area with a certain pixel width into a single-pixel-width central connected region through iterative erosion operations. An eight-neighbor chain code tracking strategy is used to traverse this single-pixel skeleton, converting discrete grid coordinates into an ordered skeleton vector sequence. Based on differential geometry principles, the tangent vector and normal vector of each node in this sequence are calculated, thereby generating curvature change rate data. The module searches for the mask edge boundaries along the normal vector direction of each node to both sides, calculates the Euclidean distance to construct a weld width distribution sequence, and performs gradient calculation on this sequence along the skeleton extension direction to generate perspective gradient rate data. Based on the curvature change rate and perspective gradient rate, a mapping model between the relative position and attitude deviation between the weld and the magnetic robot is constructed. This process recovers three-dimensional spatial information through monocular visual geometric constraints. The module uses the perspective gradient rate data, combined with the camera's intrinsic parameter matrix, to inversely solve for the spatial angle between the camera's optical axis and the weld surface normal, thereby determining the pitch attitude components of the magnetic robot. Using this spatial angle, the module generates a distortion correction matrix (homography matrix) and performs a reprojection transformation on the skeleton vector sequence, correcting the originally distorted "trapezoidal" field of view caused by perspective effects into an "orthographic topographic" field of view, restoring the true topological morphology of the weld in the local plane. Within this corrected space, the angle between the skeleton tangent and the robot's forward axis (vertical axis of the image center) is measured to resolve the yaw angle deviation; simultaneously, the vertical distance from the image principal point to the skeleton centerline is measured to resolve the lateral position deviation. These two deviations are input as feedback signals to a fuzzy PID controller, which uses a fuzzy logic algorithm to dynamically adjust the weights of the proportional, integral, and derivative coefficients based on the magnitude of the deviation. For example, in cases of large lateral deviations exceeding 10 mm, the algorithm automatically increases the weight of the lateral correction component to quickly return to center; while in the stable tracking phase with deviations less than 2 mm, the weight of the integral term is increased to eliminate steady-state errors. The module synthesizes the weighted and fused control quantities into a target angular velocity command, and performs inverse kinematics calculation based on the differential drive kinematics model of the magnetic robot (involving wheelbase and wheel diameter parameters), converting the target angular velocity and the reference inspection linear velocity into target rotational speed values ​​for the left and right drive wheels. Trapezoidal acceleration / deceleration smoothing and amplitude limiting are then applied to this rotational speed value to generate the final motor drive command, enabling the navigation magnetic robot to accurately inspect along the weld seam trajectory.

[0063] The background noise filtering module in this embodiment separates industrial noise (rust, oil) from target features (weld geometry) through total variational regularization decomposition. It preserves the edge sharpness of the weld, providing high-quality input data for subsequent neural network segmentation. Through illumination equalization processing, it effectively eliminates drastic illumination changes (such as highlights or shadows) caused by variations in the robot's crawling angle, ensuring that the output structural layer data has consistent grayscale distribution characteristics in both direct sunlight and backlit, dark areas, thereby guaranteeing the stability of the inspection system's all-weather operation.

[0064] The designed weld seam feature recognition module uses Kalman filtering to predict the region of interest, focusing the deep network's computation on the local area near the weld seam, meeting the high frame rate inspection requirements in industrial settings. A dual-stream architecture combined with an attention gating mechanism effectively improves the common problem of false targets in industrial weld seam inspection. Contextual semantic information is used to suppress linear scratches similar to the weld seam shape or edge artifacts caused by uneven lighting, maintaining segmentation accuracy even under harsh conditions with complex noise such as rust and oil stains.

[0065] The visual tracking inspection module utilizes perspective gradient as a depth cue to calculate the robot's pitch attitude from a 2D image. Reprojection transformation eliminates the distortion of path geometry (curvature, angle) caused by perspective distortion, enabling the robot to obtain accurate yaw angles and lateral errors, significantly improving tracking accuracy. The combination of fuzzy PID control and kinematic calculation achieves flexible control of the robot's trajectory. Trapezoidal acceleration and deceleration effectively avoids sudden changes in motor speed and mechanical vibration caused by fluctuations in visual signals, ensuring the smoothness of the inspection process.

[0066] Example 3: This embodiment fully deploys the aforementioned magnetic robot inspection method and system based on weld seam visual tracking onto a model X-200 magnetic wall-climbing robot, referring to... Figure 3 This enables automated inspection of weld seams on the surface of large petrochemical spherical tanks.

[0067] Furthermore, the magnetic robot adheres to the wall of the spherical tank, and an industrial camera captures continuous frames of raw images, including the weld seam, at a rate of 30 frames per second. Given the uneven illumination caused by the curvature of the tank surface, the system maps the raw RGB image to the YCbCr color space using a linear transformation matrix, separating the luminance component Y. A contrast-limited adaptive histogram equalization algorithm is used to enhance the luminance component, setting the grid size to 8×8 pixels and the clipping threshold to 2.5. Bilinear interpolation is used to suppress blockiness, and the image is then merged with the original chrominance component to reconstruct an illumination-equalized image. To remove complex texture noise, an energy minimization model including data fidelity terms and total variational regularization terms is constructed. This model is solved using an primal-dual iterative algorithm, setting the regularization weight parameter to 0.15 and the number of iterations to 80. During this process, the algorithm utilizes the sparse gradient characteristics of the total variational term to decompose the image gradient into structural and textural components. The final output structural layer data successfully removed rust spots and high-frequency noise from oil stains with a diameter of less than 5 pixels, while fully preserving the geometric step features of the weld edge, providing a high signal-to-noise ratio grayscale matrix input for subsequent processing.

[0068] Furthermore, the Kalman filter algorithm is used to dynamically track the weld position. A four-dimensional state vector containing the horizontal and vertical coordinates of the weld center and velocity components is established. Based on the geometric center of the weld parsed from the previous historical frame image (time t-1), combined with a preset uniform motion state transition matrix, the prior position of the weld in the current frame (time t) is predicted. Using this prior position as the center, the system crops a 512×512 pixel rectangular region of interest from the structural layer data to weaken irrelevant background interference at the edge of the field of view. The cropped data is fed into a dual-stream feature fusion semantic segmentation network. The spatial detail branch of this network uses stacked convolutional layers with a stride of 1 to extract high-frequency edge information while maintaining the 512×512 resolution of the feature map; the contextual semantic branch compresses the feature map size to 32×32 pixels through continuous downsampling convolution to obtain global semantic features covering the overall direction of the weld. In the fusion stage, the features from the two branches are weighted using an attention gating mechanism. A weight matrix (ranging from 0 to 1) generated from semantic information is used to multiply the spatial features point by point. This suppresses the weights of linear scratches disguised as welds (which typically lack global continuity semantics) to near 0, while the weights of the real weld regions tend to be 1. Finally, the network outputs a binarized mask image that clearly segments the weld regions.

[0069] Furthermore, skeleton vectorization extraction and pose calculation are performed on the mask image. A morphological thinning algorithm is used to shrink the weld mask of a certain width into a connected component skeleton of single-pixel width, and an eight-neighbor chain code tracking strategy is employed to transform the skeleton points into an ordered vector sequence. The system traverses this sequence, calculating the rate of curvature change of each node to identify the degree of weld curvature. Simultaneously, the weld edge is detected along the skeleton normal direction to obtain weld width distribution data. By analyzing the perspective gradient rate data and combining it with camera intrinsic parameters, the pitch angle between the camera optical axis and the weld tangent plane is inversely solved. This angle is used to generate a homography matrix to perform inverse perspective projection transformation on the skeleton pixels, correcting the originally trapezoidal distorted field of view to an orthographic top-down field of view.

[0070] Furthermore, the control deviation is calculated in the corrected three-dimensional Euclidean space. By measuring the angle between the corrected skeleton tangent and the vertical central axis of the image, the robot's yaw angle deviation is determined to be -4.5 degrees (i.e., yaw to the left); by measuring the vertical physical distance from the principal point of the image (optical center projection point) to the skeleton line, the lateral position deviation is determined to be 3.2 mm. These two deviation values ​​are input as feedback signals to the fuzzy PID controller. The fuzzy logic dynamically allocates weights according to the deviation amplitude: given the current large lateral deviation (greater than 2 mm), the controller automatically assigns a weight of 0.7 to the lateral correction component and a weight of 0.3 to the rotation correction component, synthesizing the target angular velocity command. Based on the robot's differential drive kinematic model (wheel track 300 mm, wheel diameter 100 mm), the target angular velocity and the reference inspection line velocity of 0.1 m / s are used to calculate the target rotational speed of the left and right drive wheels. The bottom control unit performs trapezoidal acceleration and deceleration smoothing on the target rotational speed, adjusts the speed of the drive motor, guides the magnetic robot to smoothly return and center the weld seam trajectory, and realizes automated inspection of the weld seam on the surface of a large petrochemical spherical tank.

[0071] This embodiment effectively separates common industrial texture noises such as rust and oil stains from weld structure information through total variational regularization, improving the failure problem under complex surface textures and significantly enhancing the robustness of image preprocessing. It introduces joint analysis of perspective gradient rate and skeleton curvature, and eliminates the impact of perspective distortion on path planning through reprojection transformation. This allows the robot to calculate the true lateral deviation and yaw angle even when encountering weld surface undulations or its own tilted posture, avoiding tracking oscillations caused by visual deception. Combining Kalman filtering for region of interest prediction with parallel processing of a two-stream neural network reduces computational load while ensuring a global understanding of weld semantics. Coupled with a fuzzy PID control strategy, it achieves rapid perception and flexible control of weld path changes, effectively preventing target loss during weld inspection.

[0072] It should be clarified that the embodiments described above are merely exemplary and are intended to aid in understanding the present invention, not to limit it. Those skilled in the art can make various changes and modifications after grasping the core ideas of the present invention. Therefore, the scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A magnetic suction robot inspection method based on weld seam visual tracking, characterized in that, include: The continuous frame original images acquired by the magnetic suction robot are subjected to illumination equalization processing, and the texture layer data containing rust, oil stains and paint noise in the continuous frame original images are decomposed and filtered using total variational regularization, while retaining the structural layer data containing weld geometry information. The region of interest (ROI) in the current frame is predicted using the Kalman filter algorithm based on the motion vectors of historical frame images. The structural layer data within the ROI is then input into a dual-stream feature fusion semantic segmentation network. Weld seam features are extracted using the spatial detail branch and contextual semantic branch of the dual-stream feature fusion semantic segmentation network, and non-weld seam features are suppressed using an attention gating mechanism to output a mask image of the weld seam region. The mask image is vectorized and extracted to calculate the curvature change rate of the skeleton pixels and the perspective gradient rate of the weld mask width along the skeleton direction. Based on the curvature change rate and perspective gradient rate, a mapping model between the relative position and posture deviation of the local weld and the magnetic robot is constructed. Combining the lateral position deviation and angular deviation of the skeleton centerline, a visual tracking control vector for the navigation magnetic robot inspection is generated.

2. The magnetic robot inspection method based on weld seam visual tracking according to claim 1, characterized in that, The specific implementation process of performing illumination equalization on continuous frame original images and decomposing and filtering texture layer data using total variational regularization to retain structural layer data includes: converting the acquired continuous frame original images to a luminance-chrominance color space and extracting the luminance component; processing the luminance component using a contrast-limited adaptive histogram equalization algorithm to generate an enhanced luminance component and reconstructing it into an illumination-equalized image; constructing an energy minimization model containing a total variational regularization term; solving the energy minimization model using an primal-dual iterative algorithm; preserving the edge geometric discontinuity of the weld by minimizing the total variational function; classifying rust, oil stains, and paint noise into texture layer data and filtering them; and outputting structural layer data containing only weld geometric information.

3. The magnetic robot inspection method based on weld seam visual tracking according to claim 1, characterized in that, The specific implementation process of predicting the region of interest (ROI) of the current frame using the Kalman filter algorithm based on the motion vectors of historical frame images, and inputting the structural layer data of the ROI into the dual-stream feature fusion semantic segmentation network includes: using the geometric center of the weld in the previous historical frame image as the observation input, correcting the prediction bias using the state update equation of the Kalman filter, and calculating the prior position estimate of the weld in the current frame based on the state transition matrix; using the prior position estimate as the anchor point, generating a rectangular window covering the potential displacement range of the weld as the ROI; using the spatial coordinates of the ROI to perform matrix clipping on the structural layer data, and inputting it into the dual-stream feature fusion semantic segmentation network.

4. The magnetic robot inspection method based on weld seam visual tracking according to claim 1, characterized in that, The specific implementation process of extracting weld seam features using the spatial detail branch and contextual semantic branch in the dual-stream feature fusion semantic segmentation network, and suppressing non-weld seam features through an attention gating mechanism to output a mask image of the weld seam region includes: feeding the structural layer data as an input tensor to the spatial detail branch, extracting spatial feature maps that preserve edge details while maintaining spatial resolution through stacked span convolutional layers; transmitting the input tensor in parallel to the contextual semantic branch, reducing the feature map size by continuous downsampling convolution and downsampling strategies to expand the effective receptive field, obtaining semantic contextual features encoding the weld seam topology, concatenating the spatial feature map with the upsampled semantic contextual features to generate multi-scale fusion features; the attention gating mechanism calculates the spatial attention weight matrix using the semantic contextual features as a guiding signal, and performs element-wise multiplication with the multi-scale fusion features to suppress non-weld seam background regions and enhance the weld seam region, and obtains the mask image of the weld seam region after binarization.

5. The magnetic robot inspection method based on weld seam visual tracking according to claim 1, characterized in that, The specific implementation process of extracting the skeleton vector from the mask image and calculating the curvature change rate of the skeleton pixels and the perspective gradient rate of the weld mask width along the skeleton direction includes: extracting the single-pixel connected domain skeleton reflecting the center direction of the weld using a morphological thinning algorithm; traversing the single-pixel connected domain skeleton using an eight-neighbor chain code tracking strategy to convert the discrete pixel coordinates into an ordered skeleton vector sequence; calculating the tangent vector and normal vector of each node on the skeleton vector sequence based on the differential geometry principle to generate the curvature change rate characterizing the degree of weld bending; searching the mask edges to both sides along the normal vector direction of each node in the skeleton vector sequence, calculating the Euclidean distance between the intersection of the normal and the mask contour, and constructing a weld width distribution sequence; performing gradient calculation on the weld width distribution sequence along the skeleton extension direction, analyzing the trend of width data changing with spatial depth, and generating perspective gradient rate data characterizing the visual perspective effect.

6. The magnetic robot inspection method based on weld seam visual tracking according to claim 1, characterized in that, Based on the curvature change rate and perspective gradient rate, the specific implementation process of constructing a mapping model between the relative position and attitude deviation of the local weld and the magnetic robot includes: by analyzing the attenuation trend of the width data in the longitudinal direction of the image, the spatial angle between the normal vector of the weld surface and the optical axis of the camera is solved, and the pitch attitude component of the magnetic robot relative to the current local weld cutting plane is calculated; a distortion correction matrix is ​​generated using the spatial angle, and a reprojection transformation is performed on the coordinate sequence of the skeleton pixels to map the image coordinates of the oblique view to the plane coordinates of the normal view, restoring the true topological shape of the weld in the local plane; the angle between the tangent direction vector of the true topological shape and the vertical axis of the image center is calculated, and the yaw angle deviation of the magnetic robot is analyzed; the normal distance from the principal point of the image to the center line of the reprojected skeleton is measured as the lateral position deviation; the pitch attitude component, yaw angle deviation, and lateral position deviation are associated to establish a mapping model.

7. The magnetic robot inspection method based on weld seam visual tracking according to claim 1, characterized in that, The specific implementation process of generating the visual tracking control vector for the inspection of the magnetic suction robot, combining the lateral position deviation and angular deviation of the skeleton centerline, includes: inputting the lateral position deviation and angular deviation as error feedback signals to a multivariable proportional-integral-derivative controller to calculate the lateral correction component and the rotational correction component; based on a fuzzy logic weighted algorithm, dynamically allocating weight coefficients according to the deviation amplitude, fusing the lateral correction component and the rotational correction component to synthesize a target angular velocity command; according to the differential drive kinematics model of the magnetic suction robot, performing inverse kinematics calculation on the target angular velocity command and the reference inspection linear velocity to convert it into the target rotational speed value of the left and right drive wheels; performing trapezoidal acceleration and deceleration smoothing processing and amplitude limiting on the target rotational speed value to generate a visual tracking control vector and guide the inspection motion of the magnetic suction robot.

8. A magnetic suction robot inspection system based on weld seam visual tracking, characterized in that, include: The background noise filtering module performs illumination equalization processing on the continuous frame original images collected by the magnetic robot, and uses total variational regularization to decompose and filter the texture layer data containing rust, oil stains and paint noise in the continuous frame original images, while retaining the structural layer data containing weld geometry information. The weld seam feature recognition module uses the motion vectors of historical frame images and the Kalman filter algorithm to predict the region of interest in the current frame. The structural layer data of the region of interest is input into the dual-stream feature fusion semantic segmentation network. The spatial detail branch and the context semantic branch in the dual-stream feature fusion semantic segmentation network are used to extract weld seam features, and non-weld seam features are suppressed by an attention gating mechanism to output a mask image of the weld seam region. The visual tracking inspection module extracts the skeleton vector from the mask image, calculates the curvature change rate of the skeleton pixels and the perspective gradient rate of the weld mask width along the skeleton direction; based on the curvature change rate and perspective gradient rate, it constructs a mapping model between the relative position and posture deviation of the local weld and the magnetic robot; and combines the lateral position deviation and angular deviation of the skeleton centerline to generate the visual tracking control vector for the navigation magnetic robot inspection.

Images