A method and system for visual inspection of discontinuous weld features guided by geometric continuity

Abstract

The present application provides a kind of discontinuous weld feature enhancement visual detection method and system guided by geometric continuity. First, the shallow detail features and deep semantic features of the weld image are extracted, the deep semantic features are mapped to the Hough parameter space, and the continuous geometric trend features are obtained. Then, the weld hybrid features are obtained by fusing the semantic-detail double-flow features, and the dynamic convolution kernel weights are generated using the continuous geometric trend features. The spatial sampling offset and sampling modulation generated by the weld hybrid features are combined to perform dynamic deformable convolution processing. Finally, the weld centerline coordinate point sequence is obtained by weld geometric trend regression. This method guides local feature adaptive enhancement with continuous geometric trend, taking into account the overall extension trend and local detail changes of the weld. It is generally applicable to discontinuous welds, especially in difficult working conditions such as very thin welds, weak features and local missing.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning, and specifically relates to a geometric continuity-guided method and system for enhancing visual detection of discontinuous weld features. Background Technology

[0002] In nuclear power plant construction, aerospace, and other industrial manufacturing sectors, welds are prone to discontinuous or missing sections due to factors such as spot welding, surface grinding, and partial masking. The actual width of such welds can vary depending on the joint type, plate thickness, machining precision, and imaging scale. In the context of fine welding of thin plates, the initial physical seam of butt joints can be as narrow as less than 0.2 mm. Spot welding and surface grinding before welding can further mask local gaps, thus forming representative extremely fine discontinuous welds.

[0003] For the detection of intermittent welds, local defects disrupt the continuous response of the weld in the image. Metal reflections, dust adhesion, motion blur, and low-contrast backgrounds further contribute to weak responses and background confusion. Existing laser scanning vision sensors are limited by spot size and resolution, making it difficult to stably capture the geometric contours of the weld, and the equipment costs are high. Contact sensing and arc sensing methods rely on the continuous physical response of the weld, which is prone to signal interruption and positioning drift in areas with local grinding defects, discontinuities, or weak features. In contrast, two-dimensional vision inspection has advantages such as being non-contact, providing rich information, and having lower deployment costs. It can perceive the weld location through image texture, grayscale changes, and edge details, making it suitable for online detection of such intermittent welds.

[0004] Existing two-dimensional visual inspection methods are still insufficient to fully meet the needs of weld seam inspection. Traditional image processing methods typically rely on local features such as grayscale thresholding, edge gradients, connected components, or geometric fitting, which lack robustness to changes in lighting, metal reflection, and grinding texture interference. While conventional pixel-level segmentation networks can improve feature representation, they essentially treat the weld seam as a set of pixel probabilities, lacking explicit constraints on the overall topological structure and geometric trend of slender weld seams, and are prone to trajectory breaks in discontinuous or curvature-varying regions. Although some structured inspection methods can utilize geometric priors to model the overall line shape, their ability to uniformly adapt to straight weld seams, curved weld seams, and local deformation changes remains limited. Therefore, there is an urgent need for a visual inspection method and system that can guide local feature enhancement using overall geometric continuity and remain adaptable to discontinuous weld seams of different widths and shapes.

[0005] To address the aforementioned problems, this invention proposes a geometrically continuous guided visual detection method and system for enhancing the features of intermittent welds, specifically targeting issues such as local missing features, varying feature strengths, and complex deformations during the detection of intermittent welds. This method obtains hybrid weld features through semantic-detail dual-stream feature fusion and generates dynamic convolution kernel weights using continuous geometric trend features. Combined with spatial sampling offsets and sampling modulation values ​​generated from the hybrid weld features, the hybrid weld features undergo dynamic deformable convolution processing. This guides adaptive enhancement of local features through continuous geometric trends, balancing the overall extension trend of the weld with local detail variations. The method is applicable to intermittent welds of different widths and shapes, exhibiting good detection stability and trajectory continuity, especially under challenging conditions such as extremely fine welds, weak features, and local missing features. Summary of the Invention

[0006] To address the aforementioned problems, this invention proposes a geometric continuity-guided method and system for enhancing visual inspection of discontinuous weld features. The method comprises the following steps:

[0007] Step S1: Obtain the image of the weld to be inspected;

[0008] Step S2: Input the image of the weld to be detected into the backbone feature extraction network to extract multi-scale image features including shallow detail features and deep semantic features;

[0009] Furthermore, step S2 specifically includes the following steps:

[0010] Step 2-1: Input the image of the weld to be detected into the backbone feature extraction network, and extract multi-scale image features through the backbone feature extraction network; wherein, the backbone feature extraction network includes at least one of convolutional neural network and Transformer-based feature extraction network;

[0011] Step 2-2: The multi-scale image features include first-scale image features, second-scale image features, third-scale image features, and fourth-scale image features, ranging from shallow to deep.

[0012] Steps 2-3: Use the first-scale image features as the shallow detail features to characterize the weld edge, texture, grayscale changes near the center line, and local spatial location information;

[0013] Steps 2-4: Use the second-scale image features, the third-scale image features, and the fourth-scale image features as the deep semantic features to characterize the overall structural semantics of the weld, long-distance contextual relationships, and macroscopic extension trends.

[0014] Step S3: Input the deep semantic features into the continuous geometric trend feature extraction module, and map the deep semantic features to the Hough parameter space through deep Hough transform to obtain continuous geometric trend features including Hough space feature map and Hough response map.

[0015] Furthermore, step S3 specifically includes the following steps:

[0016] Step 3-1: Input the deep semantic features into the continuous geometric trend feature extraction module;

[0017] Step 3-2: Map the deep semantic features from the image feature space to the Hough parameter space using a deep Hough transform;

[0018] Step 3-3: In the Hough parameter space, aggregate the responses that are in the neighborhood of the same or similar trend parameters to obtain the Hough space feature map and the Hough response map.

[0019] Step S4: Input the shallow detail features and the deep semantic features into the semantic-detail dual-stream feature fusion neck; wherein, the deep semantic features are used to extract the semantic feature stream through the semantic branch, the shallow detail features are used to extract the detail feature stream through the detail branch, and the semantic feature stream and the detail feature stream are fused through the fusion branch to obtain the weld seam hybrid features;

[0020] Furthermore, step S4 specifically includes the following steps:

[0021] Step 4-1: Input the deep semantic features into the semantic branch, and obtain the semantic feature stream after scale alignment, channel concatenation and convolution mapping;

[0022] Step 4-2: Input the shallow detail features into the detail branch, and obtain the detail feature flow after convolutional mapping and coordinate attention processing;

[0023] Step 4-3: Input the semantic feature stream and the detail feature stream into the fusion branch, and obtain the weld seam hybrid feature after channel splicing and convolution fusion.

[0024] Step S5: Input the weld seam hybrid feature and the continuous geometric trend feature into the continuous geometric guided dynamic deformable convolution module, generate spatial sampling offset and sampling modulation amount according to the weld seam hybrid feature, generate dynamic convolution kernel weights according to the continuous geometric trend feature, and input the spatial sampling offset, the sampling modulation amount and the dynamic convolution kernel weights into the dynamic deformable convolution unit to perform dynamic deformable convolution processing on the weld seam hybrid feature to obtain the weld seam enhancement feature;

[0025] Further, in step S5, the continuous geometrically guided dynamic deformable convolution module includes a weight mapping branch, an offset prediction branch, a modulation prediction branch, and the dynamic deformable convolution unit. Step S5 specifically includes the following steps:

[0026] Step 5-1: Input the continuous geometric trend features into the weight mapping branch. The weight mapping branch determines the Hough parameter points corresponding to the weld trend based on the Hough response map, and extracts the Hough feature vector from the Hough space feature map at the position corresponding to the Hough parameter points. Then, input the Hough feature vector into the parameter mapping network to generate the dynamic convolution kernel weights.

[0027] Step 5-2: Input the weld seam hybrid features into the offset prediction branch, generate multiple sampling point offset components in the horizontal and vertical directions through convolution mapping, and form the spatial sampling offset by the offset components;

[0028] Step 5-3: Input the weld seam hybrid features into the modulation prediction branch, and generate the sampling modulation amount corresponding to multiple sampling points through convolution mapping and Sigmoid activation;

[0029] Step 5-4: Input the dynamic convolution kernel weights, the spatial sampling offset, and the sampling modulation amount into the dynamic deformable convolution unit; the dynamic deformable convolution unit determines the offset sampling position according to the spatial sampling offset, adjusts the response intensity of the corresponding sampling position according to the sampling modulation amount, and performs weighted fusion of the sampling features according to the dynamic convolution kernel weights to obtain the weld enhancement features.

[0030] Step S6: Input the weld enhancement features into the weld geometric trend regression head, first infer the weld semantic segmentation response map, then extract the trend features from the weld semantic segmentation response map through local detail branch and continuous repair branch, and regress the weld centerline coordinate point sequence based on the extracted trend features.

[0031] Furthermore, step S6 specifically includes the following steps:

[0032] Step 6-1: Input the weld enhancement features into the curve decoder in the weld geometric trend regression head. The curve decoder improves the feature map resolution through transposed convolution and aligns the feature map to the size of the weld image to be detected through upsampling. Then, it generates the weld semantic segmentation response map through convolution mapping.

[0033] Step 6-2: Input the weld semantic segmentation response map into the local detail branch, and extract the local edge change features of the weld centerline neighborhood through one-dimensional asymmetric convolution with a short receptive field;

[0034] Step 6-3: Input the weld semantic segmentation response map into the continuous repair branch, and extract the continuous trend repair features in the weld extension direction through pooling and one-dimensional asymmetric convolution with a long receptive field;

[0035] Step 6-4: Perform residual fusion of the local edge change features and the continuous trend repair features to obtain the weld trend enhancement response map;

[0036] Step 6-5: Use the Soft-Argmax coordinate regression method to perform weighted coordinate regression on the weld trend enhancement response map along the image width direction to obtain the horizontal coordinate of the weld center point corresponding to each image row, and generate the weld center line coordinate point sequence based on the horizontal coordinate of the weld center point corresponding to each image row.

[0037] Furthermore, training is performed using a joint loss function, specifically including the following steps:

[0038] The joint loss function includes continuous geometric structure representation loss, weld semantic segmentation response map localization loss, and weld trend regression loss.

[0039] The continuous geometric structure characterization loss To constrain the network's response distribution in the Hough parameter space, enabling the network to learn the overall geometric trend and continuous topology of the weld seam, the loss representing the continuous geometric structure is expressed using a binary cross-entropy loss function based on the Logits form, whose expression is:

[0040] ;

[0041] in, Describe the first in the Hough parameter space The predicted Logit value for each location, This represents the actual label at the corresponding location. This represents the Sigmoid function. This indicates the number of locations in the Hough parameter space response plot;

[0042] The localization loss of the semantic segmentation response map of the weld. To constrain the consistency between the weld semantic segmentation response map and the actual weld centerline annotation, so as to improve the pixel-level positioning accuracy of the weld center position, the positioning loss of the weld semantic segmentation response map adopts a binary cross-entropy loss function based on Logits, the expression of which is:

[0043] ;

[0044] in, The first element in the semantic segmentation response graph of the weld seam The predicted Logit value for each location, The label indicates the actual weld centerline at the corresponding location. This represents the number of positions in the semantic segmentation response map of the weld seam;

[0045] The weld trend regression loss The trend prediction results output by the regression head used to constrain the geometric trend of the weld are used to enable the network to fit the local bending, deflection, and irregular trend changes of the weld. The weld trend regression loss adopts the smoothed L1 loss function, the expression of which is:

[0046] ;

[0047] in, Indicates the first The weld seam trend value predicted by the network at each location. This represents the actual weld trend value at the corresponding location. This represents the smoothing coefficient, used to control the switching range of the loss function between the quadratic and linear intervals;

[0048] The joint loss function Represented as:

[0049] ;

[0050] in, , , These represent the weighting coefficients of each loss term.

[0051] The present invention also provides a geometric continuity-guided visual inspection system for enhancing the features of discontinuous welds, comprising:

[0052] A weld seam image acquisition unit is used to acquire weld seam images; wherein, the weld seam image acquisition unit includes an industrial camera, a lens, and an optional filter component or auxiliary light source, the filter component is used to suppress welding arc light, metal reflection or ambient light interference, and the auxiliary light source is used to supplement the lighting of the weld seam area;

[0053] The system also includes an edge computing unit for receiving the weld image acquired by the weld image acquisition unit and running a weld visual inspection program; wherein the edge computing unit includes an image preprocessing module, a weld visual inspection module, a coordinate calibration and transformation module, a trajectory stitching and anomaly repair module, and a detection result output module.

[0054] The image preprocessing module is used to perform format conversion, size adjustment, normalization processing, and region of interest cropping on the weld image to obtain the weld image to be detected.

[0055] The weld visual inspection module is used to perform weld visual inspection on the image of the weld to be inspected, and obtain the sequence of coordinate points of the weld centerline in the image coordinate system;

[0056] The coordinate calibration and conversion module is used to establish a coordinate mapping relationship between the image coordinate system and the robot coordinate system based on the camera calibration parameters of the industrial camera, the calibration mapping relationship between the image coordinate system and the workpiece surface coordinate system where the weld is located, and the fixed installation calibration relationship between the industrial camera and the actuator, and to convert the weld centerline coordinate point sequence in the image coordinate system into a weld trajectory point sequence in the robot coordinate system.

[0057] The trajectory stitching and anomaly repair module is used to cache, match and stitch the sequence of weld trajectory points corresponding to multiple consecutive frames of the weld seam image to be detected, and to remove or repair abnormal weld seam trajectory points, abnormal frame detection results or discontinuous trajectory segments to obtain a continuous weld seam detection trajectory.

[0058] The detection result output module is used to output the continuous weld detection trajectory information to the host computer according to a preset data format.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] 1. This invention addresses the problems of weak features, local missing features, and local complex deformation in intermittent welds by creatively constructing a dynamic deformable feature enhancement mechanism guided by geometric continuity, thereby improving the adaptability to welds of different widths and shapes.

[0061] 2. This invention obtains weld seam hybrid features through semantic-detail dual-stream feature fusion, and uses continuous geometric trend features as global guidance to perform dynamic deformable convolution processing on the weld seam hybrid features, thereby enhancing the model's ability to jointly perceive the overall extension trend of the weld seam and local detail changes, thus improving the accuracy, stability and trajectory continuity of intermittent weld seam centerline detection.

[0062] 3. This invention is applicable to intermittent welds of different widths and shapes. It has good detection stability and trajectory continuity, especially in difficult working conditions such as extremely fine welds, weak features, and local defects. It can provide a visual perception basis for weld positioning and tracking control. Attached Figure Description

[0063] Figure 1 This is a network structure diagram of the method involved in this invention;

[0064] Figure 2 It is a neck structure map fused from semantic and detail dual-stream features;

[0065] Figure 3This is a structural diagram of a continuously geometrically guided dynamically deformable convolution module;

[0066] Figure 4 It is a regression diagram of the weld geometry trend;

[0067] Figure 5 These are comparison charts of weld inspection results under different working conditions;

[0068] Figure 6 This is a flowchart of the online testing phase. Detailed Implementation

[0069] To fully illustrate the geometric continuity-guided visual inspection method and system for enhancing the features of discontinuous welds proposed in this invention, a specific embodiment is introduced: This invention is applicable to the detection task of extremely fine discontinuous welds in submerged arc welding trolley operation scenarios, and is suitable for the geometric continuity-guided visual inspection method and system for enhancing the features of discontinuous welds. The invention will be further described in detail below with reference to the accompanying drawings.

[0070] In this embodiment, the system includes a weld seam image acquisition unit and an edge computing unit, wherein the edge computing unit runs a weld seam visual inspection program. Specifically:

[0071] A weld seam image acquisition unit is used to acquire weld seam images. This unit includes an industrial camera, a lens, and optionally a filter assembly or an auxiliary light source. The filter assembly is used to suppress interference from welding arc light, metal reflection, or ambient light. The auxiliary light source is used to provide supplemental lighting to the weld seam area. In this embodiment, the industrial camera is an MV-CE020-10 industrial camera, fixedly mounted near the welding execution end of the submerged arc welding carriage. The imaging plane of the industrial camera is substantially parallel to the surface of the workpiece to be welded, and its field of view covers the weld seam area to be inspected. This allows for real-time acquisition of the weld seam image on the surface of the workpiece before welding to obtain weld seam trend information.

[0072] An edge computing unit is used to receive the weld seam image acquired by the weld seam image acquisition unit and run the weld seam visual inspection program. The edge computing unit, according to the weld seam visual inspection program, includes an image preprocessing module, a weld seam visual inspection module, a coordinate calibration and transformation module, a trajectory stitching and anomaly repair module, and a detection result output module. In this embodiment, the edge computing unit uses the Jetson Xavier edge computing platform to receive the weld seam image acquired by the industrial camera and deploy a trained visual inspection model to achieve online perception and inference of the weld seam trend. The Jetson Xavier edge computing platform is connected to the industrial camera via USB 3.0 and interacts with the control module of the submerged arc welding carriage via a ModBus bus.

[0073] During the model training preparation phase, weld images containing both straight and curved welds were collected to construct a weld image dataset, which was then divided into a training dataset and a test dataset in an 8:2 ratio. The weld image dataset included weld images under five different conditions: no interference, occlusion, strong light, blur, and dust. The weld centerline in the weld images was labeled to obtain weld centerline coordinate labels. Based on these centerline coordinate labels, a weld semantic segmentation response map label was generated, and further, a continuous geometric trend label was generated to constrain the Hough parameter space response distribution.

[0074] During the model training phase, based on Figure 1 The network structure shown is used to construct the visual detection model. The training process of the visual detection model specifically includes the following steps:

[0075] Step S1: Obtain the image of the weld to be inspected; specifically, the image preprocessing module receives the weld image, converts the weld image to JPG format, adjusts it to 1920×1080 pixels, and performs normalization processing to obtain the image of the weld to be inspected.

[0076] Step S2: Input the image of the weld to be inspected into the backbone feature extraction network to extract multi-scale image features including shallow detail features and deep semantic features; wherein, the backbone feature extraction network is as follows: Figure 1 The backbone feature extraction network of the network structure shown.

[0077] Furthermore, step S2 specifically includes the following steps:

[0078] Step 2-1: Input the image of the weld to be detected into the backbone feature extraction network, and extract multi-scale image features through the backbone feature extraction network; wherein, the backbone feature extraction network includes at least one of convolutional neural network and Transformer-based feature extraction network;

[0079] Step 2-2: The multi-scale image features include first-scale image features, second-scale image features, third-scale image features, and fourth-scale image features, ranging from shallow to deep.

[0080] Steps 2-3: Use the first-scale image features as the shallow detail features to characterize the weld edge, texture, grayscale changes near the center line, and local spatial location information;

[0081] Steps 2-4: Use the second-scale image features, the third-scale image features, and the fourth-scale image features as the deep semantic features to characterize the overall structural semantics of the weld, long-distance contextual relationships, and macroscopic extension trends.

[0082] Step S3: Input the deep semantic features into the continuous geometric trend feature extraction module, and map the deep semantic features to the Hough parameter space through deep Hough transform to obtain continuous geometric trend features including the Hough space feature map and the Hough response map; wherein, the continuous geometric trend feature extraction module is as follows: Figure 1 The module for extracting continuous geometric trend features of the network structure shown.

[0083] Furthermore, step S3 specifically includes the following steps:

[0084] Step 3-1: Input the deep semantic features into the continuous geometric trend feature extraction module;

[0085] Step 3-2: Map the deep semantic features from the image feature space to the Hough parameter space using a deep Hough transform;

[0086] Step 3-3: In the Hough parameter space, aggregate the responses that are in the neighborhood of the same or similar trend parameters to obtain the Hough space feature map and the Hough response map.

[0087] Step S4: Input the shallow detail features and the deep semantic features into the semantic-detail dual-stream feature fusion neck; wherein, the deep semantic features are processed through a semantic branch to extract a semantic feature stream, the shallow detail features are processed through a detail branch to extract a detail feature stream, and the semantic feature stream and the detail feature stream are fused through a fusion branch to obtain the weld seam hybrid features; wherein, the semantic-detail dual-stream feature fusion neck is as follows: Figure 1 The semantic-detail dual-stream feature fusion neck of the network structure shown is as follows: Figure 2 As shown.

[0088] Furthermore, step S4 specifically includes the following steps:

[0089] Step 4-1: Input the deep semantic features into the semantic branch, and obtain the semantic feature stream after scale alignment, channel concatenation and convolution mapping;

[0090] Step 4-2: Input the shallow detail features into the detail branch, and obtain the detail feature flow after convolutional mapping and coordinate attention processing;

[0091] Step 4-3: Input the semantic feature stream and the detail feature stream into the fusion branch, and obtain the weld seam hybrid feature after channel splicing and convolution fusion.

[0092] Step S5: Input the weld seam hybrid feature and the continuous geometric trend feature into the continuous geometrically guided dynamic deformable convolution module. Generate a spatial sampling offset and a sampling modulation amount based on the weld seam hybrid feature, and generate dynamic convolution kernel weights based on the continuous geometric trend feature. Input the spatial sampling offset, the sampling modulation amount, and the dynamic convolution kernel weights into the dynamic deformable convolution unit to perform dynamic deformable convolution processing on the weld seam hybrid feature to obtain the weld seam enhancement feature; wherein, the continuous geometrically guided dynamic deformable convolution module is as follows: Figure 1 The network structure shown is a continuous geometrically guided dynamically deformable convolutional module. The specific structure of the continuous geometrically guided dynamically deformable convolutional module is as follows: Figure 3 As shown.

[0093] Further, in step S5, the continuous geometrically guided dynamic deformable convolution module includes a weight mapping branch, an offset prediction branch, a modulation prediction branch, and the dynamic deformable convolution unit. Step S5 specifically includes the following steps:

[0094] Step 5-1: Input the continuous geometric trend features into the weight mapping branch. The weight mapping branch determines the Hough parameter points corresponding to the weld trend based on the Hough response map, and extracts the Hough feature vector from the Hough space feature map at the position corresponding to the Hough parameter points. Then, input the Hough feature vector into the parameter mapping network to generate the dynamic convolution kernel weights.

[0095] Step 5-2: Input the weld seam hybrid features into the offset prediction branch, generate multiple sampling point offset components in the horizontal and vertical directions through convolution mapping, and form the spatial sampling offset by the offset components;

[0096] Step 5-3: Input the weld seam hybrid features into the modulation prediction branch, and generate the sampling modulation amount corresponding to multiple sampling points through convolution mapping and Sigmoid activation;

[0097] Step 5-4: Input the dynamic convolution kernel weights, the spatial sampling offset, and the sampling modulation amount into the dynamic deformable convolution unit; the dynamic deformable convolution unit determines the offset sampling position according to the spatial sampling offset, adjusts the response intensity of the corresponding sampling position according to the sampling modulation amount, and performs weighted fusion of the sampling features according to the dynamic convolution kernel weights to obtain the weld enhancement features.

[0098] Step S6: Input the weld enhancement features into the weld geometric trend regression head, first infer the weld semantic segmentation response map, then extract trend features from the weld semantic segmentation response map through local detail branches and continuous repair branches, and regress the weld centerline coordinate point sequence based on the extracted trend features; wherein, the weld geometric trend regression head is as follows: Figure 1 The weld geometry trend regression head of the network structure shown, the specific structure of the local detail branch and the continuous repair branch in the weld geometry trend regression head is as follows: Figure 4 As shown.

[0099] Furthermore, step S6 specifically includes the following steps:

[0100] Step 6-1: Input the weld enhancement features into the curve decoder in the weld geometric trend regression head. The curve decoder improves the feature map resolution through transposed convolution and aligns the feature map to the size of the weld image to be detected through upsampling. Then, it generates the weld semantic segmentation response map through convolution mapping.

[0101] Step 6-2: Input the weld semantic segmentation response map into the local detail branch, and extract the local edge change features of the weld centerline neighborhood through one-dimensional asymmetric convolution with a short receptive field;

[0102] Step 6-3: Input the weld semantic segmentation response map into the continuous repair branch, and extract the continuous trend repair features in the weld extension direction through pooling and one-dimensional asymmetric convolution with a long receptive field;

[0103] Step 6-4: Perform residual fusion of the local edge change features and the continuous trend repair features to obtain the weld trend enhancement response map;

[0104] Step 6-5: Use the Soft-Argmax coordinate regression method to perform weighted coordinate regression on the weld trend enhancement response map along the image width direction to obtain the horizontal coordinate of the weld center point corresponding to each image row, and generate the weld center line coordinate point sequence based on the horizontal coordinate of the weld center point corresponding to each image row.

[0105] Furthermore, training is performed using a joint loss function, specifically including the following steps:

[0106] The joint loss function includes continuous geometric structure representation loss, weld semantic segmentation response map localization loss, and weld trend regression loss.

[0107] The continuous geometric structure characterization loss To constrain the network's response distribution in the Hough parameter space, enabling the network to learn the overall geometric trend and continuous topology of the weld seam, the loss representing the continuous geometric structure is expressed using a binary cross-entropy loss function based on the Logits form, whose expression is:

[0108] ;

[0109] in, Describe the first in the Hough parameter space The predicted Logit value for each location, This represents the actual label at the corresponding location. This represents the Sigmoid function. This indicates the number of locations in the Hough parameter space response plot;

[0110] The localization loss of the semantic segmentation response map of the weld. To constrain the consistency between the weld semantic segmentation response map and the actual weld centerline annotation, so as to improve the pixel-level positioning accuracy of the weld center position, the positioning loss of the weld semantic segmentation response map adopts a binary cross-entropy loss function based on Logits, the expression of which is:

[0111] ;

[0112] in, The first element in the semantic segmentation response graph of the weld seam The predicted Logit value for each location, The label indicates the actual weld centerline at the corresponding location. This represents the number of positions in the semantic segmentation response map of the weld seam;

[0113] The weld trend regression loss The trend prediction results output by the regression head used to constrain the geometric trend of the weld are used to enable the network to fit the local bending, deflection, and irregular trend changes of the weld. The weld trend regression loss adopts the smoothed L1 loss function, the expression of which is:

[0114] ;

[0115] in, Indicates the first The weld seam trend value predicted by the network at each location. This represents the actual weld trend value at the corresponding location. This represents the smoothing coefficient, used to control the switching range of the loss function between the quadratic and linear intervals;

[0116] The joint loss function Represented as:

[0117] ;

[0118] in, , , These represent the weighting coefficients of each loss term.

[0119] To verify the detection performance of the proposed visual detection model, after model training, the method of this invention was compared with existing related detection methods using the test dataset. Evaluation metrics included F1-score, root mean square error (RMSE), first-order difference error between adjacent rows (FDE), and processing time; where F1-score evaluates overall detection accuracy, RMSE evaluates overall localization bias, and FDE evaluates overall fluctuation and continuity. Detection results are compared as follows: Figure 5 As shown in Table 1, the quantitative evaluation results are as follows.

[0120] Table 1 Quantitative Evaluation Results

[0121]

[0122] During the deployment phase, the model weight parameters obtained during the training phase are first saved. Then, based on the network structure of the visual detection model and the model weight parameters, an ONNX model file containing the network structure and weight parameters is exported. Subsequently, the TensorRT inference acceleration engine is used to parse the ONNX model file, optimize the computation graph, and build the inference engine, generating a TensorRT model file suitable for running on the Jetson Xavier edge computing platform. FP16 half-precision floating-point inference mode is enabled to improve edge computing throughput. After the conversion is complete, the TensorRT model file is loaded onto the Jetson Xavier edge computing platform.

[0123] During the online testing phase, the process is as follows: Figure 6 As shown, the specific steps include the following:

[0124] Image acquisition. The industrial camera acquires real-time images of the weld seam on the surface of the workpiece to be welded and transmits the weld seam images to the edge computing unit.

[0125] Image preprocessing. The image preprocessing module in the edge computing unit receives the weld image, converts it to JPG format, adjusts it to 1920×1080 pixels, and then performs normalization processing to obtain the weld image to be detected.

[0126] Weld centerline detection. The weld visual detection module in the edge computing unit calls the trained visual detection model to perform forward inference on the weld image to be detected, and outputs the sequence of coordinate points of the weld centerline in the image coordinate system. Unlike the model training stage, the online detection stage only performs forward computation and does not update the model parameters.

[0127] Coordinate calibration transformation. After obtaining the weld centerline coordinate point sequence in the image coordinate system, the coordinate calibration transformation module in the edge calculation unit converts the pixel offset of the weld centerline coordinate point sequence relative to the image center position into the actual physical offset based on the proportional relationship between 1080 pixels in the image offset detection direction and the actual length of 41.6mm on the surface of the workpiece to be welded. Combined with the fixed installation calibration relationship of the industrial camera relative to the submerged arc welding carriage, a weld trajectory point sequence in the robot coordinate system is generated.

[0128] Anomaly removal and repair. The trajectory stitching and anomaly repair module in the edge computing unit caches the weld trajectory point sequence corresponding to the weld image to be detected in the current frame, removes low response intensity trajectory points based on response intensity, removes abrupt trajectory points based on trajectory curvature changes, and then repairs missing positions through interpolation to obtain the corrected weld trajectory point sequence for the current frame. In this embodiment, since a global positioning relationship between the submerged arc welding carriage and the world coordinate system has not been established, and the industrial camera moves with the submerged arc welding carriage, the cross-frame trajectory points do not have a unified coordinate reference. Therefore, multi-frame trajectory accumulation stitching is not performed. Instead, the relative offset relationship of the weld with respect to the industrial camera and the submerged arc welding carriage actuator is determined based on the detection results of the current frame.

[0129] The detection result output module in the edge computing unit outputs the corrected weld trajectory point sequence of the current frame to the host computer according to a preset data format for display, recording, and weld tracking control.

[0130] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A geometrically continuous guided visual inspection method for enhancing the features of discontinuous welds, characterized in that, Includes the following steps: Step S1: Obtain the image of the weld to be inspected; Step S2: Input the image of the weld to be detected into the backbone feature extraction network to extract multi-scale image features including shallow detail features and deep semantic features; Step S3: Input the deep semantic features into the continuous geometric trend feature extraction module, and map the deep semantic features to the Hough parameter space through deep Hough transform to obtain continuous geometric trend features including Hough space feature map and Hough response map. Step S4: Input the shallow detail features and deep semantic features into the semantic-detail dual-stream feature fusion neck; wherein, the deep semantic features are used to extract the semantic feature stream through the semantic branch, and the shallow detail features are used to extract the detail feature stream through the detail branch. The semantic feature stream and the detail feature stream are fused through the fusion branch to obtain the weld seam hybrid features; Step S5: Input the weld seam hybrid features and continuous geometric trend features into the continuous geometric guided dynamic deformable convolution module, generate spatial sampling offset and sampling modulation based on the weld seam hybrid features, generate dynamic convolution kernel weights based on the continuous geometric trend features, and input the spatial sampling offset, sampling modulation, and dynamic convolution kernel weights into the dynamic deformable convolution unit to perform dynamic deformable convolution processing on the weld seam hybrid features to obtain weld seam enhancement features; Step S6: Input the weld enhancement features into the weld geometric trend regression head, first infer the weld semantic segmentation response map, then extract the trend features from the weld semantic segmentation response map through local detail branch and continuous repair branch, and regress the weld centerline coordinate point sequence based on the extracted trend features.

2. The method for enhancing visual inspection of discontinuous weld features guided by geometric continuity according to claim 1, characterized in that, Step S2 specifically includes the following steps: Step 2-1: Input the image of the weld to be detected into the backbone feature extraction network, and extract multi-scale image features through the backbone feature extraction network; wherein, the backbone feature extraction network includes at least one of convolutional neural network and Transformer-based feature extraction network; Step 2-2: The multi-scale image features include first-scale image features, second-scale image features, third-scale image features, and fourth-scale image features, ranging from shallow to deep. Steps 2-3: Use the first-scale image features as shallow detail features to characterize weld edges, textures, grayscale changes near the center line, and local spatial location information; Steps 2-4: Use the second-scale image features, third-scale image features, and fourth-scale image features as deep semantic features to characterize the overall structural semantics of the weld, long-distance contextual relationships, and macroscopic extension trends.

3. The method for enhancing visual inspection of discontinuous weld features guided by geometric continuity according to claim 1, characterized in that, Step S3 specifically includes the following steps: Step 3-1: Input the deep semantic features into the continuous geometric trend feature extraction module; Step 3-2: Map the deep semantic features from the image feature space to the Hough parameter space using a deep Hough transform; Step 3-3: In the Hough parameter space, aggregate the responses that are in the neighborhood of the same or similar trend parameters to obtain the Hough space feature map and the Hough response map.

4. The method for enhancing visual inspection of discontinuous weld features guided by geometric continuity according to claim 1, characterized in that, Step S4 specifically includes the following steps: Step 4-1: Input the deep semantic features into the semantic branch, and obtain the semantic feature stream after scale alignment, channel concatenation and convolution mapping; Step 4-2: Input the shallow detail features into the detail branch, and obtain the detail feature flow after convolutional mapping and coordinate attention processing; Step 4-3: Input the semantic feature stream and detail feature stream into the fusion branch, and obtain the weld seam hybrid feature after channel splicing and convolution fusion.

5. The method for enhancing visual inspection of discontinuous weld features guided by geometric continuity according to claim 1, characterized in that, In step S5, the continuous geometrically guided dynamic deformable convolution module includes a weight mapping branch, an offset prediction branch, a modulation prediction branch, and a dynamic deformable convolution unit. Step S5 specifically includes the following steps: Step 5-1: Input the continuous geometric trend features into the weight mapping branch. The weight mapping branch determines the Hough parameter points corresponding to the weld trend based on the Hough response map, and extracts the Hough feature vectors from the Hough space feature map at the positions corresponding to the Hough parameter points. Then, input the Hough feature vectors into the parameter mapping network to generate dynamic convolution kernel weights. Step 5-2: Input the weld seam hybrid features into the offset prediction branch, generate multiple sampling point offset components in the horizontal and vertical directions through convolution mapping, and form a spatial sampling offset by the offset components; Step 5-3: Input the weld seam hybrid features into the modulation prediction branch, and generate the sampling modulation amount corresponding to several sampling points through convolution mapping and Sigmoid activation; Step 5-4: Input the dynamic convolution kernel weights, spatial sampling offset, and sampling modulation into the dynamic deformable convolution unit; the dynamic deformable convolution unit determines the offset sampling position according to the spatial sampling offset, adjusts the response intensity of the corresponding sampling position according to the sampling modulation, and performs weighted fusion of the sampling features according to the dynamic convolution kernel weights to obtain the weld enhancement features.

6. The method for enhancing visual inspection of discontinuous weld features guided by geometric continuity according to claim 1, characterized in that, Step S6 specifically includes the following steps: Step 6-1: Input the weld enhancement features into the curve decoder in the weld geometric trend regression head. The curve decoder improves the feature map resolution through transposed convolution and aligns the feature map to the size of the weld image to be detected through upsampling. Then, it generates a weld semantic segmentation response map through convolution mapping. Step 6-2: Input the weld semantic segmentation response map into the local detail branch, and extract the local edge change features of the weld centerline neighborhood through one-dimensional asymmetric convolution with a short receptive field; Step 6-3: Input the weld semantic segmentation response map into the continuous repair branch, and extract the continuous trend repair features in the weld extension direction through pooling and one-dimensional asymmetric convolution with a long receptive field; Step 6-4: Perform residual fusion of the local edge change features and continuous trend repair features to obtain the weld trend enhancement response map; Step 6-5: Use the Soft-Argmax coordinate regression method to perform weighted coordinate regression on the weld trend enhancement response map along the image width direction to obtain the horizontal coordinate of the weld center point corresponding to each image row, and generate a sequence of weld center line coordinate points based on the horizontal coordinate of the weld center point corresponding to each image row.

7. The method for enhancing visual inspection of discontinuous weld features guided by geometric continuity according to claim 1, characterized in that, This method also includes training using a joint loss function, as detailed below: The joint loss function includes continuous geometric structure representation loss, weld semantic segmentation response map localization loss, and weld trend regression loss. The continuous geometric structure characterization loss To constrain the network's response distribution in the Hough parameter space, enabling the network to learn the overall geometric trend and continuous topology of the weld seam, the loss representing the continuous geometric structure is achieved using a binary cross-entropy loss function based on the Logits form, the expression of which is: ； in, Describe the first in the Hough parameter space The predicted Logit value for each location, This represents the actual label at the corresponding location. This represents the Sigmoid function. This indicates the number of locations in the Hough parameter space response plot; The localization loss of the semantic segmentation response map of the weld. To constrain the consistency between the weld semantic segmentation response map and the actual weld centerline annotation, so as to improve the pixel-level positioning accuracy of the weld center position, the positioning loss of the weld semantic segmentation response map adopts a binary cross-entropy loss function based on Logits, the expression of which is: ； in, The first element in the semantic segmentation response graph of the weld seam The predicted Logit value for each location, The label indicates the actual weld centerline at the corresponding location. This represents the number of positions in the semantic segmentation response map of the weld seam; The weld trend regression loss The trend prediction results output by the regression head used to constrain the geometric trend of the weld are used to fit the local bending, deflection, and irregular trend changes of the weld. The weld trend regression loss adopts the smoothed L1 loss function, the expression of which is: ； in, Indicates the first The weld seam trend value predicted by the network at each location. This represents the actual weld trend value at the corresponding location. This represents the smoothing coefficient, used to control the switching range of the loss function between the quadratic and linear intervals; The joint loss function Represented as: ； in, , , These represent the weighting coefficients of each loss term.

8. A geometrically continuous guided visual inspection system for enhancing the features of discontinuous welds, characterized in that, The system includes: A weld seam image acquisition unit is used to acquire weld seam images. The weld seam image acquisition unit includes an industrial camera, a lens, and an optional filter component or auxiliary light source. The filter component is used to suppress welding arc light, metal reflection, or ambient light interference, and the auxiliary light source is used to supplement the lighting of the weld seam area. And an edge computing unit, used to receive the weld image acquired by the weld image acquisition unit and perform weld visual inspection; the edge computing unit includes an image preprocessing module, a weld visual inspection module, a coordinate calibration and transformation module, a trajectory stitching and anomaly repair module, and a detection result output module; The image preprocessing module is used to perform format conversion, size adjustment, normalization processing, and region of interest cropping on the weld image to obtain the weld image to be detected. The weld visual inspection module is used to perform weld visual inspection on the image of the weld to be inspected, and obtain the sequence of coordinate points of the weld centerline in the image coordinate system. The coordinate calibration and conversion module is used to establish a coordinate mapping relationship between the image coordinate system and the robot coordinate system based on the camera calibration parameters of the industrial camera, the calibration mapping relationship between the image coordinate system and the workpiece surface coordinate system where the weld is located, and the fixed installation calibration relationship between the industrial camera and the actuator, and to convert the weld centerline coordinate point sequence in the image coordinate system into a weld trajectory point sequence in the robot coordinate system. The trajectory stitching and anomaly repair module is used to cache, match and stitch the sequence of weld trajectory points corresponding to multiple consecutive frames of weld images to be detected, and to remove or repair abnormal weld trajectory points, abnormal frame detection results or discontinuous trajectory segments to obtain continuous weld detection trajectory. The detection result output module is used to output the continuous weld detection trajectory information to the host computer according to a preset data format.

Images