A robot welding guidance method and system based on weld geometric feature points

Abstract

The present application relates to the technical field of robot welding guidance, and proposes a robot welding guidance method and system based on weld geometric feature points, which comprises: collecting a weld image of a workpiece to be welded and preprocessing; labeling corresponding geometric feature points in the weld image data and performing data enhancement; constructing a lightweight multi-scale feature extraction network; inputting the enhanced weld image data into the trained network to output weld geometric feature point candidate results; matching and screening the candidate results according to the weld geometric topological relationship to obtain a weld geometric feature point set; completing weld geometric analysis to obtain weld geometric information and convert it into structured robot welding guidance parameters; and sending the guidance parameters to a robot controller to control the welding gun actuator to complete welding starting positioning, trajectory tracking and posture adjustment. The present application solves the problems of unstable weld feature point positioning, insufficient structural integrity and low reliability of guidance parameter generation in a complex environment.

Description

Technical Field

[0001] This invention relates to the field of robot welding guidance technology, and in particular to a robot welding guidance method and system based on weld geometric feature points. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Robotic welding guidance is a technology in robotic welding operations that uses sensors to collect scene information and algorithms to process it to obtain the actual position and shape of the weld seam. This information is then used to correct the robot's trajectory and guide the welding torch to accurately complete the welding process. Existing robotic welding guidance methods mainly include teach-and-playback, traditional visual inspection, and weld seam recognition methods based on deep learning.

[0004] Teaching and reproduction rely on preset trajectories, making it difficult to adapt to workpiece clamping errors, welding thermal deformation, and changes in weld position, resulting in insufficient flexibility and adaptability. Traditional visual inspection typically extracts weld features through threshold segmentation, edge extraction, centerline fitting, or template matching. The effectiveness depends on manually set parameters and rules. During the welding process, strong arc light, spatter, smoke, overexposure reflection, and local occlusion can affect the laser stripes and weld edges, causing them to easily break, blur, or even overlap with noise, leading to unstable extraction of weld feature points.

[0005] In recent years, deep learning has improved the ability to extract weld features in complex backgrounds, but it still has shortcomings: First, most methods only output weld regions, laser stripe segmentation results, or detection boxes, and subsequent rule processing is still required to obtain the welding start point, centerline, and robot guidance parameters. Second, weld geometric feature points are small in scale, have weak texture, are densely distributed, and have complex structural relationships. Ordinary detection networks are prone to losing shallow details after multiple downsampling, resulting in missed detection of key points, positioning misalignment, or confusion of adjacent key points. At the same time, some deep models have a large number of parameters and computational load, which is not conducive to real-time deployment in industrial sites. In addition, existing data augmentation methods mostly use general transformation methods, which cannot effectively simulate the actual interference in the welding site and may also damage the key geometric structure of the weld, ultimately leading to unstable positioning of weld feature points and insufficient structural integrity in complex environments, and low reliability of guidance parameter generation. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention provides a robot welding guidance method and system based on weld geometric feature points, which solves the problems of unstable positioning of weld feature points, insufficient structural integrity, and low reliability of guidance parameter generation in complex environments.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a robot welding guidance method based on weld geometric feature points, comprising: Acquire and preprocess weld seam images of the workpiece to be welded to obtain standardized weld seam image data; Based on the weld geometry, corresponding geometric feature points are marked in the weld image data, and online physical noise data enhancement is performed; A lightweight multi-scale feature extraction network is constructed, and joint loss is used to complete the network training; The enhanced weld image data is input into the trained network to enhance the expression of small-scale weld geometric feature points, retain the detailed information of weld edges, laser stripes and key feature point neighborhoods, and output the candidate results of weld geometric feature points. Based on the geometric topology of the weld, the candidate results are matched and filtered to obtain the set of geometric feature points of the weld; Weld geometry is analyzed based on the set of weld geometric feature points to obtain weld geometric information and convert it into structured robot welding guidance parameters. The guidance parameters are sent to the robot controller, which controls the welding torch actuator to complete the welding start positioning, trajectory tracking and attitude adjustment based on the weld start point, center line and attitude correction amount.

[0008] As a further technical solution, the lightweight multi-scale feature extraction network includes a backbone network feature extraction part, a neck network feature fusion part, and a multi-scale output part; The backbone network feature extraction part includes a shallow feature extraction part and a deep feature extraction part. The shallow feature extraction part is equipped with an SPDConv module and a high-resolution feature layer. The deep feature extraction part and the neck network feature fusion part are equipped with a Faster_C3K2 module. The multi-scale output part is equipped with multiple Detect branches.

[0009] As a further technical solution, the specific process for outputting candidate results of weld geometric feature points includes: The weld image is input into the lightweight multi-scale feature extraction network to obtain weld feature fusion results at different scales. The multi-scale output part is used to predict the weld feature fusion results and output the center coordinates, confidence level, and weld geometric feature point number of the local response window corresponding to each weld geometric feature point. The center coordinates of the local response window are used as the position of the weld geometric feature point, and low confidence results are removed according to the confidence threshold to obtain candidate results of weld geometric feature points.

[0010] As a further technical solution, the geometric feature points include, but are not limited to, the intersection of weld edges, the intersection of bevels, the lap edge points, the blunt edge endpoints, and the relevant positions of the welding start points.

[0011] As a further technical solution, the joint loss includes WIoU v3 loss and NWD loss. WIoU v3 loss is used to perform dynamic weighted regression based on the overlap quality and center distance between the predicted box and the ground truth box. NWD loss is used to measure the distributional similarity between the predicted box and the ground truth box from the perspective of box center distance and width-height difference.

[0012] As a further technical solution, the matching and screening of candidate results includes: for duplicate points, retaining the ordered set of feature points that are most consistent with the weld structure or have a high confidence level; for locally missing points, performing structural compensation or anomaly judgment by combining the positions of adjacent feature points and weld geometric constraints. The weld geometry analysis includes: based on the set of weld geometric feature points after matching and filtering, establishing the topological connection relationship between feature points according to the weld type, analyzing the weld edge, bevel structure, lap boundary or fillet weld intersection position, calculating the weld centerline, welding start point, weld tracking direction and local weld posture information through geometric relationships, and generating guiding parameters based on the weld geometry analysis results.

[0013] As a further technical solution, the guiding parameters include, but are not limited to, welding start point, weld centerline trajectory point, local weld tracking direction, welding torch target posture matrix, position deviation, posture correction, and quality mark.

[0014] A second aspect of the present invention provides a robot welding guidance system based on weld geometric feature points, comprising: The weld seam image acquisition module is configured to: acquire weld seam images of the workpiece to be welded and preprocess them to obtain standardized weld seam image data; The geometric feature point localization module is configured to: mark corresponding geometric feature points in the weld image data according to the weld geometry, and perform online physical noise data enhancement; The model building and training module is configured to: build a lightweight multi-scale feature extraction network and use joint loss to complete network training; The candidate result output module is configured to: input the enhanced weld image data into the trained network, enhance the expression of small-scale weld geometric feature points, retain the detailed information of weld edges, laser stripes and key feature point neighborhoods, and output the candidate results of weld geometric feature points; The candidate result filtering module is configured to: match and filter candidate results based on weld geometric topology to obtain a set of weld geometric feature points; The guidance parameter generation module is configured to: perform weld geometry analysis based on the set of weld geometry feature points, obtain weld geometry information, and convert it into structured robot welding guidance parameters; The welding torch execution module is configured to send guidance parameters to the robot controller, which then controls the welding torch execution mechanism to complete welding start positioning, trajectory tracking, and attitude adjustment based on the weld start point, centerline, and attitude correction amount.

[0015] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a robot welding guidance method based on weld geometric feature points as described in the first aspect of the present invention.

[0016] The fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a robot welding guidance method based on weld geometric feature points as described in the first aspect of the present invention.

[0017] The above one or more technical solutions have the following beneficial effects: This invention improves the stability and engineering usability of weld positioning results in complex welding scenarios by performing feature point localization, matching and screening, and parameterized representation of various types of weld geometry. By introducing high-resolution shallow feature branches, non-destructive downsampling structures, lightweight feature extraction modules, and joint localization loss for small-scale geometric boxes, the model's ability to represent weak textures, small scales, and dense geometric feature points is enhanced. Combined with an online physical noise enhancement strategy for welding scenarios, the model's adaptability to interference from arc light, spatter, and occlusion is improved. Furthermore, matching and screening candidate feature points based on weld geometry topology suppresses the influence of false positives, missed positives, and isolated anomalies on weld structure analysis, improving the completeness and consistency of the weld feature point set. The screened weld geometry feature points are further converted into structured guiding parameters such as weld start point, centerline, welding direction, and posture correction, improving the reliability of converting visual inspection results into robot control commands. Finally, the robot controller drives the welding torch actuator to complete welding start positioning, trajectory tracking, and posture adjustment, reducing reliance on manual teaching and correction, and improving the automation and execution stability of the robot welding process.

[0018] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0020] Figure 1This is a schematic diagram of the robot welding guidance method of the first embodiment; Figure 2 A schematic diagram of geometric feature points corresponding to different weld types in the first embodiment; Figure 3 This is a schematic diagram of online physical noise data enhancement for the first embodiment; Figure 4 A schematic diagram of the lightweight multi-scale feature extraction network structure of the first embodiment; Figure 5 This is a schematic diagram of the robot welding guidance system in the second embodiment. Detailed Implementation

[0021] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0022] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0023] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0024] The following problems exist with existing technologies: (1) In the actual welding process, the laser stripe image will be affected by arc light, spatter, reflection, smoke, local overexposure and dynamic occlusion, which will cause the local structure of the weld groove edge, lap edge and intersection to be unstable. Traditional vision processing methods or general networks are difficult to stably output weld guide points that can be directly used by the robot.

[0025] (2) Geometric feature points of welds are usually represented by a very small local response window, and even a tiny pixel-level offset can cause a large guiding error. The general overlap constraint is prone to problems such as oversensitivity to displacement, gradient instability or zero overlap failure on a small-scale response window, and it is difficult to fully reflect the center point coordinate error.

[0026] (3) Different welds have different geometric structures. For example, the number of geometric points and topological relationships required for output are different for butt welds, lap welds, V-welds, V-welds with blunt edges, and fillet welds. Existing methods only output weld identification results or categories and cannot directly form the structured parameters required for welding torch posture adjustment, welding start point positioning, and weld tracking path.

[0027] (4) Robot welding scenarios have high real-time requirements, and the deployment end is usually not suitable for models with a large number of parameters, high computational load, and large inference latency. Therefore, it is necessary to reduce computational complexity while maintaining positioning accuracy and noise resistance, so that the system can run in real time in the industrial field.

[0028] This invention aims to provide a robust method for locating weld geometric feature points and generating guidance parameters for robot welding guidance. It outputs weld geometric feature point coordinates, weld centerline, welding start point, weld tracking direction, and guidance parameters usable by the robot control system. By locating, matching, filtering, and parameterizing feature points for various weld geometries, it improves the stability and engineering usability of weld location results in complex welding scenarios. Simultaneously, by introducing high-resolution shallow feature branches, lossless downsampling structures, lightweight feature extraction modules, and joint localization loss for small-scale geometric boxes, it enhances the model's ability to represent weak textures, small scales, and dense geometric feature points. Furthermore, by combining an online physical noise enhancement strategy for welding scenarios, it strengthens the model's adaptability to interference from arc light, spatter, and occlusion. Therefore, the purpose of this invention is to achieve stable extraction of multiple types of weld geometric feature points and reliable generation of welding guidance parameters while ensuring real-time performance. The specific method is illustrated through the following embodiments.

[0029] Example 1 like Figure 1 As shown, this embodiment discloses a robot welding guidance method based on weld geometric feature points, including: S1: Acquire and preprocess the weld seam image of the workpiece to be welded to obtain standardized weld seam image data.

[0030] S1.1: Constructing the weld dataset: In this embodiment, structured light stripes are projected onto the weld area of ​​the workpiece using a line laser. A weld image containing the weld area, laser stripes, arc light, spatter, and workpiece surface texture is obtained using an industrial camera, laser vision sensor, robotic welding vision acquisition device, or offline weld image acquisition system. A weld dataset is then constructed as input data for subsequent weld geometric feature point localization.

[0031] In a preferred embodiment, the weld image is a grayscale laser stripe weld image, where the grayscale laser stripes reflect the weld structure outline and geometric features. The weld image also includes complex interference generated by the welding site or acquisition environment. This complex interference includes one or more of the following: strong reflections, localized overexposure, spatter highlights, arc residue, smoke, dust, scratches, vertical bright stripes, background noise, and localized occlusion. In other embodiments, the weld image may also be a color weld image or a grayscale image converted from a color weld image. Regardless of whether a grayscale or color image is used, the key locations of the weld structure in the image are used as the geometric feature point detection targets.

[0032] S1.2: Weld image preprocessing: The acquired weld images are preprocessed, including image size adjustment, grayscale or brightness normalization, noise suppression, and format conversion, to reduce the impact of image brightness fluctuations, random noise, and local grayscale unevenness on subsequent feature extraction, thus obtaining standardized weld image data.

[0033] S2: Based on the weld geometry, mark the corresponding geometric feature points in the weld image data and perform online physical noise data enhancement.

[0034] S2.1: Classification of weld types: Based on the weld geometry, the acquired weld images are categorized. Weld types include, but are not limited to, fillet welds, lap welds, butt welds, V-welds, and Y-welds, and can also be applied to other weld types with clearly defined structural critical locations.

[0035] In this embodiment, for different weld types, the number and category of corresponding weld geometric feature points are set according to the weld structure characteristics and laser stripe morphology. Each weld geometric feature point corresponds to a preset geometric structure position in the weld image. The preset geometric structure position includes laser stripe endpoints, laser stripe intersections, bevel boundary points, lap boundary points, blunt edge structure points, or fillet weld positioning points. A fillet weld corresponds to 1 geometric feature point, a lap weld to 2 geometric feature points, a V-weld to 3 geometric feature points, a butt weld to 4 geometric feature points, and a Y-weld to 6 geometric feature points.

[0036] S2.2: Constructing and annotating geometric feature points of the weld: like Figure 2 As shown, based on the geometric structure of different weld types, corresponding geometric feature points are marked in the weld image. These geometric feature points include, but are not limited to, weld edge intersections, bevel intersections, lap edge points, blunt edge endpoints, and the location of the welding start point. Each geometric feature point is represented by a small-scale geometric bounding box, and its center coordinates serve as the basis for subsequent weld geometry analysis.

[0037] Specifically, in the weld geometric feature point annotation stage, small-scale rectangular annotation frames are used to annotate the geometric feature points, with the center of the annotation frame serving as the location of the weld geometric feature point. The small-scale rectangular annotation frame can be a rectangle of a fixed size, preferably 40×40 pixels. It should be noted that this size is only used to illustrate one specific implementation method. In practical applications, other sizes of geometric annotation frames can be set according to image resolution, camera field of view, laser stripe width, and weld scale.

[0038] This embodiment does not limit the final output to object detection boxes or category recognition results, but instead defines geometrically meaningful locations on the weld as weld geometric feature points. To facilitate network learning, each geometric feature point can be represented during the training phase as a local response window centered on that point, such as a rectangular window with sides of 40 pixels. This response window serves only as an intermediate representation for coordinate regression and local region constraints; its central coordinates are the core output.

[0039] Table 1 shows the correspondence between weld type, number of geometric feature points, geometric meaning, and output guidance parameters.

[0040] Table 1. Representation of Geometric Feature Points of Welds

[0041] S2.3: Perform online physical noise data augmentation: like Figure 3 As shown, during model training, online physical noise data augmentation is performed on the weld seam images to suit the welding scene. Augmentation includes operations such as arc simulation, spatter simulation, brightness perturbation, dynamic occlusion, local reflection, and random noise superposition. During augmentation, the relationship between the occluded area and key geometric feature points of the weld seam is controlled to avoid unreasonable damage to critical structures, thereby improving the model's adaptability to real welding noise. At least the following three types of perturbation training are included: (1) Spatter perturbation training: The system generates granular, wire-drawing, or mixed spatter on the image. The spatter can be distributed around the center of several random clusters or discretely distributed in the image. For wire-drawing spatter, a trailing line segment with a thicker front and a thinner back is used to simulate the trajectory of welding spatter, making its shape closer to the real scene.

[0042] (2) Arc light perturbation training: The system uses multiple offset bright spots superimposed and Gaussian blurring to generate irregular arc light halos, avoiding the distortion caused by a single regular ellipse.

[0043] (3) Dynamic Occlusion Perturbation Training: The system selects more visible occlusion areas based on image brightness and texture intensity, and avoids real geometric feature point areas with a preset probability, so that the enhancement can simulate occlusion without significantly damaging key training labels. As a preferred implementation, dynamic occlusion can forcibly avoid real geometric feature point areas with a probability of 80%, so that occlusion mainly enhances background noise, reflections, and non-critical structural areas, while retaining a certain proportion of difficult occlusion samples. This setting can reduce invalid supervision while enhancing robustness. This probability can also be adjusted within the range of 60% to 100% according to the noise intensity of the dataset.

[0044] In this embodiment, the enhancement strategy does not pre-fix the enhancement image, but generates perturbations in real time during each training sampling, so that the network model is continuously exposed to welding interference of different shapes, positions and intensities during the training process, thereby improving the network model's ability to adapt to real welding noise.

[0045] S3: Construct a lightweight multi-scale feature extraction network and use joint loss to complete network training.

[0046] S3.1: Constructing a lightweight multi-scale feature extraction network: like Figure 4 As shown, this embodiment constructs a lightweight multi-scale feature extraction network based on YOLO11n to extract the positional features of weld geometric feature points from weld images. The network includes a backbone network feature extraction part, a neck network feature fusion part, and a multi-scale output part.

[0047] In the backbone network feature extraction section, the input weld image first passes through a convolutional layer for initial feature extraction. Since weld geometric feature points are usually small in scale, and their location information is mainly concentrated in local areas such as laser stripe endpoints, intersections, bevel boundaries, and lap boundaries, an SPDConv module is set at the initial downsampling position to output shallow high-resolution features. This is used to retain local neighborhood information while reducing the feature map size, thereby reducing the impact of early downsampling on weld geometric details.

[0048] To enhance the representation of small-scale weld geometric features, a high-resolution feature layer P2 is introduced into the backbone network and used as an additional shallow output for subsequent feature fusion. The high-resolution feature layer P2, obtained from the shallower output of the backbone network, possesses higher spatial resolution and richer fine-grained positional information, effectively characterizing geometric features in laser stripe endpoints, fine bevel boundaries, and weakly textured regions. By introducing the high-resolution feature layer P2, the network improves its ability to perceive small-scale targets while maintaining its overall lightweight structure.

[0049] In the deep feature extraction part of the backbone network and the feature fusion part of the neck network, the Faster_C3K2 module is designed as a lightweight feature processing unit.

[0050] In the deep feature extraction part of the backbone network, the Faster_C3K2 module mainly serves as a lightweight feature extraction unit. It is used to extract local geometric features such as weld edges, laser stripes, bevel intersections, and blunt edge structures, and to reduce redundant computations in the network. This allows the model to maintain its ability to express the local geometric structure of the weld while meeting real-time processing requirements.

[0051] In the neck network feature fusion section, the Faster_C3K2 module primarily serves as the feature integration and refinement unit after fusion. It is positioned after upsampling and feature concatenation to perform lightweight fusion processing on the concatenated shallow high-resolution features and deep semantic features. Specifically, in the neck network feature fusion section, the network fuses the shallow high-resolution features extracted in the earlier downsampling stage of the backbone network with the deep semantic features extracted in the later downsampling stage of the backbone network through upsampling, feature concatenation, and Faster_C3K2 fusion operations. In addition to the original shallow high-resolution features, the output features of the introduced high-resolution feature layer P2 are fused step-by-step with other scale features through the upsampling path and connected to the highest resolution output branch, thereby further enhancing the ability to locate small-scale weld geometric feature points. Shallow features are used to preserve fine-grained positional information of weld geometric feature points, while deep features are used to provide information distinguishing the overall weld structure from background interference.

[0052] Multiple Detect branches at different scales are set in the multi-scale output section to output the local response window, center coordinates, confidence information, and weld geometric feature point numbers of weld geometric feature points. Among them, the high-resolution feature layer P2 corresponds to the newly added high-resolution output branch, which is used for fine localization of weld geometric feature points at small scales. The remaining output branches are used to supplement weld structure information at different scales. The local response window only serves as an intermediate expression for the learning of weld geometric feature point positions, and its center coordinates are used as the position result of the weld geometric feature points, which are then used for subsequent weld geometry analysis, centerline extraction, and guide parameter generation.

[0053] With the above structure, the network can extract local geometric details and multi-scale structural information in weld images while maintaining a low number of parameters and computational cost. In particular, it improves the detection capability of small-scale weld geometric feature points, providing a foundation for the localization of weld geometric feature points in complex welding scenarios.

[0054] In this embodiment, the improved YOLO11n network structure replaces the original Conv module with the SPDConv early downsampling module and the original C3K2 module with the Faster_C3K2 lightweight feature extraction module. The original network structure's FPN / PAN multi-scale feature fusion structure is retained, and a P2 detection head is added to the original P3–P5 multi-scale detection head. By sampling, stitching, and fusing shallow high-resolution features through the neck network, and finally integrating them into the newly added high-resolution output branch in the multi-scale output section, the expressive and localization capabilities of small-scale weld geometric feature points are enhanced. Combined with the lossless downsampling structure and the lightweight convolutional feature extraction module, the detailed information of weld edges, laser stripes, and the neighborhood of geometric feature points is preserved while reducing the number of parameters and computational load.

[0055] In this embodiment, to address the issues of small scale, precise location, and easy loss of local geometric features in weld geometric feature points during downsampling, the network adopts a structure of "shallow high-resolution feature enhancement, lightweight backbone feature extraction, multi-scale feature fusion, and multi-scale detection head output". While preserving the detailed information of small-scale weld geometric feature points, it reduces the number of model parameters and computational complexity, making it suitable for real-time weld geometric feature point detection tasks in complex welding scenarios.

[0056] S3.2: Introducing a joint loss gating mechanism for small-scale weld geometric feature points: In this embodiment, weld geometric feature points are represented by small-scale rectangular bounding boxes, and the center of the predicted small-scale rectangular bounding box is used as the position of the weld geometric feature points. Therefore, the center offset between the predicted box and the ground truth box directly affects the weld geometric feature point localization result. In the scenario of small-scale rectangular bounding boxes, when relying solely on traditional IoU-type loss for bounding box regression, the overlap area between the predicted box and the ground truth box is quite sensitive to positional offset. When the sizes of the predicted box and the ground truth box are small or the overlap area varies greatly, the IoU loss is insufficient to fully reflect the impact of the center distance of the predicted box and the difference in box scale on the localization of geometric feature points.

[0057] To address the aforementioned issues, this embodiment introduces a joint loss during the network training phase to optimize the network. The joint loss includes geometric feature point localization loss, bounding box regression loss, and a structural consistency constraint term. This enables the model to not only output the positions of weld geometric feature points but also enhance the spatial consistency between adjacent geometric feature points, improving localization stability under complex weld structures. The WIoU v3 loss is combined with the NWD loss to optimize the regression process of the predicted weld geometric feature point bounding boxes. Specifically, the WIoU v3 loss corresponds to the main regression optimization part of the geometric feature point localization loss, used for dynamic weighted regression based on the overlap quality and center distance between the predicted and ground truth bounding boxes, enhancing center localization accuracy. The NWD loss corresponds to the distribution consistency regression term in the bounding box regression loss, used to measure the distribution similarity between the predicted and ground truth bounding boxes from the perspective of center distance and width-height difference, enhancing the distribution consistency of small-scale bounding boxes. The structural consistency constraint term constrains the topological relationships between multiple weld geometric feature points.

[0058] S3.2.1: During training, let the predicted bounding box be... The real frame is Intersection over Union (IoU) loss of the corresponding current sample As shown in the following formula:

[0059]

[0060] S3.2.2: Let the center point of the prediction frame be... The center point of the true bounding box is Then the distance between their centers Expressed as follows:

[0061] Let the minimum bounding box width and height of the predicted bounding box and the ground truth bounding box be respectively... and Then the outer frame size As shown in the following formula:

[0062] The basic loss term of WIoU v3 Expressed as follows.

[0063]

[0064] in, To prevent extremely small constants with a denominator of zero.

[0065] S3.2.3: In WIoU v3, a non-monotonic dynamic focusing factor is further introduced, and the mean IoU loss during training is recorded using a moving average method:

[0066] in, This is the momentum coefficient. In one embodiment, . This refers to the mean IoU loss in the current sample, i.e., the average IoU loss.

[0067] The non-monotonic dynamic focusing gain G is calculated based on the ratio of the current sample's IoU loss to the average IoU loss.

[0068]

[0069]

[0070] in, It is a numerically stable term, avoiding a denominator of 0. It is a very small constant; To normalize the IoU loss for the current sample. and To focus on adjusting parameters. In one implementation, , .

[0071] WIoU v3 loss It can be represented as:

[0072] S3.2.4: Calculate the NWD loss between the predicted bounding box and the ground truth bounding box. .

[0073] First, the following formula is used to calculate the width-height difference term. :

[0074] Distance between predicted bounding box and ground truth bounding box The formula for measuring it is as follows:

[0075] Similarity It is the following formula:

[0076] in, This is a normalization constant. In one implementation, NWD loss for:

[0077] S3.2.5: Combine the WIoU v3 loss and the NWD loss to obtain the bounding box localization loss. :

[0078] in, and The loss weight coefficients can be set according to the training configuration.

[0079] In this embodiment, through the above-mentioned joint localization loss optimization scheme, the detection network simultaneously focuses on the overlap quality, center distance and scale difference between the predicted box and the real box during the training process. This is beneficial to improve the center localization stability of the predicted box of small-sized weld geometric feature points and provide a more stable bounding box regression result for subsequent calculation of the weld geometric feature point position based on the center of the predicted box.

[0080] S4: After training, the enhanced weld seam image data to be detected is input into the trained network. The expression of small-scale weld seam geometric feature points is enhanced through shallow high-resolution feature branches. Combined with a lossless downsampling structure and a lightweight convolutional feature extraction module, detailed information about weld seam edges, laser stripes, and the neighborhood of key feature points is preserved, outputting candidate results for weld seam geometric feature points. Specifically, the following steps are included: S4.1: The network input is a weld seam image, which is first processed by the backbone network for feature extraction. In the shallow feature extraction stage of the backbone network, to reduce the loss of small-scale geometric feature point information caused by early downsampling, an SPDConv module is introduced at the front of the backbone network to output shallow high-resolution features. This module preserves local neighborhood information through spatial-to-channel rearrangement, minimizing the loss of weld seam edges, laser stripe intersections, and neighborhood details of geometric feature points while reducing the feature map size. Subsequently, the network continues to extract semantic features at different levels through multi-layer convolutional structures.

[0081] S4.2: In the deep feature extraction part of the backbone network, Faster_C3K2 is used to replace some of the original feature extraction modules. Shallow high-resolution features are input, and the Faster_C3K2 module extracts deep semantic feature representations. This module introduces partial convolution ideas based on the C3K2 structure, performing spatial convolution calculations only on a portion of the channels, while the remaining channels are directly retained and participate in feature fusion, thereby reducing redundant computation. Through this design, the network can maintain its ability to represent features of local weld edges, bevel intersections, and dense geometric feature point regions while reducing the number of parameters and computational cost.

[0082] S4.3: In the neck network feature fusion section, a top-down and bottom-up multi-scale fusion is performed using the FPN / PAN multi-scale feature fusion structure to fuse shallow high-resolution features with deep semantic features. Shallow high-resolution features have high resolution and are suitable for preserving the precise location information of weld geometric feature points; deep semantic features have stronger semantic expressive power and help distinguish weld structure from interference such as spatter, arc light, and reflective textures. Through multi-scale fusion, the network can obtain multi-scale fused features that contain both local geometric details and overall weld structure information.

[0083] S4.4: In the detection output section, considering that the geometric feature points of the weld are represented by the center of a small-scale detection box, a P2 detection head is added to the original multi-scale detection head. This P2 head, together with the P3, P4, and P5 detection heads, forms a multi-scale output structure, with different detection heads processing features at different scales. Specifically, the P2 detection head is responsible for detecting small-scale geometric feature points at higher resolution, while the P3–P5 detection heads supplement structural information at different scales. Finally, the network outputs candidate geometric feature points corresponding to different weld types.

[0084] In this embodiment, the candidate results of weld geometric feature points include center coordinates, confidence level, and weld geometric feature point number, providing spatial geometric information for subsequent MCE calculation, Acc@t evaluation, and weld geometric analysis.

[0085] S4.5: To verify the positioning capability during the experimental phase, the Mean Center Point Error (MCE) and the positioning accuracy within the tolerance range (Acc@t) are used for evaluation. MCE represents the pixel distance between the predicted geometric point center and the actual geometric point center, and Acc@t represents the proportion of geometric points whose error does not exceed the threshold t. For multi-point weld structures, the Hungarian matching method can be used to establish a one-to-one correspondence between predicted and actual points within the topological constraints of the same weld type, avoiding greedy matching that could lead to local optimal matching errors in dense point regions.

[0086] The quality assessment results can be used for model selection after training, and can also serve as a basis for quality feedback during field operation. For example, when the number of output points in a certain frame does not meet the weld topology requirements, the reliability is lower than the threshold, or the positioning error is abnormal for multiple consecutive frames, the system can trigger re-acquisition, reduce the welding speed, or prompt manual confirmation.

[0087] S5: Based on the geometric topology of the weld, the candidate results are matched and filtered to obtain the set of geometric feature points of the weld.

[0088] After the network model outputs candidate geometric points, the candidate feature points output by the network are matched, filtered and deredundant processed according to the geometric topology relationship (corresponding number of points, spatial order and geometric relationship) of different weld types. For each type of weld, only the ordered set of feature points that conform to the geometric structure relationship is retained, and duplicate points, isolated false detection points and candidate points that do not conform to the spatial relationship are suppressed.

[0089] In this embodiment, for repeated prediction points, candidate points that are most consistent with the weld structure or have a high confidence level are retained; for locally missing points, structural compensation or anomaly judgment is performed by combining the positions of adjacent feature points and weld geometric constraints, thereby obtaining a stable set of weld geometric feature points.

[0090] S6: Based on the set of weld geometric feature points, perform weld geometric analysis to obtain weld geometric information and convert it into structured robot welding guidance parameters.

[0091] S6.1: Geometric Analysis of Welds: Based on the selected set of weld geometric feature points, the topological connection relationship between feature points is established according to the weld type, and the intersection of weld edges, groove structure, lap boundary or fillet weld is analyzed.

[0092] Let the set of geometric features of the selected welds be:

[0093] in, This represents the center coordinates of the geometric feature points of the weld in the image coordinate system. Indicates the number of geometric feature points of the weld. This indicates the corresponding confidence level. Based on the weld type, the corresponding topology template is invoked, and the weld geometric feature points with corresponding numbers are grouped into a structure point set. Then, according to the preset connection relationships, the weld centerline, welding start point, weld tracking direction, and local weld attitude information are calculated.

[0094] For weld types with multiple center reference points, the center reference points are first calculated based on paired boundary points. Let the first... The corresponding boundary points of the group are and Then the center reference point of this set of boundary points is:

[0095] The set of center points consists of all the central reference points:

[0096] When there are at least two center points, the weld centerline is obtained by linear fitting. The weld centerline can be represented as:

[0097] in, , , Here are the centerline parameters. They can be solved using the least squares method, minimizing the sum of the squared distances from each center reference point to the centerline:

[0098] in, Central reference point coordinates The corresponding weights can be determined by the confidence levels of the weld geometric feature points involved in the calculation of the center reference point. The weld normal vector is obtained based on the centerline parameters.

[0099] The weld seam tracking direction vector is:

[0100] Alternatively, it can be calculated based on the direction of the first and last points of the center point set:

[0101] The welding start point is determined based on the projection of the preset welding direction onto the set of center points. Let... Assuming a preset welding direction vector, the projection of each central reference point in that direction is:

[0102] When the welding direction changes from a smaller projection value to a larger projection value, the welding starting point is:

[0103] When the welding directions are opposite, the welding starting point is:

[0104] (1) For butt welds, the system calculates the weld center position and lateral offset based on the four edge intersection points. Let the four edge intersection points be in topological order as follows: ,in For one end, corresponding to the boundary point, If the other end is the corresponding boundary point, then the center reference point is:

[0105] Depend on and The centerline of the butt weld is determined, and the weld tracking direction is as follows:

[0106] Lateral offset of the weld can be determined by the current welding torch reference point. The normal distance to the centerline is represented as:

[0107] in, The coordinates of the welding torch reference point or the image reference center in the image coordinate system.

[0108] (2) For lap welds, the system determines the lap boundary and the starting point of the welding torch based on the two lap edge points. Let the two lap edge points be... Then the direction of the overlapping boundary is:

[0109] The normal vector of the overlapping boundary is:

[0110] According to the preset offset distance of the welding process The welding torch guide centerline can be obtained:

[0111] in, These are the path parameters along the lap joint boundary. The welding start point can be determined according to the preset welding direction. , Select from the options, or determine the center line after offset.

[0112] (3) For V-shaped welds, the system calculates the bevel centerline and opening direction based on the intersection of the left and right bevel endpoints and the center point. Let the left bevel endpoint be... The right bevel endpoint is The central intersection point is The center point of the bevel opening is:

[0113] The centerline direction of the V-weld is:

[0114] The bevel opening width is:

[0115] The bevel angle can be calculated based on the direction of the left and right bevel edges:

[0116] in, It can be used as a reference point for the root center. It can be used as a reference point for the opening center, and the two can be used to determine the weld centerline and the welding torch tracking direction.

[0117] (4) For V-shaped welds with blunt edges, the system calculates the groove boundary, blunt edge position, root center, and welding torch posture correction based on six weld geometric feature points. Assume the six points form three sets of boundary points according to their left-right correspondence. , , If these correspond to the upper part of the bevel, the middle part of the bevel, and the obtuse edge or root, respectively, then the corresponding center reference point is:

[0118] Depend on To fit the centerline of a V-shaped weld with a blunt edge, the weld tracking direction can be taken as follows:

[0119] The width of the blunt edge can be calculated from the distance between the corresponding points at the root:

[0120] The welding torch posture correction angle can be determined based on the centerline direction and the robot's current welding torch projection direction. The included angle is determined as follows:

[0121] in, Used to characterize the directional deviation of the welding torch relative to the weld centerline.

[0122] (5) For fillet welds, the system generates weld guide coordinates based on a single corner point. Let the geometric feature point of the fillet weld be... Then As a local reference point for the weld. Since a single corner point is insufficient to uniquely determine the weld tracking direction, the weld tracking direction can be determined by a preset welding direction, the workpiece boundary direction, the motion direction of adjacent frame corner points, or the robot teaching direction. If the motion direction of adjacent frame corner points is used, then:

[0123] in, and These represent the coordinates of the geometric feature points of the fillet weld in the current frame and the previous frame, respectively. According to... and It can generate welding torch guide coordinates for fillet welds.

[0124] Through the above weld geometry analysis, the system obtains the weld geometry analysis results in the image coordinate system, including the weld type indicator. The set of geometric feature points of the weld after screening The set of center reference points calculated from paired boundary points or bevel points weld centerline Welding start reference point Weld tracking direction weld normal vector Weld width or bevel width And the groove opening angle or local geometric direction angle calculated from the weld geometry. .

[0125] The geometric analysis results of the weld in the above image coordinate system can be expressed as follows:

[0126] It should be noted that the result obtained in step S6.1 The geometric analysis results, still belonging to the weld seam image space or workpiece plane, describe the geometric relationships of the weld seam structure itself and are not directly used as the final input of the robot control system. Subsequent step S6.2 will perform coordinate transformation, trajectory sampling, deviation calculation, and welding torch posture correction calculation based on the weld seam geometric analysis results, thereby generating robot welding guidance parameters.

[0127] S6.2: Generating Guide Parameters: Based on the geometric analysis results of the weld This process generates structured robot welding guidance parameters that can be called upon by the robot control system. Instead of re-establishing the topological connections between weld geometric feature points, this step completes coordinate transformation, trajectory sampling, deviation calculation, attitude correction calculation, and quality mark generation based on the weld centerline, welding start reference point, weld tracking direction, weld normal vector, weld width or bevel width, and local geometric direction angles.

[0128] First, the weld geometry information in the image coordinate system is transformed to the robot coordinate system. Let any weld geometric feature point in the image coordinate system be:

[0129] Its homogeneous coordinates are:

[0130] In the case of using line laser vision measurement, let the camera intrinsic parameter matrix be... Then the ray direction in the camera coordinate system corresponding to the pixel is:

[0131] Let the equation of the line laser plane in the camera coordinate system be:

[0132] in, The normal vector of the line laser plane. For line laser plane parameters, Let be any 3D point on the laser plane in the camera coordinate system. The specific 3D intersection point corresponding to the i-th pixel. It can be obtained by finding the intersection of the camera ray and the line laser plane:

[0133] Let the hand-eye transformation matrix from the camera coordinate system to the robot base coordinate system be... Then the spatial coordinates of the geometric feature points of the weld in the robot coordinate system are:

[0134] Therefore, the set of geometric feature points of the weld can be obtained. Central reference point set Welding start reference point and weld centerline The corresponding results converted to the robot coordinate system are denoted as follows: , , and .

[0135] In another embodiment, if the weld is located on the surface of an approximately planar workpiece, a homography transformation from the image plane to the workpiece plane can also be used. Let the homography matrix be... Then we have:

[0136] in, This represents the homogeneous plane coordinates of the workpiece. This is the scaling factor. For After normalization, the points in the workpiece coordinate system are obtained. Then, through the transformation matrix from the workpiece coordinate system to the robot coordinate system Convert to spatial coordinates in the robot coordinate system.

[0137] Secondly, the weld centerline trajectory points are generated based on the weld centerline in the robot coordinate system. For an approximately straight weld, the welding starting point in the robot coordinate system is set as... The unit vector for weld seam tracking direction is The preset trajectory sampling interval is Then the first The trajectory points of the centerline of each weld are:

[0138] in, Represents the first in the robot coordinate system The trajectory points of the centerline of each weld seam. The sampling interval is determined based on the weld length and sampling interval. If the weld is described by multiple center reference points, segmented interpolation can be performed based on adjacent center reference points. Let the adjacent center reference points be... and Then the segmented trajectory points can be represented as:

[0139] in, For the k-th segmented trajectory point at the adjacent center reference point and The linear interpolation coefficients between them correspond , correspond .

[0140] The set of weld centerline trajectory points is composed of all segmented trajectory points:

[0141] The local weld tracking direction between adjacent trajectory points is:

[0142] in, For the first Tracking direction of local weld seams at each trajectory point.

[0143] Next, calculate the welding deviation based on the current position of the robot's welding torch tip. Let the current position of the welding torch tip be... The corresponding target weld trajectory point is Then the position error vector is:

[0144] Let the workpiece surface normal vector or the welding height direction be... The transverse direction of the weld can be determined by the height direction and the weld tracking direction:

[0145] Therefore, the longitudinal deviation, lateral deviation, and height deviation are calculated:

[0146]

[0147]

[0148] in, Indicates the longitudinal deviation along the weld direction. This indicates the lateral deviation perpendicular to the weld direction. This indicates the deviation in the height direction of the welding torch. This deviation is used by the robot control system for path compensation and welding torch position correction.

[0149] The welding torch posture correction is calculated based on the weld seam tracking direction and the robot's current welding torch posture. Let the projection direction of the current welding torch onto the workpiece plane be... Then the welding direction angle is:

[0150] in, It is a two-parameter arctangent function. and respectively weld seam tracking direction The components of the vector along the y-axis and x-axis in the robot coordinate system.

[0151] The welding torch direction correction angle is:

[0152] in, This indicates the angular deviation of the current welding torch direction relative to the weld seam tracking direction. Further, based on the weld seam tracking direction... Horizontal direction and height direction Construct the target attitude matrix of the welding torch:

[0153] This attitude matrix is ​​used to constrain the welding torch to move along the weld direction, align it laterally with the weld centerline, and maintain the preset welding height direction.

[0154] To improve the reliability of the guiding parameters, the reliability score and quality indicator are further calculated. Let the average confidence level of the weld geometric feature points involved in the geometric analysis be:

[0155] Let the average fitting residual from the center reference point to the weld centerline be:

[0156] in, With the central reference point, This represents the distance from the center reference point to the weld centerline. Let the topology integrity evaluation value be... The reliability score, used to characterize whether the required weld geometry features for the current weld type are complete, can be expressed as:

[0157] in, These are the weighting coefficients, and , This represents the maximum allowable fitting error. If there are missing weld geometric feature points, centerline fitting residuals exceeding a threshold, reliability scores below a threshold, or abrupt changes in weld direction between adjacent frames exceeding a threshold, then the quality indicator... Set to an abnormal state and output a resampling or alarm signal; otherwise, a quality flag will be displayed. Set to active status.

[0158] Finally, the robot welding guidance data packet is output:

[0159] in, For weld type marking, This is the set of geometric feature points of the weld in the robot coordinate system. This is the set of points representing the trajectory of the weld centerline in the robot coordinate system. The welding start point in the robot coordinate system. For local weld seam tracking direction, The target attitude matrix of the welding torch. , , These are lateral deviation, longitudinal deviation, and height deviation. This is the amount of correction for the welding torch posture. For reliability score, This serves as a quality indicator to determine whether resampling or an alarm is required. The robot welding guidance data package can be invoked by the robot control system for welding path correction, welding torch posture adjustment, and abnormal status handling. S7: Guiding parameter output and welding torch execution S7.1: Transfer the robot welding guidance data package generated in S6.2 The data is sent to the robot controller. The robot controller reads the welding start point, weld centerline trajectory point, local weld tracking direction, welding torch target posture matrix, position deviation, posture correction, and quality indicators, and controls the welding torch actuator to complete welding start positioning, weld trajectory tracking, and welding torch posture adjustment.

[0160] During the initial positioning phase of welding, the robot controller reads the welding start point in the robot coordinate system. and the corresponding welding torch target posture Generate the initial target pose of the welding torch:

[0161] in, This indicates the initial target pose of the welding torch. Based on this initial target pose, the robot controller drives the end of the welding torch to move to the welding start position, aligning the welding torch with the weld start point and satisfying the preset welding posture.

[0162] During the weld seam trajectory tracking phase, the robot controller follows the trajectory point set along the weld seam centerline. The target pose of the welding torch is generated sequentially. For the first... If a trajectory point corresponds to a welding torch target posture as follows: Then the target pose is:

[0163] in, For the first The robot controller generates joint space or Cartesian space motion commands based on the target pose, driving the welding torch to move along the weld centerline.

[0164] During the welding torch position correction phase, the robot controller adjusts the lateral deviation output by S6.2. Longitudinal deviation and height deviation The target position of the welding torch is compensated. Let the position compensation amount obtained from the above deviation be... The corrected target position of the welding torch can then be represented as:

[0165] in, The target position can be determined by weighting the deviations in the transverse, longitudinal, and height directions according to welding process requirements. and target posture The corrected welding torch target pose is obtained:

[0166] The robot controller controls the welding torch actuator to perform trajectory tracking based on the corrected target pose.

[0167] During the welding torch posture adjustment phase, the robot controller adjusts the welding torch target posture matrix output by S6.2. Or attitude correction amount Adjust the welding torch orientation so that the direction of the welding torch movement is consistent with the direction of weld seam tracking, and ensure that the height of the welding torch meets the preset welding process requirements.

[0168] In one implementation, the robot controller performs the aforementioned pose correction using a position closed-loop or visual servoing method. Let the current pose of the welding torch end be... The corrected target pose is Then the pose error can be expressed as:

[0169] The robot controller based on pose error The system generates joint space control commands, Cartesian space position commands, speed commands, or welding trajectory correction commands to gradually bring the welding torch tip closer to the target pose. This embodiment does not limit the specific control method of the robot controller.

[0170] S7.2: Closed-loop update and anomaly handling in the welding process.

[0171] During the welding process, the system continuously acquires weld images according to a preset sampling period and repeatedly performs the steps of weld geometric feature point localization, candidate point screening, weld geometric analysis, and guidance parameter generation to obtain continuous frames of robot welding guidance data packets, thereby realizing closed-loop updates of weld visual guidance.

[0172] Let the first The robot welding guidance data packet generated by the frame is The previous valid robot welding guidance data packet was When the first Frame quality flags It is in an effective state, and the reliability score is... Not lower than the preset threshold At that time, the robot controller updates the welding path and welding torch posture using the current frame guidance parameters; when Abnormal state or In such cases, the robot controller does not directly use the current frame guidance parameters, but instead enters the exception handling process.

[0173] To reduce the impact of single-frame positioning fluctuations on the welding torch movement, the system can smooth the guidance parameters for consecutive frames. Let the trajectory point of the current frame be... The smooth trajectory points of the previous frame are The smoothed trajectory points are:

[0174] in, It is the smoothing coefficient, and The weld seam tracking direction can also be smoothed:

[0175] in, The direction for tracking the smoothed weld seam.

[0176] To determine whether the guiding parameters are stable across consecutive frames, the system calculates the change in welding start point and the change in weld direction between adjacent frames. The change in welding start point between adjacent frames is:

[0177] The change in weld direction between adjacent frames is:

[0178] when , When the reliability score is below the threshold, the centerline fitting residual exceeds the threshold, the number of weld geometric feature points does not meet the topology requirements for the corresponding weld type, or multiple consecutive frames of quality flags are in an abnormal state, the system determines that the current guidance parameters are not suitable for direct use in updating the welding torch motion; among these, This is the threshold for the maximum allowable positional change of the welding start point between adjacent frames. The threshold for the maximum allowable angle change in the weld direction between adjacent frames; both are preset based on the image sampling period, welding speed, and allowable control error.

[0179] In abnormal situations, the robot controller executes protection strategies based on the type of abnormality. These strategies include one or more of the following: reacquiring weld images, maintaining valid guidance parameters from the previous frame, reducing welding speed, pausing welding, outputting an alarm signal, or prompting for manual confirmation. If the current frame only exhibits short-term noise fluctuations, the controller can maintain valid parameters from the previous frame and continue execution. If a set of weld geometric feature points that meet the topology requirements cannot be obtained for multiple consecutive frames, or if the reliability remains below the threshold, the controller pauses welding and issues an alarm signal.

[0180] Through the aforementioned closed-loop update and anomaly handling, the system can continuously correct the welding torch position and posture based on real-time visual results during the welding process. At the same time, it can suppress the impact of abnormal guidance parameters caused by single-frame false detection, missed detection, spatter occlusion, and local reflection on welding execution, thereby improving the stability of robot welding start positioning, trajectory tracking, and posture adjustment.

[0181] As a specific implementation method, this embodiment takes a weld laser stripe image with a resolution of 960×960 acquired by an industrial camera as an example. During the training phase, each weld geometric feature point is labeled as a local response window of a fixed size, with the center of the window being the coordinates of the real geometric point. Before the training data is input into the network, an online data augmentation strategy generates spatter, arc light, and dynamic occlusion perturbations with set probabilities, and sets the occlusion avoidance probability according to the real geometric point region.

[0182] During the network extraction phase, the input image first enters a lightweight network containing high-resolution feature branches. SPDConv performs spatial rearrangement downsampling on the features, and the Faster_C3K2 module performs local convolution modeling on some channels, while other channels are directly preserved. During training, WIoU v3 and NWD joint loss constrain the positional relationship between the predicted bounding box and the ground truth bounding box. During inference, the system obtains the coordinates of candidate geometric points from the center of the response window and obtains an ordered set of points through weld geometry analysis.

[0183] For V-shaped welds with blunt edges, the system outputs six ordered geometric points and calculates the bevel centerline, root center point, and welding torch offset direction based on the upper boundary point, middle bevel point, and bottom blunt edge point. For fillet welds, the system outputs the coordinates of a single corner point and generates local welding torch guidance coordinates. These outputs can be directly used by the robot controller for weld tracking or initial point guidance.

[0184] Example 2 like Figure 5 As shown, this embodiment discloses a robot welding guidance system based on weld geometric feature points, including: The weld seam image acquisition module is configured to: acquire weld seam images of the workpiece to be welded and preprocess them to obtain standardized weld seam image data; The geometric feature point localization module is configured to: mark corresponding geometric feature points in the weld image data according to the weld geometry, and perform online physical noise data enhancement; The model building and training module is configured to: build a lightweight multi-scale feature extraction network and use joint loss to complete network training; The candidate result output module is configured to: input the enhanced weld image data into the trained network, enhance the expression of small-scale weld geometric feature points, retain the detailed information of weld edges, laser stripes and key feature point neighborhoods, and output the candidate results of weld geometric feature points; The candidate result filtering module is configured to: match and filter candidate results based on weld geometric topology to obtain a set of weld geometric feature points; The guidance parameter generation module is configured to: perform weld geometry analysis based on the set of weld geometry feature points, obtain weld geometry information, and convert it into structured robot welding guidance parameters; The welding torch execution module is configured to send guidance parameters to the robot controller, which then controls the welding torch execution mechanism to complete welding start positioning, trajectory tracking, and attitude adjustment based on the weld start point, centerline, and attitude correction amount.

[0185] In this embodiment, the weld seam image acquisition module includes a laser vision acquisition unit and an image preprocessing unit; the geometric feature point localization module includes an online welding noise perturbation unit; the model building and training module includes a lightweight multi-scale feature extraction unit and a coordinate regression and loss gating unit; the guidance parameter generation module includes a weld seam geometric analysis unit and a guidance parameter generation unit; and the welding torch execution module includes a quality assessment unit.

[0186] The laser vision acquisition unit includes a line laser, an industrial camera, a narrowband filter, and a mounting bracket. The line laser projects laser stripes onto the surface of the workpiece to be welded, and the industrial camera acquires grayscale or color images containing weld bevels, lap edges, corners, and laser stripes. The images are then fed into an image preprocessing unit for size normalization, grayscale channel adaptation, and intensity range conversion. An online welding noise perturbation unit, enabled only during the training phase, simulates random occurrences in the field such as spatter, arcing, overexposure, and dynamic occlusion. A lightweight multi-scale feature extraction unit receives the preprocessed images and outputs local geometric response features at multiple scales. A coordinate regression and loss gating unit constrains the center coordinates of the local response window to match the true geometric points during the training phase and outputs the positions and reliability of candidate geometric points during the inference phase. A weld geometry analysis unit transforms candidate points into an ordered set of geometric feature points based on weld structure rules. A guiding parameter generation unit further calculates the weld centerline, starting point, direction vector, offset, and quality indicators.

[0187] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0188] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a robot welding guidance method based on weld geometric feature points as described in Embodiment 1 of this disclosure.

[0189] Example 4 The purpose of this embodiment is to provide an electronic device.

[0190] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in a robot welding guidance method based on weld geometric feature points as described in Embodiment 1 of this disclosure.

[0191] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0192] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0193] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A robot welding guidance method based on weld geometric feature points, characterized in that, include: Acquire and preprocess weld seam images of the workpiece to be welded to obtain standardized weld seam image data; Based on the weld geometry, corresponding geometric feature points are marked in the weld image data, and online physical noise data enhancement is performed; A lightweight multi-scale feature extraction network is constructed, and joint loss is used to complete the network training; the lightweight multi-scale feature extraction network includes a backbone network feature extraction part, a neck network feature fusion part, and a multi-scale output part; The backbone network feature extraction part includes a shallow feature extraction part and a deep feature extraction part. The shallow feature extraction part is equipped with an SPDConv module and a high-resolution feature layer. The deep feature extraction part and the neck network feature fusion part are equipped with a Faster_C3K2 module. The multi-scale output part is equipped with multiple Detect branches. The joint loss includes WIoU v3 loss and NWD loss. WIoU v3 loss is used to perform dynamic weighted regression based on the overlap quality and center distance between the predicted box and the ground truth box. NWD loss is used to measure the distributional similarity between the predicted box and the ground truth box from the perspective of box center distance and width-height difference. The enhanced weld image data is input into the trained network to enhance the expression of small-scale weld geometric feature points, retain the detailed information of weld edges, laser stripes and key feature point neighborhoods, and output the candidate results of weld geometric feature points. Based on the geometric topology of the weld, the candidate results are matched and filtered to obtain the set of geometric feature points of the weld; Weld geometry is analyzed based on the set of weld geometric feature points to obtain weld geometric information and convert it into structured robot welding guidance parameters. The weld geometry analysis includes: based on the matched and filtered set of weld geometric feature points, establishing the topological connection relationship between feature points according to the weld type, analyzing the weld edge, groove structure, lap boundary or fillet weld intersection position, calculating the weld centerline, welding start point, weld tracking direction and local weld posture information through geometric relationships, and generating guiding parameters based on the weld geometry analysis results; The guidance parameters are sent to the robot controller, which controls the welding torch actuator to complete the welding start positioning, trajectory tracking and attitude adjustment based on the weld start point, center line and attitude correction amount.

2. The robot welding guidance method based on weld geometric feature points as described in claim 1, characterized in that, The specific process of outputting candidate results for weld geometric feature points includes: The weld image is input into the lightweight multi-scale feature extraction network to obtain weld feature fusion results at different scales. The multi-scale output part is used to predict the weld feature fusion results and output the center coordinates, confidence level, and weld geometric feature point number of the local response window corresponding to each weld geometric feature point. The center coordinates of the local response window are used as the position of the weld geometric feature point, and low confidence results are removed according to the confidence threshold to obtain candidate results of weld geometric feature points.

3. The robot welding guidance method based on weld geometric feature points as described in claim 1, characterized in that, The geometric feature points include, but are not limited to, the intersection of weld edges, the intersection of bevels, the lap edge points, the blunt edge endpoints, and the relevant positions of the welding start points.

4. The robot welding guidance method based on weld geometric feature points as described in claim 1, characterized in that, The matching and filtering of candidate results includes: for duplicate points, retaining the ordered set of feature points that are most consistent with the weld structure or have a high confidence level; for locally missing points, performing structural compensation or anomaly judgment by combining the positions of adjacent feature points and weld geometric constraints.

5. The robot welding guidance method based on weld geometric feature points as described in claim 1, characterized in that, The guiding parameters include, but are not limited to, welding start point, weld centerline trajectory point, local weld tracking direction, welding torch target attitude matrix, position deviation, attitude correction, and quality indicators.

6. A robotic welding guidance system based on weld geometric feature points, characterized in that, include: The weld seam image acquisition module is configured to: acquire weld seam images of the workpiece to be welded and preprocess them to obtain standardized weld seam image data; The geometric feature point localization module is configured to: mark corresponding geometric feature points in the weld image data according to the weld geometry, and perform online physical noise data enhancement; The model building and training module is configured to: build a lightweight multi-scale feature extraction network and use joint loss to complete network training; the lightweight multi-scale feature extraction network includes a backbone network feature extraction part, a neck network feature fusion part, and a multi-scale output part; The backbone network feature extraction part includes a shallow feature extraction part and a deep feature extraction part. The shallow feature extraction part is equipped with an SPDConv module and a high-resolution feature layer. The deep feature extraction part and the neck network feature fusion part are equipped with a Faster_C3K2 module. The multi-scale output part is equipped with multiple Detect branches. The joint loss includes WIoU v3 loss and NWD loss. WIoU v3 loss is used to perform dynamic weighted regression based on the overlap quality and center distance between the predicted box and the ground truth box. NWD loss is used to measure the distributional similarity between the predicted box and the ground truth box from the perspective of box center distance and width-height difference. The candidate result output module is configured to: input the enhanced weld image data into the trained network, enhance the expression of small-scale weld geometric feature points, retain the detailed information of weld edges, laser stripes and key feature point neighborhoods, and output the candidate results of weld geometric feature points; The candidate result filtering module is configured to: match and filter candidate results based on weld geometric topology to obtain a set of weld geometric feature points; The guidance parameter generation module is configured to: perform weld geometry analysis based on the set of weld geometry feature points, obtain weld geometry information, and convert it into structured robot welding guidance parameters; the weld geometry analysis includes: based on the matched and filtered set of weld geometry feature points, establishing the topological connection relationship between feature points according to the weld type, analyzing the weld edge, bevel structure, lap boundary, or fillet weld intersection position, calculating the weld centerline, welding start point, weld tracking direction, and local weld posture information through geometric relationships, and generating guidance parameters based on the weld geometry analysis results; The welding torch execution module is configured to send guidance parameters to the robot controller, which then controls the welding torch execution mechanism to complete welding start positioning, trajectory tracking, and attitude adjustment based on the weld start point, centerline, and attitude correction amount.

7. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps of the robot welding guidance method based on weld geometric feature points as described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the robot welding guidance method based on weld geometric feature points as described in any one of claims 1-5.

Images