A welding robot gasp physical positioning and vision fusion method and system

Abstract

The application relates to the technical field of welding control, and discloses a welding robot air claw physical positioning and vision fusion method and system. The method comprises the following steps: obtaining a physical reference coordinate by grabbing a standard part through an air claw; performing degradation calculation on a weld seam image to obtain an image degradation index; extrapolating the physical reference coordinate and the image degradation index to obtain a comprehensive early warning moment; aligning the comprehensive early warning moment with a beat idle window of a welding program to obtain a repositioning execution moment; grabbing an updated physical reference coordinate according to the repositioning execution moment; and performing weighted fusion on the index and a weld seam vision coordinate to obtain a corrected welding torch path. The application improves the stability of weld seam positioning during the whole weld pass operation and the continuity of the welding torch path correction.

Description

Technical Field

[0001] This application relates to the field of welding control technology, and in particular to a method and system for physical positioning and vision fusion of a welding robot gripper. Background Technology

[0002] In arc welding of medium-thick plate structures, welding robots need to determine the spatial position of the weld before arc ignition and continuously correct the welding torch trajectory during movement. Current methods fall into two categories. One is contact-based physical positioning, where the robot's end effector or gripper contacts a fixed reference point at the workstation. After multiple contacts in different postures, the position and posture of the tool's center point relative to the flange are determined through coordinate transformation, serving as the position reference in the robot's base coordinate system. This method is widely used in tool coordinate system calibration and has led to automated calibration schemes using calibration plates and photoelectric sensors. The other is non-contact visual positioning, which uses cameras or laser sensors to acquire weld images, extracts the positional relationship between the bevel centerline and the welding torch projection point, maps it to the robot's base coordinate system through hand-eye calibration transformation, and controls the welding torch to remain above and along the weld. For three-dimensional workpieces, path planning schemes using structured light measurement and 3D vision guidance have also been developed. To balance both approaches, the industry further synthesizes the physical reference coordinates and visual weld coordinates using weighted averaging or Kalman filtering, using the synthesized result to drive the welding torch path.

[0003] The above approach has revealed several shortcomings in actual workstations. Physical positioning relies primarily on single-point contact, lacking geometric constraints. The relative posture between the tool tip and the reference point varies each time, affecting the repeatability of the calculation results. More importantly, this reference is only acquired once before arc ignition and is not updated during the entire weld seam operation. Visual positioning is constrained by the environment. Arc light causes localized overexposure, compressing the grayscale dynamic range; spatter and smoke blur the grayscale steps at the bevel edge; and molten pool heat radiation introduces scattered noise into the sensor. The combined effect of these three factors results in jumps in the extraction of the bevel center feature point between adjacent frames. Furthermore, existing fusion processes generally use fixed weights or scene-independent general covariance recursion. The weight values ​​do not change with the actual degree of interference on-site, and their proportion in the synthesized result does not decrease even when visual data is distorted.

[0004] Since the physical reference is acquired only once and not updated, the workpiece expansion caused by welding heat input, the tooling deformation due to heat, and the vibration drift of the robot body will gradually cause the reference to deviate from the true value. This deviation has no corresponding observable measurement within the robot; the controller still uses the values ​​before arc ignition for synthesis, and the deviation is treated as the true value. Furthermore, the period of physical reference distortion highly overlaps with the period of visual distortion because both are driven by the same heat source and fumes. At this time, the visual side cannot detect the deviation on the physical side, and the physical side cannot be verified by the visual side. The synthesized result loses reliable support from either side. If a threshold trigger method is used to constantly re-grab the standard part to refresh the reference, the refresh action must cause the welding torch to lift, stop wire and gas feeding, and leave the weld. Each refresh adds one arc termination and one arc initiation on the weld bead, increasing porosity, lack of fusion, and abrupt changes in weld formation. The action intended to correct positioning deviations instead creates new process defects on the weld. Therefore, when and whether to refresh become unavoidable issues. Summary of the Invention

[0005] This application provides a method and system for physical positioning and visual fusion of a welding robot's pneumatic gripper. It solves the problems of distorted visual weld coordinates under strong light, dust, and high-temperature interference, and the undetectable shift of the physical reference acquired in a single operation due to welding heat input. It also addresses the issue that refreshing the physical reference using threshold triggering increases the number of arc initiation and termination operations, leading to joint defects in the weld bead. This improves the stability of weld positioning throughout the entire welding operation and the continuity of welding torch path correction.

[0006] In a first aspect, this application provides a method for physical positioning and visual fusion of a welding robot gripper, the method comprising: Step S1: Use a pneumatic gripper to grasp the standard part and obtain its physical reference coordinates; Step S2: Perform image quality degradation calculation based on the weld image to obtain the image degradation index; Step S3: Perform trend extrapolation processing on the positioning failure time based on the physical reference coordinates and the image degradation index to obtain the comprehensive early warning time; Step S4: Obtain the idle window of the cycle time by filtering from the segmented path of the welding program. Align the comprehensive warning time with the idle window of the cycle time by using a time matching algorithm to obtain the repositioning execution time before the comprehensive warning time. Re-capture the standard part at the repositioning execution time to obtain the updated physical reference coordinates. Then, perform weighted fusion processing on the updated physical reference coordinates and the weld visual coordinates according to the image degradation index to obtain the corrected welding gun path.

[0007] Secondly, this application provides a physical positioning and vision fusion system for a welding robot gripper, the welding robot gripper physical positioning and vision fusion system comprising: The gripping module is used to grip standard parts using a pneumatic gripper to obtain physical reference coordinates; The calculation module is used to perform image quality degradation calculations based on the weld seam image to obtain image degradation indices. The extrapolation module is used to perform trend extrapolation processing on the positioning failure time based on the physical reference coordinates and the image degradation index to obtain the comprehensive early warning time; The alignment module is used to filter out the idle window of the cycle time from the segmented path of the welding program, and to align the comprehensive warning time with the idle window of the cycle time through a time matching algorithm to obtain the repositioning execution time before the comprehensive warning time. At the repositioning execution time, the standard part is re-captured to obtain the updated physical reference coordinates, and the updated physical reference coordinates and the weld visual coordinates are weighted and fused according to the image degradation index to obtain the corrected welding torch path.

[0008] In the technical solution provided in this application, the traditional single-point contact positioning is replaced by a pneumatic gripper gripping standard parts. The gripping action itself forms geometric self-constraint through multi-line contact between the V-shaped positioning groove on the inner side of the gripper fingers and the cylindrical surface. The axis of the standard part is forced to converge to the same relative position each time it closes. The resulting physical reference coordinates no longer depend on the repeatability of the operating posture, and the dispersion of the contact link is constrained within the mechanical precision of the pneumatic gripper itself. On this basis, the image degradation index is not taken from a general image quality evaluation method, but rather the imaging consequences of three types of interference in the welding site are quantified separately: arc light compresses the grayscale dynamic range, smoke blurs the bevel edges, and thermal radiation introduces scattered noise. Each of the three corresponds to an independently calculable component, which is then normalized to a unified direction and synthesized. This structure makes the reliability of the vision side an observable measure directly linked to the welding process state, rather than merely a statistical byproduct within the algorithm. This is why the subsequent weighted fusion processing allows the ratio between the physical reference coordinates and the weld visual coordinates to automatically increase or decrease according to the actual strength of on-site interference, avoiding the situation where a fixed-weight scheme still accepts distorted data according to the original ratio when there is visual distortion. This increase or decrease of the fusion weights is driven entirely by the single variable of the image degradation index, without any manually set levels or thresholds. The same algorithm can be used at workstations with different plate thicknesses, different shielding gas ratios, and different dust concentrations without needing to readjust the weight parameters.

[0009] The introduction of a comprehensive early warning system transforms the assessment of positioning reliability from a current judgment to one with a time margin. The physical reference coordinates and image degradation indicators converge here; whichever approaches its respective reliability boundary first takes precedence, allowing the controller to determine the length of the margin before the reference coordinates truly fail. The degradation trend fitting window supporting this judgment is not fixed. Its duration automatically switches between 2 and 4 seconds depending on the slope of the degradation trend. When the slope is relatively flat, the window is widened to 4 seconds for a more stable fitting result; when the slope becomes steep, it is narrowed to 2 seconds to keep up with the sudden changes in the arc light, balancing the stability of the early warning system with its response speed. The value of this margin is precisely realized in the time-matching algorithm. The idle window of the cycle time comes from the segmented path of the welding program. These are the non-arc-starting periods already programmed in the program, such as the welding torch lifting, positioner changing position, and torch cleaning and wire trimming. The repositioning execution time is determined by aligning the comprehensive warning time with these periods. This ensures that the re-grabbing process of refreshing the reference falls within the original pauses of the weld bead, thus updating the physical reference coordinates without adding extra arc initiation and termination times. Even in cases where a long straight weld bead is completed in one go and there are no non-arc-starting periods along the entire weld bead, the controller will directly insert a grab at the comprehensive warning time, sacrificing one more arc initiation and termination time to update the reference. This ensures that the algorithm has a definite output under any condition and there is no unsolvable state. The solution also has corresponding handling for possible abnormal situations in these two information paths: if the visual side exceeds the preset threshold for three consecutive frames, it is judged as distorted and the fusion weight is given to the physical reference. If the physical side grabbing action itself fails, it will automatically retry once, avoiding the occasional failure of a single link from dragging down the positioning of the entire weld bead. Regarding the contribution of algorithm features to the solution, the reason why trend extrapolation and time matching are effective is based on the data association unique to the welding scenario, the causal relationship between heat input and workpiece expansion, and the correspondence between program segmentation and non-arc-starting periods. Outside of this application domain, the same algorithm structure does not have an executable input source. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of an embodiment of the physical positioning and vision fusion method for welding robot pneumatic grippers in this application. Figure 2 This is a schematic diagram showing the weighted superposition of contrast attenuation component, sharpness attenuation component and noise increase component in the embodiments of this application; Figure 3This is a schematic diagram showing the changes of the three normalized components and image degradation index with welding time in an embodiment of this application; Figure 4 This is a schematic diagram illustrating how the modified heat input drift conversion coefficient changes with the number of relocation execution rounds in the embodiments of this application. Detailed Implementation

[0012] This application provides a method and system for physical positioning and vision fusion of a welding robot gripper. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0013] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the physical positioning and vision fusion method for welding robot grippers in this application includes: Step S1: Use a pneumatic gripper to grasp the standard part and obtain its physical reference coordinates; Specifically, a 10mm diameter cylindrical pin is chosen as the standard part because when the 90-degree V-groove on the inside of the gripper's fingers contacts the cylindrical surface, the position of the contact line depends only on the cylinder diameter and is independent of the pin's axial insertion depth. If the pin diameter is too small, the distance between the two contact lines will be insufficient, and slight slippage may easily occur under clamping force. If the pin diameter is too large, it will exceed the opening stroke of commonly used grippers. The tolerance is limited to ±0.002mm to account for this error being transmitted to the gripping calculation coordinates as is. The gripper's own repeatability deviation limit is ±0.02mm. The pin's form and position error must be kept within one-tenth of this order of magnitude to avoid becoming the main source of error. The coordinates of the pre-calibrated reference point are measured once during the installation and commissioning of the coordinate measuring machine. After that, they are stored in the controller as fixed quantities. The coordinates of the gripping solution obtained each time are subtracted from it axis by axis. If any axial deviation exceeds 0.02 mm, the gripping is deemed invalid. Common causes are welding spatter particles attached to the surface of the pin, or slight interference between the gripper fingers and the edge of the tooling. In this case, the gripping action can be repeated.

[0014] Step S2: Perform image quality degradation calculation based on the weld image to obtain the image degradation index; Specifically, the normalized upper limits for the three image degradation indices are not empirical values. Instead, they are calculated under undisturbed conditions where the welding station is free of arc and dust, and lighting is provided by conventional industrial light sources. Twenty consecutive frames of images are captured on the same weld section, and the values ​​for local contrast, edge gradient sharpness, and grayscale noise variance are calculated. The arithmetic mean of each value is then taken, and these three arithmetic means are recorded as the normalized upper limits for contrast, sharpness, and noise variance, respectively. The reason for using undisturbed conditions as the benchmark is that image quality at the welding site can only be considered to have degraded relative to a clean state. For the three sub-indices, lower values ​​for contrast and sharpness indicate worse image quality, while noise variance indicates the opposite. Therefore, during normalization, the first two are calculated by subtracting the ratio of the measured value to the upper limit from 1, while the third is directly calculated as the ratio of the measured value to the upper limit. Only after unifying the directions can they be added together. The weighting coefficients are set to 0.4, 0.4, and 0.2 because overexposure of the arc light and dust obstruction directly swallow the information of the weld edge, which has the most significant impact on the extraction of the feature points of the weld center. Thermal radiation noise is mainly manifested as scattered points, and the weld direction can still be identified after median filtering, so the weights are reduced.

[0015] Step S3: Based on the physical reference coordinates and image degradation index, perform trend extrapolation on the positioning failure time to obtain the comprehensive early warning time; Specifically, the correlation between heat input and physical reference drift is based on the fact that the workpiece expands due to heat. The product of welding current and voltage, integrated over time, represents the heat delivered to the workpiece. The more heat, the greater the workpiece expansion, resulting in a positional shift of the standard parts clamped on the same fixture. The initial value of the conversion coefficient was obtained during the debugging phase. The method involved first capturing the standard part and recording its coordinates, then performing a no-load welding process for at least 30 seconds according to the current and voltage parameters of formal production, capturing the part again and recording its coordinates. The difference between the two coordinates on each axis, divided by the accumulated heat during this period, yielded the initial value. The drift unreliability threshold was set at 0.15 mm, based on the allowable misalignment of a conventional fillet weld. Exceeding this value would cause the welding torch to deviate from the bevel. The visual degradation unreliability threshold was set at 0.6, derived from control experiments during the debugging phase. After the index exceeded 0.6, the weld edge became discontinuous in the image, and the center point extraction results began to change abruptly. The trend slope is fitted using least-squares data from the most recent 10 frames (2 seconds). A window that is too short will be skewed by interference from single-frame splashes, while a window that is too long will fail to keep up with the sudden changes in the arc light. The earlier outward time of each of the two branches is used, and the one that fails first takes over the warning.

[0016] Step S4: Select the idle window of the cycle from the segmented path of the welding program. Align the comprehensive warning time with the idle window of the cycle using the time matching algorithm to obtain the repositioning execution time before the comprehensive warning time. Re-grab the standard part at the repositioning execution time to obtain the updated physical reference coordinates. Then, perform weighted fusion processing on the updated physical reference coordinates and the weld visual coordinates according to the image degradation index to obtain the corrected welding gun path.

[0017] Specifically, the idle window is read from the motion planning layer of the robot controller. During the programming phase, the welding program breaks down the entire weld seam into several program segments. These segments inherently include actions that prevent arcing, such as lifting the welding torch, changing the positioner, and cleaning the torch and cutting the wire. The start time and duration of each such action are recorded in pairs, forming two attributes of the window. This data does not require additional sensor acquisition; it exists within the program itself. A safety margin of 0.5 seconds is added to the wire feeding stop time (0.08 seconds), gas stop time (0.06 seconds), arc extinguishing time (0.06 seconds), and a reserved 0.30 seconds. This covers the response time from issuing the stop wire feeding and gas supply command to the complete extinguishing of the arc. The alignment cutoff time is obtained by subtracting this margin from the overall warning time. Only windows with a start time earlier than this cutoff time and a duration sufficient to complete one grabbing action are considered candidates. The candidate window with the closest time to the current window is selected. If no candidate window is available, the algorithm stops waiting and directly inserts a grabbing action at the overall warning time. This branch ensures a definite output under any operating condition. After the capture is completed, the obtained coordinates replace the original physical reference coordinates. Simultaneously, the measured axis-by-axis difference is divided by the cumulative heat in this interval, and the conversion coefficient is updated exponentially smoothed with a smoothing factor of 0.3 to balance the tracking speed during the workpiece heating stage and the suppression of fluctuations in a single measurement. The fusion weight is calculated from the current image degradation index; a higher index reduces the visual weight and correspondingly increases the physical weight, with the sum of the two always being 1. The fused positioning coordinates obtained by weighted summation are subtracted axis-by-axis from the original coordinates of the interpolation points that have not yet been executed in the program. The difference is then added back to these interpolation points. After the welding torch resumes arc ignition, it moves along the corrected coordinates. This solves the problem of inaccurate weld start point positioning caused by visual coordinate distortion under strong light, dust, and high temperature, and also solves the problem of the physical reference obtained from a single capture gradually distorting with thermal deformation without being noticeable. Furthermore, the capture action of refreshing the reference is placed within the program's existing pauses, without adding extra arc ignition and termination times.

[0018] In one specific embodiment, step S1 includes: The cylindrical surface of the standard part is closed and gripped by the V-shaped positioning grooves on the inner side of the two gripping fingers of the pneumatic gripper, resulting in a self-centering contact relationship of the pneumatic gripper constrained by four symmetrical contact lines. Based on the self-centering contact relationship of the pneumatic gripper, the angles of each joint of the welding robot are read, and the angles of each joint are input into the forward kinematics model for pose calculation, so as to obtain the pose transformation matrix of the pneumatic gripper coordinate system relative to the robot base coordinate system. Based on the pose transformation matrix, the design center coordinates of the standard part in the pneumatic gripper coordinate system are transformed to obtain the gripping solution coordinates for this gripping operation. Based on the comparison of the grasping and solving coordinates with the pre-stored standard part pre-calibration reference point coordinates, the coordinates with deviations not exceeding the repeated grasping deviation limit corresponding to the self-centering contact relationship of the pneumatic gripper are selected from the grasping and solving coordinates and used as physical reference coordinates. The standard part is a cylindrical pin with a diameter of 10mm and a tolerance of ±0.002mm.

[0019] Specifically, a V-shaped groove with a 90-degree included angle is machined on the inner side of each of the two grippers of the pneumatic gripper. The symmetry plane of the groove coincides with the closing motion direction of the gripper. When the gripper closes, the two inclined surfaces of each V-shaped groove are tangent to the outer cylindrical surface of the cylindrical pin once. The two grooves together form four contact lines parallel to the pin axis. These four lines are projected as four points on a section perpendicular to the pin axis, symmetrical about the left and right of the gripper closing direction and about the top and bottom of the pin axis. Since the position of the contact point tangent to the cylindrical surface at the 90-degree angle is determined only by the radius of the pin, once the pin is clamped, its axis must fall on the intersection line of the symmetry planes of the two V-shaped grooves. The two radial translational degrees of freedom and the rotation about the two radial axes are simultaneously locked. This is the essence of the self-centering contact relationship of the pneumatic gripper, which is fundamentally different from the single-point contact calibration method where the tool tip and the reference point only form one contact point and the contact state itself has no constraint capability. The pin's axial positioning is achieved by a shoulder machined at the pin's root. When the gripper fingers are fully engaged, their lower end faces rest against this shoulder, thus fixing the pin's axial position. After four radial constraints and one axial constraint, the pin's center point in the gripper coordinate system is uniquely determined by the gripper fingers' geometry. This position can be read from the gripper's part drawing, i.e., the design center coordinates, without requiring on-site measurement. The pin diameter is set to 10 mm, with cylindricity and diameter tolerances limited to ±0.002 mm. This is because pin diameter deviations are directly converted into the calculated pin axis position through the tangent point location, while the gripper's repeatability deviation limit is ±0.02 mm. Controlling the pin's form and position error to one-tenth of this order of magnitude prevents it from becoming a primary source of error.

[0020] After the gripper fingers are fully closed and the air pressure switch confirms the clamping, the controller reads six joint angles from the absolute encoders of each joint. The forward kinematics model uses the link parameter table provided in the robot's factory documentation. Each joint corresponds to four quantities: link length, link angle, link offset, and joint angle. Homogeneous transformations are constructed joint by joint and multiplied to obtain the pose transformation matrix of the gripper coordinate system relative to the robot's base coordinate system. This is a 4x4 homogeneous matrix, with the top 3 rows and 3 columns describing the posture and the first 3 elements of the right column describing the position. The design center coordinates are written as homogeneous column vectors and multiplied by this matrix to obtain the three-axis values ​​of the pin center in the robot's base coordinate system, which are the gripping calculation coordinates for this grasping operation. The pre-calibration reference point coordinates are measured once by a coordinate measuring machine on the pin mounted on the tooling during the assembly and debugging phase of the entire line. After the measurement, it is written into the controller as a fixed quantity and will not be measured again unless the tooling is disassembled or reassembled. Each gripping operation yields a calculated coordinate difference from the reference point's coordinates in the X, Y, and Z directions. The largest absolute value is compared to 0.02 mm. If the difference does not exceed 0.02 mm, the gripping operation is considered valid, and the coordinates are adopted as the physical reference coordinates. If the difference exceeds 0.02 mm, the operation is invalidated, the value is not adopted, the gripper retracts, and another gripping attempt is performed. The 0.02 mm limit is derived from repeatable measurements of the gripper paired with the pin. This is achieved by performing several consecutive gripping operations on a cold fixture and calculating the maximum dispersion of the relative mean of the calculated coordinates for each gripping operation. A typical cause of exceeding this limit is welding spatter adhering to the pin's conical surface or slight interference between the gripper's side and the fixture's vertical plate. In these two cases, the contact line is incomplete, the self-centering relationship is invalid, and the obtained value cannot be used as a reference.

[0021] In one specific embodiment, step S2 includes: Based on the grayscale distribution of the weld image, the candidate region of the weld is cropped to obtain the weld degradation calculation region; The gray-level extreme values ​​in the weld degradation calculation area are processed by Michelson contrast calculation, the gray-level gradient along the weld edge direction in the weld degradation calculation area is processed by Sobel operator amplitude mean calculation, and the high-frequency response after Laplace operator filtering in the weld degradation calculation area is processed by variance calculation to obtain local contrast, edge gradient sharpness and gray-level noise variance. Based on the pre-stored upper limits of contrast normalization, sharpness normalization, and noise variance normalization under the interference-free working conditions, the local contrast, edge gradient sharpness, and grayscale noise variance are normalized respectively to obtain the contrast attenuation component, sharpness attenuation component, and noise increase component. The interference-free working conditions are those without arc, dust, and conventional industrial lighting. The upper limits of contrast normalization, sharpness normalization, and noise variance normalization are obtained by continuously acquiring 20 frames of weld seam images under the interference-free working conditions and taking the arithmetic mean of their corresponding values. The image degradation index is obtained by linearly weighting and summing the contrast attenuation component, sharpness attenuation component, and noise enhancement component according to preset weighting coefficients.

[0022] Specifically, the weld image is acquired at 5 frames per second by a vision sensor mounted on the side of the welding torch. In the entire image, the bevel area has a rough surface and weak reflectivity, resulting in a generally lower grayscale level than the base material plane. The controller calculates the grayscale value line by line along the horizontal direction of the image, selecting the connected bands with the lowest grayscale values ​​as candidate weld areas. Then, using the center line of this connected band as the axis, it expands by several pixels on both sides to form a rectangular block. This rectangular block is the weld degradation calculation area, with a fixed size of 128 pixels by 48 pixels, and the long side follows the weld direction. This size is chosen because the short side needs to be sufficient to cover the edge transition zones on both sides of the bevel, while the long side cannot be long enough to include background elements such as tooling fixtures and positioning blocks; otherwise, the calculated grayscale extreme values ​​would reflect the background rather than the weld. After defining the region, the maximum and minimum gray levels within this rectangular block are taken. The difference between these two values ​​is divided by their sum and then multiplied by 255 to obtain the local contrast, measured in gray levels. The Sobel operator is used to perform convolution in this region to obtain the horizontal and vertical gradient components of each pixel. The normal direction of the edge is determined according to the direction of the bevel centerline. Only the gradient components in the normal direction are taken to calculate their amplitudes. The arithmetic mean of the amplitudes of all pixels within the region is then taken to obtain the edge gradient sharpness. A second-order difference filter is then applied to the same region using the Laplacian operator. The filtering result preserves the high-frequency transitions in gray levels. The variance of these response values ​​is calculated to obtain the gray-level noise variance. These three terms correspond to the light and dark compression caused by arc light, the edge blurring caused by dust, and the scattered noise caused by thermal radiation, respectively.

[0023] The three normalization upper limits were obtained all at once during the entire production line commissioning phase. The method involved arranging all tooling, workpieces, and sensors in production mode, with no arcing, no gas supply, and no current flow. Only standard industrial lighting was used. Twenty consecutive frames were captured on the same bevel section. For each frame, the weld degradation calculation area was extracted using the method described above, and three values ​​were calculated. The arithmetic mean of each of the 20 frames was recorded as the contrast normalization upper limit, sharpness normalization upper limit, and noise variance normalization upper limit, respectively. The reason for using this clean state as the benchmark is that image quality can only be assessed relative to the level under interference-free conditions; using absolute values ​​would be affected by lens, light source, and bevel material, making comparisons meaningless. During normalization, for local contrast and edge gradient sharpness, lower values ​​indicate worse image quality. Therefore, the ratio of the measured value to its respective upper limit was subtracted from 1 to obtain the contrast attenuation component and sharpness attenuation component. For grayscale noise variance, higher values ​​indicate worse image quality; the ratio of the measured value to its upper limit was directly taken to obtain the noise increase component. All three components are dimensionless, with values ​​ranging from 0 to 1, indicating that higher values ​​indicate more severe degradation. Their summation is meaningful only when all three are weighted. The contrast attenuation and sharpness attenuation components are each weighted at 0.4, while the noise boost component is weighted at 0.2. This is because overexposure from arc light and dust obstruction directly smooth out the grayscale abrupt changes at the bevel edge, rendering the extraction of weld center feature points ineffective. Thermal radiation noise, on the other hand, manifests as isolated scattered points, and the bevel direction can still be discerned after median filtering, thus having a relatively minor impact. The image degradation index is obtained after weighted summation of the three components, with values ​​also ranging from 0 to 1. A value closer to 1 indicates poorer weld imaging quality in the current frame.

[0024] Figure 2 This is a schematic diagram showing the weighted superposition of contrast attenuation component, sharpness attenuation component and noise enhancement component in the embodiments of this application. Figure 2 The image shows the superposition of three normalized components at several sampling moments during the welding operation. The horizontal axis represents the sampling moment, and the vertical axis represents the component values ​​after being weighted by preset weighting coefficients. The bars from bottom to top represent the contrast attenuation component, the sharpness attenuation component, and the noise increase component. The total height of the bars is the image degradation index obtained at that moment. As the arc continues to burn and spatter and smoke gradually accumulate above the bevel, the three components each increase, and the superposition height increases accordingly. The contrast attenuation component and the sharpness attenuation component have similar increases due to their equal weights, while the noise increase component has a relatively gentle increase due to its lower weight.

[0025] Figure 3 This is a schematic diagram showing the changes of the three normalized components and image degradation index with welding time in an embodiment of this application. Figure 3The figure shows the curves of the three dimensionless normalized components—contrast attenuation, sharpness attenuation, and noise increase—as a function of welding time t during the welding process. It also shows the image degradation index curve obtained by linearly weighting these three components with weights of 0.4, 0.4, and 0.2. A pre-stored degradation unreliability threshold of 0.6 is marked as a reference line in the figure. As can be seen from the figure, the contrast attenuation and sharpness attenuation components show similar upward trends and have the steepest slopes during welding, while the noise increase component shows a relatively gentle upward trend. All three components together drive the image degradation index to monotonically increase with time, approaching the degradation unreliability threshold at approximately 60 seconds. This verifies, through measured data, that the weighting coefficients of 0.4 for contrast and sharpness and 0.2 for noise match the actual impact of each component on the extraction of weld center feature points. It also confirms that a degradation unreliability threshold of 0.6 provides an observable basis for judgment.

[0026] In one specific embodiment, step S3 includes: The degradation index sequence, which is composed of image degradation indices according to the acquisition time sequence, is subjected to least squares linear fitting within a sliding time window to obtain the degradation trend slope. The image degradation index at the current moment is linearly extrapolated based on the pre-stored degradation unreliability threshold and degradation trend slope to obtain the visual failure moment. The heat input drift conversion coefficient is calculated based on the axial difference of the physical reference coordinates in two adjacent gripping processes and the product time integral of the welding current and welding voltage between the two gripping processes. The initial value of the heat input drift conversion coefficient is calibrated as follows: a standard part is gripped once in a cold state to obtain the calibration coordinates. After a no-load welding process of no less than 30 seconds with preset welding current and welding voltage parameters, the standard part is gripped again to obtain the recalibrated coordinates. The difference between the calibration coordinates and the recalibrated coordinates is divided by the product time integral accumulated during the no-load welding process to obtain the initial value. The product time integral accumulated since the last gripping process is drift converted according to the heat input drift conversion coefficient to obtain the physical reference drift accumulation and physical reference drift rate. The physical baseline drift accumulation is linearly extrapolated based on the pre-stored drift unreliability threshold and the physical baseline drift rate to obtain the physical failure time. The earlier value is compared with the visual failure time to obtain the comprehensive warning time.

[0027] Specifically, an image degradation index is calculated for each frame and sequentially placed into a first-in-first-out (FIFO) buffer according to the acquisition time, forming a degradation index sequence. The sliding time window length is set to 2 seconds, and with the visual sensor displaying 5 frames per second, the window can accommodate exactly 10 data points. The controller uses the acquisition time of each data point as the independent variable and the image degradation index at that point as the dependent variable to perform a univariate least squares linear fit. The slope of the fitted line is the degradation trend slope, measured in seconds. The 2-second window is chosen because a shorter window would be skewed by single-frame splashes or instantaneous arc light fluctuations, with a single outlier potentially increasing the slope several times over; a longer window would be constrained by older data from two seconds prior, failing to keep up with the surge in degradation indices when the arc light suddenly intensifies. Each time a new frame of data arrives, the oldest frame is discarded, and the fit is re-performed. The degradation unreliability threshold was set at 0.6, derived from control experiments during the debugging phase. The index value was compared one by one with the extracted weld center point results in the same frame. After the index exceeded 0.6, the bevel edge became discontinuous in the image, and the center point began to jump between adjacent frames; previously, it was continuously usable. The extrapolation algorithm subtracted the image degradation index of the current frame from 0.6, divided the difference by the degradation trend slope to obtain the remaining reliable duration, and added to the current time to obtain the visual failure time. If the fitted slope was not greater than zero, it indicated that the imaging quality was improving or remaining stable. In this case, no extrapolation was performed, and the visual failure time was set to a value much larger than the remaining working time of this weld, equivalent to the visual side not participating in the early warning for this round.

[0028] The physical conversion is based on the thermal expansion of the workpiece. The welding current multiplied by the voltage equals the instantaneous power. Integrating this over time yields the heat delivered to the workpiece, which is the product-time integral, measured in kilojoules. The controller samples the current and voltage feedback from the welding machine at millisecond levels and performs trapezoidal accumulation, starting from the moment the standard part was last gripped and continuing up to the current time. As the workpiece expands due to heat, the standard part clamped on the same fixture shifts accordingly; the more heat, the greater the shift, and the two are approximately proportional. The proportionality constant is the heat input drift conversion coefficient, measured in millimeters per kilojoule. The initial value of this coefficient is calibrated during the debugging phase. The method involves gripping the standard part once and recording its coordinates, then performing a no-load welding process for at least 30 seconds according to the current and voltage parameters for actual production, gripping again and recording the coordinates. The physical reference coordinates obtained from the two grippings are subtracted axis by axis in the X, Y, and Z directions. The difference along the axis with the largest deviation is divided by the product-time integral accumulated during this no-load welding period; the quotient is the initial value. During formal welding, the data is no longer captured. Instead, the accumulated time integral is multiplied by this coefficient to obtain the accumulated physical reference drift. The instantaneous power is multiplied by this coefficient to obtain the physical reference drift rate, in millimeters per second. The unreliable drift threshold is set at 0.15 mm, based on the allowable misalignment of a conventional fillet weld. If the reference offset exceeds this value, the welding torch will deviate from the root of the bevel. The extrapolation method is isomorphic to the visual side: 0.15 mm minus the accumulated physical reference drift, the difference divided by the physical reference drift rate, and the current time added to obtain the physical failure time. When welding is paused or the current is zero, the rate is 0, and the physical failure time is also set to a value much larger than the remaining working time. The two failure times are directly compared, and the one earlier on the timeline is taken as the comprehensive warning time. Whichever reaches the unusable state first takes precedence. If both are set to far values, no warning is generated for this round.

[0029] In one specific embodiment, step S4 involves filtering out idle windows from the segmented paths of the welding program, aligning the comprehensive warning time with the idle windows using a time matching algorithm, and obtaining the repositioning execution time before the comprehensive warning time, including: Based on the segmented path of the welding program, the non-arc-starting time period occupied by the welding torch lifting action, positioner changing action and torch cleaning and wire cutting action between two adjacent weld program segments is processed to obtain the beat idle window composed of the window start time and the window duration. The comprehensive warning time and the idle window of the clock are input into the time matching algorithm. The comprehensive warning time is pre-offset according to the pre-stored safety margin to obtain the aligned cutoff time. Based on the alignment deadline, the window start times of the beat idle window are compared and processed. The window start time is not later than the alignment deadline and the window duration is not less than the pre-stored re-grabbing action time. The window is selected from the beat idle window as the usable alignment window. Based on the available alignment window, the windows are sorted from nearest to farthest according to the interval between the window's start time and the current time. The start time of the first window in the sorted window is selected from the available alignment window and used as the relocation execution time. When the available alignment window is empty, the comprehensive warning time is used as the relocation execution time.

[0030] Specifically, during the teaching or offline programming phase, the welding program breaks down the entire weld seam into several program segments. Each segment corresponds to a continuous arc-initiating straight line or circular arc trajectory. The segments are pre-programmed with actions that do not initiate an arc, such as lifting the welding torch, changing the positioner, and cleaning the torch and cutting the wire. During program download, the controller's motion planning layer expands the entire path into a time-stamped instruction table. The expected start time and duration of each instruction are in the table. Entries marked as non-arc initiation are selected, and their start time and duration are used to form idle time windows. These windows have the following attributes: start time and duration. This data does not require any additional sensors or on-site measurements; it is already present in the program. The time reference is unified to the controller's operation timer, which is reset to zero at the moment of arc initiation. The integrated warning time derived from both the visual and physical sides uses the same timer reading, allowing for direct comparison. The safety margin is 0.5 seconds, which is composed of the wire feeding stop response of 0.08 seconds, the gas stop response of 0.06 seconds, the arc extinguishing response of 0.06 seconds, and the reserved margin of 0.30 seconds. The alignment cutoff time is obtained by subtracting this margin from the comprehensive warning time. This means that the grasping action must be started before this time in order to complete a baseline refresh before the warning arrives.

[0031] The screening process involves two criteria. Window start times later than alignment deadlines are directly rejected, as a warning will have already occurred by then. Window start times that are acceptable but whose duration is less than the pre-stored re-grabbing action time are also rejected, as they cannot accommodate a complete grab. The re-grabbing action time is calculated by adding four segments: the travel time of the gripper moving from its current position to above the standard part (0.12 seconds), the time for the gripper fingers to close and be confirmed by the pressure switch (0.05 seconds), the time for encoder reading and position calculation (0.04 seconds), and the time for the gripper fingers to open and retract (0.04 seconds). The total time is 0.25 seconds, measured once during line commissioning and then hardcoded into the controller. Windows that pass both criteria are added to the available alignment window set. Within the set, windows are sorted in ascending order of the interval between their start time and the current time, and the start time of the first window in the set is taken as the repositioning execution time. The reason for choosing the closest value instead of the one closest to the alignment cutoff time is that the changes in arc light and dust during welding are not strictly linear, and the slope of the degradation trend can become steep at any time. Refreshing the benchmark earlier leaves room for error. The situation where the set is empty does exist, typically when a long, straight weld is completed continuously without any repositioning or torch cleaning. In this case, we no longer wait but directly insert a capture at the comprehensive warning time. The cost is one more arc termination and arc initiation, but the benchmark will not lose its update. The algorithm has a definite output under any working condition, and there is no unsolvable state.

[0032] In one specific embodiment, step S4 involves re-grabbing the standard part at the repositioning execution time to obtain updated physical reference coordinates, including: The countdown is decremented based on the difference between the repositioning execution time and the current time. When the difference decreases to the pre-stored arc-ending preparation time, the wire feeding and gas supply actions of the welding power source are stopped, resulting in the welding gun being in a non-arc-starting state and in a paused state. Based on the welding torch paused state, the air gripper is displaced along the pre-stored standard part approach path, and the standard part is gripped by the air gripper to obtain the gripping coordinates of this gripping. Based on the capture solution coordinates and physical reference coordinates captured in this capture, an axis-by-axis difference calculation is performed to obtain the measured physical reference drift. Then, the heat input drift conversion coefficient is calculated by multiplying the measured physical reference drift with the cumulative time integral from the last capture process to the start of this capture process, and the corrected heat input drift conversion coefficient is obtained. Based on the captured coordinates, the reference coordinates are replaced to obtain the updated physical reference coordinates. Then, the accumulated physical reference drift is reset to zero according to the corrected heat input drift conversion coefficient to obtain the initial drift calculation value for the next monitoring cycle.

[0033] Specifically, the countdown is based on the controller's operation timer, calculating the difference between the repositioning execution time and the current time every interpolation cycle. The pre-stored arc-extinguishing preparation time is 0.2 seconds, which is obtained by superimposing three segments: the mechanical response of the wire feeder from receiving the stop command to the wire tip stopping (0.08 seconds), the lag of the shielding gas solenoid valve de-energizing to the gas flow cutoff (0.06 seconds), and the time from the arc extinguishing to the solidification of the molten pool surface (0.06 seconds). When the difference decreases to 0.2 seconds, the stop command for wire feeding and gas supply is issued. When the difference reaches zero, the arc has been completely extinguished, the molten pool has been contained, and the welding torch enters the non-arc-starting state, that is, the welding torch is in a paused state. The reason for stopping 0.2 seconds in advance instead of stopping at the exact time is that the moving gripper with the arc will drag a biting edge on the bevel. The approach path for standard parts is entered via teaching during line commissioning. Starting from a safe height after the welding torch is lifted, the path first translates along the horizontal plane above the fixture to directly above the standard part, then descends vertically to the clamping height. Several transition points are set in between to bypass the workpiece upright plate and clamping mechanism. The entire process does not interfere with any fixtures. After being recorded, it is stored as a fixed trajectory in the controller, and the same path is followed every time it is called. After the pneumatic gripper reaches its position, it performs this gripping operation by closing the V-shaped positioning groove to grasp the cylindrical pin. The angles of each joint are read, the pose transformation matrix is ​​obtained through positive kinematics calculation, and then the design center coordinates of the pin are transformed to the robot base coordinate system to obtain the gripping solution coordinates for this gripping operation, which are completely consistent with the initial gripping operation in the cold state.

[0034] The captured coordinates are subtracted from the previously adopted physical reference coordinates along the X, Y, and Z axes, and the difference with the largest absolute value is taken as the measured physical reference drift. The axis with the largest absolute value is used instead of a three-axis composite because the welding torch deviation from the bevel depends on the offset in a single direction, not spatial distance. The corresponding product time integral is accumulated over an interval starting from the moment the last capture was completed. The product of welding current and welding voltage is accumulated using a trapezoidal method in each interpolation cycle until the capture action within the current available alignment window is initiated. The heat within this interval corresponds exactly to the measured drift. The measured physical reference drift is divided by this product time integral to obtain the ratio of the current measurement. This ratio is then combined with the original heat input drift conversion coefficient using exponential smoothing, with the new value accounting for 0.3 and the old value for 0.7, to obtain the corrected heat input drift conversion coefficient. The smoothing factor of 0.3 is chosen because the expansion rate of the workpiece changes during its transition from a cold state to a steady state, and the coefficient needs to keep pace; a smaller factor would not be sufficient. Furthermore, each gripping action inherently carries a repetitive deviation of ±0.02 mm; a larger factor would amplify this deviation directly into the coefficient, causing the physical failure time in the next extrapolation to be premature. The gripping coordinates from this gripping action immediately overwrite the original physical reference coordinates, becoming the updated physical reference coordinates. The accumulator of the product time integral is reset to zero, and calculations restart from this point. The accumulated physical reference drift is simultaneously reset to zero, meaning the initial drift calculation value for the next monitoring cycle is set to 0 mm, and the physical side restarts accumulation from this new zero point.

[0035] Figure 4 This is a schematic diagram illustrating how the modified heat input drift conversion coefficient changes with the number of relocation execution rounds in the embodiments of this application. Figure 4 This diagram illustrates the variation of the heat input drift conversion coefficient when a welding robot performs multiple repositioning operations on the same weld seam. The horizontal axis represents the number of repositioning rounds, and the vertical axis represents the value of the coefficient. The solid step line represents the correction result obtained by combining the measured physical reference drift amount with the cumulative time integral amount of the interval after each round of gripping, and the original coefficient with a smoothing factor. The dashed line below represents the initial value obtained from calibration on the cold fixture during the debugging phase, and the dotted line above represents the level of the coefficient after the workpiece has been heated to a steady state. Since the workpiece is still in the heating process at the initial stage of arc initiation, the expansion rate is higher than the level at the cold calibration. The correction result of the first few rounds increases from the initial value round by round, and after entering a steady state, the fluctuations between rounds are only within a small range.

[0036] In one specific embodiment, step S4 involves weighted fusion of the updated physical reference coordinates and the weld visual coordinates based on image degradation indices to obtain the corrected welding torch path, including: The image degradation index corresponding to the last weld seam image before the relocation execution time is used to perform weight conversion processing to obtain the visual fusion weight. Then, the normalization and complementarity operation is performed on the visual fusion weight to obtain the physical fusion weight. The coordinate mapping process is performed based on the pixel coordinates of the weld center feature points in the weld image and the pre-stored hand-eye calibration transformation relationship to obtain the visual coordinates of the weld in the robot base coordinate system. The hand-eye calibration transformation relationship is obtained by taking pictures in multiple poses using a calibration board when the welding robot is installed, and then calculating the results based on the pictures. The visual coordinates of the weld are weighted axis by axis based on the visual fusion weight, and the updated physical reference coordinates are weighted axis by axis based on the physical fusion weight. The weighted axis results of the two are summed to obtain the fused positioning coordinates. Based on the fusion positioning coordinates and the original coordinates of the currently uninterpolated points in the segmented path of the welding program, the path correction offset vector is obtained by subtracting the coordinates of each axis. Then, the original coordinates of the currently uninterpolated points are superimposed on the path correction offset vector to obtain the corrected welding gun path.

[0037] Specifically, the weight conversion is obtained by subtracting the image degradation index from 1 to get the visual fusion weight. The image degradation index falls between 0 and 1, and the visual fusion weight also falls between 0 and 1, with lower weights for poorer imaging. The physical fusion weight is obtained by subtracting the visual fusion weight from 1, and the sum of the two is always 1, which is the normalization complementarity. The image degradation index of the last frame before the repositioning execution time is taken because the welding torch has been lifted and the arc has been extinguished during the capture action. At this time, there is no arc light or smoke in the captured image, and the degradation index will be abnormally low. The weight calculated using it cannot represent the imaging condition after welding recovery. The last frame before capture is the frame where the arc is still burning and interference still exists, which is closest to the working condition after arc recovery. The physical side provides the position of the pin center. To locate it on the weld, a fixed offset vector, entered during the teaching phase, is used. This offset vector represents the three-axis distance from the pin center to the starting point of the weld. The workpiece and standard part are clamped on the same fixture, and their relative positions are maintained by the fixture. Adding this offset vector to the updated physical reference coordinates yields the weld position derived from the physical side. This vector is the key factor in the fusion process. The weld center feature point on the visual side is extracted from the weld degradation calculation area. It searches for grayscale troughs column by column along a direction perpendicular to the bevel direction. A point at the starting end of the center line connecting the troughs is taken as the feature point, and its row and column pixel coordinates are read. This is then combined with the triangulation reading of the laser line at that point to obtain the depth. All three are substituted into the pre-stored hand-eye calibration transformation relationship and mapped onto the robot's base coordinate system to obtain the weld visual coordinates. The hand-eye calibration transformation relationship is performed once during installation using a calibration board. The robot, carrying the sensor, takes pictures of the same calibration board from several poses to solve the pose of the sensor coordinate system relative to the flange. This is then stored as a fixed quantity, and the calibration is not repeated without disassembling the tooling.

[0038] Fusion is performed separately along each axis. The X-value of the weld visual coordinates is multiplied by the visual fusion weight, and the X-value of the weld position derived from the physical side is multiplied by the physical fusion weight. The two are added together to obtain the X-value of the fused positioning coordinates. The Y and Z values ​​are processed similarly. When the arc light is strong and the dust is heavy, the image degradation index is high, and the visual component is suppressed, resulting in more reliance on the recently refreshed physical reference. When the image quality improves, the weights automatically revert, and the visual component regains dominance. The segmented path of the welding program is expanded into a series of interpolation points during download. Each point has three-axis coordinates and an estimated execution time. The points after the job timer reading are the currently unexecuted interpolation points. The original coordinates of the first unexecuted interpolation point are subtracted from the fused positioning coordinates, and the difference is calculated axis by axis to obtain the path correction offset vector, which is a three-axis displacement. This vector is added axis by axis to the original coordinates of all remaining unexecuted interpolation points of this weld to obtain the corrected welding torch path. Points already traversed remain unchanged. The entire weld seam is shifted instead of just one point because thermal deformation of the workpiece and drift of the reference point cause a positional shift along the entire weld seam. Correcting each point individually would cause the trajectory to bend at angles. After the gripper retracts, wire and gas feed resumes, and the arc is reignited, the welding torch continues to move downwards from the pause point along the corrected coordinates.

[0039] In one specific embodiment, the method further includes: When the image degradation index exceeds the preset threshold for 3 consecutive frames, the visual fusion weight is reset to 0, and the comprehensive warning time is set to the current time. If the air pressure switch does not activate within the preset time limit after the gripper's fingers close, the gripping process is deemed invalid. The gripper is then controlled to open and retract, and the gripping process is re-executed along the pre-stored standard part approach path. If the process is still deemed invalid after retrying, the physical reference coordinates of the previous attempt are retained. When the measured physical reference drift obtained from two consecutive rounds of repositioning processing both exceed twice the drift unreliability threshold, the thermal input drift conversion coefficient is not updated.

[0040] When the vision sensor drops frames or the entire image is overexposed, the controller records an out-of-limit flag in the degradation index sequence. Three consecutive out-of-limit frames indicate that the vision side is unusable, the vision fusion weight is set to 0, and the fusion positioning coordinates are degraded to the weld position given separately by the physical side. Simultaneously, the vision failure time is set to the current time, and the comprehensive warning time is also set to the current time. The controller performs repositioning within the nearest idle window according to the original alignment processing method. If the pressure switch does not activate within the preset time limit after the gripper fingers close, it indicates that the pin is not clamped or the gripper fingers' stroke is obstructed. This gripping judgment is invalid. After the gripper opens and retracts, it retryes once along the standard part approach path. If the retry is still unsuccessful, the previously adopted physical reference coordinates are retained and a prompt is output to the operator. This weld line is then completed according to the existing reference. If the measured physical reference drift obtained from two consecutive repositioning rounds exceeds twice the drift unreliability threshold, it usually means that the tooling has been moved or the clamping mechanism has loosened. In this case, the thermal input drift conversion coefficient is no longer updated to avoid solidifying a clamping anomaly into the coefficient.

[0041] When the welding procedure is a long straight single segment and there is no non-arc start period in the entire weld bead, the alignment window can be an empty set. The controller directly inserts a grab at the comprehensive warning moment. This branch exchanges the reference update for the cost of adding one arc termination and arc initiation. The joint position is selected at the original positioning weld point on the weld, so that the new joint coincides with the original joint.

[0042] The heat input drift conversion coefficient is corrected round by round with a smoothing factor of 0.3. This is a part of the scheme that automatically adjusts according to the workpiece temperature. The expansion rate of the workpiece changes continuously from a cold state to a steady state. This coefficient increases round by round and converges after the steady state, without the need for operator intervention. The length of the sliding time window is also adjusted with the slope of the degradation trend. When the absolute value of the slope is lower than the preset smoothness criterion for three consecutive fitting cycles, the window is widened from 2 seconds to 4 seconds for a total of 20 data points, resulting in a more stable fitting result. After the slope crosses the smoothness criterion, the window falls back to 2 seconds, following the sudden change of the arc light.

[0043] The sum of the visual fusion weight and the physical fusion weight is always 1. The fluctuation of the two is entirely determined by the image degradation index of the current frame. There are no manual settings, and the weights do not need to be readjusted on the same equipment at workstations with different plate thicknesses, different protective gas ratios, and different dust concentrations. The normalization upper limit allows for a refresh every 20 frames when the first piece of each shift is idle and arc-free. The overall image degradation caused by lens contamination or illumination attenuation is thus included in the baseline and is not misjudged as welding interference.

[0044] The above describes the physical positioning and vision fusion method of the welding robot gripper in the embodiments of this application. The following describes the physical positioning and vision fusion system of the welding robot gripper in the embodiments of this application. One embodiment of the physical positioning and vision fusion system of the welding robot gripper in the embodiments of this application includes: The gripping module is used to grip standard parts using a pneumatic gripper to obtain physical reference coordinates; The calculation module is used to perform image quality degradation calculations based on the weld seam image to obtain image degradation indices. The extrapolation module is used to perform trend extrapolation processing on the positioning failure time based on the physical reference coordinates and the image degradation index to obtain the comprehensive early warning time; The alignment module is used to filter out the idle window of the cycle time from the segmented path of the welding program, and to align the comprehensive warning time with the idle window of the cycle time through a time matching algorithm to obtain the repositioning execution time before the comprehensive warning time. At the repositioning execution time, the standard part is re-captured to obtain the updated physical reference coordinates, and the updated physical reference coordinates and the weld visual coordinates are weighted and fused according to the image degradation index to obtain the corrected welding torch path.

[0045] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for physical positioning and vision fusion of a welding robot gripper, characterized in that, The method includes: Step S1: Use a pneumatic gripper to grasp the standard part and obtain its physical reference coordinates; Step S2: Perform image quality degradation calculation processing on the weld image to obtain image degradation indices, including: performing weld candidate region cropping processing based on the grayscale distribution of the weld image to obtain the weld degradation calculation region; performing Michelson contrast calculation processing on the grayscale extreme values ​​within the weld degradation calculation region; performing Sobel operator amplitude mean calculation processing on the grayscale gradient along the weld edge direction within the weld degradation calculation region; and performing variance calculation processing on the high-frequency response after Laplacian operator filtering within the weld degradation calculation region to obtain local contrast, edge gradient sharpness, and grayscale noise variance; and performing contrast normalization upper limit and sharpness normalization upper limit under pre-stored interference-free working conditions. The local contrast, edge gradient sharpness, and grayscale noise variance are normalized to obtain contrast attenuation, sharpness attenuation, and noise increase components, respectively. The interference-free working condition is one without arcing, dust, and conventional industrial lighting. The upper limit of contrast normalization, the upper limit of sharpness normalization, and the upper limit of noise variance normalization are obtained by taking the arithmetic mean of 20 consecutively acquired weld seam images under the interference-free working condition. The image degradation index is obtained by linearly weighting and summing the contrast attenuation, sharpness attenuation, and noise increase components according to preset weight coefficients. Step S3: Perform trend extrapolation processing on the positioning failure time based on the physical reference coordinates and the image degradation index to obtain the comprehensive warning time, including: performing least squares linear fitting processing on the degradation index sequence formed by the acquisition time sequence within a sliding time window to obtain the degradation trend slope; performing linear extrapolation processing on the image degradation index at the current time based on the pre-stored degradation unreliability threshold and the degradation trend slope to obtain the visual failure time; performing thermal input drift conversion coefficient calculation processing based on the axial difference of the physical reference coordinates in two adjacent grasping processes and the time integral of the product of welding current and welding voltage between the two grasping processes, the initial value of the thermal input drift conversion coefficient is calibrated in the following way: performing a grasping process on the standard part in a cold state to obtain the calibration coordinates, according to... After a no-load welding process of no less than 30 seconds using preset welding current and welding voltage parameters, the standard part is gripped again to obtain recalibrated coordinates. The difference between the calibrated coordinates and the recalibrated coordinates is divided by the product time integral accumulated during the no-load welding process to obtain the initial value. The product time integral accumulated since the last gripping process is then converted according to the heat input drift conversion coefficient to obtain the physical reference drift accumulation and physical reference drift rate. The physical reference drift accumulation is linearly extrapolated according to the pre-stored drift unreliability threshold and the physical reference drift rate to obtain the physical failure time. The earlier value is compared with the visual failure time to obtain the comprehensive warning time. The workpiece and the standard part are clamped on the same tooling. Step S4: Obtain the idle window of the cycle time by filtering from the segmented path of the welding program. Align the comprehensive warning time with the idle window of the cycle time by using a time matching algorithm to obtain the repositioning execution time before the comprehensive warning time. Re-capture the standard part at the repositioning execution time to obtain the updated physical reference coordinates. Then, perform weighted fusion processing on the updated physical reference coordinates and the weld visual coordinates according to the image degradation index to obtain the corrected welding gun path.

2. The method for physical positioning and vision fusion of a welding robot gripper according to claim 1, characterized in that, Step S1 includes: The standard part's cylindrical surface is closed and gripped by the V-shaped positioning grooves on the inner sides of the two gripping fingers of the pneumatic gripper, resulting in a self-centering contact relationship of the pneumatic gripper constrained by four symmetrical contact lines. Based on the self-centering contact relationship of the gripper, the angles of each joint of the welding robot are read, and the angles of each joint are input into the positive kinematics model for pose calculation to obtain the pose transformation matrix of the gripper coordinate system relative to the robot base coordinate system. The design center coordinates of the standard part in the pneumatic gripper coordinate system are transformed according to the pose transformation matrix to obtain the gripping solution coordinates for this gripping operation. Based on the grasping calculation coordinates and the pre-stored standard part pre-calibration reference point coordinates, a step-by-step deviation comparison is performed. The coordinates whose deviation does not exceed the repeated grasping deviation limit corresponding to the self-centering contact relationship of the pneumatic gripper are selected from the grasping calculation coordinates and used as the physical reference coordinates. The standard part is a cylindrical pin with a diameter of 10mm and a tolerance of ±0.002mm.

3. The method for physical positioning and vision fusion of the welding robot gripper according to claim 2, characterized in that, Step S4, which involves filtering the segmented path of the welding program to obtain a time-idle window, and aligning the comprehensive warning time with the time-idle window using a time matching algorithm to obtain the repositioning execution time before the comprehensive warning time, includes: Based on the non-arc-starting time period occupied by the welding torch lifting action, positioner changing action and torch cleaning and wire cutting action between two adjacent weld program segments in the segmented path of the welding program, the time period extraction process is performed to obtain the beat idle window composed of the window start time and the window duration. The comprehensive early warning time and the beat idle window are input into the time matching algorithm. The comprehensive early warning time is pre-offset according to the pre-stored safety margin to obtain the alignment cutoff time. Based on the alignment cutoff time, the window start time of the beat idle window is compared and processed. The window start time is not later than the alignment cutoff time and the duration of the window is not less than the pre-stored re-grabbing action time, and the window is selected from the beat idle window as the available alignment window. Based on the available alignment window, the windows are sorted from nearest to farthest according to the interval between the window start time and the current time. The start time of the first window in the sorted order is selected from the available alignment window and used as the relocation execution time. When the available alignment window is an empty set, the comprehensive warning time is used as the relocation execution time.

4. The method for physical positioning and vision fusion of a welding robot gripper according to claim 3, characterized in that, Step S4, which involves re-grabbing the standard part at the repositioning execution time to obtain updated physical reference coordinates, includes: Based on the difference between the repositioning execution time and the current time, a countdown is performed to decrease. When the difference decreases to the pre-stored arc-ending preparation time, a stop command is issued to the wire feeding and gas supply actions of the welding power source to obtain a welding torch paused state in a non-arc-starting state. Based on the welding torch paused state, the air gripper is displaced along the pre-stored standard part approach path, and the standard part is gripped by the air gripper to obtain the gripping coordinates of this gripping. Based on the capture solution coordinates and the physical reference coordinates captured in this capture, an axis-by-axis difference calculation is performed to obtain the measured physical reference drift. Then, the measured physical reference drift and the product time integral accumulated from the last capture process to the start of this capture process are processed to obtain the corrected heat input drift conversion coefficient. Based on the captured and calculated coordinates, a reference coordinate replacement process is performed to obtain the updated physical reference coordinates. Then, the accumulated physical reference drift is reset to zero according to the corrected heat input drift conversion coefficient to obtain the initial drift calculation value for the next monitoring cycle.

5. The method for physical positioning and vision fusion of a welding robot gripper according to claim 4, characterized in that, Step S4, which involves weighted fusion of the updated physical reference coordinates and the weld visual coordinates based on the image degradation index to obtain the corrected welding torch path, includes: Based on the image degradation index corresponding to the last frame of the weld image before the relocation execution time, a weight conversion process is performed to obtain the visual fusion weight, and a normalized complementary operation is performed based on the visual fusion weight to obtain the physical fusion weight. The coordinate mapping process is performed based on the pixel coordinates of the weld center feature points in the weld image and the pre-stored hand-eye calibration transformation relationship to obtain the visual coordinates of the weld in the robot base coordinate system. The hand-eye calibration transformation relationship is obtained by taking pictures in multiple poses using a calibration board when the welding robot is installed, and calculating the results based on the pictures. The visual coordinates of the weld are weighted axis by axis based on the visual fusion weight, and the updated physical reference coordinates are weighted axis by axis based on the physical fusion weight. The weighted axis results of the two are summed to obtain the fused positioning coordinates. Based on the fused positioning coordinates and the original coordinates of the currently uninterpolated points in the segmented path of the welding program, a path correction offset vector is obtained by subtracting the coordinates from each axis. The original coordinates of the currently uninterpolated points are then superimposed on the path correction offset vector to obtain the corrected welding torch path.

6. The method for physical positioning and vision fusion of a welding robot gripper according to claim 5, characterized in that, The method further includes: When the image degradation index exceeds the preset threshold for 3 consecutive frames, the visual fusion weight is reset to 0, and the comprehensive warning time is set to the current time. If the air pressure switch does not activate within a preset time limit after the gripper's fingers close, the gripping process is deemed invalid. The gripper is then controlled to open and retract, and the gripping process is re-executed along the pre-stored standard part approach path. If the retry is still deemed invalid, the previously adopted physical reference coordinates are retained. When the measured physical reference drift amount obtained from two consecutive rounds of repositioning processing both exceed twice the drift unreliability threshold, the thermal input drift conversion coefficient is not updated.

7. A physical positioning and vision fusion system for a welding robot pneumatic gripper, characterized in that, For implementing the physical positioning and vision fusion method for welding robot grippers as described in any one of claims 1-6, the welding robot gripper physical positioning and vision fusion system comprises: The gripping module is used to grip standard parts using a pneumatic gripper to obtain physical reference coordinates; The calculation module is used to perform image quality degradation calculation processing based on the weld image to obtain image degradation indicators, including: cropping candidate regions of the weld based on the grayscale distribution of the weld image to obtain a weld degradation calculation region; performing Michelson contrast calculation on the grayscale extreme values ​​within the weld degradation calculation region; performing Sobel operator amplitude mean calculation on the grayscale gradient along the weld edge direction within the weld degradation calculation region; and performing variance calculation on the high-frequency response within the weld degradation calculation region after Laplace operator filtering to obtain local contrast, edge gradient sharpness, and grayscale noise variance; and performing contrast normalization upper limit and sharpness normalization based on pre-stored upper limit of contrast and sharpness normalization under interference-free conditions. The upper limit and the noise variance normalization upper limit are used to normalize the local contrast, the edge gradient sharpness, and the gray-scale noise variance, respectively, to obtain the contrast attenuation component, the sharpness attenuation component, and the noise increase component. The interference-free working condition is a working condition without arcing, dust, and conventional industrial lighting. The upper limit of contrast normalization, the upper limit of sharpness normalization, and the upper limit of noise variance normalization are obtained by continuously acquiring 20 frames of the weld seam image under the interference-free working condition and taking the arithmetic mean of their respective values. The image degradation index is obtained by linearly weighting and summing the contrast attenuation component, the sharpness attenuation component, and the noise increase component according to a preset weight coefficient. The extrapolation module is used to perform trend extrapolation processing on the positioning failure time based on the physical reference coordinates and the image degradation index to obtain a comprehensive early warning time. This includes: performing least-squares linear fitting processing on the degradation index sequence formed by the acquisition time sequence within a sliding time window to obtain the degradation trend slope; performing linear extrapolation processing on the image degradation index at the current time based on a pre-stored degradation unreliability threshold and the degradation trend slope to obtain the visual failure time; and calculating the thermal input drift conversion coefficient based on the axis-by-axis difference of the physical reference coordinates in two adjacent capture processes and the time integral of the product of the welding current and welding voltage between the two capture processes. The initial value of the thermal input drift conversion coefficient is calibrated as follows: performing a capture process on the standard part in a cold state to obtain the calibration coordinates. After performing a no-load welding process for no less than 30 seconds according to preset welding current and welding voltage parameters, the standard part is gripped again to obtain recalibrated coordinates. The difference between the calibrated coordinates and the recalibrated coordinates is divided by the product time integral accumulated during the no-load welding process to obtain the initial value. The product time integral accumulated since the last gripping process is then converted according to the heat input drift conversion coefficient to obtain the physical reference drift accumulation and physical reference drift rate. The physical reference drift accumulation is linearly extrapolated according to the pre-stored drift unreliability threshold and the physical reference drift rate to obtain the physical failure time. The earlier value is compared with the visual failure time to obtain the comprehensive warning time. The workpiece and the standard part are clamped on the same tooling. The alignment module is used to filter out the idle window of the cycle time from the segmented path of the welding program, and to align the comprehensive warning time with the idle window of the cycle time through a time matching algorithm to obtain the repositioning execution time before the comprehensive warning time. At the repositioning execution time, the standard part is re-captured to obtain the updated physical reference coordinates, and the updated physical reference coordinates and the weld visual coordinates are weighted and fused according to the image degradation index to obtain the corrected welding torch path.

Images