Welding robot spot image processing method based on vision sensor

Abstract

This invention discloses a welding robot weld point image processing method based on vision sensors, belonging to the field of welding vision control technology. The method includes: pre-welding 3D weld seam location and establishment of weld seam mileage coordinates; simultaneous acquisition of local images, robot pose, and controller timing information during welding; extraction of local geometric features of the weld seam; determination of future action points based on look-ahead distance and total time delay; hierarchical control of trajectory, posture, and oscillation corrections based on image reliability and coordinate drift detection results; and post-weld geometric inspection of the weld seam, determination of quality rule sets according to joint type and welding process type, identification of defect categories and quality levels, and writing back to the rework section. This method achieves unified integration of pre-welding mapping, in-welding predictive control, and post-weld quality closed-loop, improving tracking stability, consistency of quality judgment, and accuracy of rework positioning, while reducing welding deviations and rework rates caused by miscorrections and delayed corrections.

Description

Technical Field

[0001] This invention relates to the field of welding vision control technology, specifically to a welding robot weld point image processing method based on vision sensors. Background Technology

[0002] It is mainly applicable to medium and heavy plate components, pressure vessels, engineering machinery, steel structures, rail transportation, and ship sections. It is suitable for automated workstations that require continuous tracking of joint position and forming quality in multi-layer multi-pass arc welding, laser welding, and laser-arc hybrid welding. The workpiece materials in such scenarios are mostly carbon steel, low alloy steel, and stainless steel, and the joint types include V-groove butt joints, U-groove butt joints, corner joints, and closed curve butt joints.

[0003] Current processes typically involve first locating or teaching the weld joint, then continuously acquiring images of the weld joint as it changes during the welding process using a vision sensor. This process extracts features such as gaps, edges, center lines, or stripes to drive the welding torch trajectory correction. Post-weld, the weld geometry and defects are inspected and evaluated online or offline. Abb's publicly available application manuals indicate that industrial robots already support optical weld tracking and pre-processing tracking, and online structured light inspection can also identify and grade weld geometric defects. This demonstrates that basic methods for in-weld tracking and post-weld inspection have been established in this field.

[0004] Chinese patent document CN114734143A discloses a weld seam tracking method based on image processing. This method is mainly aimed at the laser welding process. The workpieces to be welded are placed together on a worktable, forming a welding gap. The laser welding head's output port is positioned above one end of the gap. A camera on the welding head acquires a local gap image located on one side of the output port and transmits it to a control unit. The control unit processes the image using Python-OpenCV, including Gaussian filtering, grayscale histogram equalization, triangular thresholding binarization, extraction of the gap edge contour and width value from the binary image, skeletonization and contour averaging composite algorithm for centerline extraction, and feature point location detection using slope analysis. Then, the control unit sets welding parameters such as laser power, spot diameter, welding head movement speed, and wire feed speed based on the processing results, controlling the welding head to move along the gap for welding. During continuous welding, the camera repeatedly acquires images at regular time intervals. The control unit compares the image processing results of each round with the previous round, calculates the deviation, and transmits it to the controller via the motion system to correct the welding head movement. This shows that the basic workflow of existing technology is: local image acquisition - image preprocessing - gap contour and centerline extraction - deviation comparison between rounds - welding parameters and welding head motion control.

[0005] The above method can achieve basic tracking in planar welds or when local gaps are relatively stable, but it still has limitations in multi-layer, multi-pass arc welding of medium and thick plates or laser-arc hybrid welding workstations. First, this method calculates the deviation by comparing the "currently acquired local image" with the "result of the previous image processing round." Optical weld tracking sensors in industrial applications are generally forward-looking trackers, and the sensor's observed position naturally has a look-ahead relationship with the actual arc ignition or correction position of the welding torch. The ABB manual points out that this look-ahead relationship is also affected by welding speed, sensor frequency, and internal controller delay; therefore, the weld position corresponding to the current image is not necessarily equivalent to the actual position at the moment of welding execution. Second, under multi-layer, multi-pass welding, strong arc light, spatter, and fumes, local weld images will exhibit saturation, occlusion, blurring, and stripe distortion. Studies by Xu et al. have confirmed that arc light and spatter increase visual tracking errors, thus affecting the stability of the deviation calculation based on image round comparison. Furthermore, in-weld tracking and post-weld quality evaluation are usually in different processing chains. Although existing studies have shown that post-weld geometric measurement and ISO5817 quality level evaluation can be achieved, their results are merely inspection results rather than directly executable rework positioning information.

[0006] Therefore, under complex working conditions, existing technologies are prone to problems such as tracking lag, local overshoot, weld formation fluctuations, difficulty in rework positioning, and lack of automated quality closed-loop, ultimately affecting welding consistency, cycle time, and rework costs. Therefore, it is necessary to research a welding robot vision processing and decision-making method that simultaneously considers look-ahead measurement and execution time difference, complex interference environments, and the utilization of post-weld quality information during the welding process. Summary of the Invention

[0007] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a visual sensor-based image processing method for welding robot weld points. This method extracts local geometric features of the weld seam and determines the future action point by combining the look-ahead distance and total time delay. Based on image reliability and coordinate drift detection results, it performs layered control on trajectory, posture, and oscillation corrections. Post-weld weld seam geometric inspection is performed, and a quality rule set is determined according to joint type and welding process type. Defect categories and quality levels are identified, and the rework section is written back. This method achieves unified integration of pre-weld mapping, in-weld predictive control, and post-weld quality closed-loop, improving tracking stability, consistency of quality judgment, and accuracy of rework positioning, thus solving the technical problems described in the background art.

[0008] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: The image processing method for welding robot weld points based on vision sensors includes: performing three-dimensional seam finding before welding the joint to be welded, extracting the weld centerline, weld tangent, local curvature, gap distribution and joint type, establishing weld mileage coordinates, and simultaneously acquiring local images during welding, robot pose, welding speed, swing phase and robot controller timing information. Based on the local geometric features of the weld seam extracted from the local images during the welding process, and combined with the total time delay determined by the look-ahead distance and time series information, and the weld seam mileage coordinates, the future point of action is determined. Based on the reliability of local images during welding and the coordinate drift detection results, the trajectory, attitude, and oscillation correction corresponding to the future action point are controlled in layers. After welding, geometric inspection is performed on the weld seam. Based on the joint type and welding process type, a quality rule set is determined, the defect category and quality level are judged, and the rework section is written back.

[0009] Furthermore, the weld mileage coordinates are established, including: discretizing the weld centerline obtained from the three-dimensional weld search before welding to form continuous centerline nodes; sorting the centerline nodes according to the extension direction from the weld start point to the weld end point; determining the weld mileage position corresponding to each centerline node based on the cumulative path length between adjacent centerline nodes; and associating the acquisition position of each frame of local image during welding with the corresponding weld mileage position.

[0010] Furthermore, the timing information of the robot controller includes the image acquisition time, the image processing completion time, the controller receiving time, the controller sending time, and the robot execution feedback time. The total delay is determined based on the time difference between the image acquisition time, the image processing completion time, the controller receiving time, the controller sending time, and the robot execution feedback time.

[0011] Furthermore, local geometric features of the weld are extracted from local images during welding, including: extracting lateral deviation, vertical deviation, weld tangent, local curvature, gap width, and welding torch relative attitude error within the region of interest corresponding to the weld mileage position; determining the future action point, including interpolating or extrapolating the mileage difference between the currently measured position and the position to be corrected by combining the look-ahead distance, total time delay, and weld mileage coordinates.

[0012] Furthermore, the corresponding trajectory correction, attitude correction, and oscillation parameter correction are generated, including: determining the target position, target attitude, and target oscillation correction requirements based on the weld tangent, local curvature, gap width, and relative attitude error of the welding torch corresponding to the future point of action, and generating the trajectory correction, attitude correction, and oscillation parameter correction from the target position, target attitude, and target oscillation correction requirements, respectively.

[0013] Furthermore, a credibility evaluation is established for local images during welding, including determining the credibility evaluation based on the proportion of saturated pixels, occlusion rate, image blur, stripe continuity, and consistency of historical residuals. The tiered control includes: performing full correction when the credibility rating is high, performing amplitude limiting correction when the credibility rating is medium, and performing degradation control when the credibility rating is low.

[0014] Furthermore, when the system is configured with a primary channel and a backup channel, the reliability evaluation is also corrected based on the consistency comparison results between the primary channel output and the backup channel output. The coordinate drift detection results are determined based on the deviation of the reference edge, fixture reference, or nozzle profile relative to the weld reference map; when the deviation exceeds the preset boundary, the coordinate transformation relationship between the visual coordinates and the weld mileage coordinates is updated.

[0015] Furthermore, degradation control includes reducing welding speed, maintaining the current weld tangential, pausing oscillation, and triggering local search recapture; when arc light, spatter, smoke, or obstruction causes the confidence evaluation to continuously fall below the low confidence boundary for a preset number of frames, the system switches from closed-loop tracking to protection mode, and only retains safe movement and welding abort judgment.

[0016] Furthermore, geometric inspection is performed on the weld after welding, including extracting weld width, reinforcement height, leg length, misalignment, undercut, lap joint, and incomplete weld. A quality rule set is determined based on the joint type and welding process type, including: matching the corresponding first quality rule set when the welding process type is ordinary fusion welding, and matching the corresponding second quality rule set when the welding process type is laser-arc hybrid welding. The defect category and quality level are determined based on the quality rule set.

[0017] Furthermore, the write-back of the repair section includes merging adjacent weld sections with the same defect category into the same repair section and outputting the repair location point or repair trajectory fragment; and reversing the update of the rescanning section, risk section or control parameters for the next round of welding, including: marking the repair section with repeated occurrence of the same type of defect as the rescanning section for the next round of pre-weld three-dimensional seam finding, marking the repair section corresponding to the low confidence state as the high sensitivity section during welding, and using the repair section that overlaps with the predicted high-risk section to update the correction weight or safety boundary of the future action point.

[0018] (III) Beneficial Effects This invention provides a method for processing weld point images of welding robots based on vision sensors, which has the following beneficial effects: By generating weld seam mileage coordinates through 3D seam finding before welding, the local images during welding, robot pose, and controller timing information are unified into the same weld seam mileage coordinates. This ensures that the weld seam corresponds to the same spatial object before, during, and after welding, avoiding positioning errors caused by inconsistent reference benchmarks. This provides a continuous data foundation for subsequent determination of action points and rewriting of rework sections.

[0019] By extracting local geometric features of the weld and determining subsequent action points based on look-ahead distance and total time delay, the welding robot no longer corrects itself solely based on the instantaneous position in the current local image during welding. Instead, it controls itself according to the target position, target posture, and target oscillation correction needs during execution, reducing lag corrections and overshoot caused by time misalignment in curved segments, thermal deformation segments, and gap fluctuation segments. By establishing a credibility evaluation for local images during welding and combining it with layered control based on coordinate drift detection results, trajectory correction, posture correction, and oscillation correction have clear control boundaries in different image states and installation states, avoiding the direct conversion of abnormal images such as arc light, spatter, smoke, obstruction, or drift into control actions. Through geometric inspection of the weld after welding, quality rules are determined based on the joint type and welding process type, ensuring that the judgment of defect type and quality level is consistent with the actual welding situation. This avoids result mismatch caused by using a single judgment caliber for different joints and different welding processes, ensuring consistency in post-weld quality judgment. Attached Figure Description

[0020] Figure 1 This is a diagram showing the overall architecture of the weld closed-loop tracking and repair system of the present invention. Figure 2 This is a schematic diagram illustrating the weld entity reconstruction and weld mileage coordinate establishment of the present invention. Figure 3 This is a schematic diagram of the timing synchronization acquisition and entry data packet generation of the present invention; Figure 4 This is a schematic diagram illustrating the prediction of future action points and the generation of target states in this invention. Figure 5 This is a schematic diagram illustrating the image reliability, drift compensation, and control mode switching of the present invention; Figure 6 This is a schematic diagram of the post-weld quality mapping, rework task generation, and front-end feedback closed loop of the present invention. Figure 7 This is a schematic diagram illustrating the extraction of local geometric states for different joint types according to the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Please see Figures 1-7 This invention provides a method for processing weld point images of a welding robot based on a vision sensor, comprising: Step 1: Before and after the welding action begins, first establish a unified weld reference map for the head to be welded, and then assemble the visual acquisition during welding, robot motion status and controller transmission and reception times into a timing synchronization data package that can be directly called for the next step, so as to ensure that the subsequent calculation object, reference coordinates and time baseline all point to the same weld entity.

[0023] In the case of multi-layer, multi-pass arc welding of medium and thick plates and laser-arc hybrid welding, the joint is not a constant bright line or dark seam in a single frame image, but a three-dimensional joint area affected by clamping errors, beveling deviations, thermal warping of the plate, local reflections, edge gaps, and pre-weld contamination. If only the gap boundary in a small segment of the image in front of the welding torch is used as the tracking object after welding begins, this object has neither a global starting point nor a global direction, and will disappear briefly due to local occlusion. This will cause the lateral and vertical deviations extracted in the next step to become disconnected from the overall trend of the weld.

[0024] Therefore, the joint surface contour, bevel edge, and gap distribution are first converted into a continuous centerline extending along the weld by pre-weld 3D seam finding, and the weld mileage coordinates are established using this centerline as the framework. In this way, any local weld fragment in any subsequent image frame can be substituted back to the same main weld line, and each deviation, orientation, and defect location used in subsequent steps is no longer an isolated pixel result, but a local state attached to the unified weld entity.

[0025] In a workstation used for corner welding of box girders, after the operator clamps two steel plates, the pre-weld three-dimensional seam-finding unit first scans the joint area along the direction the welding torch will travel. This three-dimensional seam-finding unit preferably uses a line laser profile sensor, but a structured light projection head or a binocular vision head can also be used. When using a line laser profile sensor, its mounting bracket is fixed to a rigid connecting seat in front of the welding torch. The connecting seat is made of a heat-stable metal material. The sensor's field of view covers the base material on both sides of the weld and the bevel edge, allowing both the bevel edge and the bottom profile of the seam to be seen in the same frame. If there are flash, oxide scale, or obstructions from positioning fixtures on the workpiece surface, the industrial control / edge calculation unit first performs a continuity screening of the original profile, retaining only the profile segments that continuously appear in space along the joint direction and satisfy the bevel development relationship in the height direction, to avoid mistaking the fixture edge for the weld edge.

[0026] When performing a pre-weld 3D seam finding unit, it does not perform indiscriminate sampling of the entire workpiece. Instead, it limits the scanning bandwidth based on the pre-defined workpiece boundaries, expected arc initiation positions, and expected arc termination positions in the welding task, ensuring that the main sampling band extends along the expected weld direction. This is because the joint area often contains reinforcing rib edges, positioning block edges, and welding torch clearance slots. If the scanning bandwidth is not limited beforehand, strong reflective boundaries unrelated to the weld will be mixed in during contour cleaning, causing abrupt changes in the subsequently established centerline. After receiving the contour flow, the industrial control / edge calculation unit first layers the height abrupt changes according to the base material normal direction, and then connects adjacent contour segments end-to-end according to the joint extension direction, thereby reorganizing the bevel edges, the bottom edge of the gap, and the edge of the local notch into candidate weld bands. For segments with short interruptions and continuous tangential direction in the candidate weld band, they are joined according to the contour direction. For segments with long interruptions and significant tangential changes, they are kept disconnected and marked as segments to be re-scanned to avoid forcibly constructing a non-existent continuous centerline at the current stage.

[0027] In one embodiment, if the joint to be welded is a V-groove butt joint, the industrial control / edge calculation unit uses the middle trajectory after the projection of the bevel edges and the bottom of the bevel on both sides as the candidate center line; if the joint to be welded is a U-groove butt joint, the symmetrical center zone of the arc segments of the bevels on both sides is used as the candidate center line; if the joint to be welded is a corner joint, the projection zone of the intersection line of the two base materials is used as the candidate center line.

[0028] Furthermore, within the same weld entity recognition framework, a more stable centerline extraction basis is specified for different geometric shapes. Through this step, the system outputs no longer scattered contour points, but a candidate weld centerline extending spatially along the weld and its two boundary zones.

[0029] After obtaining the candidate weld centerline, the industrial control / edge computing unit discretizes it into sequentially arranged centerline nodes and establishes the weld mileage value with the weld start point as the zero point. Weld mileage value The calculation uses the cumulative form of centerline arc length: ; Among them, weld mileage value : No. The cumulative length of each centerline node relative to the weld start point is used as a position marker for subsequent steps; the node's lateral coordinates : No. The lateral position of each centerline node in the workstation's reference coordinate system is determined by the workpiece width and the sensor's field of view; the longitudinal coordinate of the node... : No. The position of each centerline node along the weld extension direction, and its value range is determined by the total length of the weld; node height coordinates : No. The height of each centerline node relative to the reference tooling surface is determined by the plate thickness, bevel depth, and clamping posture.

[0030] Using weld seam mileage values The reason for not directly using simple image row numbers or scan sequences is that image row numbers only reflect the sampling position inside the sensor and cannot stably reflect the true spatial distance on the weld seam, while weld seam mileage values... The 3D path length is directly fixed as a location label that can be inherited in subsequent steps.

[0031] Establishing weld mileage values Simultaneously, the industrial control / edge computing unit also writes the joint type, bevel form, and weld bead layer into the weld reference map. This allows subsequent steps to reference the same weld mileage value. At that time, we not only know which section of the weld it is, but also what type of joint it belongs to, what type of bevel area it is in, and which pass it corresponds to.

[0032] For example, on a closed-curve butt weld of a ship plate, if the initial arc segment is a U-shaped bevel and the middle section transitions to a V-shaped bevel after partial grinding, the weld reference map will correlate the bevel shape change with the weld mileage value. Corresponding records. If a certain mileage segment is subsequently identified as a boundary uncertainty area by post-weld inspection, this information can be accurately reverted to the corresponding bevel segment recorded during the pre-weld mapping stage, rather than remaining only in an isolated inspection image.

[0033] The local weld feature points output by the visual acquisition unit during welding are first represented in pixel coordinates. The industrial control / edge computing unit calls the camera intrinsic parameters to backproject the pixel coordinates into camera coordinates, and then converts the camera coordinates into tooling coordinates according to the pre-calibrated extrinsic parameter matrix from the camera coordinate system to the tooling coordinate system. Afterwards, the tooling coordinate points are projected onto the weld centerline in the weld reference map, the centerline node number corresponding to the projected point is obtained, and the corresponding weld mileage value is obtained through interpolation between adjacent centerline nodes. When the projection point falls between two adjacent centerline nodes, the weld mileage value is calculated according to the ratio of the arc length from that point to the two centerline nodes. .

[0034] In use, the weld object is reconstructed as a continuous, indexable, and segmentable weld entity, and any subsequent local image results can be located to a unified weld mileage value. Different joint types are bound to the weld reference map during the mapping stage, so there is no need to re-determine the basic geometric category when calling it in subsequent steps; if a certain centerline cannot be formed continuously, the segment is marked as a segment to be re-scanned in the current stage, thus avoiding sending unstable boundaries into the next step from the source.

[0035] The welding images, robot pose, controller transmission and reception times, and execution feedback times all originate from a single workstation, but are physically separated into several links: the vision acquisition link is responsible for imaging and transmission, the controller link is responsible for command reception and issuance, and the execution link is responsible for robot movement and feedback. If packaging is not performed before proceeding to the next step, the local geometric features obtained in the next step cannot be distinguished as either the weld position at the image acquisition time or the weld position at the controller correction time.

[0036] Therefore, all temporal quantities related to the local image of the weld are summarized into a traceable original time series, and then compared with the weld mileage value. The data is bound and encapsulated to form a timing synchronization data packet that can be directly invoked in the next step.

[0037] In one embodiment for longitudinal seam welding of pressure vessel sections, the in-weld vision acquisition unit is mounted on a fixed support in front of the welding torch, with its viewing axis pointing towards the local weld area in front of the welding torch's forward direction. The robot controller interface unit and the industrial control / edge computing unit transmit the current pose, current welding speed, and swing phase via an industrial communication link. Execution feedback consists of joint positioning markers or end-effector pose feedback output by the robot controller. The in-weld vision acquisition unit is responsible for seeing the weld seam, the robot controller interface unit is responsible for knowing where the welding torch has moved, and the execution feedback is responsible for confirming when the control command is actually applied to the end of the welding torch; all three are indispensable. If only the in-weld vision acquisition unit and the robot's current pose are present, but the controller's receiving, sending, and executing feedback moments are missing, subsequent steps can only yield a vague concept of delay, rather than a clearly segmented total delay.

[0038] When each frame of welding image enters the system, the industrial control / edge computing unit immediately writes the image acquisition time and the image processing completion time after image preprocessing is completed. When the robot controller interface unit receives the correction request issued in the previous cycle or the status query of the current cycle, it writes the controller receiving time and the controller issuing time, respectively. When the robot actuator completes the corresponding action and generates feedback, it writes the execution feedback time.

[0039] Therefore, the system has a total delay. The original components: ; Among them, total latency The complete time chain from the initial formation of the welding image to the robot's feedback return is used by subsequent steps to determine the time misalignment between the current image and the actual correction position of the welding torch; acquisition delay. The time taken for the visual acquisition unit to complete exposure, readout, and transmission to the industrial control / edge computing unit during welding is affected by camera frame rate, exposure strategy, and link bandwidth; processing latency. The time taken for the industrial control / edge computing unit to complete the current frame preprocessing, region truncation, and coarse extraction of local geometric features is affected by the operating environment, thread scheduling, and algorithm path length. Receive delay The time overhead for the robot controller to receive information related to the current loop, the range of which is affected by the communication link and the controller's buffering strategy; scheduling delay. The time overhead for queuing and issuing correction requests within the controller, the range of which is affected by the control cycle and task priority; execution delay. The time taken for the robot end effector to go from receiving an action command to outputting identifiable feedback is affected by the robot's motion state and the granularity of the feedback.

[0040] Using total delay The fractional encapsulation, rather than simply recording the current loop duration, has two direct benefits. First, subsequent steps can clearly identify whether the delay originates from the acquisition, processing, or execution side, thus enabling unified handling of delays from different sources during future point-of-effect prediction, rather than conflating all delays into an uninterpretable black box. Second, if a component exhibits continuous anomalies in a local segment, such as execution delay... If the time lag is consistently large in a certain corner segment, the next step will not only know that there is a time lag in that segment, but also that the time lag is closer to the motion execution bottleneck than the visual acquisition bottleneck.

[0041] After completing the timing chain writing, the industrial control / edge computing unit does not directly send all image frames to the next step, but first performs an entry validity check. This check includes three layers: the first layer checks the weld mileage value. Can it be matched with a unique weld section in the current image? The second layer checks the total delay. The first layer checks whether all components are complete, especially whether the controller receiving time, controller sending time, and execution feedback time are missing; the third layer checks whether the image frame has a whole frame missing, partial coverage, obvious overexposure, or central area occlusion caused by spatter adhesion. If the first layer fails, it means that the current image cannot be bound to a unified weld entity, and the frame is marked as a mismatch frame that cannot be used for geometric prediction; if the second layer fails, it means that although the current image sees the weld, it cannot enter the future action point inference, and the frame is marked as a low confidence input, only its weld mileage value is retained. The original image data is used for subsequent degradation paths; if the third layer fails, it means that the current frame is not suitable for full correction. The system retains the stable state of the previous moment and encapsulates the frame as an abnormal frame for credibility arbitration in step three.

[0042] In one approach, the entry data packet in step two should at least include: a weld reference map and the weld mileage value corresponding to the current image. Connector type, bevel type, image frame number, total delay It includes information on its components, robot pose, welding speed, swing phase, frame validity, and causes of anomalies.

[0043] The reason is that step two requires not isolated images or isolated moments, but a composite object that indicates which weld seam it is located on, which type of joint it belongs to, which time chain it is in, and whether it is currently complete and usable. If this information is stored separately, step two must reassemble it when it is called, which can easily lead to object mismatch; by encapsulating it once, step two can perform local geometric calculations and future action point predictions around the same entry data packet.

[0044] In use, the image during welding, the robot's pose, and the time of transmission and reception by the controller correspond to the same object. The next step accepts a weld segment with timestamps and spatial indexes, not an image. Abnormal frames are isolated or marked as low confidence before step two to prevent the next step from performing a full extrapolation of distorted input.

[0045] As an alternative embodiment, the pre-weld three-dimensional seam finding unit is not limited to a line laser contour sensor. When using a structured light projection head, a binocular vision head, or a device with equivalent spatial reconstruction capabilities, as long as it can output a continuous centerline that can extend along the weld and establish weld mileage values, it is acceptable. All of these can be incorporated into the same technical concept; the visual acquisition unit in welding is not limited to a single visible light camera. When using a paraxial high dynamic range imaging head, an off-axis local contour acquisition head, or a main / backup dual-channel imaging combination, as long as the image frame and total latency can be compared, it is acceptable. The data packets are encapsulated together as the entry data packet for step two, and are consistent with this implementation method. The controller interface link is not limited to a certain brand of controller. As long as it can stably provide the controller receiving time, controller sending time, and execution feedback time, it can support the timing synchronization acquisition logic of this step.

[0046] Step 2: In response to the weld reference map, weld mileage coordinates and timing synchronization data packets output in Step 1, a local geometric state consistent with the weld mileage coordinates is first formed on the local weld image. Then, the look-ahead distance and total time delay are converted into the mileage difference propagating along the weld mileage coordinates, thereby obtaining the target position, target attitude and target swing correction requirements at the future point of action, and organizing them into a prediction result set that can be directly called in Step 3.

[0047] Although step one provides the weld reference map and weld mileage coordinates, the visual acquisition unit during welding still sees a transient local weld fragment obscured by arc light, spatter, reflections, local shadows, and smoke. If this fragment is not geometrically calculated to correspond to the weld reference map, step two can only arrive at a vague conclusion that the current frame roughly contains a weld, failing to pinpoint which segment of the weld this local fragment corresponds to, and unable to firmly bind subsequent trajectory corrections, attitude corrections, and sway corrections to the same spatial object. Therefore, the primary task is not simply edge extraction, but rather, under the constraint of the weld mileage coordinates, constructing a local geometric state oriented towards the welding torch control chain, ensuring that the weld appearance in the local image, the weld reference map from step one, and the current pose of the robot controller all fall within the same reference relationship.

[0048] Optimal extraction path for local geometry: Lateral deviation: the normal distance from the local weld centerline to the current projection point of the welding torch; Vertical deviation: the height difference between the current height of the welding torch and the target weld surface or the bottom of the target bevel; Tangential deviation: obtained by performing a first-order difference on three or five consecutive centerline nodes near the current weld mileage; Curvature: parameterized by the arc length of the local centerline, and then obtained from the rate of change of adjacent tangential deviations; Gap width: the distance between the two edges measured on the normal section of the local weld; Attitude error: obtained by the angle between the welding torch axis direction and the local weld normal or the desired welding attitude direction.

[0049] In one embodiment for longitudinal seam welding of large storage tank walls, the following actions are all performed by the industrial control / edge computing unit. The in-weld vision acquisition unit is responsible for continuously outputting image frames, and the robot controller interface unit is responsible for outputting the current pose, current welding speed, and swing phase. After receiving a frame image, the industrial control / edge computing unit first reads the initial weld seam mileage value corresponding to that frame according to the data packet number written in step one, and then extracts a local weld seam area near the initial mileage value. During arc welding or laser-arc hybrid welding, bright arc areas, welding wire ends, nozzle edges, and local spatter highlights often appear simultaneously near the welding torch. If the local area is not first defined by the initial weld seam mileage value, image processing can easily involve bright boundaries unrelated to the weld seam in subsequent calculations. After the local area is defined, the industrial control / edge computing unit rearranges the image coordinates according to the tangential direction of the corresponding segment in the weld seam reference map, so that the main direction in the local image is consistent with the tangential direction of the weld seam. After this rearrangement, the same weld section can still maintain a similar directional expression under different welding torch postures, and the definitions of subsequent lateral deviation, vertical deviation, local curvature and welding torch relative posture error no longer change with screen rotation.

[0050] The extraction of local geometry is based on the weld mileage values ​​given in step one. Instead of using the center of the image as the default location for the weld center, the industrial control / edge computing unit first retrieves the weld mileage value from the weld reference map. Based on the reference tangent, reference curvature, and joint type of the corresponding section, the current local weld area is then subjected to direction normalization and boundary cleanup. For V-groove butt joints, the local geometry is preferably determined by the bevel edges on both sides, the bottom zone of the gap in the middle, and the angle between the welding torch and the bevel. For U-groove butt joints, the local geometry is preferably determined by the arc transition edge, the center zone at the bottom of the gap, and the arc normal of the welding torch. For corner joints, the local geometry is preferably determined by the intersection boundary of the two base materials, the transition zone of the weld leg, and the orientation of the welding torch relative to the intersection line. The purpose of this setting is that local weld segments of different joint types have different geometric stability elements. If all joints are processed using the same image feature extraction rules, some joints will experience center position drift under local brightness changes. By introducing the joint type into the local geometry determination in advance, the same weld mileage section can always use a solution path that matches its geometry.

[0051] After completing direction normalization and connector type binding, the industrial control / edge computing unit extracts the lateral deviation. Vertical deviation Local tangential Local curvature Gap width and attitude error Among them, lateral deviation Used to characterize the lateral deviation of the current projected position of the welding torch relative to the centerline of the local weld; vertical deviation. Used to characterize the deviation of the current height of the welding torch relative to the target weld surface or target bevel depth; local tangential Used to characterize the main extension direction of the current weld section; local curvature Used to characterize the degree of bending of the weld centerline at the current position; gap width Used to characterize the degree of opening at the current weld joint or bottom; attitude error This is used to characterize the deviation between the welding torch axis and the local weld geometry normal. The quantities mentioned above are not isolated but rather represent the same weld mileage value. This establishes a set of local geometric states simultaneously. Therefore, if welding torch oscillation becomes unstable in a certain weld section, the weld center is not simply shifted to one side; instead, the gap width of that section can be used simultaneously. Local curvature and attitude error This explains why this section is difficult to track.

[0052] After the local geometric state group is extracted, the industrial control / edge computing unit does not directly output the correction command for the current position. Instead, it first utilizes the total delay already written in the data packet from step one. Compared with the current welding speed Calculate the weld seam displacement corresponding to the time misalignment, and then compare it with the effective look-ahead distance of the visual acquisition unit during welding. By comparing the distances between the current observation point and the point where the correction is performed, the distance difference can be obtained. : ; Among them, mileage difference The remaining or leading distance along the main direction of the weld between the weld position currently seen in the image and the actual position where the welding torch performs corrections; the value can be positive, zero, or negative; effective look-ahead distance. The effective distance along the weld seam direction between the center of the main observation area of ​​the visual acquisition unit during welding and the point of action of the welding torch is determined by the sensor installation position, the center of the field of view and the geometric relationship of the welding torch. Obtained from sensor mounting pose calibration; when the welding torch pose changes... The current attitude is projected onto the tangential direction of the weld and recalculated; the initial value is given by offline calibration and corrected in real time by the current attitude during operation.

[0053] Current welding speed The current speed of the robot's end effector along the weld seam, the range of which is determined by the process recipe and the current state of the controller; total delay. The unified definition inherited from step one represents the complete time chain from image formation to the return of execution feedback; Using mileage difference Used as an entry point for subsequent predictions, rather than directly based on total latency. The reason for this judgment is that the total delay itself only provides the duration, and only when compared with the current welding speed... and effective forward distance Only after joint conversion can it truly be mapped to the weld seam mileage coordinates, becoming the spatial quantity that can be executed subsequently.

[0054] When the mileage difference When positive, it indicates that the current image observation point is still ahead of the position where the welding torch will reach in the future, and the search for the future point of action should proceed forward along the weld reference map; when the mileage difference... When the time difference is close to zero, it indicates a high degree of time alignment between the observation point and the execution point, allowing for approximate direct referencing of the local geometric state group; when the mileage difference... A negative value indicates that the welding torch execution chain is significantly lagging behind, and the weld section seen in the current image is no longer suitable as a direct control target. Future action points should be determined more reliant on the weld reference map and local curvature extrapolation. (Look-ahead distance) Total latency Compared with the current welding speed Converted into mileage difference This transforms temporal misalignment into spatial misalignment; after the local geometric state and mileage difference are coupled, the solution object and solution direction of the future point of action are clearly defined.

[0055] Even if the local geometric state set and odometer difference have been obtained The system still cannot directly send this set of data to the control chain unchanged. The reason is that the future point of action is not a specific pixel in the current image, but rather the point in time the welding torch experiences after the total delay. The actual weld location that will be reached and requires correction will then be determined. This location is constrained by both the global continuity of the weld reference map from step one and the local instantaneous changes in the current local geometry. If extrapolation is made solely based on the weld reference map, the subtle shifts in the current segment caused by heat input, local warping, or gap changes will be ignored. If only short-range translations are made based on the current local geometry, the global orientation will be lost in closed curve segments and segments with abrupt changes in local curvature.

[0056] Therefore, the task is to perform a local interpolation extrapolation constrained by the weld reference map, simultaneously incorporating global continuity and local instantaneity into the solution of future action points.

[0057] In an implementation of a closed-curve butt welding method for a boom body of engineering machinery, when the welding torch travels along the rounded transition section, the weld seam seen in the current image is not a straight line, but rather in a continuous bending state. At this time, the industrial control / edge computing unit first uses the current weld seam mileage value in the weld seam reference map. Centered on a point, a continuous curvature segment is extracted forward as a candidate prediction band, and then the obtained local curvature is... Local tangential and gap width Write this into the candidate prediction band. The weld reference map gives the pre-weld static geometry, while the local curvature... and gap width It reflects the instantaneous state during welding. When the two are superimposed, the future point of action will not deviate from the original direction of the weld, and can also reflect the local deformation characteristics of the current section.

[0058] Furthermore, the industrial control / edge computing unit uses the current weld mileage value Starting from the distance difference and local curvature Calculate the future action point weld mileage value : ; Among them, the future action point weld mileage value The weld position that should be corresponding to when the welding torch performs corrections, used to subsequently extract the target tangent and target normal at that position from the weld reference map; current weld mileage value. The weld location currently bound to the image is inherited from the entry data packet in step one; mileage difference. Used to represent the offset along the weld direction obtained from time misalignment conversion; local curvature The degree of bending in the current weld section is determined by the bending shape of the joint path; if If it falls below the threshold, then Setting it to 0 degenerates into linear progression.

[0059] Scale factor Used to adjust mileage difference The basic weight in mileage advancement is preferably a positive value and is fixed in the process formula after system calibration; Defined as the mileage difference-based propulsion coefficient, initially set to 1, calibrated by unloaded path tracking test welding; coupling factor Used to adjust local curvature For future action point weld mileage values The additional effects, preferably non-negative values, are considered. Defined as the curvature coupling coefficient, its initial value is determined by trial welding of a closed-curve joint, and it is only used when... It takes effect when the curvature threshold is exceeded; when the weld shows obvious bending in a localized area, it is only based on the mileage difference. Linear propagation will cause the future point of action to fall on the tangential extension line before the bend, thus reducing the local curvature. Once introduced together, the future action point weld mileage value It will be moved forward appropriately along the actual weld seam direction, so as to more closely match the actual running path of the closed curve segment and the rounded transition segment.

[0060] In practical implementation, if the industrial control / edge computing unit detects that the curvature change within the current candidate prediction band is gradual, then the weld mileage value at the future application point will be adjusted accordingly. Piecewise linear interpolation is used; if continuous bending is detected within the candidate prediction band, the weld mileage value at the future action point is adjusted. The corresponding spatial points are interpolated using spline interpolation or arc-length-preserving interpolation. The parallel implementation paths are presented here to ensure that the inventive principle of this step is not limited by a single solver, as long as the future weld mileage value at the point of application is considered. The results were derived by combining the weld reference map and the current local geometry, and all belong to the same technical concept.

[0061] Future action point weld mileage value Once determined, the industrial control / edge computing unit extracts the reference tangent and reference normal corresponding to the mileage location from the weld reference map, and then superimposes them onto the current local geometric state group to form the future target state of the action point. The future target state of the action point includes at least the target lateral correction, target vertical correction, target attitude correction, and target sway correction requirements. Among them, the target lateral correction inherits the lateral deviation. The mileage value of the weld seam at the future action point The local boundary changes of the corresponding section; the target vertical correction inherits the vertical deviation. Variations in bevel depth or weld formation height; target attitude correction amount inherits attitude error. The tangential change at the future point of action; the target oscillation correction requirement is determined by the gap width. It is determined together with the curvature change of the segment where the future point of action will be located.

[0062] In the first embodiment, when the gap width of the segment where the future point of action is located... When the width increases, the industrial control / edge computing unit provides a large swing coverage for the segment; when the local curvature of the segment where the future action point is located... When the value increases, the industrial control / edge computing unit imposes constraints on the rate of change of the attitude correction amount, ensuring that the full correction or amplitude limiting correction in step three has a clear upstream target.

[0063] To prevent the target state of future action points from fluctuating due to instantaneous disturbances in a single frame, the industrial control / edge computing unit also performs a sequential consistency check on the target state over several consecutive frames. This check does not change the weld mileage value of future action points. Instead of defining the target state in step two, it checks whether the target lateral correction, target vertical correction, target attitude correction, and target sway correction requirements generated in adjacent frames change continuously along the weld seam mileage coordinates. If continuity is established, step two outputs a complete prediction result set; if continuity is not established, but the local geometric state group of the current frame has not been marked as an abnormal frame in step one, the target state of the current frame is retained and a continuity review mark is added for further processing in the credibility arbitration in step three. The purpose of this approach is that step two is only responsible for forming the target state of the future action point, without deleting all anomalies in this step, but explicitly passing the target states that need further verification to step three, ensuring clear responsibilities and uninterrupted connection between steps.

[0064] When used, the current local geometry of the weld is mapped to the weld mileage value at the future point of action. Subsequent control no longer depends on the current image pixel position. The target position, target attitude and target swing correction requirements are generated within the same weld seam mileage coordinates, enabling step three to perform credibility arbitration and drift compensation with a unified object. The interpolation extrapolation process simultaneously absorbs the global continuity of the weld seam reference map and the instantaneous changes of the current local geometric state, so that the closed curve segment, thermal deformation segment and gap fluctuation segment can all obtain an implementable future action point description.

[0065] Step 2 output includes the weld mileage value at the future action point. The prediction result set includes the target lateral correction amount, target vertical correction amount, target attitude correction amount, target sway correction requirement, and continuity marker. This result set can be directly used as the input for the confidence arbitration, drift compensation, and robustness control in step three.

[0066] As a parallel example, the extraction of local geometric states is not limited to the single-edge method. For the main channel of line structured light, it is preferable to form the lateral deviation by combining fringe center extraction with bevel boundary positioning. With gap width For high dynamic range imaging channels, it is preferable to form a lateral deviation through the relative relationship between the weld light-dark transition zone and the nozzle edge. With attitude error For a dual-channel layout, it is preferable to use the main channel to provide a local tangential direction. and local curvature In preparation for vertical deviation compensation of the passage and gap width Future action point weld mileage value The solution is not limited to a single mathematical form, as long as it also references the current weld mileage value. Mileage difference and local curvature The output of subsequent executable target states all belong to the same inventive concept. The purpose of this setting is to prove that the inventive principle of step two has universality. Its core does not lie in a specific image operator, but in transforming the spatial and temporal references locked in step one into future action point states that the welding torch can use.

[0067] lateral deviation : Normal distance from the current projection point of the welding torch to the center line of the local weld; backup channel to compensate for vertical deviation. The height difference between the current height of the welding torch and the target weld surface or the bottom of the target bevel; local tangential direction. : The unit tangential vector obtained from the first-order difference of the local continuous centerline nodes; local curvature : Obtained from the first-order rate of change of adjacent tangential vectors with respect to the local arc length; gap width : The distance between the two edges on the local normal section; attitude error : The angle between the direction of the welding torch axis and the direction of the desired local welding posture.

[0068] Step 3: Based on the future action point state output in Step 2, establish a robust control chain driven by image credibility, channel consistency, coordinate drift, and execution boundary. The welding torch can only perform full correction when the future action point state meets the three conditions of visibility, location, and executability. If the conditions are not met, amplitude limiting correction or degradation control is performed to prevent subsequent control actions on the same weld entity from being amplified by single-frame anomalies, continuous contamination, and installation deviations.

[0069] While the future action point state generated in step two has achieved spatial and temporal alignment, it is still based on the welding image. This welding image is continuously affected by variations in arc intensity, spatter obstruction, shielding window adhesion, flue gas, nozzle edge intrusion, and high-brightness reflection from the welding wire tip within the workstation. If the future action point state is directly sent to the robot controller, a short-term overexposure, a stripe break, or a misjudgment of obstruction will be directly converted into welding torch movement correction. Therefore, identifying the weld seam is not the sole prerequisite; instead, it is first determined whether the weld seam segment possesses sufficient image credibility to support control actions at the current moment, and then whether the future action point state is permitted to enter the control chain.

[0070] In an implementation for multi-layer, multi-pass longitudinal seam welding of wind turbine towers, the following actions are performed by the industrial control / edge computing unit: the welding vision acquisition unit continuously outputs the main channel image; if the workstation is configured with a backup channel, the backup channel simultaneously outputs auxiliary images. The industrial control / edge computing unit receives the future action point weld mileage value output in step two. Next, return to the original local weld area that produced the result and check the proportion of bright areas, the degree of stripe continuity, the length of local blur, the occlusion area, and the residual continuity with the previous stable frame in turn.

[0071] Although the future action point state has been interpolated and extrapolated, its basis is still the observability of the current local weld area. When the area is covered by spatter, or the stripe only retains one side, the target lateral correction and target attitude correction given in step two should not be regarded as high-permission inputs.

[0072] As a supplement: when the local curvature change near the current weld mileage value is lower than a preset smoothing threshold, the industrial control / edge calculation unit uses piecewise linear interpolation to obtain the position of the future action point on the weld centerline; when the local curvature change near the current weld mileage value is higher than the smoothing threshold, spline interpolation or arc-length-preserving interpolation is used to obtain the position of the future action point. The interpolation input is the current weld mileage value. Nearby continuous centerline nodes, local tangent Local curvature and future action point weld mileage value The output is the three-dimensional coordinates of the future point of action in the tooling coordinate system.

[0073] The industrial control / edge computing unit constructs image credibility for each frame's local weld area. Image credibility Instead of using a single threshold comparison, the saturation pixel ratio, occlusion rate, image blur, stripe continuity, and historical residual consistency are combined into a continuous weight, specifically expressed as follows: ; Among them, image credibility The current local weld area's support for the control chain is preferably within the range of 0 to 1; saturated pixel ratio. The ratio of the area of ​​pixels exceeding the upper grayscale threshold within a local weld seam region to the total area of ​​the region, with a value ranging from 0 to 1; Occlusion rate. : The percentage of the target area covered by spatter, nozzle edges, or welding wire tips, ranging from 0 to 1; continuous length This represents the ratio of the effective stripe or boundary length to the local window length, maintaining geometric consistency along the main direction of the weld, preferably ranging from 0 to 1; the ratio of the length of the pixel chain that continuously satisfies boundary geometric consistency along the main direction of the weld to the local window length; ambiguity. : Local boundary diffusion degree, preferably ranging from 0 to 1, with a larger value indicating greater boundary diffusion; the result of normalized local boundary grayscale transition band width; Residual consistency The similarity between the current local weld area and the previous stable frame after normalization in the main weld direction, preferably ranging from 0 to 1; the normalized similarity between the current frame and the most recent stable reference frame after direction normalization of the local weld area.

[0074] Weighting coefficient Weighting coefficients Weighting coefficients Weighting coefficients Weighting coefficients These factors are used to adjust the image credibility of each image. The contribution of each component is preferably positive and sums to 1. The weights are obtained from trial welding calibration and written into the process recipe according to joint type, process type, and sensor arrangement; they are initialized with equal weights by default and subsequently updated offline based on the statistics of erroneous correction events.

[0075] Among these factors, image distortion at the welding site occurs gradually. When there is a thin layer of spatter on the protective window surface, local boundaries will first show blurring and diffusion, and contrast reduction; variations in arc intensity at different current levels will lead to changes in the saturated pixel ratio. By continuously recording these changes, the reliability of the image can be determined. The process has evolved from stable availability to the need for amplitude limiting control, and then to allowing degradation control.

[0076] When the workstation is configured with a primary channel and a backup channel, the industrial control / edge computing unit obtains the image reliability of the primary channel. Subsequently, the weld mileage values ​​of the main and backup channels at the same future action point were also determined. The consistency of the results is checked. This consistency comparison is not between the two images themselves, but rather between the deviations between the target lateral correction, target vertical correction, and target pose correction calculated by each of the two channels. To this end, the industrial control / edge computing unit constructs a consistency deviation measurement. : ; Among them, consistency deviation : The degree of difference between the main channel and the backup channel regarding the state of the same future point of action; the larger the value, the more inconsistent the conclusions of the two channels; lateral deviation of the main channel. Lateral deviation of the backup channel These represent the lateral deviations calculated for the main channel and the backup channel, respectively; and the vertical deviation of the main channel. Vertical deviation of the backup channel These represent the estimates of vertical deviation for the two channels respectively; the attitude error of the main channel. and backup channel attitude error These represent the estimates of the relative attitude error of the welding torches in the two channels, respectively. Weighting coefficient Weighting coefficients Weighting coefficients These are used to adjust the deviation of different deviation terms from the consistency amount. The contribution of the signal should preferably be positive. Even if the grayscale of the primary and backup channels differs greatly, as long as they describe the target state of the same future point of action in the same way, it means that the current image distortion does not affect the control conclusion. If the conclusions of the two channels are significantly different, it means that the current state of the future point of action has lost its support.

[0077] Based on image credibility and consistency deviation The future action point states are divided into high-permission, medium-permission, and low-permission states. The high-permission state corresponds to full correction, the medium-permission state corresponds to amplitude limiting correction, and the low-permission state corresponds to degradation control. In this way, step three transforms the purely image-related issue of whether it is clear or unclear into the control chain's actions of allowing, restricting, or stopping traffic.

[0078] In practice, the future action point state is assigned image confidence before entering the control chain. and consistency deviation The primary and backup channels cross-verify the state of the same future point of action, and subsequent steps deal with the layered control permission state.

[0079] Even if the future point of action status has passed the verification of image reliability and consistency deviation, the system may still experience coordinate drift due to factors such as thermal expansion of the sensor bracket, slight deflection due to welding torch collision, re-clamping of the fixture, workpiece thermal deformation, and installation differences after replacement of the protective window. If such drift is not identified, the future point of action weld mileage value output in step two will be lost. Although it appears reasonable within the image, its corresponding spatial location has deviated from the weld reference map. Even if subsequent control is performed under high permissible conditions, the welding torch will still be pulled away from the actual weld entity. Therefore, before switching control modes, an online drift check must be performed, and the coordinate transformation relationship must be updated or the mode switched to protection mode when the drift reaches the boundary.

[0080] Among them, when Above the upper threshold and Below the consistency threshold and When the drift is below the first drift threshold, full correction is applied; when... Located between the upper and lower thresholds, or Exceeding the consistency threshold but not the mismatch threshold, or When the amplitude is between the first and second drift thresholds, amplitude limiting correction is applied; when Below the lower threshold, or Exceeding the mismatch threshold, or Exceeding the second drift threshold, and with consecutive abnormal counts. When the preset frame rate is reached, degradation control is initiated. The preset threshold is obtained from trial welding calibration and stored in the process recipe according to joint type and process type. This closes the logic.

[0081] In one implementation of automated welding for long welds in bridge box girders, the welding duration is relatively long, and the welding torch support and sensor mounting base gradually heat up due to heat input. Within each preset weld mileage range, the industrial control / edge computing unit compares the reference edge, nozzle profile, or fixture reference identified by the visual acquisition unit during welding with the reference relationship of the same mileage segment in the weld reference map from step one. If the comparison result shows a smooth shift of the reference edge along the lateral or vertical direction, it indicates a slow drift in the coordinate transformation relationship; if the comparison result changes abruptly within a short period, it indicates a loose support, collision, or relative displacement of the workpiece. This drift does not always manifest as severe distortion; in many cases, it only manifests as a continuous bias of the future action point to one side. Without the constraint of the reference edge or fixture reference, this state is often mistakenly interpreted as a lateral change in the weld itself.

[0082] Preferably, the reference object is at least one of the welding torch nozzle leading edge profile and the edge of the fixture reference hole. The position and orientation of the currently observed reference object are directly extracted from the welding image, and the position and orientation of the reference object are taken from the corresponding mileage section of the weld reference map established in step one. (Reference lateral offset) The horizontal difference between the two is taken into account, with reference to the vertical offset. The vertical difference between the two, with reference angle offset. This represents the difference in the angle between the two directions.

[0083] Among them, the industrial control / edge computing unit constructs the drift amount based on the deviation of the reference edge within the same weld seam mileage segment from the baseline map and the current observation. : ; Among them, drift amount : The degree of deviation of the current coordinate transformation relationship relative to the weld reference map; reference lateral offset The amount of change in the lateral position of the currently observed reference edge or nozzle profile relative to the base map; the reference vertical offset. The change in the vertical position of the corresponding reference object; reference angle offset. The change in the orientation of the reference object relative to the orientation of the base map; weighting coefficient. Weighting coefficients Weighting coefficients These are used to adjust the drift amount of the three types of offsets respectively. The contribution, preferably a positive value, is used. The drift amount is employed. Instead of judging based on a single deviation, the drift at the welding site is often a composite deviation that includes both positional and orientation drift. Single indicators are prone to overlooking the combined effects caused by slight rotation of the support or slight lifting of the workpiece.

[0084] When drift amount When within the allowable range, the industrial control / edge computing unit maintains the original coordinate transformation relationship; when the drift amount When the drift exceeds the first boundary but not the second boundary, the industrial control / edge computing unit progressively updates the transformation relationship between the visual coordinate system and the weld seam mileage coordinates, and downgrades the current future action point state to a first-level control permission; when the drift amount When the second boundary is exceeded, the industrial control / edge computing unit no longer trusts the original coordinate transformation relationship, directly blocks high-permission and medium-permission control, and only retains the protection mode.

[0085] Image credibility Consistency deviation and drift amount Under the combined effect, the industrial control / edge computing unit assigns execution modes based on the current and future states of action. When image credibility... Stability, consistency deviation Smaller and drift amount When the image is within the allowable range, perform a full correction; when the image confidence level is... Decrease or consistency deviation Increased but not reaching the protection boundary, or drift amount When entering the progressive update interval, perform amplitude limiting correction; when image confidence... Persistently low, deviation from consistency Sudden increase or drift When the second boundary is exceeded, degradation control is executed. Degradation control includes at least the following action chain: The industrial control / edge computing unit first locks the local tangential direction corresponding to the current weld mileage value, allowing only the robot controller to maintain the current tangential direction; then, it limits the target lateral correction and target attitude correction within the safety boundary; subsequently, it switches the swing motion to hold or pause state; if it remains in a low-permission state for several consecutive frames, the robot controller performs deceleration and enters a local search and recapture; if the local search and recapture fails, it enters the welding abort determination.

[0086] To prevent the system from frequently switching between high and low permission states, the industrial control / edge computing unit is configured with a continuous anomaly counter. Continuous anomaly count This represents the cumulative number of frames that are in a low-permission state or drift out of bounds for a certain number of consecutive frames; its value is a positive integer. Current image confidence level. Consistency deviation and drift amount Although there may be short-term fluctuations within a single frame, only when there are consecutive abnormal counts The system only officially switches from clipping correction to degradation control or protection mode when the preset frame rate is reached. A continuous anomaly count is set. The reason is that spatter and arc fluctuations during welding often last only a very short time, and if continuous anomaly counting is not set... The system may overreact due to a single momentary occlusion.

[0087] The current frame is determined to be in a low-permission state, or the drift amount is... When the second boundary is exceeded, the number of consecutive anomalies is counted. add When the current frame recovers to a high-permission or medium-permission state, the continuous anomaly count is... Reset to zero; when consecutive abnormal counts are reached. When the preset frame rate threshold is reached, the system switches from amplitude limiting correction to degradation control or protection mode. The preset frame rate threshold is written into the process recipe based on the trial welding calibration.

[0088] In use, coordinate drift is identified and graded during the welding process; the future action point state is bound to a defined execution mode before execution, and the control chain no longer fluctuates drastically due to single-frame anomalies; continuous anomaly counting... This enables the degradation control and protection modes to have persistent criteria. As a result, the output of step three is no longer the exposed future action point state, but a constrained control result set with control modes, drift update flags, anomaly persistence flags, and safety boundary flags. This result set will serve as the upstream basis for step four to perform post-weld quality mapping, rework write-back, and closed-loop feedback.

[0089] As an alternative embodiment, the reference edge is not limited to the nozzle profile, fixture reference, or weld side fixed mark; any positional object that can be stably reproduced under weld mileage coordinates can be used as the basis for drift inspection; image reliability The constituent items are not limited to saturated pixel ratio, occlusion rate, continuity length, blurriness, and residual consistency; the action chain of degradation control is not limited to deceleration, maintaining the current tangential direction, pausing swing, and local search recapture, as long as its execution purpose is to compress the high-risk future action point state within the safe boundary, it belongs to the equivalent implementation of this step. Through these parallel extensions, the inventive concept of this step remains singular, that is, coupling the future action point state with control permission, coordinate validity, and execution boundary.

[0090] Step 4: In response to the constrained control result set output in Step 3 and the weld reference map, weld mileage coordinates, joint type identifier, and process type identifier output in Step 1, perform rule set mapping, defect segment generation, rework task encapsulation, and front-end feedback marking on the post-weld geometric inspection results. This transforms the post-weld results from static inspection conclusions into executable rework objects and inheritable front-end constraints.

[0091] The weld surface contour, weld cross-sectional contour, or weld surface image collected by the post-weld geometric inspection unit are essentially just geometric representations of the weld bead after formation. If these representations are not aligned with the weld mileage coordinates established in step one, nor are they associated with the joint type identifier, process type identifier, and the constrained control result set output in step three, then even if defect identification is subsequently completed, only scattered post-weld abnormal areas can be obtained, without clearly identifying the start and end positions of these abnormal areas in the entire weld, the corresponding layer relationships, and their association with pre-weld and during-weld conditions.

[0092] Therefore, the post-weld geometric inspection results are remapped to the same weld mileage coordinates, so that each post-weld formation corresponds one-to-one with a specific weld mileage segment, and on this basis, a quality rule set that matches the current process is selected.

[0093] In one embodiment for automated welding of longitudinal seams in pressure vessel sections, the following actions are all performed by the industrial control / edge computing unit: the post-weld geometry detection unit is responsible for moving along the weld direction to collect the weld surface contour; the robot controller interface unit is responsible for providing the end pose corresponding to the post-weld scanning path; and the weld reference map formed in step one by the pre-weld 3D seam finding unit serves as the alignment reference for this step. The post-weld geometry detection unit preferably uses a line laser contour sensor, but a structured light detection head or a detection device with equivalent contour reconstruction capabilities can also be used. After receiving the post-weld contour sequence, the industrial control / edge computing unit first performs mileage registration on the contour sequence according to the weld reference map in step one, and then maps each contour segment to the corresponding weld mileage value range.

[0094] The reason is that the sampling pitch of the post-weld contour sequence is affected by the scanning speed, sensor trigger cycle and robot end posture changes. If it is not repositioned in the weld seam mileage coordinates, the same physical location will have inconsistent numbers in different workpieces, different passes or different rework rounds. The rework object generated in step four cannot form a unified reference with the previous steps.

[0095] Rule set mapping: First, select the rule set according to the process type identifier; then select the core geometric feature group to participate in the judgment according to the joint type; then compare the corresponding threshold intervals according to the defect category; finally, output the defect category, quality level and rework section.

[0096] After completing the initial alignment of the post-weld contour sequence with the weld reference map, the industrial control / edge computing unit extracts the weld width segment by segment. , Yu Gao Leg length Misaligned edges Biting edge Overlap and not fully welded .

[0097] Among them, weld width This represents the lateral distance between the outer edges of both sides of the weld surface in the current weld mileage section; its value range is determined by the number of process passes and the joint type; reinforcement height. This represents the convexity of the weld surface relative to the base metal reference surface; its value range is determined by both the welding heat input and the filler material; leg length. Primarily used in corner joint scenarios, it indicates the projected length of the weld feet on both sides along the surface of the base material; misalignment. This indicates the degree of relative misalignment of the edges of the base material on both sides of the joint in the height direction; undercut. Indicates the depth or width of a localized groove formed at the interface between the weld and the base metal; lap joint This indicates a localized area where the weld metal extends beyond the base metal surface but does not form a smooth transition with the base metal; incomplete welding. This indicates localized depressions or cross-sectional defects caused by insufficient weld filling. These geometric features are not recorded as isolated points, but rather as groups of weld mileage segments.

[0098] For example, on a corner weld of a box girder, when the post-weld profile sequence enters a certain corner, the industrial control / edge calculation unit first extracts several consecutive sampling sections from the weld mileage section corresponding to that corner, and then calculates the weld width on these sampling sections respectively. and leg length Then, the misaligned edges of that section were... and bite edge They are all attached to the same section object. Therefore, step four focuses on the overall forming characteristics of which weld section, rather than the instantaneous value of a particular section.

[0099] Furthermore, to ensure a single-chain connection between the post-weld geometric features and subsequent rule set mapping, the industrial control / edge computing unit further constructs segment mapping quantities. : ; Among them, the segment mapping amount The comprehensive geometric representation of the current weld mileage section is used as a unified descriptive quantity for the rule set entry point; weld width. , Yu Gao Leg length Misaligned edges Biting edge Overlap and not fully welded Same meaning as before; weighting coefficient To weighting coefficient These are used to adjust the mapping amount of different geometric features in the segment. The contribution value should preferably be non-negative and predefined in the joint type and process type configuration file. Construct the segment mapping quantity. Its role is not to replace specific rule-based judgments, but to compress multiple geometric features within the same segment into a unified entry point that can be sorted, screened, and compared with previous risk segments. In this way, before rule set selection, the industrial control / edge computing unit has already obtained a complete geometric profile of each weld seam mileage segment.

[0100] In segment mapping amount After generation, the industrial control / edge computing unit selects a quality rule set based on the joint type identifier and process type identifier output in step one. If the process type identifier corresponds to ordinary fusion welding, the corresponding section judgment template of ISO5817 is loaded; if the process type identifier corresponds to laser-arc hybrid welding, the corresponding section judgment template of ISO12932 is loaded. The joint type identifier is not redundant information here, because the key geometric features of butt welding and fillet welding are different; the former focuses more on the reinforcement height. Misaligned edges and not fully welded The latter focuses more on leg length Biting edge and overlapping .

[0101] The industrial control / edge computing unit first reads the process type identifier written in step one; when the process type identifier is ordinary fusion welding, it calls the corresponding rule template of ISO5817; when the process type identifier is laser-arc hybrid welding, it calls the corresponding rule template of ISO12932. Then, it reads the joint type identifier and selects the corresponding core geometric feature groups for butt welds and fillet welds respectively to participate in the determination.

[0102] The industrial control / edge computing unit compares the geometric features with the corresponding templates one by one in the current weld seam mileage section and outputs the defect category. and quality grade Among them, defect categories Indicates the forming anomaly category and quality level corresponding to the current section. This indicates the grade result of the current segment within the selected rule set. If multiple geometric anomalies occur simultaneously in the current segment, the anomaly with the stronger impact on weld continuity is preferentially assigned to the defect category. And write the remaining exceptions as auxiliary markers to the current section object.

[0103] When the defect category of two adjacent segment objects Same, and quality level When the weld does not cross the preset grade boundary, it is merged into the same repair section; when the same weld mileage section repeatedly shows the same defect category within the most recent preset number of times. When the defect category of the same section is specified, it is marked as a key area for re-scanning; When the low-permission state in step three co-occurs continuously for a preset number of times, it is marked as a high-sensitivity segment.

[0104] For example, in a closed curve butt weld of a rail vehicle sidewall, the industrial control / edge computing unit performs rule set mapping on three consecutive weld mileage segments, where the first segment is the residual height. Fluctuation, the second segment is a biting edge Add, the third segment is a misaligned edge and not fully welded When superimposed, the system will not merge the three sections into a single defect; instead, it will generate three separate section objects. Each section object includes the weld mileage start point, weld mileage end point, and defect category. Quality grade And the corresponding geometric feature group, so that the subsequent rework and rewriting corresponds to a weld segment with a clear start and end range and defect attributes, rather than an abstract weld evaluation result.

[0105] In use, post-weld geometric inspection results are re-bound to the weld mileage coordinates; joint type and process type identifiers solidify the rule set selection into a repeatable process; each weld segment forms a sequence simultaneously containing geometric features and defect categories. and quality grade The segment object provides explicit input for generating rework tasks.

[0106] If step four only outputs the post-weld quality level, it can only indicate that the forming of some weld mileage sections is abnormal. It cannot directly drive the robot controller to rework, nor can it form a loop with the re-sweeping requirements of step one, the future action point weight of step two, and the degradation control sensitivity of step three. In other words, it cannot re-enter the welding control chain.

[0107] Therefore, after obtaining the segment object, it is necessary to further transform the segment object into a rework task package, and form a front-end feedback mark based on the correspondence between the segment object and the previous running state.

[0108] In one implementation for automated welding of long weld seams in ship sections, after the industrial control / edge computing unit generates the section objects, it does not immediately issue a rework command for the entire weld seam to the robot controller interface unit. Instead, it first merges the section objects based on weld seam mileage values. If two adjacent section objects have the same defect category... And quality level If the directions of continuous change are consistent, they should be merged into the same repair section; if two adjacent sections have consecutive mileage but different defect categories... If they differ, they should remain independent to prevent rework actions from mixing two types of anomalies that should be handled separately into the same process action. Rework tasks deal with robot motion chains and welding process chains, not abstract inspection chains. Only when the weld seam mileage section and defect category... and quality grade Only when the continuity requirement is met together can the rework action have the significance of being executed in a single stage.

[0109] Furthermore, after the industrial control / edge computing unit completes the continuous merging of segment objects, it forms the starting point of the rework segment. and the end of the repair section and build a rework task intensity : ; Among them, the intensity of rework tasks : The overall priority of the current rework section in rework sequencing and rework process selection; section mapping quantity Used to represent the comprehensive geometric characteristic strength of the current repair section; defect level quantity : By quality grade The mapped grade weights, whose value ranges are predefined by the process rule set; are mapped to monotonic values ​​from high risk to low risk according to the rule set grades; historical frequency quantity. : The cumulative degree of repeated anomalies in the same weld mileage segment in the previous pass or rework round of the current workpiece. Its value is non-negative and is the normalized result of the number of repeated anomalies in the current weld mileage segment in the most recent pass or workpiece; weighting coefficient. Weighting coefficients Weighting coefficients These three types of information are used to adjust the intensity of rework tasks. The contribution of the value should preferably be non-negative.

[0110] In practice, rework is not always performed segment by segment in the natural order of welds. In scenarios involving multiple welds operating in parallel or with limited rework windows, it is necessary to sort multiple rework segments. Geometric features, grade results, and historical frequency are uniformly incorporated into the rework task intensity. This allows the rework task package to reflect both the current test results and the historical characteristics of repeated instability in the section.

[0111] The rework task package should at least include the weld number and the starting point of the rework section. End of repair section Defect categories Quality grade Intensity of rework tasks The system includes suggestions for correcting the welding torch posture, indicating necessary re-sweeping marks, and specifying the execution sequence. For example, on a corner weld of a box girder, if the starting point of a certain repair section... End of the repair section The main difference is in leg length. Insufficient and localized undercut edges The industrial control / edge computing unit then writes the start and end mileage values ​​of that section and the current defect category into the rework task package. and quality grade Based on the target attitude correction amount, control mode, and drift state saved in steps two and three, the robot controller interface unit adds attitude correction direction and rescan marks to the rework section. After receiving the rework task package, the robot controller interface unit can directly generate the rework path without having to search for abnormal locations in the post-weld images again.

[0112] After the rework task package is generated, the industrial control / edge computing unit also needs to feed back the rework section object to the front-end status that steps one and two depend on. This feedback is not based on fuzzy experience, but rather on specifying the rework section and defect category. Quality grade And the control mode record in step three is mapped to the front-end feedback marker. When the front-end feedback marker is written back to the weld reference map, at least three types of results are formed: First, if the same defect category repeatedly occurs in the current workpiece or consecutive workpieces in the same rework section. If a certain type of repair section frequently appears simultaneously with a low-permissibility state or degradation control in step three, then that section is designated as a high-sensitivity section in subsequent operations of step three. If repair sections are concentrated in specific local curvature change areas, then that section is designated as the future weld mileage value in step two. The key constraint section during the solution process. Therefore, step four does not supersede steps one through three, but rather converts the post-weld inspection results into marked objects that can be directly understood by the preceding steps.

[0113] To avoid redundant coverage of closed-loop feedback within the same workpiece, the industrial control / edge computing unit sets exit conditions in this step. If the entire length of the current weld has completed post-weld inspection and no rework section has been generated, the process outputs a normal completion status; if a rework section exists, a rework task package is output and the feedback marker in the current weld reference map is maintained; if a rework section is in a boundary uncertainty area in the rule set mapping, that section is output as a section requiring review and is not directly sent to automatic rework. Such exit conditions ensure that the object of step four is always a clearly defined section, and will not directly send sections with unstable rule boundaries into the automatic welding action chain.

[0114] When used, the segment object is converted into an executable rework task package; the rework task package carries the weld mileage range, defect attributes and front-end scan requirements; the post-weld inspection results are re-entered into the weld reference map, future action point solution chain and control permission chain through front-end feedback markers.

[0115] As a parallel embodiment, the rework task package is not limited to being directly sent to the robot controller interface unit; it can also be sent to the upper-level process management unit first, which will then sort the multiple weld seams before issuing the rework task. The front-end feedback markers are not limited to key rescanning sections, high-sensitivity sections, and sections requiring verification; they can also be extended to thermal input focus sections, nozzle posture focus sections, or fixture verification sections. The post-weld geometry detection unit is not limited to a line laser contour sensor, as long as it can output sufficient data to support the weld width. , Yu Gao Leg length Misaligned edges Biting edge Overlap and not fully welded The reconstructed test data can all be incorporated into the implementation path of this step. Therefore, this step always maintains a single overall inventive concept, namely, to reconnect the post-weld results with the pre-weld and in-weld processing chains through rule set mapping, rework object generation, and front-end feedback.

[0116] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0117] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for processing weld point images of a welding robot based on a vision sensor, characterized in that: include, Before welding, a three-dimensional seam tracing is performed on the joint to be welded, extracting the weld centerline, weld tangent, local curvature, gap distribution, and joint type, establishing weld mileage coordinates, and simultaneously acquiring local images during welding, robot pose, welding speed, oscillation phase, and robot controller timing information. Local geometric features of the weld are extracted from local images during welding. Combined with the total time delay and weld mileage coordinates determined by the look-ahead distance and time series information, the future action point is determined. Based on the reliability of the local images during welding and the coordinate drift detection results, the trajectory, attitude and oscillation correction corresponding to the future action point are controlled in layers. After welding, geometric inspection of the weld is carried out. The quality rule set is determined according to the joint type and welding process type, the defect category and quality level are determined, and the rework section is written back. Establishing weld mileage coordinates includes: discretizing continuous centerline nodes based on the weld centerline obtained from the three-dimensional weld locator before welding; sorting the centerline nodes according to the extension direction from the weld start point to the weld end point; determining the weld mileage position corresponding to each centerline node based on the cumulative path length between adjacent centerline nodes; and associating the acquisition position of each frame of local image during welding with the corresponding weld mileage position. The timing information of the robot controller includes the image acquisition time, the image processing completion time, the controller receiving time, the controller sending time, and the robot execution feedback time. The total delay is determined based on the time difference between the image acquisition time, the image processing completion time, the controller receiving time, the controller sending time, and the robot execution feedback time. Extracting local geometric features of the weld from local images during welding, including: extracting lateral deviation, vertical deviation, weld tangent, local curvature, gap width, and welding torch relative attitude error within the region of interest corresponding to the weld mileage position; determining the future action point, including interpolating or extrapolating the mileage difference between the currently measured position and the position to be corrected by combining the look-ahead distance, total time delay, and weld mileage coordinates. Generate corresponding trajectory correction, attitude correction, and oscillation parameter correction values, including: determining the target position, target attitude, and target oscillation correction requirements based on the weld tangent, local curvature, gap width, and relative attitude error of the welding torch corresponding to the future point of action, and generating trajectory correction, attitude correction, and oscillation parameter correction values ​​from the target position, target attitude, and target oscillation correction requirements respectively. A credibility evaluation is established for local images during welding, including determining the credibility evaluation based on the proportion of saturated pixels, occlusion rate, image blur, stripe continuity, and consistency of historical residuals. The tiered control includes: performing full correction when the credibility rating is high, performing amplitude limiting correction when the credibility rating is medium, and performing degradation control when the credibility rating is low. When the system is configured with a primary channel and a backup channel, the reliability evaluation is also corrected based on the consistency comparison results between the primary channel output and the backup channel output. The coordinate drift detection results are determined based on the deviation of the reference edge, fixture reference, or nozzle profile relative to the weld reference map; when the deviation exceeds the preset boundary, the coordinate transformation relationship between the visual coordinates and the weld mileage coordinates is updated.

2. The welding robot weld point image processing method according to claim 1, characterized in that: Degradation control includes reducing welding speed, maintaining the current weld tangent, pausing oscillation, and triggering local search recapture. When arc light, spatter, smoke, or obstruction causes the confidence evaluation to continuously fall below the low confidence boundary for a preset number of frames, the system switches from closed-loop tracking to protection mode, retaining only safe movement and welding abort judgments.

3. The welding robot weld point image processing method according to claim 2, characterized in that: Geometric inspection of the weld seam after welding is performed, including extracting weld width, reinforcement height, leg length, misalignment, undercut, lap joint, and incomplete weld. A quality rule set is determined based on the joint type and welding process type, including: matching the corresponding first quality rule set when the welding process type is ordinary fusion welding, matching the corresponding second quality rule set when the welding process type is laser-arc hybrid welding, and determining the defect category and quality level based on the quality rule set.

4. The welding robot weld point image processing method according to claim 3, characterized in that: Write back the repair section, including merging adjacent weld sections with the same defect category into the same repair section, and outputting the repair location point or repair trajectory segment; The reverse update of the rescanning section, risk section or control parameters for the next round of welding includes: marking the rescanning section where the same type of defect recurs as the rescanning section for the three-dimensional seam finding before the next round of welding; marking the rescanning section corresponding to the low confidence state as the high sensitivity section during welding; and using the rescanning section that coincides with the predicted high risk section to update the correction weight or safety boundary of the future action point.

Images