A robot welding quality detection method and system based on machine vision

Abstract

The present application relates to the technical fields of image processing and robot welding detection, in particular to a robot welding quality detection method and system based on machine vision. The method acquires a sequence of weld images and path progress data collected by a robot along a welding path, extracts a weld centerline and a weld boundary, establishes a weld material coordinate system, decomposes deformation between adjacent images into global projection deformation and local non-rigid deformation, and solves a mapping relationship under the constraints of path progress, centerline continuity and boundary sequence to generate a continuous weld strip image and segment candidate abnormal regions, and then determines a detection result according to forward mapping consistency, reverse re-projection consistency and cross-frame shape continuity. The present application can establish image correspondence under a unified material coordinate, distinguish real weld defects from transient abnormalities caused by arc light, spatter, reflection and occlusion, and reduce defect position deviation, repeated identification and false determination.

Description

Technical Field

[0001] This invention relates to the field of image processing and robotic welding inspection technology, specifically to a robotic welding quality inspection method and system based on machine vision. Background Technology

[0002] Robotic welding quality inspection typically employs post-weld visual imaging, where an imaging module installed at the robot's end effector or near the welding station continuously acquires images of the weld surface. These images are then processed with grayscale correction, noise suppression, contrast enhancement, and region of interest extraction. Existing solutions often use single-frame images as the processing object, determining the weld center region through edge detection or region segmentation, and extracting features such as weld width, boundary undulations, grayscale abrupt changes, texture direction, connected region area, and aspect ratio. Preset thresholds, template matching, or classification models are then used to identify surface anomalies such as cracks, porosity, undercut, weld beads, and lack of fusion. For straight welds, images are typically numbered at fixed intervals, and defect locations are recorded using pixel coordinates. For curved welds, the inspection results are arranged sequentially according to the robot's operating order. While this approach has a clear implementation path and can perform local weld appearance inspection, each frame remains within independent pixel coordinates, resulting in a lack of uniform representation of the same weld material location across different images. Defect determination primarily depends on the texture and contour features of a single frame.

[0003] To utilize repetitive information in continuous images, existing technologies also employ image stitching, multi-frame tracking, or robot pose-assisted registration to process weld seam image sequences. Image stitching schemes typically extract corner points, texture blocks, or weld seam edge features from adjacent images, calculate translation, rotation, or projection transformations based on feature correspondences, and then superimpose the images into a long strip weld seam image. Multi-frame tracking schemes maintain target numbers based on the overlap and motion direction of candidate anomaly regions in adjacent images. Robot pose-assisted schemes estimate the longitudinal displacement of adjacent images based on the robotic arm's motion trajectory, capture time, and image sequence number, and then use the estimation results as initial values ​​for stitching. Some schemes also assign lower weights to areas with brightness saturation, motion blur, or smoke occlusion to reduce the impact of low-quality images on the fusion result. The above processing can compensate for the lack of continuity in single-frame detection, but ordinary image features are easily affected by weld seam texture repetition, surface reflection, and local low texture. Simple pose transformation cannot describe weld seam undulations, curvature changes, and imaging scale changes, so the obtained stitching relationship may still deviate from the actual material position on the weld seam surface.

[0004] When a robot moves along a curved or spatial weld seam and continuously acquires images, changes in robot pose, weld seam curvature, and surface undulations can cause projected displacement, scale changes, and local non-rigid deformation of the same material location in adjacent images. Arc saturation, metal spatter, smoke obstruction, and surface reflection can also create transient abnormal regions that resemble welding defects. This makes it impossible for existing single-frame recognition, ordinary image stitching, or processing methods based solely on robot trajectory conversion to stably establish cross-frame correspondences under a unified material coordinate system. Since candidate abnormal regions cannot be reliably traced back to the same weld seam material location, their boundaries, areas, and extension directions in previous, current, and subsequent images cannot be consistently compared. Consequently, there is a lack of distinguishable continuous evidence between the real defect and visual interference that appears, migrates, or disappears with changes in imaging conditions, leading to positional shifts, duplicate recognition, or incorrect judgments in the detection results. Summary of the Invention

[0005] The purpose of this invention is to provide a robot welding quality inspection method and system based on machine vision, which can solve the problems in the background art mentioned above.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A robot welding quality inspection method based on machine vision includes: acquiring a sequence of weld seam images continuously collected as the robot moves along the welding path and path progress data corresponding to each weld seam image; Extract the weld centerline and weld boundary from each weld image, and establish a weld material coordinate system with the welding direction as the longitudinal direction and the weld cross-section as the transverse direction based on the path progress data; The deformation between adjacent weld seam images is decomposed into global projection deformation and local non-rigid deformation, and the mapping relationship between each weld seam image and the weld seam material coordinate system is obtained under the constraints of the path progress data, weld seam centerline continuity and weld seam boundary sequence. A continuous weld strip map is generated based on the mapping relationship, and candidate abnormal regions are segmented. The candidate abnormal regions are projected onto the preceding and subsequent weld images, and the welding quality inspection results are determined based on the consistency of forward mapping, the consistency of reverse reprojection, and the continuity of cross-frame morphology.

[0007] Preferably, establishing the weld material coordinate system includes: monotonically sorting the weld image sequence according to the path progress data, and selecting a longitudinal anchor point corresponding to the path progress data on the weld centerline of each weld image; Calculate the tangential direction of the weld centerline at the longitudinal anchor point, and take the direction that is orthogonal to the tangential direction and points to the two sides of the weld as the transverse direction; Establish lateral normalized coordinates based on the intersection of the weld boundaries on both sides with the lateral direction; The centerline arc length between adjacent longitudinal anchor points is accumulated, the accumulated arc length is corrected using the path progress data, and the corrected longitudinal coordinates are combined with the transverse normalized coordinates to form the weld material coordinate system.

[0008] Preferably, obtaining the mapping relationship includes: constructing the initial transformation of the global projection deformation based on the path progress difference and the tangential change of the weld centerline corresponding to adjacent weld images; After removing the initial transformation, multiple local control points are set along the center line of the weld, and the grayscale structure similarity, boundary distance residual and center line offset of each local control point in the adjacent weld images are calculated respectively. The grayscale structure similarity, the boundary distance residual, and the centerline offset constitute a joint matching cost, and the local non-rigid deformation is obtained under the constraints that the longitudinal order of the local control points is not reversed, the weld boundaries on both sides do not intersect, and the local transverse scale is continuous. The global projection deformation and the local non-rigid deformation are combined to form a mapping relationship.

[0009] Preferably, determining the welding quality inspection result includes: establishing a morphological description of the candidate abnormal region, including the region centroid, boundary contour, main extension direction, and area distribution; Based on the mapping relationship, the morphological description is forward-mapped from the current weld image to the subsequent weld image, and then reprojected backward from the subsequent weld image back to the current weld image; Calculate the region overlap after forward mapping, the boundary distance after reverse reprojection, and the variation sequence of the main extension direction and area distribution in continuous weld images; The calculation results are assigned confidence weights based on the image saturation, blurriness, and occlusion ratio corresponding to the candidate abnormal regions. The cross-frame consistency judgment value of the candidate abnormal regions is formed based on the weighted region overlap, boundary distance, and change sequence.

[0010] Preferably, the method of correcting the cumulative arc length using the path progress data includes: selecting the position of abrupt change in centerline curvature, the position of stable weld width, and the position of stable texture in the overlapping area of ​​adjacent images as joint correction nodes in the weld image sequence; The longitudinal deviation between the visual cumulative arc length of each joint correction node and the path progress data is calculated respectively, and the correction weight of each longitudinal deviation is determined according to the confidence of centerline extraction, boundary integrity and texture matching stability. A piecewise correction function is established with constraints such as continuous vertical scaling between adjacent joint correction nodes, monotonically increasing vertical coordinates, and consistent corresponding positions in the overlapping areas of the images. The longitudinal coordinates of each longitudinal anchor point are updated using the piecewise correction function, and the local cross-sectional position corresponding to the transverse normalized coordinates is updated synchronously.

[0011] Preferably, the global projection deformation and the local non-rigid deformation are solved by alternating constraints, specifically including: fixing the local non-rigid deformation, and correcting the global projection deformation according to the updated longitudinal anchor points, the corresponding positions of the overlapping areas of adjacent images, and the overall orientation of the weld boundaries on both sides; The fixed global projection deformation is used to update the local non-rigid deformation based on the joint matching cost of the local control points. After each update, calculate the local rate of change of the composite mapping in the longitudinal and transverse directions of the weld, and apply position backoff constraints to local control points where the longitudinal rate of change is less than zero, the transverse rate of change is flipped, or the boundary mapping is out of bounds. Repeat the correction and update process until the difference in control point displacement between two adjacent composite mappings meets the preset convergence condition.

[0012] Preferably, forming the cross-frame consistency determination value further includes: converting the boundary contour of the candidate abnormal region into a topological description composed of endpoints, turning points, bifurcation points and closed boundary segments, and recording the material coordinates of each topological point relative to the weld centerline and the weld boundaries on both sides; Based on the composite mapping, topological points are matched between the preceding weld image, the current weld image, and the subsequent weld image, and the topological point connectivity retention degree and material coordinate offset are calculated. For topological points covered by saturated, blurred, or occluded areas, set missing markers and use the connection order of adjacent visible topological points to limit the allowable location range of the missing topological points; The connection relationship retention, material coordinate offset, and allowable position interval compliance are incorporated into the cross-frame consistency judgment value.

[0013] Preferably, generating the continuous weld strip image includes: dividing each weld image into multiple material coordinate grids according to the composite mapping, and calculating the grid confidence for each material coordinate grid, which consists of imaging sharpness, local saturation ratio, boundary integrity, mapping residual and topological point visibility; When multiple weld images are mapped to the same material coordinate grid, the image data with the highest grid confidence is retained as the base data, and the remaining image data are weighted and fused according to the consistency of grayscale structure and boundary direction between the base data and the base data. For topological points with missing labels, only image data with overlapping location intervals are fused; Based on the fused mesh data, the weld centerline, weld boundary, and candidate abnormal region are re-extracted, and the composite mapping and cross-frame consistency judgment value are updated.

[0014] Preferably, determining the welding quality inspection result further includes: sorting the candidate abnormal regions according to the longitudinal coordinates of the weld material coordinate system, and merging the candidate abnormal regions with a longitudinal interval smaller than the corresponding local weld width and a continuous topological point connection relationship into an abnormal segment; For each abnormal segment, the cross-frame consistency judgment value and mesh confidence of each candidate abnormal region constituting the abnormal segment in different weld images are collected, and candidate abnormal regions whose material coordinate offset exceeds the corresponding allowable position interval are eliminated. The start and end positions of the abnormal segment are determined based on the coordinate range of the overlapping materials in the remaining candidate abnormal regions; The lateral range of the abnormal section is determined based on the fused boundary contour, and the start and end positions, lateral range, morphological description, and cross-frame consistency judgment value are combined to form the welding quality inspection result.

[0015] A robot welding quality inspection system based on machine vision includes: an image acquisition module, which acquires a sequence of weld images continuously collected as the robot moves along the welding path, as well as path progress data corresponding to each weld image; The material coordinate construction module extracts the weld centerline and weld boundary from each weld image, and establishes a weld material coordinate system with the welding direction as the longitudinal direction and the weld cross-section as the transverse direction based on the path progress data. The deformation mapping module decomposes the deformation between adjacent weld seam images into global projection deformation and local non-rigid deformation, and obtains the mapping relationship between each weld seam image and the weld seam material coordinate system under the constraints of path progress, centerline continuity and boundary sequence. The strip generation module generates a continuous weld strip map and segments candidate abnormal regions based on the mapping relationship; The consistency determination module projects the candidate abnormal region onto the preceding weld image and the subsequent weld image, and determines the welding quality detection result based on the forward mapping consistency, the reverse reprojection consistency, and the cross-frame morphological continuity.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By using the robot path progress, weld centerline, and weld boundary together to establish a weld material coordinate system, the pixel positions in continuous images can be converted into unified material positions distributed along the welding direction and the weld transversely. Changes between adjacent images are decomposed into global projection deformation and local non-rigid deformation. Path progress limits the longitudinal displacement range, while centerline continuity, boundary order, and local width changes limit the local deformation range, allowing robot motion information and weld image geometric information to participate in the mapping process. After candidate anomaly regions are forward-mapped to subsequent images and then reprojected back to the original image, cross-frame comparisons can be performed based on region overlap, boundary distance, area change, and continuity of extension direction. Defects fixed to the weld surface maintain position and shape correlation in the unified material coordinate system. Anomalies caused by changes in arc light, spatter, reflection, or occlusion manifest as coordinate drift, boundary discontinuity, or reprojection deviation, thus forming a continuous image judgment criterion for distinguishing between real defects and transient visual interference.

[0017] 2. Based on composite mapping, multi-frame weld images are unfolded into continuous strip images. The reliability of the material coordinate grid is determined according to imaging sharpness, local saturation ratio, boundary integrity, mapping residual, and topological point visibility. When multiple frames of data exist at the same material location, image information with higher reliability can be retained, and the remaining images are fused according to the consistency of grayscale structure and boundary direction. After the endpoints, turning points, bifurcation points, and closed boundary segments of candidate anomaly regions are converted into topological descriptions, their positions relative to the weld centerline and the two side boundaries can be combined to limit the range of local boundary loss caused by occlusion or blurring. Adjacent candidate anomaly regions are merged according to material coordinates, and anomaly evidence exceeding the allowable position range is removed. This forms the start and end positions, lateral range, and morphological information of continuous anomaly segments, reducing the repeated recording of the same defect in adjacent images and reducing the impact of single-frame local distortion on the determination of defect length and location. Attached Figure Description

[0018] Figure 1 This is an overall flowchart of a robot welding quality inspection method based on machine vision; Figure 2 Flowchart for constructing the coordinate system of weld material and obtaining composite deformation mapping; Figure 3 Curves showing the energy component variations in composite deformation mapping; Figure 4 Comparison images of real weld seams before and after processing; Figure 5 A cross-frame consistency scoring map for candidate anomaly regions; Figure 6 A graph showing the distribution of longitudinal position segmental correction values; Figure 7 This is a graph showing the fusion curve of the continuous weld seam strip mesh. Detailed Implementation

[0019] In one embodiment, please refer to Figure 1 A machine vision-based robotic welding quality inspection method includes: acquiring a sequence of weld images continuously collected as the robot moves along the welding path, and path progress data corresponding to each weld image; extracting the weld centerline and weld boundary from each weld image, and establishing a weld material coordinate system with the welding direction as the longitudinal direction and the weld transverse section as the transverse direction based on the path progress data; decomposing the deformation between adjacent weld images into global projection deformation and local non-rigid deformation, and calculating the weld material coordinates from each weld image under the constraints of path progress data, weld centerline continuity, and weld boundary sequence. The method establishes the mapping relationship between the material coordinate system; generates a continuous weld strip image based on the mapping relationship and segments candidate abnormal regions; projects the candidate abnormal regions onto the preceding and subsequent weld images; and determines the welding quality detection results based on the consistency of forward mapping, the consistency of reverse reprojection, and the continuity of cross-frame morphology. In this embodiment, the method uses the weld material position as a common index for continuous images, so that the perspective changes caused by robot movement, the local deformation caused by weld surface undulations, and the transient abnormalities formed by arc light, spatter, reflection, and occlusion are included in the same image processing link for constraint and judgment.

[0020] Specifically, the weld seam image sequence is saved according to the image acquisition time. Each weld seam image is associated with a path progress record. The path progress record represents the cumulative travel position of the robot welding reference point on the predetermined welding path at the time of image acquisition. The path progress record is obtained through the time correspondence between the image acquisition trigger time and the robot motion record. When an acquisition time is located between two adjacent robot motion record times, the corresponding path progress is obtained by time interpolation of the two adjacent cumulative travel positions. When there are repeated acquisitions, short pauses or local backtracking in the image sequence, a monotonic path progress sequence is established based on the main motion direction of the welding operation. Images that do not meet the main motion direction are marked as supplementary images. Supplementary images do not participate in the initial sorting of the longitudinal coordinates, but can provide additional image evidence of the same material position during subsequent reprojection and confidence fusion.

[0021] In this embodiment, brightness range normalization, local noise suppression, and weld region cropping are performed on each weld image. Brightness range normalization determines the transformation interval based on the grayscale distribution of the adjacent area of ​​the weld, and does not directly use the maximum and minimum grayscale of the entire image to avoid local arc saturation changing the grayscale level of the normal weld texture. The weld centerline is formed by connecting the response peak positions of each transverse section in the weld candidate area. The weld boundary is determined by the gradient extreme value positions on both sides of the centerline and the region continuity. When only one side boundary is detected for a certain transverse section, the boundary direction of the front and rear transverse sections is used to limit the selectable position of the missing boundary, and a lower boundary integrity is added to the completion position. The centerline and boundary are saved in sub-pixel coordinates to reduce the cumulative offset caused by integer rounding during multiple mappings.

[0022] Please refer to Figure 2 When establishing the coordinate system of the weld material, longitudinal anchor points are selected on the centerline according to the corresponding positions of the path progress. The arrangement direction of the longitudinal anchor points along the weld centerline is taken as the welding direction. The local tangent of the centerline is calculated at each longitudinal anchor point. The lateral direction is taken as the direction that is orthogonal to the local tangent and passes through the weld boundaries on both sides. The longitudinal coordinate represents the cumulative position of the material along the welding direction, and the lateral coordinate represents the normalized distance of the material position relative to the centerline. The lateral coordinate of the centerline is 0. The lateral coordinate of the left weld boundary is taken as the negative boundary value, and the lateral coordinate of the right weld boundary is taken as the positive boundary value. Pixels in different images but corresponding to the same weld material position have the same or similar longitudinal and lateral coordinates after mapping, thus avoiding the direct comparison of candidate abnormal areas using the original pixel row and column numbers.

[0023] Preferably, the longitudinal material coordinates of the i-th longitudinal anchor point are determined according to the following formula: , ; in, This represents the longitudinal material coordinate of the i-th longitudinal anchor point. This represents the visual cumulative arc length obtained based on the centerline point sequence. This represents the two-dimensional image coordinates of the q-th centerline sampling point. This represents the path progress corresponding to the i-th vertical anchor point. The cumulative visual arc length is represented by a weight between 0 and 1. When the center line is continuous and the two side boundaries are intact, it can be set as follows: Take a value close to 1; when the image has occlusion or a broken center line, you can set... Take values ​​close to 0, for example, the visual cumulative arc length at a certain location is 120 normalized length units, and the path progress is 116 normalized length units. When the value is 0.75, the longitudinal material coordinates are set to 119 normalized length units, which makes the image geometric information dominant and preserves the constraint of path progress on local drift.

[0024] The deformation between adjacent weld seam images is decomposed into global projection deformation and local non-rigid deformation. Global projection deformation describes the overall image displacement, rotation, scale changes, and perspective changes caused by the robot's translation, turning, and changes in shooting direction along the path. Local non-rigid deformation describes the internal deformation of the weld seam region caused by weld seam curvature, weld seam surface height changes, solidification texture undulations, and local imaging angle differences. When solving, the initial value of longitudinal displacement is first given based on the path progress difference between adjacent images, and then the initial value of rotation direction is given using the tangential change of the centerline. The global projection deformation is corrected by the correspondence between the weld seam centerline and the two side boundaries. After the overall alignment is completed, local control points are arranged in the weld seam region, and the non-rigid deformation is represented by the displacement field of the local control points.

[0025] In a preferred embodiment, the joint solution of global projective deformation and local non-rigid deformation is limited by the following formula: ; Where H represents the global projection transformation parameters, and U represents the local control point displacement field. This represents the deviation between the path progress difference and the global longitudinal displacement. This term represents the joint deviation between the grayscale structure and the centerline position within the overlapping area of ​​adjacent images. This indicates the distance deviation between corresponding points on both sides of the weld boundary. This indicates constraints related to the longitudinal order of local control points, non-intersecting boundaries, and smooth displacement. , , , These represent the non-negative weights of each deviation term, and their sum is 1. For example, when arc light causes instability in the grayscale structure, the weights are reduced. and increase and This allows the path progress and boundary geometry to have a higher constraint ratio in the mapping results.

[0026] Please refer to Figure 3 , Figure 3 The horizontal axis represents the iteration order of adjacent weld seam image pairs in the composite mapping solution process, and the vertical axis represents the normalized energy value. The curves correspond to the total energy, path term, structure term, boundary term, and smoothing term, respectively. The legend uses different line types, gray levels, and hollow markers to distinguish each term. Figure 3 The energy is calculated using the composite deformation mapping formula shown in the text, where the path term constrains the global longitudinal displacement, the structure term constrains the grayscale structure and centerline position, the boundary term constrains the correspondence between the weld boundaries on both sides, and the smoothing term restricts the order and smoothness of the displacement field of the local control points.

[0027] from Figure 3 It can be observed that the total energy curve exhibits local peaks with different image pairs and iteration states. These peaks mainly increase synchronously with the path term, indicating that when the robot's perspective changes significantly or the local path progress varies considerably, the global projection deformation contributes more strongly to the total energy. The structure and boundary terms are generally in the lower range, but visible fluctuations still appear in local image pairs, reflecting that arc light, spatter, reflections, or missing weld boundaries can make the grayscale structure and boundary correspondence unstable. The smoothness term is relatively stable, indicating that local non-rigid control points do not exhibit severe deformation in most areas.

[0028] Based on the obtained mapping relationship, each weld image is transformed into a weld material coordinate grid. The vertical resolution of the material coordinate grid is determined based on the overlap range between consecutive images, and the horizontal resolution is determined based on the weld width and the minimum boundary expression range required for candidate defects. When there are multiple source pixels in the same material coordinate grid, the original image number, mapping residual, sharpness, saturation state and boundary visibility state of each source pixel are recorded. The continuous weld strip image is formed by arranging the material coordinate grid according to the vertical coordinate. The weld center line in the strip image remains continuous after normalization, and the two sides of the weld are expanded according to the local width. The curved weld is thus converted into a strip representation suitable for continuous segmentation and length quantization.

[0029] Candidate anomalous regions are jointly determined by local grayscale anomalies, boundary anomalies, and texture direction anomalies in the continuous weld strip image. Local grayscale anomalies indicate that the grayscale distribution of the region has abrupt changes relative to the texture of the adjacent normal weld. Boundary anomalies indicate that the region causes local concavity, convexity, or discontinuity interruption of the weld boundary. Texture direction anomalies indicate that the main texture direction of the region is inconsistent with the solidification texture direction of the weld. The segmentation process retains the pixel set, boundary contour, centroid, area, main extension direction, and relative position of each candidate anomalous region in the material coordinate grid with respect to the center line and the two side boundaries. Instead of making a defect conclusion directly based on the single-frame classification result in the segmentation stage, the candidate anomalous regions are used as input for cross-frame consistency determination.

[0030] Please refer to Figure 4 , Figure 4 Sample 1 shows a before-and-after comparison of the rib weld sample, while Sample 2 shows a before-and-after comparison of the highly reflective weld surface sample. Each group's before-and-after image represents the original weld acquisition image, and the after-and-after image represents the result after brightness normalization, weld region clipping, centerline extraction, boundary enhancement, material coordinate mesh expansion, and candidate segment labeling. In the after-and-after image, the green curve represents the weld centerline, the yellow curve represents the two side boundaries or candidate boundary trajectories, the light-colored mesh represents material coordinate segments, and the red rectangular areas represent candidate abnormal segments that need to undergo cross-frame consistency and topology compliance verification.

[0031] from Figure 4 Sample 1 reveals that the original rib weld image suffers from issues such as tilted viewing angle, localized weld bead protrusion, and insufficient background contrast. Directly comparing the location of abnormal areas in the original image is easily affected by the shooting angle. The processed image unifies the weld centerline, side boundaries, and candidate areas under the material coordinate index, making the segmentation along the welding direction clearer. In Sample 2, the original weld surface exhibits significant reflection, abrupt changes in texture direction, and localized shadows. The processed image enhances the centerline and boundary trajectories and labels multiple lateral candidate segments, ensuring that verifiable evidence of anomalies is preserved even under strong reflective conditions.

[0032] When projecting candidate anomaly regions onto preceding and subsequent weld images, the system first searches for images covering the material coordinate range of the candidate anomaly region. Then, it uses the inverse mapping from material coordinates to pixel coordinates of each image to obtain the corresponding search region. Forward mapping consistency is determined by the overlap between the candidate region in the current image and the anomaly response region in the subsequent image after mapping. Backward reprojection consistency is determined by the boundary deviation between the corresponding region in the subsequent image and the original candidate region after back mapping. Cross-frame morphological continuity is determined by the changes in region area, main extension direction, contour transition structure, and position relative to the weld boundary in the preceding, current, and subsequent images. When a covering image is missing, the absence is not considered inconsistency; instead, partial evidence is formed based on available images, and the corresponding judgment weight is reduced.

[0033] In one embodiment, establishing a weld material coordinate system includes: monotonically sorting the weld image sequence according to path progress data; selecting a longitudinal anchor point corresponding to the path progress data on the weld centerline of each weld image; calculating the tangential direction of the weld centerline at the longitudinal anchor point, and using the direction orthogonal to the tangential direction and pointing towards the two sides of the weld as the lateral direction; establishing lateral normalized coordinates based on the intersection of the two sides of the weld boundary and the lateral direction; accumulating the arc length of the centerline between adjacent longitudinal anchor points, correcting the accumulated arc length using path progress data, and combining the corrected longitudinal coordinates with the lateral normalized coordinates to form a weld material coordinate system. In this embodiment, monotonically sorting is based on the main welding motion direction. When there are paused images, multiple images are allowed to correspond to the same longitudinal interval. When there are rewinding and re-captured images, the rewinding and re-captured images are associated with the established material coordinate interval.

[0034] Specifically, the vertical anchor point is not simply taken as the geometric center of each image, but rather selected within the area where the center line intersects with the effective area of ​​the image, corresponding to the path progress. When selecting, the distance from each candidate position of the center line in the image to the predicted position of the center line of the adjacent image is calculated, and the anchor point confidence is determined by combining the visibility of the two side boundaries. When the anchor point confidence is low, the tangential interpolation of the adjacent high-confidence anchor point is used to determine the local tangential. The horizontal normalized coordinates are normalized according to the actual distance from the center line to the left and right boundaries, so that the true proportional relationship on both sides of the center line can still be preserved when the width of the left and right welds is asymmetrical. For example, when the distance from the center line to the left boundary is less than the distance to the right boundary, the same absolute horizontal distance corresponds to different normalized coordinate changes on the left and right sides.

[0035] In this embodiment, the centerline arc length accumulation and path progress correction adopt local consistency control. If the difference between the visual arc length and path progress increment between adjacent anchor points is within the interpretable range of the current image overlap area, the visual arc length is retained to reflect the direction of the weld curve. If the difference accumulates continuously and is accompanied by centerline breakage, boundary loss or texture matching instability, the proportion of path progress in the vertical coordinate is increased. The correction process keeps the vertical coordinate of the anchor point monotonically increasing and limits the continuous change of vertical scaling between adjacent anchor points to avoid local correction causing the same material interval to be folded or repeatedly mapped. The corrected vertical coordinate and the horizontal normalized coordinate together constitute a two-dimensional material coordinate. The length, position and shape of the subsequent candidate abnormal region are calculated in the two-dimensional material coordinate.

[0036] In one embodiment, obtaining the mapping relationship includes: constructing an initial transformation of the global projection deformation based on the path progress difference and the tangential change of the weld centerline corresponding to adjacent weld images; after removing the initial transformation, setting multiple local control points along the weld centerline, and calculating the grayscale structural similarity, boundary distance residual, and centerline offset of each local control point in adjacent weld images; using grayscale structural similarity, boundary distance residual, and centerline offset to form a joint matching cost, and obtaining the local non-rigid deformation under the constraints of no reversal of the longitudinal order of the local control points, no intersection of the weld boundaries on both sides, and continuity of the local lateral scale; and combining the global projection deformation and the local non-rigid deformation to form a mapping relationship.

[0037] Local control points are set along the centerline with equal arc length, and boundary control points are added near the boundaries on both sides of the centerline. The candidate positions of the control points in adjacent images are limited to the local search area after global projection transformation. The gray-scale structure similarity is obtained by comparing the gray-scale arrangement relationship after removing the mean in the neighborhood of the control point to reduce the influence of overall brightness changes. The boundary distance residual is calculated by the difference between the left and right boundary positions on the corresponding horizontal section of the control point. The centerline offset is calculated by the normal distance between the transformed centerline and the centerline of the target image. During joint matching, the gray-scale structure similarity in the saturated or blurred areas is reduced by a certain proportion, and the geometric deviation term is increased for areas with complete boundaries and continuous centerlines, so that the local displacement field is jointly constrained by available image evidence.

[0038] Preferably, the cross-frame consistency value of the r-th candidate anomaly region is calculated according to the following formula: ; in, This represents the cross-frame consistency value of the r-th candidate anomalous region. This indicates the overlap of the regions after forward mapping. Represents the normalized back-projection boundary distance. This represents the change in area between consecutive images. This indicates the amount of change in the main extension direction. Indicates the degree of preservation of topological connectivity. to These are non-negative combination coefficients that sum to 1, for example, when the area information is incomplete due to local occlusion. And improve based on the visibility of the boundary. or This makes the consistency value consist of currently observable evidence of material location and morphology.

[0039] Please refer to Figure 5 , Figure 5 The horizontal axis represents the candidate anomaly region number sorted by material coordinates, and the vertical axis represents the normalized consistency score. The curves correspond to the cross-frame consistency total value, overlap component, boundary component, area component, direction component, and topological component, respectively. The legend uses different line types and labels to indicate each component. Figure 5 The cross-frame consistency formula shown in the text is used to calculate the following: the region overlap is used to evaluate the overlap after forward mapping, the boundary distance is used to evaluate the reverse reprojection deviation, the area and orientation changes are used to evaluate the morphological continuity, and the topology preservation is used to evaluate whether the contour connection relationship is stable.

[0040] from Figure 5It can be seen that the overall consistency scores of different candidate anomaly regions vary. Regions with higher scores typically have good overlap, boundary, and topological components, indicating that these regions can repeatedly appear in previous, current, and subsequent images with similar material coordinates and contour structures. Although some candidate regions show obvious local grayscale anomalies, their area or orientation components are low, indicating that their morphology may change significantly with changes in viewing angle or reflective migration, requiring a reduction in the confidence level of directly classifying them as real defects.

[0041] In one embodiment, determining the welding quality inspection result includes: establishing a morphological description of the candidate abnormal region, including the region centroid, boundary contour, main extension direction, and area distribution; mapping the morphological description from the current weld image to the subsequent weld image according to the mapping relationship, and reprojecting the subsequent weld image back to the current weld image; calculating the region overlap after forward mapping, the boundary distance after reverse reprojection, and the change sequence of the main extension direction and area distribution in the continuous weld images; assigning confidence weights to each calculation result according to the image saturation, blurring degree, and occlusion ratio corresponding to the candidate abnormal region, and forming a cross-frame consistency judgment value of the candidate abnormal region based on the weighted region overlap, boundary distance, and change sequence.

[0042] In the morphological description, the centroid of the region is calculated in the material coordinate system. The boundary contour is represented by a set of contour points arranged in sequence. The main extension direction is determined by the main direction of the covariance of the material coordinate points of the candidate region. The area distribution is composed of multiple cross-sectional areas divided along the longitudinal material coordinate of the candidate region. During forward mapping, the set of contour points is projected onto the subsequent image and corresponds to the abnormal response boundary in the subsequent image. During reverse reprojection, the inverse matrix of the forward mapping is not directly used to replace the local mapping. Instead, a two-level mapping from the subsequent image to the material coordinate and from the material coordinate to the current image is called to incorporate local non-rigid deformation and material coordinate constraints into the reprojection process.

[0043] Image saturation is determined by the proportion of pixels in the candidate region and its neighborhood that reach the upper limit of brightness. Blur is determined by the concentration of edge response within the region. Occlusion ratio is determined by the proportion of material coordinate grids in which the candidate region is not visible in adjacent images. Confidence weights are attached to overlap, boundary distance, area change, and direction change, respectively. When a candidate region in a subsequent image is occluded by smoke or dust, the current candidate region is not directly rejected based on low overlap. Instead, the confidence of overlap is reduced, and the reverse visible boundary, the shape of the preceding image, and the topological connection relationship are retained as the basis for judgment. When the arc reflection moves in position in a subsequent image, its material coordinate offset and boundary reprojection distance are still included in the consistency judgment.

[0044] In one embodiment, correcting the cumulative arc length using path progress data includes: selecting locations of abrupt changes in centerline curvature, stable weld width locations, and stable texture locations within overlapping areas of adjacent images as joint correction nodes in the weld image sequence; calculating the longitudinal deviation between the visual cumulative arc length of each joint correction node and the path progress data, and determining the correction weight of each longitudinal deviation according to the centerline extraction confidence, boundary integrity, and texture matching stability; establishing a piecewise correction function with constraints such as continuous longitudinal scaling between adjacent joint correction nodes, monotonically increasing longitudinal coordinates, and consistent corresponding positions in overlapping image areas; updating the longitudinal coordinates of each longitudinal anchor point using the piecewise correction function, and synchronously updating the local cross-sectional positions corresponding to the transverse normalized coordinates.

[0045] The joint correction nodes are selected in a complementary manner based on different image evidence. The position of the curvature change of the center line is used to provide the geometric mark of the curved weld, the position of the weld width stability is used to provide the correspondence of the transverse section, and the position of the texture stability in the overlapping area of ​​adjacent images is used to provide the correspondence of local material texture. When the same position meets multiple node conditions at the same time, it is merged into one node and the node credibility is increased. The longitudinal deviation is the difference between the position represented by the visual cumulative arc length and the position represented by the path progress. The correction weight is determined by whether the center line is continuous, whether the left and right boundaries are visible at the same time, and whether the texture correspondence is unique. Even if the texture response is strong in the repeated fish scale pattern area, the texture matching stability is reduced because the correspondence is not unique.

[0046] Preferably, the correction amount at any longitudinal position z is calculated according to the following formula: ; in, This represents the correction amount at the longitudinal position z. Denotes the set of joint correction nodes located in the z-neighborhood. This indicates the original longitudinal position of the nth joint correction node. This represents the longitudinal deviation of the nth joint correction node. This represents the correction weight of the nth joint correction node. This represents a distance kernel function with an effective range of h. The smaller the distance, the larger the kernel function value. For example, when there are high-weight nodes with the same deviation direction in the neighborhood of a certain position, the position will receive the same direction correction. If the deviation directions of the neighboring nodes are opposite, a continuous transition will be formed according to the node distance and weight to avoid jumps in the vertical coordinate.

[0047] Please refer to Figure 6 , Figure 6 The horizontal axis represents the longitudinal position in the weld material coordinate system, and the vertical axis represents the corresponding correction amount or related correction contribution. The curves show the piecewise correction distribution formed by weighting the joint correction nodes using a distance kernel function, and the legend distinguishes different output terms. Figure 6 The correction amount at any position is calculated using the longitudinal correction formula shown in the text. The longitudinal deviation, node weight, and distance kernel function of each joint correction node in the neighborhood jointly determine the correction amount at any position. The closer the node is and the higher its confidence level, the greater its influence on the current position.

[0048] from Figure 6 It can be seen that the correction amount changes continuously along the longitudinal material coordinates without abrupt jumps between adjacent material positions. In the interval where the joint correction nodes are dense and the deviation directions are consistent, the correction curve forms a peak in the same direction, indicating that there is an interpretable system offset between the visual cumulative arc length and the robot path progress in this area. In the interval where the deviation directions are opposite or the node weights are low, the correction amount gradually falls back and forms a smooth transition, avoiding distortion of the entire material coordinates due to anomalies in a single local node.

[0049] In one embodiment, global projection deformation and local non-rigid deformation are solved by alternating constraints, specifically including: fixing the local non-rigid deformation, correcting the global projection deformation based on the updated longitudinal anchor points, the corresponding positions of adjacent image overlap areas, and the overall orientation of the weld boundaries on both sides; fixing the corrected global projection deformation, updating the local non-rigid deformation based on the joint matching cost of local control points; calculating the local change rate of the composite mapping in the longitudinal and transverse directions of the weld after each update, and applying position backoff constraints to local control points where the longitudinal change rate is less than zero, the transverse change rate sign is flipped, or the boundary mapping exceeds the limit; repeating the correction and update until the displacement difference of the control points between two adjacent composite mappings satisfies the preset convergence condition.

[0050] At the start of the alternating constraint solution, the displacement of local control points is set to zero. The initial transformation formed by the path progress difference and the change in the centerline tangential direction is used as the global projection deformation. When correcting the global projection deformation, the longitudinal anchor point correspondence after piecewise correction is used, and the perspective change is restricted by the overall direction of the weld boundary on both sides. When updating the local non-rigid deformation, the joint matching cost of the candidate displacement of the control point is calculated one by one. At the same time, a smoothing constraint is set on the displacement difference between adjacent control points. After each round of update, the longitudinal and transverse arrangement of the material coordinate mesh after mapping is checked. If the local control point causes the next longitudinal mesh to be mapped to the previous longitudinal mesh, the control point is backed to the nearest position that satisfies the longitudinal monotonicity.

[0051] Furthermore, the stopping condition for alternating solutions is calculated according to the following formula: ; in, This represents the average change in control point displacement after the h-th update, where M represents the total number of local control points. This represents the displacement vector of the m-th local control point after the h-th update. This represents the displacement vector corresponding to the previous round, when Updates will stop when the data falls below the preset convergence limit and the composite mapping does not exhibit vertical reversal, horizontal flipping, or boundary overflow. For example, if some control points are located in smoke-covered areas and continue to fluctuate, the control points can be frozen and their displacements can be obtained by interpolation using adjacent visible control points, thus preventing a small number of unobservable areas from preventing the overall mapping from converging.

[0052] In one embodiment, forming a cross-frame consistency determination value further includes: converting the boundary contour of the candidate abnormal region into a topological description consisting of endpoints, turning points, bifurcation points, and closed boundary segments, and recording the material coordinates of each topological point relative to the weld centerline and the weld boundaries on both sides; matching topological points between the preceding weld image, the current weld image, and the subsequent weld image based on the composite mapping, and calculating the topological point connectivity retention degree and material coordinate offset; setting missing markers for topological points covered by saturated areas, blurred areas, or occluded areas, and using the connection order of adjacent visible topological points to limit the allowable position range of the missing topological points; and incorporating the connectivity retention degree, material coordinate offset, and allowable position range conformity into the cross-frame consistency determination value.

[0053] When converting boundary contours into topological descriptions, local peaks of contour curvature are used as inflection points, the two ends of slender candidate regions are used as endpoints, and the connection points where multiple branches extend within a region are used as bifurcation points. When contours are connected end to end and contain closed regions, closed boundary segments are recorded. In addition to storing material coordinates, each topological point also stores the connection order and boundary direction with adjacent topological points. During cross-frame matching, corresponding points are first searched based on material coordinates, and then the connection order and boundary direction are compared to avoid mismatching of non-corresponding points in duplicate textures simply because they are close in distance. After local occlusion, closed boundary segments can appear as two visible endpoints. At this time, the allowable position range of missing boundaries is established based on the occlusion range and the original connection direction.

[0054] Preferably, the topological conformity of the r-th candidate anomaly region is determined according to the following formula: ; in, This represents the topological conformity of the r-th candidate anomaly region. This indicates the degree of preservation of topological connectivity between preceding, current, and subsequent images. This represents the normalized material coordinate offset of the matching topology point. This indicates the proportion of topological points with missing markers that fall within the allowed location range. The coefficients are non-negative and sum to 1; for example, they increase when the candidate region boundary is intact. and reduce Improved when candidate region is partially occluded The proportion of the topology conformity is used to make the topology conformity correspond to the actual visible state.

[0055] In one embodiment, generating a continuous weld strip map includes: dividing each weld image into multiple material coordinate grids based on a composite mapping, and calculating the grid confidence level for each material coordinate grid, which consists of image sharpness, local saturation ratio, boundary integrity, mapping residual, and topological point visibility; when multiple weld images are mapped to the same material coordinate grid, retaining the image data with the highest grid confidence level as the base data, and performing weighted fusion according to the grayscale structure consistency and boundary direction consistency between the remaining image data and the base data; for topological points with missing markers, only image data with overlapping allowed position intervals are fused; and re-extracting the weld centerline, weld boundary, and candidate abnormal regions based on the fused grid data, and updating the composite mapping and cross-frame consistency judgment values.

[0056] The image sharpness in the mesh confidence is represented by the concentration of edge response within the material coordinate mesh; the local saturation ratio is represented by the proportion of source pixels that reach the brightness limit; the boundary integrity is represented by whether the center line and both side boundaries exist simultaneously in the corresponding transverse section; the mapping residual is represented by the coordinate difference of the source pixels after forward mapping and backward reprojection; and the topological point visibility is represented by the number of directly observable endpoints, turning points, bifurcation points, and closed boundary segments within the mesh. After the basic data is selected, it is not directly averaged over all source pixels, but the main boundary structure of the basic data is retained, and fusion is only performed when other images have the same material position and consistent boundary direction as the basic data.

[0057] Furthermore, the fused image value of the g-th material coordinate grid is calculated according to the following formula: ; in, This represents the fused image value of the g-th material coordinate grid. This represents the set of source images covering the g-th material coordinate grid. This represents the image value mapped from the f-th source image to the g-th material coordinate grid. Indicates the confidence level of the corresponding grid. This represents a combined value indicating the consistency of grayscale structure and boundary orientation between the source image and the underlying data. It is indicated when the source image exhibits saturation or inconsistent boundary orientation. or When multiple source images are clear and their boundaries correspond consistently, the smaller value is selected, and the fusion result is determined jointly by the multiple source images.

[0058] Please refer to Figure 7 , Figure 7 The horizontal axis represents the material coordinate grid position in the continuous weld strip image, and the vertical axis represents the normalized grid value. The curves show the grid confidence and the image intensity after multi-source fusion, respectively. The legend distinguishes between the two types of output using different line styles and markers. Figure 7The image value is calculated by the mesh fusion formula shown in the text. The image value after each source image is mapped to the same material mesh needs to be multiplied by the mesh confidence, as well as the grayscale structure consistency and boundary direction consistency with the base data. Then, the fused image value is formed by weighted averaging.

[0059] from Figure 7 As can be seen, the fusion intensity varies continuously along the material coordinates, while the mesh confidence fluctuates locally. Regions with higher confidence typically correspond to material meshes with clear imaging, intact boundaries, small mapping residuals, and good topological point visibility; regions with decreasing confidence may correspond to local saturation, blurring, occlusion, or inconsistent boundary orientations. In low-confidence regions, the fusion intensity does not simply replicate images from a single source, but is constrained by both confidence and structural consistency, reducing the contamination of the strip map by low-quality sources.

[0060] When re-extracting the centerline and boundary in the fused strip image, the centerline and boundary in the base data are used as the initial positions. The gradient peak and regional continuous response of the fused image are searched in the adjacent grid. The new centerline and boundary are used to recalculate the lateral normalized coordinates and make small-scale updates to the local non-rigid deformation. The candidate anomaly region is re-segmented in the updated material coordinates. If an anomaly exists only in the low-confidence source image and has no corresponding boundary or topology in the high-confidence source image, its cross-frame consistency judgment value is reduced according to the source confidence. If an anomaly maintains the material position and boundary direction in multiple source images, its fused contour is retained for subsequent anomaly segment generation.

[0061] In one embodiment, determining the welding quality inspection result further includes: sorting candidate abnormal regions according to the longitudinal coordinates of the weld material coordinate system; merging candidate abnormal regions with a longitudinal interval smaller than the corresponding local weld width and continuous topological point connection relationship into abnormal segments; for each abnormal segment, collecting the cross-frame consistency judgment value and mesh confidence of each candidate abnormal region constituting the abnormal segment in different weld images, and removing candidate abnormal regions whose material coordinate offset exceeds the corresponding allowable position interval; determining the start and end positions of the abnormal segment according to the overlapping material coordinate range of the remaining candidate abnormal regions, determining the lateral range of the abnormal segment according to the fused boundary contour, and combining the start and end positions, lateral range, morphological description, and cross-frame consistency judgment value into the welding quality inspection result.

[0062] When merging candidate anomalous regions, the longitudinal interval is normalized using the local weld width to accommodate the variation of weld width along the path. The continuous connection relationship of topological points indicates that the end topological point of the previous candidate region and the starting topological point of the next candidate region are in the same extension direction in the material coordinates, and there is no complete separation area between them that is consistent with the normal texture of the weld. Candidate regions whose material coordinate offset exceeds the allowable position range are regarded as outlier evidence generated by mapping instability or transient interference. The elimination process only removes the region record in the corresponding source image, without deleting the valid region of the same material position in other source images, so that the anomalous segment is determined by the range of material positions that can be repeatedly observed across frames.

[0063] The start and end positions in the detection results are taken as the combined boundary of the common coverage and continuous extension range of the remaining candidate anomaly regions in the longitudinal material coordinates. The lateral range is taken as the minimum envelope range of the fused boundary contour relative to the weld centerline and the two side boundaries. The morphological description saves the continuous length, average width variation, main extension direction, number of endpoints, number of turning points, bifurcation state and closed boundary state of the anomaly segment. The cross-frame consistency judgment value is collected by the consistency values ​​of the candidate regions constituting the anomaly segment according to the grid confidence. The output record is simultaneously associated with the original weld image number and material coordinate grid range covering the anomaly segment, so as to verify the image evidence under the same coordinate reference.

[0064] In a preferred embodiment, when the weld path contains continuous curvature changes, the material coordinate system rotates continuously tangentially with the centerline. During the strip unfolding process, the order of adjacent transverse sections is maintained and the intersection of sections is restricted. When local undulations of the weld cause changes in the image width of the same transverse section, the position of the abnormal region relative to the centerline and weld boundary is preserved by transverse normalized coordinates, instead of directly judging the region size based on the original pixel width. When the robot pauses briefly at a local location and collects multiple approximate images, the approximate images are regarded as multiple source images of the same material interval. The basic data is selected based on sharpness, saturation state, and mapping residual, and the remaining images are used for cross-frame consistency and topology conformity calculation.

[0065] In another preferred embodiment, when there is a periodic fish scale pattern in the weld image, local texture matching increases the material coordinate monotonicity and path progress difference constraints, so that adjacent repeating textures cannot cross the predetermined longitudinal search range to establish a correspondence. When a local area forms a bright closed patch due to reflection, the boundary position and area of ​​the bright closed patch usually change in images from different viewing angles. Its back reprojection boundary distance, material coordinate offset and topological connectivity retention are jointly included in the judgment. When the real surface anomaly is missing its boundary only due to local occlusion in multiple images, the position interval is allowed to limit the missing topological point, and the connection order is maintained by the adjacent visible topological points.

[0066] In one embodiment, a machine vision-based robotic welding quality inspection system includes: an image acquisition module for acquiring a sequence of weld images continuously collected as the robot moves along the welding path, as well as path progress data corresponding to each weld image; a material coordinate construction module for extracting the weld centerline and weld boundary from each weld image, and establishing a weld material coordinate system with the welding direction as the longitudinal direction and the weld transverse cross section as the transverse direction based on the path progress data; a deformation mapping module for decomposing the deformation between adjacent weld images into global projection deformation and local non-rigid deformation, and obtaining the mapping relationship from each weld image to the weld material coordinate system under the constraints of path progress, centerline continuity, and boundary sequence; a strip generation module for generating continuous weld strip images and segmenting candidate abnormal regions based on the mapping relationship; and a consistency determination module for projecting the candidate abnormal regions onto the preceding and subsequent weld images, and determining the welding quality inspection result based on forward mapping consistency, reverse reprojection consistency, and cross-frame morphological continuity.

[0067] The image acquisition module receives weld images and path progress records according to a unified time base, and establishes an index record for each image containing the acquisition time, image number, path progress, and valid status. The material coordinate construction module performs weld region extraction, centerline extraction, boundary extraction, anchor point selection, longitudinal arc length accumulation, and lateral normalization processing on the images. The deformation mapping module saves the global projection transformation parameters, local control point displacement fields, mapping residuals, and reversible mapping indexes of adjacent images. The strip generation module organizes multi-source image data according to the material coordinate grid and saves the grid credibility. The consistency determination module reads the morphological and topological descriptions of candidate abnormal regions and completes the forward mapping, reverse reprojection, credibility weighting, and abnormal segment generation between the preceding, current, and subsequent images.

[0068] In this embodiment, each module can be implemented by program instructions in the same computing device, or it can be executed by processing nodes that communicate with each other. When the program runs, it reads the image sequence according to the path progress index. The material coordinate construction result serves as the common data benchmark for deformation mapping, strip generation, and consistency determination. The deformation mapping result includes both the forward mapping from the image to the material coordinates and the reverse mapping from the material coordinates to the image. After the strip map is updated, the new center line, boundary, and candidate abnormal region are fed back to the deformation mapping process for local correction. The consistency determination result is associated with the original image record by the material coordinate range, thereby forming a closed processing flow from continuous image acquisition, unified coordinate establishment, composite deformation calculation, strip fusion, candidate region segmentation to cross-frame result output.

Claims

1. A robot welding quality inspection method based on machine vision, characterized in that, include: Acquire a sequence of weld seam images continuously collected as the robot moves along the welding path, as well as path progress data corresponding to each weld seam image; The center line and boundary of each weld are extracted from the weld images, and a weld material coordinate system is established based on the path progress data, with the welding direction as the longitudinal direction and the weld cross-section as the transverse direction. The deformation between adjacent weld seam images is decomposed into global projection deformation and local non-rigid deformation, and the mapping relationship between each weld seam image and the weld seam material coordinate system is obtained under the constraints of the path progress data, weld seam centerline continuity and weld seam boundary sequence. A continuous weld strip map is generated based on the mapping relationship, and candidate abnormal regions are segmented. The candidate abnormal regions are projected onto the preceding and subsequent weld images, and the welding quality inspection results are determined based on the consistency of forward mapping, the consistency of reverse reprojection, and the continuity of cross-frame morphology.

2. The robot welding quality inspection method based on machine vision according to claim 1, characterized in that, Establishing the weld material coordinate system includes: monotonically sorting the weld image sequence according to the path progress data, and selecting a longitudinal anchor point corresponding to the path progress data on the weld centerline of each weld image; Calculate the tangential direction of the weld centerline at the longitudinal anchor point, and take the direction that is orthogonal to the tangential direction and points to the two sides of the weld as the transverse direction; Establish lateral normalized coordinates based on the intersection of the weld boundaries on both sides with the lateral direction; The centerline arc length between adjacent longitudinal anchor points is accumulated, the accumulated arc length is corrected using the path progress data, and the corrected longitudinal coordinates are combined with the transverse normalized coordinates to form the weld material coordinate system.

3. The robot welding quality inspection method based on machine vision according to claim 1, characterized in that, Determining the mapping relationship includes: constructing the initial transformation of the global projection deformation based on the path progress difference and the tangential change of the weld centerline corresponding to adjacent weld images; After removing the initial transformation, multiple local control points are set along the center line of the weld, and the grayscale structure similarity, boundary distance residual and center line offset of each local control point in the adjacent weld images are calculated respectively. The grayscale structure similarity, the boundary distance residual, and the centerline offset constitute a joint matching cost, and the local non-rigid deformation is obtained under the constraints that the longitudinal order of the local control points is not reversed, the weld boundaries on both sides do not intersect, and the local transverse scale is continuous. The global projection deformation and the local non-rigid deformation are combined to form a mapping relationship.

4. The robot welding quality inspection method based on machine vision according to claim 1, characterized in that, Determining the welding quality inspection results includes: establishing a morphological description of the candidate abnormal region, including the region centroid, boundary contour, main extension direction, and area distribution; Based on the mapping relationship, the morphological description is forward-mapped from the current weld image to the subsequent weld image, and then reprojected backward from the subsequent weld image back to the current weld image; Calculate the region overlap after forward mapping, the boundary distance after reverse reprojection, and the variation sequence of the main extension direction and area distribution in continuous weld images; The calculation results are assigned confidence weights based on the image saturation, blurriness, and occlusion ratio corresponding to the candidate abnormal regions. The cross-frame consistency judgment value of the candidate abnormal regions is formed based on the weighted region overlap, boundary distance, and change sequence.

5. The robot welding quality inspection method based on machine vision according to claim 2, characterized in that, Correcting the cumulative arc length using the path progress data includes: selecting the locations of abrupt changes in centerline curvature, stable weld width locations, and stable texture locations within the overlapping areas of adjacent images as joint correction nodes in the weld image sequence; The longitudinal deviation between the visual cumulative arc length of each joint correction node and the path progress data is calculated respectively, and the correction weight of each longitudinal deviation is determined according to the confidence of centerline extraction, boundary integrity and texture matching stability. A piecewise correction function is established with constraints such as continuous vertical scaling between adjacent joint correction nodes, monotonically increasing vertical coordinates, and consistent corresponding positions in the overlapping areas of the images. The longitudinal coordinates of each longitudinal anchor point are updated using the piecewise correction function, and the local cross-sectional position corresponding to the transverse normalized coordinates is updated synchronously.

6. The robot welding quality inspection method based on machine vision according to claim 3, characterized in that, The global projection deformation and the local non-rigid deformation are solved by alternating constraints, specifically including: fixing the local non-rigid deformation, and correcting the global projection deformation according to the updated longitudinal anchor points, the corresponding positions of the overlapping areas of adjacent images, and the overall orientation of the weld seam boundaries on both sides. The fixed global projection deformation is used to update the local non-rigid deformation based on the joint matching cost of the local control points. After each update, calculate the local rate of change of the composite mapping in the longitudinal and transverse directions of the weld, and apply position backoff constraints to local control points where the longitudinal rate of change is less than zero, the transverse rate of change is flipped, or the boundary mapping is out of bounds. Repeat the correction and update process until the difference in control point displacement between two adjacent composite mappings meets the preset convergence condition.

7. The robot welding quality inspection method based on machine vision according to claim 4, characterized in that, The process of forming the cross-frame consistency determination value also includes: converting the boundary contour of the candidate abnormal region into a topological description consisting of endpoints, turning points, bifurcation points and closed boundary segments, and recording the material coordinates of each topological point relative to the weld centerline and the weld boundaries on both sides. Based on the composite mapping, topological points are matched between the preceding weld image, the current weld image, and the subsequent weld image, and the topological point connectivity retention degree and material coordinate offset are calculated. For topological points covered by saturated, blurred, or occluded areas, set missing markers and use the connection order of adjacent visible topological points to limit the allowable location range of the missing topological points; The connection relationship retention, material coordinate offset, and allowable position interval compliance are incorporated into the cross-frame consistency judgment value.

8. The robot welding quality inspection method based on machine vision according to claim 1, characterized in that, Generating the continuous weld strip image includes: dividing each weld image into multiple material coordinate grids based on composite mapping, and calculating the grid confidence level for each material coordinate grid, which consists of imaging sharpness, local saturation ratio, boundary integrity, mapping residual, and topological point visibility. When multiple weld images are mapped to the same material coordinate grid, the image data with the highest grid confidence is retained as the base data, and the remaining image data are weighted and fused according to the consistency of grayscale structure and boundary direction between the base data and the base data. For topological points with missing labels, only image data with overlapping location intervals are fused; Based on the fused mesh data, the weld centerline, weld boundary, and candidate abnormal region are re-extracted, and the composite mapping and cross-frame consistency judgment value are updated.

9. The robot welding quality inspection method based on machine vision according to claim 7, characterized in that, Determining the welding quality inspection result further includes: sorting the candidate abnormal regions according to the longitudinal coordinates of the weld material coordinate system, and merging the candidate abnormal regions with longitudinal intervals smaller than the corresponding local weld width and continuous topological point connection relationships into abnormal segments; For each abnormal segment, the cross-frame consistency judgment value and mesh confidence of each candidate abnormal region constituting the abnormal segment in different weld images are collected, and candidate abnormal regions whose material coordinate offset exceeds the corresponding allowable position interval are eliminated. The start and end positions of the abnormal segment are determined based on the coordinate range of the overlapping materials in the remaining candidate abnormal regions; The lateral range of the abnormal section is determined based on the fused boundary contour, and the start and end positions, lateral range, morphological description, and cross-frame consistency judgment value are combined to form the welding quality inspection result.

10. A robot welding quality inspection system based on machine vision, characterized in that, include: The image acquisition module acquires a sequence of weld seam images continuously collected as the robot moves along the welding path, as well as the path progress data corresponding to each weld seam image. The material coordinate construction module extracts the weld centerline and weld boundary from each weld image, and establishes a weld material coordinate system with the welding direction as the longitudinal direction and the weld cross-section as the transverse direction based on the path progress data. The deformation mapping module decomposes the deformation between adjacent weld seam images into global projection deformation and local non-rigid deformation, and obtains the mapping relationship between each weld seam image and the weld seam material coordinate system under the constraints of path progress, centerline continuity and boundary sequence. The strip generation module generates a continuous weld strip map and segments candidate abnormal regions based on the mapping relationship; The consistency determination module projects the candidate abnormal region onto the preceding weld image and the subsequent weld image, and determines the welding quality detection result based on the forward mapping consistency, the reverse reprojection consistency, and the cross-frame morphological continuity.

Images