Laser welding seam detection method and system combined with image recognition

Abstract

The invention discloses a laser welding seam detection method and system combined with image recognition, and relates to the technical field of image recognition. A plurality of pit areas are determined based on image recognition of a welding seam three-dimensional image; determining a plurality of welding seam control nodes according to the pit space of each pit area, the corresponding surface morphology and the area variation of the two adjacent pit areas, and determining a corresponding welding seam quality event based on dynamic detection of each welding seam control node. According to the welding seam quality event of each welding seam control node, the corresponding node position and the laser data combination of the laser welding head, the corresponding laser welding seam detection system is determined, and the accuracy of the laser welding seam detection system is improved. And corresponding welding data and dynamic working data are determined based on detection of the laser welding head, and a welding state curve of the laser welding head is determined according to the welding data and the dynamic working data of the laser welding head and the quality grade of each key welding line area.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more particularly to a method and system for detecting laser welds that combines image recognition. Background Technology

[0002] With the development of technology, automobiles are gradually being applied to people's lives. Gasoline cars and new energy vehicles are types of automobiles. Automobiles are composed of multiple automotive parts. In some automotive parts, the production of these parts involves laser welding processes, resulting in corresponding laser welds. In existing technologies, images of laser welds are acquired, and multiple abnormal features are identified based on the recognition of these images. The corresponding abnormal content is then marked. However, the weld quality events at each weld control node and the combination of laser data from the laser welding head are ignored, affecting the accuracy of the laser weld detection system and leading to lower accuracy in the welding state curve of the laser welding head. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a method and system for detecting laser welds that combines image recognition.

[0004] This invention provides a method for detecting laser welds using image recognition, comprising: When the laser welding head performs laser welding on the automotive workpiece, the weld seam path of the automotive workpiece is acquired, and visual inspection of the weld seam is triggered along the weld seam path to determine multiple weld seam images; a three-dimensional image of the weld seam is determined based on the weld seam path of the automotive workpiece and multiple weld seam images; the three-dimensional image of the weld seam reproduces the three-dimensional morphology of the laser weld seam, including multi-dimensional information such as pits, protrusions, and undercut. Multiple pit regions are identified based on image recognition of weld 3D images. Multiple weld control nodes are determined based on the pit space of each pit region, the corresponding surface morphology, and the regional variation of two adjacent pit regions. The weld control nodes present a complete description of the node location, the triggering event, and the correlation evidence pointing to the two original pit regions that caused the node. In multiple weld seam control nodes, the corresponding weld seam quality events are determined based on the dynamic detection of each weld seam control node, and the corresponding laser weld seam detection system is determined based on the combination of the weld seam quality events of each weld seam control node, the corresponding node position and the laser data of the laser welding head. Based on the identification of this laser weld seam detection system, multiple key weld seam areas are identified, and the quality level of each key weld seam area is determined based on the abnormal events, corresponding abnormal morphologies, and corresponding pit surfaces of each key weld seam area. Based on the detection of the laser welding head, the corresponding welding data and dynamic working data are determined. According to the welding data, dynamic working data and quality level of each key weld area of ​​the laser welding head, the welding state curve of the laser welding head is determined, and the corresponding welding optimization content is marked.

[0005] This invention provides a laser weld detection system that combines image recognition, which is applied to the above-described laser weld detection method that combines image recognition.

[0006] Compared with the prior art, the beneficial effects of the present invention are: (1) In multiple weld control nodes, the corresponding weld quality events are determined based on the dynamic detection of each weld control node. The corresponding laser weld detection system is determined based on the weld quality events of each weld control node, the corresponding node position and the laser data combination of the laser welding head. The weld quality events are further controlled, and the overall consideration of the weld quality events of each weld control node, the corresponding node position and the laser data combination of the laser welding head is realized, which improves the accuracy of the laser weld detection system.

[0007] (2) Based on the identification of the laser weld seam detection system, multiple key weld seam areas are identified. Based on the abnormal events, corresponding abnormal morphologies and corresponding pit surfaces of each key weld seam area, the quality level of each key weld seam area is determined. Based on the detection of the laser welding head, the corresponding welding data and dynamic working data are determined. Based on the welding data, dynamic working data and quality level of each key weld seam area of ​​the laser welding head, the welding state curve of the laser welding head is determined. The quality level of each key weld seam area is introduced, and the welding data and dynamic working data of the laser welding head are further controlled, improving the accuracy of the welding state curve of the laser welding head, and marking the corresponding welding optimization content. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating the laser weld detection method combining image recognition in an embodiment of the present invention. Figure 2 This is a flowchart illustrating step S11 of the laser weld detection method combining image recognition in an embodiment of the present invention. Figure 3 This is a flowchart illustrating step S12 of the laser weld detection method combining image recognition in an embodiment of the present invention. Figure 4 This is a flowchart illustrating step S13 of the laser weld detection method combining image recognition in an embodiment of the present invention. Figure 5This is a flowchart illustrating step S14 of the laser weld detection method combining image recognition in an embodiment of the present invention. Figure 6 This is a flowchart illustrating step S15 of the laser weld detection method combining image recognition in an embodiment of the present invention. Detailed Implementation

[0009] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0010] Please see Figures 1 to 6 A laser weld detection method combining image recognition is proposed and applied to image recognition scenarios. The laser weld detection method combining image recognition includes: Step S11: When the laser welding head performs laser welding on the automotive workpiece, the weld seam path of the automotive workpiece is acquired, and visual inspection of the weld seam is triggered along the weld seam path to determine multiple weld seam images; a three-dimensional image of the weld seam is determined based on the weld seam path of the automotive workpiece and the multiple weld seam images. Step S12: Based on image recognition of the weld three-dimensional image, determine multiple pit regions, and determine multiple weld control nodes according to the pit space of each pit region, the corresponding surface morphology, and the regional change of two adjacent pit regions. Step S13: In multiple weld seam control nodes, determine the corresponding weld seam quality events based on the dynamic detection of each weld seam control node, and determine the corresponding laser weld seam detection system based on the combination of weld seam quality events of each weld seam control node, the corresponding node position and the laser data of the laser welding head; Step S14: Based on the identification of the laser weld inspection system, identify multiple key weld areas, and determine the quality level of each key weld area based on the abnormal events, corresponding abnormal morphology and corresponding pit surface of each key weld area. Step S15: Determine the corresponding welding data and dynamic working data based on the detection of the laser welding head. Determine the welding state curve of the laser welding head based on the welding data, dynamic working data and quality level of each key weld area, and mark the corresponding welding optimization content.

[0011] refer to Figure 2 In step S11, the specific steps are as follows: S111: The automotive workpiece is positioned in the corresponding positioning fixture, and the current posture of the automotive workpiece is determined. The weld seam path of the automotive workpiece is determined according to the welding position of the automotive workpiece, the moving path of the laser welding head and the corresponding laser welding requirements. The camera moves along the weld seam path to determine the camera's moving path. The camera's visual inspection of the weld seam is triggered according to the camera's moving path and the current posture of the automotive workpiece. S112: Based on the visual inspection of the weld seam by the camera, determine multiple weld seam images and mark the corresponding shooting positions. Based on the multiple weld seam images, the corresponding shooting positions and the weld seam path of the automotive workpiece, determine multiple image combinations and construct a three-dimensional image of the weld seam based on the multiple image combinations.

[0012] In the embodiments of this application, physical constraints are provided by a high-precision positioning fixture tailored to the workpiece to ensure the repeatability of the workpiece's positioning accuracy each time it is placed. The system needs to determine the workpiece's six-degree-of-freedom (6-DoF) attitude in the robot's workspace through an external measurement system, namely its position (X, Y, Z) and rotation (Roll, Pitch, Yaw) in space. Commonly used attitude acquisition techniques include contact probe calibration and non-contact 3D vision calibration. The former uses a probe carried by the robot's end effector to contact known reference points on the workpiece and calculates the workpiece coordinate system through multi-point fitting. The latter uses a line laser scanner or structured light camera to quickly scan the workpiece and performs ICP (Iterative Closest Point) registration between the generated 3D point cloud data and the workpiece's CAD model to calculate the workpiece's current precise attitude.

[0013] The system directly extracts the theoretical geometric centerline of the weld from the CAD model of the workpiece. This path is initially defined in the workpiece coordinate system. Using the obtained {Part_Pose_Base} posture matrix, the system transforms this theoretical weld path from the workpiece coordinate system to the robot's base coordinate system through homogeneous coordinate transformation, generating a path {Weld_Path_Base} that the robot can directly execute. Based on the specific laser welding process requirements, such as weld speed, laser power, and wire feed rate, the system associates these process parameters with each point on the path, ultimately forming a welding path that contains both spatial position information and process instructions.

[0014] When the camera is capturing images, its optical axis maintains a fixed optimal angle (usually vertical or slightly tilted) with the normal to the weld surface to ensure that the captured images have high consistency in lighting, focal length, and perspective, thereby guaranteeing the accuracy of subsequent inspections. The camera's movement path {Camera_Path_Base} is generated based on the weld path {Weld_Path_Base}, and is usually offset by a fixed working distance in the normal direction of the weld path. In addition, the planning must also fully consider the camera's own optical performance, such as field of view (FOV) and depth of field (DOF).

[0015] The camera does not record continuously during this process. Instead, as it moves along its path to each preset shooting trigger point, it precisely controls the camera exposure through hardware trigger signals (such as TTL signals) to capture a high-quality still image. The system records in real time the precise pose {Camera_Pose_i} of the camera end in the robot's base coordinate system when each image is captured. This move-trigger-record mode ensures a strict binding between image data and spatial position information, providing indispensable geometric prior knowledge for subsequent stereo vision analysis and defect localization.

[0016] Specifically, the automotive workpiece is an aluminum alloy battery tray for a new energy vehicle. The laser weld is a straight weld approximately 1.5 meters long between its frame and base plate. The operator places the battery tray on a special fixture consisting of multiple high-precision positioning pins and hydraulic clamping cylinders. The fixture is designed to ensure that the positional deviation of the tray is less than ±0.1mm each time it is placed. The robot initiates the positioning program, and its end-effector's line laser scanner quickly scans several pre-defined feature areas of the tray (such as the corners of the frame). The system automatically registers the scanned point cloud with the standard CAD model of the battery tray stored in the database and instantly calculates the specific posture of the tray relative to the robot's base coordinate system. For example, the position is (1012.5mm, 500.3mm, 800.0mm), and the rotation is (0.05°, -0.02°, 0.1°). This precise posture data is recorded by the system as {Part_Pose_Base}, serving as the benchmark for all subsequent path calculations.

[0017] For the laser weld seam of the battery tray, the system extracts the theoretical centerline of the weld seam, with its start and end points being Start_WCS(0,0,0) and End_WCS(1500,0,0) in the workpiece coordinate system, respectively. Using the determined {Part_Pose_Base} posture matrix, the system performs coordinate transformation on this theoretical path to generate the actual welding path {Weld_Path_Base} in the robot base coordinate system. For example, the theoretical start point Start_WCS(0,0,0) is transformed into the actual robot space start point Start_Base(1012.5mm,500.3mm,800.0mm). Based on the welding process requirements, the welding speed is set to 50mm / s, and the laser power to 3000W. The system associates these parameters with each point on {Weld_Path_Base} to generate the final welding instruction set, controlling the robot to move along {Weld_Path_Base} at a speed of 50mm / s while the laser continuously emits light at a power of 3000W.

[0018] The camera is mounted to the side front of the laser welding head at a 30° angle to the welding direction to avoid direct impact from welding fumes and spatter. Based on the generated {Weld_Path_Base}, the system generates a camera movement path {Camera_Path_Base} that is completely parallel to the weld normal (perpendicular to the weld surface). The field of view of the industrial camera used can cover a weld length of 20mm at the set working distance. To ensure the accuracy of image stitching and 3D reconstruction, the system sets the image overlap rate to 80%. Therefore, the camera's shooting step distance is precisely calculated as 20mm x (1-80%) = 4mm. The system finally generates a precise shooting trigger point every 4mm on {Camera_Path_Base}, forming a complete scanning sequence.

[0019] After the robot starts, the welding head begins welding along {Weld_Path_Base}, while the camera scans synchronously along {Camera_Path_Base}. When the camera moves to the first shooting point, the robot controller sends a TTL high-level trigger signal to the camera through the I / O port. The camera immediately responds and exposes, capturing the first weld image Image_1. The system simultaneously records the precise pose of the camera at this moment {Camera_Pose_1}. Subsequently, the robot triggers the camera every time it moves precisely 4mm along {Camera_Path_Base}, sequentially capturing Image_2, Image_3, ... and recording the corresponding {Camera_Pose_2}, {Camera_Pose_3}, ... until the entire 1.5-meter weld seam is scanned. The system obtains a series of weld image sequences with precise spatial pose labels.

[0020] Furthermore, the camera sequentially acquires a series of two-dimensional weld seam images, forming an image sequence {I_1,I_2,…,I_n}. For each frame I_i in the sequence, the system must bind it to a precise spatial pose label {P_i}. This pose label {P_i} is a six-degree-of-freedom rigid body transformation matrix, which describes the precise position and orientation of the camera coordinate system relative to the robot's base coordinate system at the instant I_i is acquired. The pose data is usually provided by the robot controller at the same timestamp when the camera is triggered, ensuring temporal and spatial synchronization.

[0021] The system performs a feature point detection algorithm on each image I_i in the image sequence, extracting thousands of feature points that remain stable under illumination, scale changes, and rotation. Feature point matching is performed between adjacent images I_i and I_{i+1} to find the pixel correspondence of the same physical point captured from different viewpoints. These matched feature point pairs, combined with their respective corresponding camera poses {P_i} and {P_{i+1}}, constitute an image combination. Here, image combination refers to geometrically linking multiple views closely through shared visual features. The weld path serves as a macroscopic constraint to verify and optimize the matching results, ensuring that all image combinations unfold along the same physical path.

[0022] For the same feature point matched in two or more images, its three-dimensional position in space can be calculated by the intersection of rays emanating from the centers of different cameras. When there are a large number of feature points from different viewpoints, the system can use algorithms such as motion reconstruction structure or multi-view stereo matching to perform global optimization, and finally generate a dense three-dimensional point cloud with color or reflectivity information. This point cloud is the specific representation of the weld stereo image. The weld stereo image reproduces the three-dimensional morphology of the laser weld, including multi-dimensional information such as pits, protrusions, and undercut. This multi-dimensional information includes depth, volume, and position information.

[0023] Specifically, the camera takes pictures along a 1.5-meter weld seam path with a step distance of 4mm. The system eventually obtains a sequence of (1500mm / 4mm) + 1 = 376 images, namely {Image_001, Image_002, ..., Image_376}. For the first image Image_001, the system records the camera pose {Camera_Pose_1} at the moment of its capture. This is a data structure containing translation vectors (x,y,z) and rotation quaternions (qw,qx,qy,qz). Similarly, Image_002 is bound to {Camera_Pose_2}, and so on. At this point, the system has a complete dataset: {(Image_i,Camera_Pose_i)|i=1to376}, laying a solid data foundation for subsequent processing.

[0024] The system analyzes Image_001 and detects 5000 ORB feature points, each with a 128-dimensional descriptor to characterize its local image information. Image_002 also detects 5000 feature points. 800 pairs of matching feature points are found between Image_001 and Image_002. For example, pixels (u_1, v_1) in Image_001 and pixels (u_2, v_2) in Image_002 are identified as the same micro-pit point on the corresponding weld surface. These 800 pairs of matching points, combined with the known {Camera_Pose_1} and {Camera_Pose_2}, constitute an image combination. This combination provides strong geometric constraints: the projection position of the same spatial point observed from two different known positions on the two image planes is known. This process is repeated between all adjacent image pairs (Image_002 and Image_003, ..., Image_375 and Image_376), forming a feature chain composed of image combinations that runs through the entire weld.

[0025] The system has known {Camera_Pose_1} and {Camera_Pose_2} (i.e., the intrinsic and extrinsic parameters and spatial positions of the two cameras), as well as the pixel coordinates (u_1, v_1) and (u_2, v_2) of the point on Image_001 and Image_002. Using triangulation formulas, the three-dimensional coordinates (X, Y, Z) of the pit point in the robot's base coordinate system can be calculated. The system performs the above calculations on tens of thousands of matching points in all image combinations, and uses global optimization techniques such as bundle adjustment to minimize reprojection errors, and jointly optimizes the three-dimensional coordinates of all points and camera poses. The system outputs a point cloud, which is a stereo image of the laser weld. In the point cloud, the undulations of the weld and the pit (whose Z coordinate value is lower than the surrounding area) are clearly visible, and the depth of a certain pit is measured to be 0.15 mm and the diameter is 0.8 mm.

[0026] refer to Figure 3 In step S12, the specific steps are as follows: S121: Detect the three-dimensional image of the weld and identify multiple pits during the detection process. Trigger image recognition of the three-dimensional image of the weld based on the spatial position and corresponding extension direction of each pit to identify multiple pit areas. S122: In multiple pit regions, the corresponding pit space is determined based on the detection of each pit region, and the corresponding surface morphology is determined based on the surface recognition of each pit region, so as to collect the pit space and the corresponding surface morphology of each pit region. S123: Trigger a comparison of two adjacent pit areas along the weld path of the automotive workpiece, determine the amount of regional change during the comparison process, determine the content of the first node based on the amount of regional change and the pit space of the pit area, determine the content of the second node based on the amount of regional change and the surface morphology of the pit area, and determine multiple weld control nodes based on the content of the first node and the content of the second node.

[0027] In the embodiments of this application, a robust estimation algorithm, such as RANSAC (Random Sample Consensus), is used to fit an optimal reference plane from the entire weld point cloud. This plane represents an ideal, defect-free weld surface. During the fitting process, points that deviate far from the plane (i.e., potential defect points) are considered outliers and do not participate in the plane calculation, thus ensuring the accuracy of the reference plane. The directed distance (i.e., depth residual) from each point in the point cloud to the reference plane is calculated, and a depth threshold is set. All points with depth residuals less than this threshold are filtered out to form a candidate set of depression points. Euclidean clustering algorithm is applied to this candidate set of points to group points with spatial distances less than a certain clustering tolerance into the same cluster. Each independent cluster is a preliminarily identified depression portion.

[0028] For each point cloud cluster of a pit, its centroid is calculated, which is the average of the coordinates of all points within the cluster, representing the center position of the pit in three-dimensional space. Principal component analysis (PCA) is used to process the point cloud cluster. PCA calculates three main direction vectors of the point cloud distribution, where the first principal component (the vector corresponding to the largest eigenvalue) usually represents the main extension direction of the pit on the weld. Based on the centroid and extension direction, the system defines a standardized local three-dimensional bounding region. The orientation of this bounding region is aligned according to the extension direction, and its size is sufficient to completely cover the entire pit and a small transition area around it. The process of defining this bounding region is the signal that triggers a local image recognition, and the system will shrink the subsequent analysis scope from the entire weld point cloud to the interior of this bounding region.

[0029] The system performs fine segmentation and boundary extraction on the point cloud within each local enclosed region to obtain the corresponding pit region. Within the enclosed region established by the trigger, the system runs a more refined segmentation algorithm, such as a region growing algorithm. This algorithm starts from a seed point and gradually merges adjacent points according to the normal vector similarity and distance proximity criteria until the boundary is reached. This can more accurately separate the pit itself from the surrounding healthy weld surface. After the segmentation is completed, the system extracts the boundary point set of the pit point cloud. These boundary points accurately delineate the contour of the pit in three-dimensional space. This finely segmented point cloud set with clear boundaries is formally defined as a pit region. It not only contains the points inside the pit but also implies its precise geometric boundary and spatial pose information.

[0030] Specifically, in the 3D point cloud of the laser weld, the system runs the RANSAC algorithm and successfully fits a mathematically ideal plane by ignoring all spatter and depression points on the weld. The system calculates the distance of each point in the point cloud to this ideal plane and sets a depth threshold of -0.08 mm. Approximately 10,000 points have a distance value less than this threshold, and these points are marked as candidate depression points. The system then performs Euclidean clustering on these candidate points, which are automatically divided into 42 independent clusters. The system initially identifies 42 depression parts on the weld, each of which is an independent point cloud cluster.

[0031] For the fifth pit, the system calculates the average coordinates of all points within the point cloud cluster, obtaining its centroid position as (1015.3mm, 502.1mm, 799.8mm). PCA analysis of this point cloud cluster reveals that its first principal component direction vector is (0.98, 0.12, 0.15), which is almost parallel to the X-axis (weld length direction) of the robot's base coordinate system, indicating that the pit mainly extends along the weld direction. The system aligns a cuboid region centered on the centroid and according to the first principal component direction, setting its dimensions to 12mm x 8mm x 5mm. This region triggers the identification of the fifth pit, and subsequent system analysis will focus on the point cloud data falling within this region.

[0032] Starting from a point with the greatest depth, the algorithm continuously merges points with similar normal vectors and a distance of less than 0.5 mm, eventually accurately separating a point cloud cluster containing approximately 850 points. The system identifies those points among these 850 points that have at least one neighboring point that does not belong to the cluster, forming a boundary ring of approximately 120 points. This finely segmented independent point cloud, containing 850 points and 120 boundary points, is ultimately identified by the system as the fifth pit region. Thus, all 42 pits on the weld have undergone similar processing and have been transformed into 42 precisely defined pit regions.

[0033] Furthermore, in multiple pit regions, the corresponding pit space is determined based on the detection of each pit region, and the corresponding surface morphology is determined based on the surface recognition of each pit region. This process collects the pit space and corresponding surface morphology of each pit region, taking into account the overall consideration of surface recognition of each pit region and ensuring the accuracy of the corresponding surface morphology.

[0034] At this point, for each pit region, the system uses its boundary points or surrounding non-pitted points to refit a local reference plane. This plane fits the local weld curvature of the region better than the global reference plane, thereby improving the accuracy of the calculation. The pit volume is calculated using methods such as voxelization integration or mesh projection integration. Taking voxelization as an example, the system divides the space surrounding the pit region into tiny three-dimensional cubes (voxels). By determining whether each voxel is occupied by the pit point cloud and accumulating its depth, the approximate volume of the pit can be obtained. In addition, the system also calculates the directed distance from all points in the pit region to the local reference plane, extracting key indicators such as maximum depth, average depth, and depth standard deviation. At the same time, the projected area and perimeter are calculated by projecting the point cloud onto the local reference plane.

[0035] For each point within the pit region, a micro-tangent plane is fitted using its neighboring points to calculate the surface normal vector and curvature indices such as Gaussian curvature and mean curvature. Based on this data, surface roughness can be quantified. For example, by calculating the angular standard deviation between the normal vectors of all points within the region, a larger angular standard deviation indicates a more uneven and rougher surface. If the 3D scanning device simultaneously records the reflection intensity of each point when acquiring the point cloud, this intensity information becomes key data for analyzing surface morphology. The system can analyze the distribution, contrast, and texture features of intensity values ​​within the pit region. For example, a smooth metal surface has high and uniform reflectivity, while an oxidized rough surface has low and uneven reflectivity. Based on these quantitative features, a classifier can automatically categorize surface morphology, such as smooth, wavy, rough, or oxidized.

[0036] Create a unique ID for each pit area and store all extracted feature values ​​in an associated data structure (such as a JSON object or database record). Sort and index these structured data objects according to their spatial location on the weld path (such as the X coordinate of the centroid) to facilitate rapid retrieval and comparison of adjacent areas in subsequent steps S123. After all pit areas have been processed in the above way, the system has a complete pit feature library. Each record in this library comprehensively and quantitatively describes an independent pit defect.

[0037] Specifically, for the identified pit No. 5 on the weld, the system fitted a local reference plane using approximately 500 healthy weld surface points surrounding the pit. The system divided the area surrounding the pit into tiny voxels of 0.05mm x 0.05mm x 0.05mm. Statistically, approximately 6400 voxels were identified as pit voxels. By integrating their depths, the volume of the pit was finally calculated to be 0.8mm³. In terms of depth analysis, the system calculated the maximum depth of the pit to be 0.28mm, the average depth to be 0.13mm, and the standard deviation of the depth to be 0.04mm. The two-dimensional contour area obtained after projection is 6.2mm². These values ​​together constitute the pit spatial descriptor of pit No. 5: {Volume: 0.8mm³, Maximum Depth: 0.28mm, Average Depth: 0.13mm, Projected Area: 6.2mm²}.

[0038] The system calculated the normal vectors and principal curvatures of 850 points within the region. The results showed a very wide range of normal vector direction variations, from perpendicular to the weld surface to nearly horizontal. The calculated standard deviation of the angle between all point normal vectors was as high as 35 degrees, indicating an extremely irregular surface, far exceeding that of a normal weld surface. In terms of reflectivity analysis, the system read the point cloud intensity values ​​of the region, finding a wide distribution of intensity values ​​and a large number of low-intensity spots, consistent with typical characteristics of metal oxidation or impurity contamination. Combining the high normal vector variation rate and low-intensity spots, the system automatically labeled the surface morphology of this pit as rough and irregular with oxide spots. A data record was created for the fifth pit region, and this record was stored in the pit feature library. Similar structured data will be generated for all 42 pit regions on the weld.

[0039] Therefore, the comparison of two adjacent pit areas is triggered along the weld path of the automotive workpiece, and the amount of regional change is determined during the comparison process. The content of the first-level node is determined based on the amount of regional change and the pit space of the pit area, and the content of the second-level node is determined based on the amount of regional change and the surface morphology of the pit area. Multiple weld control nodes are determined based on the content of the first-level node and the second-level node, which takes into account the overall consideration of the content of the first-level node and the second-level node, ensuring the accuracy of multiple weld control nodes. At the same time, a three-dimensional image of the weld is introduced, and image recognition is performed on the three-dimensional image of the weld. This takes into account the pit space of each pit area, the corresponding surface morphology, and the amount of regional change between two adjacent pit areas, which improves the accuracy of multiple weld control nodes.

[0040] At this point, the system sorts the pit feature library established in S122 according to the coordinates of the centroid of each pit region on the weld path (e.g., the X-axis). The system creates a sliding window of size 2 and iterates through the sorted list to form a series of adjacent pit region pairs, such as (Pit_i, Pit_{i+1}). For each pair, the system calculates the regional change between them, which is a multi-dimensional difference vector, mainly including: spatial change (e.g., centroid distance), geometric change (e.g., volume, depth difference or relative change rate), and morphological change (e.g., roughness index difference or morphological label transition). The system combines all the above changes into a high-dimensional regional change vector Δ_i, which fully quantifies the changes in the weld state from the i-th pit to the (i+1)-th pit.

[0041] The first level of node content focuses on the geometric dimension. The system compares the geometric components (such as volume change rate and depth change rate) in the region change vector with preset process thresholds. For example, if the volume change rate threshold is set to 50%, and Δ_volume_rate > 50%, a geometric state mutation event is triggered, which is usually directly related to the instability of welding energy input. The second level of node content focuses on the physical dimension. The system compares the morphological components (such as roughness change and morphological label change) with another set of preset thresholds. For example, if the roughness change threshold is set to 15.0, and Δ_roughness > 15.0, or a morphological label transition occurs, a physical property mutation event is triggered, which is usually related to changes in the protective gas effect or material cleanliness.

[0042] The system has a built-in decision engine whose rules can be OR logic (creating a node when any event is triggered), AND logic (creating a node only when two events are triggered simultaneously), or weighted logic (creating a node when the total weight exceeds a threshold). Once the decision conditions are met, the system generates a weld control node at the midpoint between the two adjacent pit areas (along the weld path). This weld control node is a composite information entity, presenting the node location, a complete description of the triggering event, and evidence linking the two original pit areas that caused the node. All generated control nodes are stored in positional order, forming a weld control node chain. This chain is the core input for subsequent quality event analysis and the construction of the detection system.

[0043] Specifically, the system sequentially paired pit regions 5 and 6 on the weld; extracted their features: Pit_005 has a volume of 0.8 mm³, a roughness of 35.0, and a rough, irregular shape; Pit_006 has a volume of 0.2 mm³, a roughness of 8.5, and a smooth shape; the system calculated the changes: the distance between the centroids of the two along the weld direction is 18.0 mm; the relative change rate of volume is |0.8-0.2| / 0.8=75%; the change in roughness is |35.0-8.5|=26.5, and the shape label has changed; the system generates a change vector Δ_5=[18.0 mm, 75%, 26.5, ...] for this pair of regions.

[0044] The volume change rate in Δ_5 is 75%, exceeding the preset threshold of 50%; the system generates the first level of node content: {Event Type: Geometric State Abrupt Change, Specific Indicator: Volume Change Rate, Value: 75%}; In the second level of content judgment, the roughness change in Δ_5 is 26.5, exceeding the preset threshold of 15.0, and the morphology label also undergoes a transition; therefore, the system generates the second level of node content: {Event Type: Physical Property Abrupt Change, Specific Indicator: Roughness and Morphology, Value: From Rough and Irregular to Smooth}.

[0045] The system uses OR logic for decision-making. Since both the first-level node content (geometric state mutation) and the second-level node content (physical attribute mutation) are triggered, the creation conditions are met. The system creates a weld control node N_005 at the midpoint between the centroids of pits 5 and 6, with coordinates (1024.3mm, 502.5mm, 799.9mm). The data structure of this node encapsulates its location, triggering events (75% geometric mutation and physical mutation), and related evidence (Pit_005 and Pit_006). This node N_005 is added to the control node list, becoming a key control point for monitoring weld quality. The system will continue to perform the same analysis process on subsequent adjacent pit pairs (such as Pit_006 and Pit_007) until all pits have been traversed.

[0046] refer to Figure 4 In step S13, the specific steps are as follows: S131: Monitor each weld seam control node in real time, determine multiple surface features corresponding to each weld seam control node, determine the corresponding weld seam quality event based on the node position of each weld seam control node, the multiple surface features corresponding to each weld seam and the pit space, and mark the priority of the weld seam quality event. S132: Based on the identification of each weld quality event, determine the corresponding weld quality content, and determine the first weld inspection content according to the weld quality content of each weld quality event, the corresponding priority and the surface morphology of the corresponding pit area. S133: The laser data of the laser welding head is matched based on the movement position of each weld control node and the laser welding head to determine the laser data combination of the laser welding head. The second weld inspection content is determined based on the laser data combination of the laser welding head and the weld quality content of each weld quality event. The corresponding laser weld inspection system is determined based on the first and second weld inspection content.

[0047] In the embodiments of this application, each weld seam control node is monitored in real time, and multiple surface features are identified during the monitoring process. Based on the node position of each weld seam control node, the multiple surface features, and the pit space, the corresponding weld seam quality event is determined, and the priority of the weld seam quality event is marked. This approach takes into account the overall consideration of the node position of each weld seam control node, the multiple surface features, and the pit space, ensuring the accuracy of the corresponding weld seam quality event.

[0048] At this point, the system determines whether the laser welding head is about to arrive at or is in the monitoring area of ​​a certain weld seam control node by acquiring the TCP (tool center point) position of the robot controller in real time. The monitoring area is usually a spatial window with a certain length along the weld seam path centered on the node position. Once it enters the monitoring window, the system will simultaneously activate one or more high-bandwidth sensors and execute specific image processing algorithms to extract surface features. These dynamic features include the geometric features of the molten pool (length, width, area) calculated by a high-speed camera, the dynamic features of the molten pool (oscillation frequency, amplitude) extracted by optical flow or frequency domain analysis, and the plasma spectral features monitored by a spectrometer. They together reflect the instantaneous physical state of the welding process.

[0049] The system spatiotemporally aligns the real-time extracted dynamic surface feature time series with the static pit spatial data (such as volume and depth) pre-stored at the node location in S12. The node location provides a spatial anchor point to ensure that the two correspond to the same physical region. The system has a built-in rule-based or machine learning-based classifier that receives multi-source input and makes judgments based on a preset expert knowledge base. For example, the knowledge base contains the rule: IF (static pit volume > 0.5 mm³) AND (dynamic molten pool width standard deviation > 0.2 mm) THEN (event type = 'porosity aggregation caused by molten pool instability'). Once a rule is triggered, the system instantiates a weld quality event, which is a structured data object that encapsulates the event type, trigger location, and all associated dynamic and static evidence.

[0050] The system employs a multi-dimensional weighted scoring model to calculate the priority score of events. The scoring dimensions include: geometric severity based on static pit spatial data (e.g., the greater the depth and volume of the pit, the higher the score); dynamic severity based on dynamic surface feature data (e.g., the greater the fluctuation amplitude of the molten pool, the higher the score); and process criticality based on node location (if the node is located in a stress concentration zone, the score is even higher). The system maps the calculated total score to a discrete priority level (e.g., urgent, high, medium, low) and attaches this priority label to the data structure of the weld quality event.

[0051] Specifically, when the laser welding head approaches the control node N_005 at a speed of 50 mm / s, the system detects that the welding head TCP has entered the monitoring window [-10 mm, +10 mm] centered on N_005 and immediately starts synchronous monitoring; the high-speed camera starts shooting at a frame rate of 2000 fps and calculates the molten pool width in real time; before entering the node area, the molten pool width is stable at 1.2 mm ± 0.05 mm, and its main oscillation frequency is also stable at 150 Hz; however, when the welding head reaches the N_005 position, the system captures that the molten pool width fluctuates drastically from 1.2 mm to 1.8 mm within 0.05 seconds, and at the same time, FFT analysis shows that the main oscillation frequency splits into two unstable frequencies of 120 Hz and 280 Hz. These are the key surface features reflecting the abnormal process.

[0052] At control node N_005, the system correlates the real-time captured features of drastic changes in molten pool width and splitting oscillation frequency with the static data associated with N_005 (drastic volume changes in Pit_005 and Pit_006). The system's classifier receives the input: {static features: drastic volume change, dynamic features: drastic changes in molten pool width and splitting oscillation frequency}. According to the built-in rules, this strongly points to an unstable process. Therefore, the system creates a weld quality event with the ID QE_N_005, the content of which is: {event type: porosity aggregation caused by molten pool instability, trigger location: N_005, key dynamic evidence: standard deviation of molten pool width 0.3mm, frequency splitting, key static evidence: upstream and downstream pit volume change rate 75%}.

[0053] For the created event QE_N_005, the system begins priority evaluation. In terms of geometric severity, Pit_005's volume is 0.8 mm³, exceeding the severity threshold, resulting in a score of 8.5 / 10. In terms of dynamic severity, the standard deviation of the melt pool width is 0.3 mm, exceeding the severity threshold, resulting in a score of 9.0 / 10. In terms of process criticality, N_005 is located on a straight line segment, a non-critical position, resulting in a score of 5.0 / 10. Assuming weights of 0.4, 0.4, and 0.2 respectively, the total score is 8.5 x 0.4 + 9.0 x 0.4 + 5.0 x 0.2 = 8.2. The total score of 8.2 falls within the emergency range, therefore the system marks event QE_N_005 as having an emergency priority. This priority-marked event will be sent to subsequent steps, and an audible and visual alarm will be triggered simultaneously.

[0054] Furthermore, based on the identification of each weld quality event, the corresponding weld quality content is determined. The first layer of weld inspection content is determined according to the weld quality content, corresponding priority, and surface morphology of the corresponding pit area of ​​each weld quality event. This takes into account the overall consideration of the weld quality content, corresponding priority, and surface morphology of the corresponding pit area of ​​each weld quality event, ensuring the accuracy of the first layer of weld inspection content.

[0055] At this point, the system parses the data structure of the weld quality event and queries an internal event-defect mapping knowledge base to associate the event type (such as porosity aggregation caused by molten pool instability) with specific, industry-standard defect names (such as Porosity in the ISO 6520 standard). After determining the defect type, the system integrates all the data associated with the event to generate a structured description, including a qualitative description of the defect, a quantitative description referencing static geometric data and dynamic feature data, and a clear spatiotemporal location. The generated weld quality content is stored in a standardized data structure, facilitating subsequent report generation and data analysis.

[0056] The system integrates quality description, risk assessment, and surface evidence to form a visual and geometric-centric inspection report, which constitutes the first layer of weld inspection content. The system correlates information from three different dimensions: the weld quality content provides an objective description of what it is and how much it is; the priority provides the risk assessment of its importance; and the surface morphology of the pit area provides visual evidence of how it looks, serving as an important supplement and corroboration to the quality content. For example, in a porosity cluster event, a smooth surface morphology points to a different underlying cause than a rough morphology. The system encapsulates this integrated information into a data object, which is a conclusion purely from a visual inspection perspective (including static 3D vision and dynamic process vision), and does not contain any information about laser process parameters.

[0057] Specifically, for the emergency event QE_N_005 identified in S131, the system queries the knowledge base and maps the porosity cluster caused by molten pool instability to the standard defect type PorosityCluster. The system generates content: qualitatively, it is described that at node N_005, a dense porosity cluster was generated due to molten pool instability; quantitatively, it integrates data to show that the average volume of the associated static pits is 0.5 mm³; the standard deviation of the molten pool width monitored in real time reaches 0.3 mm, and the main oscillation frequency splits; the spatiotemporal location is that the event occurred at the weld path coordinates (1024.3 mm, 502.5 mm, 799.9 mm); the system generates a structured weld quality content object, which fully records the standardized information of the event.

[0058] The system continues to process event QE_N_005, performing data fusion. Its quality content is porosity clusters and molten pool instability, with an urgent priority. The surface morphology of the related pits shows that upstream Pit_005 is rough and irregular with oxide spots, while downstream Pit_006 is smooth. The system combines these three to form a richer context: the drastic change in surface morphology provides strong visual evidence for the conclusion of molten pool instability, indicating that a significant state transition occurred in the welding process near this node. The final generated first-level weld inspection content is a complete, self-contained visual inspection report entry, which will serve as input, awaiting deep correlation with the laser process data.

[0059] Therefore, the laser data of the laser welding head is matched based on the movement position of each weld control node and the laser welding head to determine the laser data combination of the laser welding head. The second layer of weld inspection content is determined based on the laser data combination of the laser welding head and the weld quality content of each weld quality event. The corresponding laser weld inspection system is determined based on the first and second layer of weld inspection content, thus incorporating the overall considerations of both layers and ensuring the accuracy of the corresponding laser weld inspection system. Simultaneously, it further controls weld quality events, achieving an overall consideration of weld quality events at each weld control node, the corresponding node position, and the laser data combination of the laser welding head, thereby improving the accuracy of the laser weld inspection system.

[0060] At this time, the system subscribes to the TCP (Tool Center Point) position data stream of the laser welding head in real time through a high-frequency communication interface. When the real-time position of the TCP matches the preset position of a certain weld seam control node, the system sends a high-precision timestamp to the laser and robot controller. The system extracts a data segment of fixed duration from the data buffers of all relevant devices, centered on this timestamp. All data segments extracted from different devices within the same time window are encapsulated into a unified data packet, namely the laser data combination, which accurately reproduces a snapshot of the complete process state at the moment the quality event occurs.

[0061] The system identifies the direct causes of quality incidents from complex process data, forming the second layer of weld inspection content. It then deeply correlates the first layer of weld inspection content (the manifestation of the quality incident) with laser data combinations (process data), typically achieved through pattern matching, correlation analysis, or rule-based expert systems. The system searches for anomaly patterns in the data combinations that are temporally consistent with the quality incident description and logically causally related. Once a potential anomaly pattern is found, the system makes inferences based on a pre-stored process knowledge base. For example, a sudden negative jump in laser power can lead to insufficient energy input to the molten pool, causing molten pool oscillation and instability, resulting in porosity. The system then structurally encapsulates the inferred process causes to form the second layer of weld inspection content.

[0062] The system pairs the first-level weld inspection content (quality manifestation) and the second-level weld inspection content (process root cause) for the same control node to form a complete knowledge entry. This entry is a strongly correlated instance of manifestation and cause. All generated knowledge entries are stored in a structured database or knowledge graph, forming a laser weld inspection system. This system not only stores data but also establishes a multi-dimensional index and has self-learning capabilities. As welding progresses, more and more knowledge entries are added. The system can analyze the entire system through machine learning algorithms to discover deeper process-quality relationships. These newly discovered patterns can be used to optimize inspection rules and scoring models, forming a closed loop of continuous improvement.

[0063] Specifically, when the laser welding head reaches the precise position of the control node N_005, the robot controller sends a trigger signal to the central system, along with a high-precision timestamp t_event=15.285s. The system immediately extracts the laser power data for the time period [15.085s, 15.585s] from the laser's data buffer, and simultaneously extracts the welding speed data for the same time period from the robot controller's buffer. By analyzing this data combination named Laser_Data_Set_N_005, the system discovers that approximately 50ms before the timestamp 15.285s, i.e., 15.235s, the laser power instantly drops from the set 3000W to 2850W and lasts for 80ms. This power interruption is the most critical anomaly in this data combination.

[0064] The system analyzes the quality content of Laser_Data_Set_N_005 and QE_N_005. The quality content is described as a porosity cluster caused by molten pool instability. In the laser data combination, the system detects an abnormal pattern of power interruption before the event occurs. The two are highly correlated in time and conform to physical logic. Based on the process knowledge base, the system infers that the power interruption is the direct process cause of molten pool instability and porosity cluster. The system generates a second weld inspection content, which clearly indicates that the quality problem was caused by a 150W power drop lasting 80ms.

[0065] The system integrates the first and second layers of N_005 to create a complete knowledge entry that firmly links the visual appearance of porosity clusters with the process cause of power interruption. This entry is added to the laser weld inspection system. Now, when operators or subsequent intelligent programs query what caused the porosity clusters, the system can immediately return this entry as evidence. When dozens of similar power interruption-porosity cluster entries accumulate in the inspection system, the system can confirm that this is a strongly correlated causal chain. If the system detects a small power fluctuation again, even if it has not yet reached the threshold for triggering a quality event, the system marks it as high risk and issues an early warning, thus achieving an intelligent upgrade from post-detection to pre-prediction.

[0066] refer to Figure 5 In step S14, the specific steps are as follows: S141: In this laser weld inspection system, the laser weld inspection system is dynamically identified, and the node critical coefficients of multiple weld control nodes are determined during the identification process. Multiple key weld areas are determined based on the node critical coefficients of multiple weld control nodes, the corresponding pit areas, and the three-dimensional image of the weld. S142: Based on the detection of each critical weld area, multiple sub-anomaly features are determined, and the abnormal events of the critical weld area are determined according to the feature positions, corresponding feature shapes, and regional shapes of the multiple sub-anomaly features. S143: Implement targeted control for each critical weld area and mark the abnormal morphology of each critical weld area. Based on the identification of each critical weld area, determine the corresponding pit surface. Determine the corresponding multidimensional quality coefficient according to the abnormal events, corresponding abnormal morphology and corresponding pit surface of each critical weld area. Determine the quality level of the critical weld area according to the matching of the mapping relationship between the multidimensional quality coefficient and the quality level.

[0067] In the embodiments of this application, the laser weld detection system is dynamically identified, and the node critical coefficients of multiple weld control nodes are determined during the identification process. Multiple key weld areas are determined based on the node critical coefficients of multiple weld control nodes, the corresponding pit areas, and the weld three-dimensional image. This approach takes into account the overall consideration of the node critical coefficients of multiple weld control nodes, the corresponding pit areas, and the weld three-dimensional image, ensuring the accuracy of multiple key weld areas.

[0068] At this point, the system performs periodic or event-driven scanning and analysis on the laser weld inspection system. This includes statistically analyzing the frequency of weld quality events associated with each control node in historical production data, calculating the average severity score of associated events for each node, and using association rule mining algorithms to analyze the failure modes of nodes. Based on the above analysis results, the system calculates a comprehensive node criticality coefficient for each control node. This coefficient is a weighted score, and its calculation model can be designed as: NCCI=w1XF_norm+w2XS_norm+w3XC_sens, where F_norm is the normalized value of event frequency, S_norm is the normalized value of average severity score, C_sens is process sensitivity, and w1, w2, and w3 are weights set according to the quality control strategy.

[0069] The system sets a critical coefficient threshold. All control nodes whose calculated critical coefficient exceeds this threshold are officially marked as critical nodes. On the weld path, the system uses a one-dimensional clustering algorithm to cluster these discrete critical nodes. The nodes in each cluster and the weld segments between them together constitute a critical weld area. To ensure the accuracy of the area, the system highlights or renders the defined critical weld area on the original weld 3D image (i.e., the complete 3D point cloud model) for the operator to visually confirm through the 3D visualization interface.

[0070] Specifically, the system dynamically identifies the laser weld seam inspection system. In the production data of the most recent 1000 products, the system found that control node N_005 was associated with 80 quality events, including 50 emergency events and 30 high-risk events. Control node N_015, on the other hand, was associated with only 40 quality events, indicating lower severity. During the coefficient calculation phase, for N_005, its event frequency normalized value F_norm was 0.08, and its average severity score normalized S_norm was approximately 0.91. Assuming its process sensitivity C_sens was 0.8, with all weights set to 1 / 3, its node criticality coefficient NCCI_N005 was calculated to be approximately 0.60. In contrast, all indicators for N_015 were lower, with a calculated NCCI_N015 of approximately 0.38.

[0071] The critical coefficient threshold is set to 0.5. Based on the calculation in the previous step, the coefficient of N_005 (0.60) exceeds the threshold, so it is marked as a critical node, while N_015 is not marked. In the area definition stage, since there is only one critical node N_005, the system takes it as the core and extends it upstream and downstream by a fixed safety distance (for example, to the adjacent non-critical nodes N_004 and N_006). The entire weld segment from N_004 to N_006 is defined as a critical weld area, named KSA_01. In the 3D visualization confirmation stage, the system renders the point cloud area corresponding to KSA_01 (approximately from 1010mm to 1030mm of the weld path) in red on the complete 3D point cloud model of the weld. The quality control engineer sees this red weld on the screen and confirms that it is indeed a region with a high incidence of defects such as porosity and lack of fusion in the past, thus confirming the validity of the critical weld area.

[0072] Furthermore, multiple sub-anomaly features are determined based on the detection of each critical weld area. The abnormal events of the critical weld area are determined according to the feature positions, corresponding feature shapes, and regional morphology of the critical weld area based on the feature positions, corresponding feature shapes, and regional morphology of the critical weld area. This approach takes into account the overall consideration of the feature positions, corresponding feature shapes, and regional morphology of the critical weld area, ensuring the accuracy of abnormal events in the critical weld area.

[0073] At this point, the system identifies a key weld area and retrieves the corresponding raw data from the database, including ultra-high density 3D point clouds, high frame rate process images, and multispectral data. The system employs a series of specialized algorithms to extract different types of sub-anomaly features: applying curvature-based segmentation algorithms to identify minute incomplete fusion and microcracks; quantifying local changes in surface roughness by calculating the normal vector change of the point cloud or applying the gray-level co-occurrence matrix to the 2D intensity map; and inferring the presence of subsurface porosity by analyzing the temperature gradient of thermal imaging data. Each detected sub-anomaly feature is instantiated as a structured data object containing its type, quantification index, and preliminary location.

[0074] The system places all extracted sub-anomaly features in the same space and engineering context for comprehensive analysis. This includes analyzing the spatial distribution relationship of each feature within the region (feature location), analyzing the morphological attributes of the features (feature morphology), and integrating the macroscopic regional morphology of the key weld area (such as curvature, plate thickness, and joint type). The system has a built-in inference engine based on expert knowledge and machine learning. This engine receives feature location + feature morphology + regional morphology as input and infers a high-level anomaly event based on a preset rule base or a trained model.

[0075] Specifically, the system performs refined detection on the critical weld area KSA_01. The system retrieves the original point cloud corresponding to KSA_01, with a point spacing of 0.02mm, as well as 4000fps high-speed video captured during welding. In the feature extraction stage, through curvature analysis, the system identifies a continuous micro-undercut with a length of 2.5mm and a depth of approximately 0.06mm at the junction of the weld and the base material (weld toe). GLCM analysis of the point cloud intensity map reveals that the contrast and entropy values ​​of this area are significantly higher than those of the normal area, indicating that the surface texture is extremely irregular and contains a large number of disordered oxide spots. In the point cloud depth data, through voxelization filtering, three subsurface micropores with a diameter of less than 0.1mm are identified. These are the three sub-anomaly features determined by the system.

[0076] The system performs a fusion analysis of the sub-anomaly features within KSA_01. In the context analysis, the system discovers the most significant micro-undercut feature, which is located precisely at a known structural corner of the workpiece, a classic stress concentration area. Simultaneously, the undercut is identified as sharp, with an extremely small radius of curvature at its root. Combined with the macroscopic morphology of the area where KSA_01 is located being identified as a high-stress concentration zone, the inference engine matches a strong rule: IF (location = stress concentration zone) AND (feature = sharp micro-undercut) THEN (anomaly event = 'fatigue crack initiation source'). The system uses irregular surfaces and micropores as supplementary evidence to further strengthen this judgment. The system identifies an anomaly event for KSA_01: the presence of a sharp micro-notch in a high-stress concentration area constitutes a potential fatigue crack initiation source with an extremely high risk of propagation.

[0077] Therefore, targeted control is implemented for each critical weld area, and abnormal morphologies of each critical weld area are marked. Based on the identification of each critical weld area, the corresponding pit surface is determined. According to the abnormal events, corresponding abnormal morphologies, and corresponding pit surfaces of each critical weld area, the corresponding multidimensional quality coefficient is determined. The quality level of the critical weld area is determined based on the matching of the mapping relationship between the multidimensional quality coefficient and the quality level. This approach takes into account the overall consideration of matching the mapping relationship between the multidimensional quality coefficient and the quality level, ensuring the accuracy of the quality level of the critical weld area.

[0078] At this point, the system establishes a dynamic control file for each critical weld area and automatically triggers a series of control actions based on the abnormal events identified in S142. For example, it strengthens monitoring in subsequent production, locks in more robust process parameters, or marks the area as requiring special inspection in the production management system. The system will classify and mark all abnormalities in the area according to their abnormality patterns. Common pattern markings include: periodic (related to equipment vibration or power ripple), sudden (related to material batch or shielding gas interruption), scalable (continuous process deterioration), and diffuse (overall process instability).

[0079] The system integrates all surface-related quantitative data from S122 and S142 to form a generalized comprehensive description of the pit surface. The system uses a weighted model to calculate the coefficient, which comprehensively evaluates the quality from multiple dimensions: risk dimension (based on the severity of abnormal events), behavioral dimension (based on the stability of abnormal morphology), physical dimension (based on the quantitative data of the pit surface, such as roughness), and geometric dimension (based on the geometric dimensions of sub-abnormal features). The final multidimensional quality coefficient is the weighted sum of the scores of each dimension, which comprehensively reflects the overall quality status of the area.

[0080] The system internally maintains a standardized multi-dimensional quality coefficient - quality grade mapping matrix, which is a digital representation of the enterprise's quality standards. The system uses the multi-dimensional quality coefficient calculated in the previous step as input, queries this mapping matrix for an exact match, and thus determines the final quality grade of this critical weld area. The determined quality grade triggers corresponding decision-making processes and forms an information closed-loop. For example, a non-conforming grade will automatically trigger the scrapping or rework process, archive all analysis data into the product's lifetime quality file, and simultaneously feedback to the detection system of S13.

[0081] Specifically, for the critical weld area KSA_01, in its control file, the system marks the fatigue crack initiation source event determined by S142 as the highest risk. The system sends an instruction to the robot controller: in subsequent welding, when entering the KSA_01 area, the welding speed is automatically reduced by 10%, and the laser power is increased by 5%. At the same time, in the MES (Manufacturing Execution System), this product is automatically marked as requiring penetrant inspection and re-inspection for the KSA_01 area. In terms of abnormal morphology marking, the system reviews historical data and finds that the micro-edge biting feature has occurred in the last 5 productions and its position is fixed, so it is marked as periodic; while the oxidation spots have different distributions and concentrations each time, and are marked as diffuse.

[0082] The system calculates the multi-dimensional quality coefficient for KSA_01. The system integrates data and determines that the comprehensive surface attributes of the pit for KSA_01 are: {Ra: 3.2μm, oxidation ratio: 15%}. In the calculation of the multi-dimensional quality coefficient, for the risk dimension, due to the fatigue crack initiation source event, the score S_risk is 0.2 (full score 1.0); for the behavior dimension, due to periodic edge biting, the score S_behavior is 0.7; for the physical dimension, since the surface roughness Ra = 3.2μm exceeds the standard, the score S_physical is 0.5; for the geometric dimension, due to the edge biting depth of 0.06mm approaching the scrapping limit, the score S_geometric is 0.6. Assuming that the weights are all 0.25, the final multi-dimensional quality coefficient MQC = (0.2 + 0.7 + 0.5 + 0.6) × 0.25 = 0.5.

[0083] The system determines the final quality grade for KSA_01. The mapping matrix for the weld is defined as: MQC > 0.85 is excellent, 0.70 < MQC ≤ 0.85 is good, 0.55 < MQC ≤ 0.70 is qualified, and MQC ≤ 0.55 is unqualified. The MQC of KSA_01 is 0.5, and this value is less than or equal to 0.55. Therefore, after system matching, the final quality grade of KSA_01 is determined to be unqualified. The system automatically generates a comprehensive report containing all analysis evidence and sends an instruction to the production execution system to transfer this product to the non-conforming product handling area. This decision result and all analysis data are fed back into the laser weld detection system.

[0084] refer to Figure 6 In step S15, the specific steps are as follows: S151: Collect the welding process of the laser welding head, determine the welding data of the laser welding head based on the matching of the welding process of the laser welding head and the corresponding welding database, and determine the corresponding dynamic working data based on the welding database of the laser welding head. The welding data and dynamic working data of the laser welding head are two data groups of different dimensions. S152: Collect the quality level of each key weld area, determine the first baryon welding state curve based on the quality level of each key weld area and the welding data of the laser welding head, determine the second baryon welding state curve based on the quality level of each key weld area and the dynamic working data of the laser welding head, and determine the welding state curve of the laser welding head based on the synthesis of the first baryon welding state curve and the second baryon welding state curve. S153: Based on the identification of the welding state curve, multiple welding state nodes are determined and the corresponding welding states are marked. Based on the node position of each welding state node, the corresponding welding state and the corresponding welding event, the optimization content of each welding state node is determined and the corresponding welding optimization content is marked.

[0085] In the embodiments of this application, the welding process of the laser welding head is collected, and the welding data of the laser welding head is determined by matching the welding process of the laser welding head with the corresponding welding database. At the same time, the corresponding dynamic working data is determined based on the welding database of the laser welding head. The welding data and dynamic working data of the laser welding head are two data groups of different dimensions, which is compatible with the overall consideration of the welding database of the laser welding head and ensures the accuracy of the corresponding dynamic working data.

[0086] At this point, the system subscribes to TCP (tool center point) pose data from the robot controller at an extremely high frequency (usually 1kHz or higher) via the industrial real-time Ethernet protocol. This includes three-dimensional spatial coordinates and attitude information. All acquired data points are stamped with a high-precision timestamp that is strictly synchronized with the system's master clock. These data form a continuous time-series data stream, i.e., the welding process. It is essentially a multi-dimensional curve that fully describes the complete motion trajectory and attitude changes of the welding head from the start of the arc to the end of the arc, providing a spatial and temporal reference system for all subsequent analyses.

[0087] The system spatially matches the TCP coordinate sequence in the welding process with the process program stored in the welding database. When a TCP point in the welding process is identified as belonging to a specific program segment, the system extracts the preset process parameter settings of that program segment from the database, such as target laser power, target welding speed, defocusing amount, etc. The system arranges these extracted parameter settings according to the time sequence of the welding process to form a new time series data stream, i.e. welding data.

[0088] The system directly collects feedback data from the controllers or sensors of each execution unit in real time, including the actual output laser power from the laser, the actual TCP speed from the robot controller, and the actual wire feeding speed from the wire feeder. Similar to the welding process, all collected dynamic data is tagged with a synchronized timestamp. The system uses this unified timestamp to precisely align the dynamic data streams from different hardware with the welding process and welding data on the timeline. These aligned dynamic feedback data streams constitute the dynamic working data.

[0089] Specifically, during the welding process, the system collects TCP data from the KUKA robot controller at a frequency of 2kHz, and attaches an NTP synchronization timestamp with microsecond precision to each data point; the system ultimately generates a complete data stream from t=0.000s to t=25.500s; for example, at t=15.285s, a data point in the welding process is {timestamp:15.285,x:1024.30,y:502.51,z:799.92,…}, and this complete data stream is the welding process of this welding operation.

[0090] When processing the welding process of the weld seam, the system matches the point {x:1024.30,y:502.51,z:799.92} in the process to the program segment PROG_B:SEG_05 in the welding database; the system queries the process settings of this program segment and obtains: {target power:3000W, target speed:50mm / s, target defocusing amount:0mm}; at t=15.285s, the corresponding welding data is the above parameter set; the entire welding process generates a complete welding data time series.

[0091] The system synchronously acquires dynamic data during weld welding. Around t=15.285s, the system reads the actual power reading from the IPG laser controller and the actual speed feedback from the KUKA controller. All readings are timestamped at the microsecond level. The system finds that at t=15.285s, the dynamic working data is {timestamp:15.285, actual_power:2850, actual_speed:49.9}. By comparison, the system clearly sees that at this moment, actual_power (2850W) is significantly lower than set_power (3000W). Thus, the system obtains two data sets with different dimensions but precise time synchronization. The difference between these two data sets is the core input for all subsequent state analysis and optimization.

[0092] Furthermore, the quality levels of each critical weld area are collected. Based on the quality levels of each critical weld area and the welding data of the laser welding head, the first baryon welding state curve is determined. Based on the quality levels of each critical weld area and the dynamic working data of the laser welding head, the second baryon welding state curve is determined. The welding state curve of the laser welding head is determined based on the synthesis of the first and second baryon welding state curves. This overall consideration of the synthesis of the first and second baryon welding state curves ensures the accuracy of the welding state curve of the laser welding head.

[0093] At this point, the system obtains the final quality level of all critical weld areas and uses the welding process to map these discrete levels onto a continuous time axis. Key parameters (such as target laser power and target welding speed) are selected from the welding data (i.e., process settings) as independent variables, and the quantified quality level is used as the dependent variable. Statistical modeling is performed to output the predicted quality level. Plotting this along the time axis forms the first baryon welding state curve, which reflects the expected quality performance under unbiased process settings.

[0094] The second baryon welding state curve evaluates the dynamic stability and control accuracy of the equipment system. The system aligns discrete quality levels onto the time axis. Key parameters and their dynamic characteristics (such as the deviation between actual and set values, and parameter fluctuations) are selected from dynamic operating data (i.e., actual operation feedback) as independent variables, and the quantified quality level is used as the dependent variable for modeling. The model output is plotted along the time axis to form the second baryon welding state curve. This curve is very sensitive to the actual performance of the equipment and can accurately capture quality degradation caused by equipment vibration, power grid fluctuations, control delays, etc.

[0095] Based on historical data, the system assigns dynamic weights to the two sub-curves and uses fusion algorithms such as weighted average or neural networks to synthesize them into a final welding status curve. The vertical axis of this curve is a comprehensive health status score (e.g., 0-100), and the horizontal axis is time or weld position. The final welding status curve reflects both the macroscopic impact of the process settings and captures the abnormalities in equipment execution. It is a benchmark curve that can comprehensively and quantitatively evaluate the status of the entire welding process.

[0096] Specifically, the critical weld area KSA_01 is located at 1024mm along the weld path, and its quality level is unqualified (quantified as 1). Based on a welding speed of 50mm / s, this level is aligned to the time point t≈20.48s. The system selects the target laser power as the main analysis feature and examines the welding data throughout the welding process. During the time period of KSA_01, the target laser power is set to 3000W, which is exactly the same as the setting value for other qualified areas of the weld. The first baryon welding state curve generated by the system does not show a significant drop near the area corresponding to KSA_01. This curve is relatively flat, indicating that from the perspective of process setting, this area has not been specially treated or has any design defects.

[0097] The non-compliance level (1) of KSA_01 is still aligned to t≈20.48s; the system selects the deviation between the actual laser power and the target power as the core analysis feature; by checking the dynamic working data corresponding to KSA_01, the system found that between t=20.45s and t=20.50s, the actual power dropped from 3000W to 2850W instantly, resulting in a significant deviation of -150W; based on this strong negative correlation, the second baryon welding state curve generated by the system showed a sharp and deep trough at t=20.48s.

[0098] Based on historical weld data, most quality fluctuations are related to the instantaneous response of the equipment. The system sets the weight of the second baryon curve to 0.8 and the weight of the first baryon curve to 0.2. At t=20.48s, the health score of the first baryon curve is as high as 95 points (because the settings are correct), while the health score of the second baryon curve is only 20 points (because the actual power is seriously abnormal). The final score after synthesis = 95 x 0.2 + 20 x 0.8 = 35. Therefore, the final welding state curve generated by the system drops sharply from the normal score of around 95 points to 35 points at t=20.48s, forming a peak. This synthesized curve clearly indicates that although the process settings are correct, a serious problem occurred at the equipment execution level, causing the welding state to deteriorate sharply at that moment, ultimately resulting in product defects.

[0099] Therefore, based on the node position, corresponding welding state, and corresponding welding event of each welding state node, the optimization content of each welding state node is determined, and the corresponding welding optimization content is marked. This approach takes into account the overall consideration of the node position, corresponding welding state, and corresponding welding event of each welding state node, ensuring the accuracy of the optimization content of each welding state node. At the same time, the quality level of each key weld area is introduced to further control the welding data and dynamic working data of the laser welding head, improve the accuracy of the welding state curve of the laser welding head, and mark the corresponding welding optimization content.

[0100] At this point, the system applies various signal processing and event detection algorithms to the welding status curve, such as threshold crossing (triggered when the curve health score is lower than a preset threshold), gradient analysis (triggered when the slope exceeds the rate of change threshold), extreme point detection (finding local minima of depth), or pattern matching (matching with preset abnormal pattern templates). Whenever the algorithm detects an event that meets the conditions, the system records a precise timestamp on the time axis, i.e., a welding status node. At the same time, the system marks a qualitative welding status for the node based on the algorithm characteristics that triggered it, such as continuous abnormality, transient deterioration, severe fault, or periodic instability.

[0101] For each defined welding state node, the system performs a data backtracking based on its timestamp, while simultaneously querying the node's location, marked welding state, and the original welding events associated with that spatiotemporal location in S131. The system inputs the associated information into a diagnostic engine, which uses logical rules or machine learning models to perform causal tracing to determine the objects that need optimization, i.e., the content to be optimized. After determining the general direction of the content to be optimized, the system further queries the knowledge base to generate more specific and actionable welding optimization content. These suggestions are targeted at specific hardware, software, or process parameters.

[0102] Specifically, the system analyzes the final welding state curve of the weld. After applying gradient analysis and threshold crossing algorithm, the system detects that the curve slope exceeds the preset threshold near t=20.48s, and the lowest point of the curve is 35 points, which is far below the fault threshold of 30 points. At the timestamp t=20.48s corresponding to the lowest point of the curve, the system determines the welding state node and names it WSN_01. Since this node is triggered by a drastic negative change and a low depth score, the system marks its welding state as a serious fault.

[0103] The system performed a diagnostic on the welding status node WSN_01. The system correlated the following information: node location (t=20.48s), welding status (severe process failure), welding event (porosity caused by molten pool instability recorded in S131), and contextual data (a 150W power interruption was shown in S151 at t=20.45s). After analysis by the diagnostic engine, the root cause was identified as insufficient laser suppression of power grid fluctuations, which is the area to be optimized. Further analysis revealed that similar power interruption events occurred multiple times in the past week, with the timing highly overlapping with the start-up time of another large stamping machine in the workshop. Based on this strong correlation, the system ultimately generated and marked specific welding optimization content: the laser is sensitive to power grid disturbances; it is recommended to install an active filter or dynamic voltage restorer (DVR) with a rated power of not less than 10kVA at the laser power input.

[0104] In another embodiment of this application, the laser weld detection system incorporating image recognition includes: The weld seam stereo imaging module is used to acquire the weld seam path of the automotive workpiece when the laser welding head performs laser welding on the automotive workpiece, and trigger visual detection of the weld seam along the weld seam path to identify multiple weld seam images; and to determine a weld seam stereo image based on the weld seam path of the automotive workpiece and multiple weld seam images. The weld seam control node module is used to determine multiple pit regions based on image recognition of the weld seam 3D image. Multiple weld seam control nodes are determined according to the pit space of each pit region, the corresponding surface morphology, and the regional change of two adjacent pit regions. The laser weld inspection system module is used to determine the corresponding weld quality events based on the dynamic detection of each weld control node in multiple weld control nodes. The corresponding laser weld inspection system is determined by combining the weld quality events of each weld control node, the corresponding node position, and the laser data of the laser welding head. The quality level module is used to identify multiple key weld areas based on the identification of the laser weld inspection system, and to determine the quality level of each key weld area based on the abnormal events, corresponding abnormal morphology and corresponding pit surface of each key weld area. The welding status curve module is used to determine the corresponding welding data and dynamic working data based on the detection of the laser welding head. It determines the welding status curve of the laser welding head based on the welding data, dynamic working data and quality level of each key weld area, and marks the corresponding welding optimization content.

[0105] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for detecting laser welds combining image recognition, characterized in that, include: When the laser welding head performs laser welding on the automotive workpiece, the weld seam path of the automotive workpiece is acquired, and visual inspection of the weld seam is triggered along the weld seam path to determine multiple weld seam images; a three-dimensional image of the weld seam is determined based on the weld seam path of the automotive workpiece and multiple weld seam images; the three-dimensional image of the weld seam reproduces the three-dimensional morphology of the laser weld seam, including multi-dimensional information such as pits, protrusions, and undercut. Multiple pit regions are identified based on image recognition of weld 3D images. Multiple weld control nodes are determined based on the pit space of each pit region, the corresponding surface morphology, and the regional variation of two adjacent pit regions. The weld control nodes present a complete description of the node location, the triggering event, and the correlation evidence pointing to the two original pit regions that caused the node. In multiple weld seam control nodes, the corresponding weld seam quality events are determined based on the dynamic detection of each weld seam control node, and the corresponding laser weld seam detection system is determined based on the combination of the weld seam quality events of each weld seam control node, the corresponding node position and the laser data of the laser welding head. Based on the identification of this laser weld seam detection system, multiple key weld seam areas are identified, and the quality level of each key weld seam area is determined based on the abnormal events, corresponding abnormal morphologies, and corresponding pit surfaces of each key weld seam area. Based on the detection of the laser welding head, the corresponding welding data and dynamic working data are determined. According to the welding data, dynamic working data and quality level of each key weld area of ​​the laser welding head, the welding state curve of the laser welding head is determined, and the corresponding welding optimization content is marked.

2. The laser weld detection method combining image recognition according to claim 1, characterized in that, When the laser welding head performs laser welding on the automotive workpiece, the weld seam path of the automotive workpiece is acquired, and visual detection of the weld seam is triggered along the weld seam path to determine multiple weld seam images. A 3D image of the weld seam is determined based on the weld seam path of the automotive workpiece and multiple weld seam images, including: The automotive workpiece is positioned in the corresponding positioning fixture, and the current posture of the automotive workpiece is determined. The weld seam path of the automotive workpiece is determined according to the welding position of the automotive workpiece, the moving path of the laser welding head, and the corresponding laser welding requirements. The camera moves along the weld seam path to determine the camera's moving path. The camera's visual inspection of the weld seam is triggered according to the camera's moving path and the current posture of the automotive workpiece. Multiple weld seam images are determined based on visual inspection of the weld seam using a camera, and the corresponding shooting positions are marked. Multiple image combinations are determined based on the multiple weld seam images, the corresponding shooting positions, and the weld seam path of the automotive workpiece. A three-dimensional image of the weld seam is constructed based on the multiple image combinations.

3. The laser weld detection method combining image recognition according to claim 1, characterized in that, The image recognition based on the weld seam 3D image determines multiple pit regions, and determines multiple weld seam control nodes based on the pit space of each pit region, the corresponding surface morphology, and the regional variation between two adjacent pit regions, including: The weld 3D image is inspected, and multiple pits are identified during the inspection process. Based on the spatial position and corresponding extension direction of each pit, image recognition of the weld 3D image is triggered to identify multiple pit areas. In multiple pit regions, the corresponding pit space is determined based on the detection of each pit region, and the corresponding surface morphology is determined based on the surface recognition of each pit region, so as to collect the pit space and corresponding surface morphology of each pit region. The comparison of two adjacent pit areas is triggered along the weld path of the automotive workpiece, and the amount of regional change is determined during the comparison. The content of the first node is determined based on the amount of regional change and the pit space of the pit area. The content of the second node is determined based on the amount of regional change and the surface morphology of the pit area. Multiple weld control nodes are determined based on the content of the first node and the content of the second node.

4. The laser weld detection method combining image recognition according to claim 1, characterized in that, In the process of determining corresponding weld quality events based on dynamic detection at multiple weld control nodes, and establishing a corresponding laser weld detection system based on the weld quality events at each control node, the corresponding node location, and the laser data from the laser welding head, the system includes: Real-time monitoring of each weld seam control node, and determination of multiple corresponding surface features during the monitoring process. Based on the node position of each weld seam control node, the corresponding multiple surface features and pit space, the corresponding weld seam quality event is determined, and the priority of the weld seam quality event is marked.

5. The laser weld detection method combining image recognition according to claim 4, characterized in that, The method of determining corresponding weld quality events based on dynamic detection at multiple weld control nodes, and determining a corresponding laser weld detection system based on the weld quality events at each weld control node, the corresponding node position, and the laser data from the laser welding head, further includes: Based on the identification of each weld quality event, the corresponding weld quality content is determined. The first weld inspection content is determined according to the weld quality content of each weld quality event, the corresponding priority, and the surface morphology of the corresponding pit area. The laser data of the laser welding head is matched based on the movement position of each weld control node and the laser welding head to determine the combination of laser data of the laser welding head. The second layer of weld inspection content is determined based on the combination of laser data of the laser welding head and the weld quality content of each weld quality event. The corresponding laser weld inspection system is determined based on the first layer of weld inspection content and the second layer of weld inspection content.

6. The laser weld detection method combining image recognition according to claim 1, characterized in that, The process of identifying multiple critical weld areas based on the laser weld inspection system, and determining the quality level of each critical weld area based on abnormal events, corresponding abnormal morphologies, and corresponding pit surfaces, includes: In this laser weld inspection system, the laser weld inspection system is dynamically identified, and the node critical coefficients of multiple weld control nodes are determined during the identification process. Based on the node critical coefficients of multiple weld control nodes, the corresponding pit areas, and the three-dimensional image of the weld, multiple key weld areas are determined.

7. The laser weld detection method combining image recognition according to claim 6, characterized in that, The method of identifying multiple critical weld areas based on the laser weld inspection system, and determining the quality level of each critical weld area based on the abnormal events, corresponding abnormal morphologies, and corresponding pit surfaces of each critical weld area, further includes: Based on the detection of each critical weld area, multiple sub-anomaly features are determined. Based on the feature location, corresponding feature shape, and regional shape of the critical weld area, the abnormal event of the critical weld area is determined. Targeted control is implemented for each critical weld area, and abnormal morphologies of each critical weld area are marked. Based on the identification of each critical weld area, the corresponding pit surface is determined. According to the abnormal events, corresponding abnormal morphologies, and corresponding pit surfaces of each critical weld area, the corresponding multidimensional quality coefficient is determined. The quality level of the critical weld area is determined by matching the mapping relationship between the multidimensional quality coefficient and the quality level.

8. The laser weld detection method combining image recognition according to claim 1, characterized in that, The welding data and dynamic working data are determined based on the detection of the laser welding head. The welding state curve of the laser welding head is determined according to the welding data, dynamic working data, and quality levels of each key weld area, and the corresponding welding optimization content is marked, including: The welding process of the laser welding head is collected, and the welding data of the laser welding head is determined by matching the welding process of the laser welding head with the corresponding welding database. At the same time, the corresponding dynamic working data is determined based on the welding database of the laser welding head. The welding data and dynamic working data of the laser welding head are two data groups of different dimensions.

9. The laser weld detection method combining image recognition according to claim 8, characterized in that, The method of determining corresponding welding data and dynamic working data based on the detection of the laser welding head, determining the welding state curve of the laser welding head based on the welding data, dynamic working data and quality level of each key weld area, and marking the corresponding welding optimization content, also includes: The quality level of each key weld area is collected. Based on the quality level of each key weld area and the welding data of the laser welding head, the first baryon welding state curve is determined. Based on the quality level of each key weld area and the dynamic working data of the laser welding head, the second baryon welding state curve is determined. The welding state curve of the laser welding head is determined based on the synthesis of the first baryon welding state curve and the second baryon welding state curve. Based on the identification of the welding state curve, multiple welding state nodes are determined and the corresponding welding states are marked. According to the node position, corresponding welding state and corresponding welding event of each welding state node, the content to be optimized for each welding state node is determined and the corresponding welding optimization content is marked.

10. A laser weld seam detection system combining image recognition, characterized in that, The laser weld detection system incorporating image recognition is applied to the laser weld detection method incorporating image recognition as described in any one of claims 1-9.

Images