Resistance welding seam defect on-line detection system and method

Abstract

This invention discloses an online detection system and method for weld defects in resistance welding, belonging to the field of defect detection technology. The method includes: acquiring real-time image sequences of the welding process and simultaneously collecting welding process parameters; extracting region coordinate data through inter-frame difference, watershed segmentation, and geometric prior knowledge; generating multimodal feature vectors by combining process parameters; and constructing weld state configuration data after decoding and mapping. Iterative optimization of convolutional neural network weights is performed until the sum of the area ratios of over-welded regions and non-fused regions is lower than a first threshold. After convergence, real-time detection is performed and pixel-level molten pool feature data is collected. The grayscale ratio is used to determine whether to adjust welding process parameters. Finally, the detection results are output according to defect judgment rules, forming a process-defect correlation database. This method achieves accurate online detection of weld defects and dynamic adjustment of welding process parameters, improving the real-time performance and accuracy of resistance welding quality detection.

Description

Technical Field

[0001] This invention belongs to the field of defect detection technology, specifically an online detection system and method for weld defects in resistance welding. Background Technology

[0002] Resistance welding is widely used in industries such as automotive and aerospace due to its high welding efficiency and high degree of automation. However, its welding process is susceptible to environmental factors, resulting in defects such as over-welding and lack of fusion, which directly affect the structural stability and service life of the welded parts. Therefore, accurate detection of weld defects is crucial. Currently, resistance welding defect detection mostly adopts post-weld offline inspection or destructive sampling inspection. Offline inspection cannot control the welding process in real time and is difficult to achieve full batch inspection, while destructive inspection results in material waste and increased inspection costs. Neither of these methods can meet the real-time quality control requirements of modern production lines.

[0003] Existing non-destructive online inspection technologies still have many limitations. Ultrasonic and X-ray inspection methods are easily affected by environmental interference, have limited accuracy, and struggle to identify minute internal defects. Single machine vision or process parameter inspection methods, lacking multi-source data fusion, cannot comprehensively characterize the dynamic welding process and are insufficiently accurate in judging key states such as weld nugget formation. Furthermore, traditional inspection methods lack the ability to dynamically adjust welding process parameters, making it impossible to adjust the process in time during the defect initiation stage, and difficult to establish the correlation between process parameters and defects, thus failing to provide data support for process optimization. Therefore, there is an urgent need for an online inspection method for resistance welding defects that can achieve multi-source data fusion, pixel-level accurate detection, and dynamic adjustment of welding process parameters. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes an online detection system and method for weld defects in resistance welding. The system acquires real-time image sequences of the welding process and simultaneously collects welding process parameters. Regional coordinate data is extracted through inter-frame differencing, watershed segmentation, and geometric prior knowledge. Multimodal feature vectors are generated by combining these data with process parameters, and weld state configuration data is constructed after decoding and mapping. The weights of the convolutional neural network are iteratively optimized until the sum of the areas of the over-welded region and the non-fused region falls below a first threshold. After convergence, real-time detection is performed, and pixel-level molten pool feature data is collected. The grayscale ratio is used to determine whether to adjust the welding process parameters. Finally, the detection results are output according to defect judgment rules, forming a process-defect correlation database. This method achieves accurate online detection of weld defects and dynamic adjustment of welding process parameters, improving the real-time performance and accuracy of resistance welding quality inspection.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] Online detection methods for weld defects in resistance welding include:

[0007] The system acquires a real-time image sequence of the resistance welding process of the workpiece to be inspected, and simultaneously collects the corresponding welding process parameters; the welding process parameters include at least welding current, electrode pressure, and welding time.

[0008] Based on the real-time image sequence and welding process parameters, a multimodal feature field of the welding process is constructed to obtain weld state configuration data;

[0009] Based on the weld state configuration data, iterative optimization is performed on the weight parameters in the preset deep learning detection model until the sum of the area ratios of the over-welded area and the non-fused area in the current detection frame output by the deep learning detection model is less than the preset first threshold, and the detection model convergence signal is output.

[0010] Based on the received convergence signal of the detection model, the resistance welding process is subjected to real-time weld defect detection at a preset frame rate, and pixel-level molten pool feature data of each detection frame is collected during the detection process; the pixel-level molten pool feature data includes the gray value of each pixel and over-welding or non-fusion markers determined based on the gray value threshold.

[0011] Specifically, the step of constructing a multimodal feature field of the welding process based on the real-time image sequence and welding process parameters to obtain weld state configuration data includes:

[0012] Inter-frame difference calculation is performed on consecutive image frames in the real-time image sequence. The dynamic welding area is located based on the pixel grayscale change. The image of the located dynamic welding area is then processed by a watershed image segmentation algorithm based on morphological gradient and preset label to obtain a preliminary segmentation result image containing multiple candidate regions. Then, combined with the geometric prior knowledge that the weld area is distributed in a concentric ring shape, the contour coordinate data of the heat-affected zone and the boundary point coordinate data of the weld nugget candidate region are identified from the preliminary segmentation result image.

[0013] The instantaneous values ​​of welding current, electrode pressure, and welding time are matched with each frame of the real-time image sequence using the timestamp alignment method to generate a welding process parameter time vector synchronized with each image frame.

[0014] The contour coordinate data of the heat-affected zone, the boundary point coordinate data of the weld nugget candidate zone, and the time-series vector of the welding process parameters are cascaded and fused to generate a multimodal feature vector.

[0015] Specifically, the step of constructing a multimodal feature field of the welding process based on the real-time image sequence and welding process parameters to obtain weld state configuration data further includes:

[0016] The multimodal feature vectors are input into a pre-trained fully connected neural network for decoding and mapping. The fully connected neural network sequentially includes an input layer, three hidden layers using a linear rectified function as the activation function, and an output layer using an S-shaped growth curve function as the activation function. The output layer outputs a normalized probability distribution. After decoding, the probability distribution is represented as a heat-affected zone distribution map containing pixel coordinates, a binary mask image representing the welding spatter area, and a weld nugget forming contour model described by a sequence of boundary point coordinates, which together constitute the weld state configuration data. The binary mask image of the welding spatter area is used to calibrate the spatter pixel area during the welding process.

[0017] Specifically, the iterative optimization of the weight parameters in the preset deep learning detection model includes:

[0018] Initialize a convolutional neural network with an encoder-decoder structure as a deep learning detection model;

[0019] The heat-affected zone distribution map, the binary mask image of the welding spatter area, and the weld nugget forming contour model from the weld condition configuration data are used as inputs to the deep learning detection model. In a single iteration, the deep learning detection model outputs a predicted defect region segmentation map and calculates the loss value between the predicted defect region segmentation map and the supervision label generated based on the weld condition configuration data. The loss value is calculated by weighted summation of the cross-entropy loss function and the Dessian loss function. The backpropagation algorithm is used to calculate the gradient of each weight parameter of the deep learning detection model based on the loss value. The adaptive moment estimation algorithm is used to update all weight parameters of the deep learning detection model based on the gradient. The iteration process is repeated until the sum of the total area of ​​the over-welded region pixels and the total area of ​​the non-fused region pixels in the current detection frame output by the deep learning detection model is less than a first threshold in proportion to the total image area.

[0020] Specifically, when calculating the loss value between the predicted defect region segmentation map and the supervision label generated based on the weld condition configuration data, the method for generating the supervision label includes:

[0021] Based on the heat-affected zone distribution map and weld nugget forming contour model in the weld condition configuration data, a category label for each pixel is generated by rule judgment.

[0022] For pixels located inside the boundary defined by the weld nugget forming contour model, if their grayscale value is higher than the preset over-welding grayscale threshold, they are marked as over-welding areas; if their grayscale value is lower than the preset non-fusion grayscale threshold, they are marked as non-fusion areas; otherwise, they are marked as normal welding areas.

[0023] For pixels located within the area defined by the heat-affected zone distribution map but not inside the weld nugget forming contour model, they are labeled as heat-affected zone category. For pixels marked by the binary mask image of the welding spatter area, they are separately labeled as spatter area category.

[0024] The category labels of all pixels constitute the supervision label.

[0025] Specifically, the real-time weld defect detection during the resistance welding process at a preset frame rate, and the acquisition of pixel-level molten pool feature data for each detection frame during the detection process, includes:

[0026] When the convergence signal of the detection model is received, the image acquisition device is started to capture real-time images of the welding area at a preset frame rate. For each detection frame image acquired, the deep learning detection model that has been iteratively optimized is called to perform forward inference, and the category label of each pixel in the corresponding detection frame image is output as belonging to the normal welding area, the over-welded area or the non-fused area.

[0027] Based on the category label, traverse all pixels in the detection frame image, record the gray value of each pixel, and add an over-soldering label to pixels classified as over-soldering areas and an un-fused label to pixels classified as unfused areas.

[0028] The grayscale values ​​of all pixels and their corresponding identifiers are aggregated to generate pixel-level melt pool feature data.

[0029] Specifically, the method further includes:

[0030] Based on the pixel-level weld pool feature data, the average gray value of all pixels with over-soldering marks in the current detection frame is calculated as the average gray value of the over-soldering area, and the average gray value of all pixels with non-fusion marks is calculated as the average gray value of the non-fusion area.

[0031] The ratio of the average grayscale of the welded area to the average grayscale of the unfused area is calculated to obtain the real-time grayscale ratio. It is then determined whether the real-time grayscale ratio is less than a preset second threshold. If the real-time grayscale ratio is less than the second threshold, the defect detection result of the current detection frame is generated and output based on the output of the current deep learning detection model and the preset defect judgment rules. If the real-time grayscale ratio is greater than or equal to the second threshold, the welding current compensation and electrode pressure adjustment are calculated based on the pixel-level molten pool feature data. The welding current compensation is sent to the welding power controller, and the electrode pressure adjustment is sent to the electrode pressure servo mechanism. After the welding process parameters are updated, the process returns to perform real-time weld defect detection at a preset frame rate during the resistance welding process until the real-time grayscale ratio is less than the second threshold.

[0032] Specifically, the calculation of welding current compensation and electrode pressure adjustment based on the pixel-level molten pool feature data includes:

[0033] Based on the pixel-level molten pool feature data, the total number of pixels with over-soldering markers and the total number of pixels with non-fusion markers are counted. The difference in pixel size of the region is calculated by the difference. At the same time, the absolute difference between the average gray level of the over-soldering region and the average gray level of the non-fusion region is calculated to obtain the absolute gray level difference.

[0034] The pixel difference in the region and the absolute difference in grayscale are respectively input into a dual-channel proportional-integral controller. The first channel of the dual-channel proportional-integral controller takes the pixel difference in the region as the first input and obtains a first control quantity through proportional and integral operations, which serves as the welding current compensation quantity. The second channel of the dual-channel proportional-integral controller takes the absolute difference in grayscale as the second input and obtains a second control quantity through proportional and integral operations, which serves as the electrode pressure adjustment quantity. The proportional operation is to multiply the first input or the second input by a preset proportional coefficient, and the integral operation is to perform discrete-time integration on the first input or the second input and then multiply it by a preset integral coefficient.

[0035] Specifically, based on the output of the current deep learning detection model and the preset defect judgment rules, the defect detection result of the current detection frame is generated and output, including:

[0036] The deep learning detection model is invoked to infer the current detection frame and obtain a pixel-level classification result map. In the pixel-level classification result map, all connected regions formed by pixels with over-soldering marks are identified and the area of ​​each connected region is calculated. All connected regions formed by pixels with non-fused marks are identified and the area of ​​each connected region is calculated.

[0037] If the area of ​​any over-welded indicator connected region exceeds the first area threshold, an over-welded defect is determined to exist. If the area of ​​any non-fused indicator connected region exceeds the second area threshold, a non-fused defect is determined to exist. Based on the determination results, a defect detection result report is generated, which includes the defect type, the contour coordinates of the connected region where the defect is located, and the defect area.

[0038] An online defect detection system for resistance welding includes:

[0039] The data acquisition module is used to acquire real-time image sequences and welding process parameters of the resistance welding process;

[0040] The feature construction module is used to perform inter-frame difference, watershed segmentation and geometric prior recognition on real-time image sequences, extract coordinate data of heat-affected areas and weld nugget candidate areas, and generate multimodal feature vectors by feature concatenation with welding process parameter time vectors. After decoding and mapping, the weld state configuration data is output.

[0041] The model iteration and optimization module is used to iteratively optimize the weight parameters of the deep learning detection model and output a converged deep learning detection model.

[0042] The defect detection module is used to capture real-time images of the welding area and output pixel-level category labels. It traverses pixels to record grayscale values, adds corresponding labels to defect pixels, and generates pixel-level molten pool feature data.

[0043] The parameter control module is used to calculate the real-time grayscale ratio between the welded area and the unfused area, and to calculate the welding current compensation and electrode pressure adjustment.

[0044] The data storage module is used to calculate the area of ​​connected regions with over-welding and non-fusion markings based on the pixel-level classification result image, generate a defect detection result report, record all defect data within the welding cycle, and construct a process-defect association database after storing the data in association with the corresponding welding process parameters.

[0045] Compared with the prior art, the beneficial effects of the present invention are:

[0046] 1. This invention proposes an online detection system for weld defects in resistance welding, and optimizes and improves its architecture, operation steps and processes. The system has the advantages of simple process, low investment and operating costs and low production costs.

[0047] 2. This invention proposes an online detection method for weld defects in resistance welding. This method achieves accurate online detection of weld defects in resistance welding. By fusing real-time image sequences of the welding process with process parameters such as welding current and electrode pressure to construct a multimodal feature field, and combining a deep learning detection model with an encoder-decoder structure and completing weight iterative optimization, it can identify defects such as over-welding and lack of fusion with pixel-level accuracy. At the same time, it collects pixel-level molten pool feature data and accurately determines the defect status through indicators such as grayscale ratio, outputting detailed detection results including defect type, location, and area, thus improving the accuracy and refinement of defect detection and effectively solving the problem that traditional detection methods are unable to capture dynamic welding defects in real time.

[0048] 3. This invention proposes an online defect detection method for resistance welding. The method has the ability to dynamically adjust welding process parameters and optimize the process. When abnormal characteristics of the molten pool are detected, the welding current compensation and electrode pressure adjustment can be calculated and output through a dual-channel proportional-integral controller to achieve closed-loop control of the welding process and reduce the generation of defects. At the same time, the defect statistics during the welding cycle are associated with the process parameters and stored to form a database. This can help to explore the sensitive range of process parameters and provide data support for resistance welding process optimization and parameter matching. This not only improves the stability of the welding process, but also reduces the incidence of weld defects from the source, combining the dual value of real-time detection and process optimization. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the online detection method for weld defects in resistance welding according to the present invention;

[0050] Figure 2 This is a flowchart illustrating the principle of the online detection method for weld defects in resistance welding according to the present invention.

[0051] Figure 3 This is a diagram illustrating the architecture of the online weld defect detection system for resistance welding according to the present invention. Detailed Implementation

[0052] Example 1:

[0053] Please see Figure 1 and Figure 2 The present invention provides an embodiment of an online detection method for weld defects in resistance welding, the method comprising S1 to S4, including the following steps:

[0054] S1: Acquire a real-time image sequence of the resistance welding process of the workpiece to be inspected, and simultaneously collect the corresponding welding process parameters; the welding process parameters include at least welding current, electrode pressure and welding time;

[0055] Furthermore, the specific steps of S1 include:

[0056] (1) A high-speed industrial camera device is deployed 200mm above the workpiece surface directly above the welding station of the resistance welding equipment. The camera device is equipped with a high dynamic range imaging lens, which can adapt to the contrast between the strong light in the molten pool area and the weak light in the workpiece substrate during the welding process. The acquisition frame rate of the camera device is adaptively adjusted according to the welding cycle of the welding process. Under the conventional resistance welding process, it is set to 200 frames / second to ensure that the details of dynamic processes such as molten pool formation, heat-affected zone changes, and welding spatter can be captured. At the same time, high-precision data acquisition sensors are deployed on the welding power supply, electrode pressure actuator, and welding time controller. The acquisition accuracy of the current sensor reaches 0.1A, the acquisition accuracy of the pressure sensor reaches 0.01MPa, and the time sensor is used to time in milliseconds. All sensors are connected to the camera device through the industrial Ethernet bus and a hard synchronization mechanism based on timestamp is adopted to control the time synchronization error between image acquisition and process parameter acquisition within 1 millisecond.

[0057] (2) Each acquisition device starts working at the same time as the welding process starts. The high-speed industrial camera continuously captures images of the welding area and generates a real-time image sequence arranged in chronological order. Each image frame is accompanied by a unique timestamp.

[0058] (3) The sensors of each process parameter synchronously collect the welding current and electrode pressure values ​​at each moment during the welding process, and combine them with the cumulative value of welding time to generate a welding process parameter sequence containing timestamps, so as to realize the synchronous acquisition of real-time image sequence and welding process parameters.

[0059] Furthermore, this embodiment takes the typical resistance welding application scenario of spot welding of automotive body steel plates as an example. The workpiece to be inspected is a 1.2mm thick cold-rolled automotive steel plate, using an AC resistance spot welding process with a preset welding current of 8kA, an electrode pressure of 3.5MPa, and a welding time of 200ms. A high-speed industrial camera is deployed next to the electrode arm at this spot welding station, with the lens focal length adjusted to clearly cover the contact area between the welding electrode and the workpiece, as well as the surrounding 50mm range. The camera acquires images of the welding process in real time at a rate of 200 frames per second, forming a real-time image sequence with a resolution of 1920×1080. Simultaneously, a Hall current sensor is installed at the output end of the spot welding power supply, a pressure transmitter is installed in the air circuit of the electrode pressure cylinder, and a time acquisition module is connected to the welding controller. All three sensors are connected to the camera via a PROFINET bus to achieve hard synchronization. During the welding process, the real-time change curves of the welding current and electrode pressure, as well as the cumulative value of the welding time, are acquired synchronously, completing the synchronous acquisition of the real-time image sequence of the welding process and the welding process parameters.

[0060] S2: Based on the real-time image sequence and welding process parameters, construct a multimodal feature field of the welding process to obtain weld state configuration data;

[0061] Furthermore, the construction of the multimodal feature field involves deeply integrating spatial features such as the molten pool, heat-affected zone, and spatter from the visual image dimension with temporal features such as current, pressure, and time from the process parameter dimension. This forms a feature system that can comprehensively characterize the dynamic state of the welding process. The weld state configuration data is the concrete expression of this feature field, containing the spatial distribution, morphological features, and temporal matching information of key parts of the welding area, serving as the core basis for training deep learning detection models and identifying defects. In this embodiment, for the real-time image sequence and welding process parameters of spot welding of automotive body steel plates, a series of steps, including inter-frame differencing, image segmentation, feature concatenation, and neural network decoding mapping, are used to complete the construction of the multimodal feature field. The final result is weld state configuration data containing a heat-affected zone distribution map, a binary mask image of the welding spatter area, and a molten nugget forming contour model. This accurately depicts the forming state of the molten nugget, the range change of the heat-affected zone, the distribution of welding spatter, and the temporal correspondence between these visual features and welding current, electrode pressure, and welding time during the spot welding process.

[0062] The process of constructing a multimodal feature field of the welding process based on the real-time image sequence and welding process parameters to obtain weld state configuration data includes:

[0063] S2.1: Perform inter-frame difference calculation on consecutive image frames in the real-time image sequence, locate the dynamic welding area based on pixel grayscale changes, and perform a watershed image segmentation algorithm based on morphological gradient and preset labels on the image of the located dynamic welding area to obtain a preliminary segmentation result image containing multiple candidate areas. Then, combined with the geometric prior knowledge that the weld area is distributed in a concentric ring shape, the contour coordinate data of the heat-affected zone and the boundary point coordinate data of the weld nugget candidate area are identified from the preliminary segmentation result image. Simultaneously, the instantaneous values ​​of welding current, electrode pressure, and welding time are matched with each frame of the real-time image sequence using the timestamp alignment method to generate a welding process parameter time sequence vector synchronized with each image frame.

[0064] Furthermore, the specific process of performing inter-frame difference calculation on consecutive image frames in the real-time image sequence includes: selecting three consecutive frames from the real-time image sequence, denoted as frame (n-1), frame n, and frame (n+1); subtracting the pixel-level grayscale values ​​of frame n from those of frame (n-1) and frame (n+1) respectively to obtain two difference images; taking the absolute value of the grayscale values ​​of the two difference images and then performing pixel-level summation to obtain an inter-frame difference fused image; setting a grayscale difference threshold; marking pixels in the inter-frame difference fused image with grayscale values ​​greater than the grayscale difference threshold as dynamically changing pixels; the connected region composed of all dynamically changing pixels is the dynamic welding region; the dynamic welding region accurately covers the areas where visual changes occur during the welding process, such as the molten pool, electrode contact area, and spatter generation area, effectively eliminating interference from static backgrounds such as the workpiece substrate. In the automotive steel plate spot welding scenario of this embodiment, the grayscale difference threshold is set to 50. Through inter-frame difference calculation, a circular dynamic welding area with a diameter of about 30mm is successfully located with the contact point between the electrode and the workpiece as the center. This dynamic welding area completely includes the weld nugget forming area and the heat-affected zone.

[0065] Furthermore, the process of performing a watershed image segmentation algorithm based on morphological gradient and preset markers on the image of the located dynamic welding area includes: first, grayscale conversion and Gaussian filtering preprocessing are performed on the dynamic welding area image. The kernel size of the Gaussian filter is set to 5×5, and the standard deviation is set to 1.5 to effectively suppress the salt-and-pepper noise generated by strong light during welding; then, the morphological gradient of the preprocessed image is calculated using a combination of dilation and erosion operations. The structural elements of both dilation and erosion are 3×3 rectangular structures, and the morphological gradient image can clearly highlight the edge contours of each region in the image; then, preset markers are set according to the morphological characteristics of the resistance welding weld area, and the contact point between the electrode and the workpiece is set as the kernel. The core marker point is set as the outer boundary of the dynamic welding area as the background marker point. The core marker point and the background marker point provide the initial region seed for watershed segmentation. Finally, based on the morphological gradient image, the watershed segmentation algorithm is executed with the preset markers as seed points. During the segmentation process, a region merging strategy is introduced to merge small segmented regions with an area of ​​less than 50 pixels into adjacent large regions to avoid over-segmentation. Finally, a preliminary segmentation result image containing candidate regions for weld nuggets, candidate regions for heat effects, candidate regions for spatter, and background regions is obtained. Among them, grayscale conversion, Gaussian filtering, dilation operation, erosion operation and watershed segmentation algorithm are existing technologies in this field and are not inventive solutions of this application, and will not be described in detail here.

[0066] It should be noted that in resistance welding, the weld seam area exhibits a concentric ring-shaped distribution with the weld nugget as the core and the heat-affected zone surrounding it due to the heat spreading from the electrode contact point. This geometric prior knowledge provides a basis for region identification.

[0067] Furthermore, based on the prior geometric knowledge that the weld area is distributed in a concentric ring shape, the process of identifying the contour coordinate data of the heat-affected zone and the boundary point coordinate data of the weld nugget candidate area from the preliminary segmentation result image includes: First, in the preliminary segmentation result image, taking the contact point between the electrode and the workpiece as the center, calculate the center coordinates and radius of each candidate area. The circular candidate area with the smallest deviation between the center coordinates and the contact point coordinates and the smallest radius is determined as the weld nugget candidate area. The edge pixels of the weld nugget candidate area are extracted, and the horizontal and vertical coordinates of the edge pixels are arranged in clockwise order to obtain the boundary point coordinate data of the weld nugget candidate area. The annular candidate area surrounding the weld nugget candidate area is determined as the heat-affected zone. The outer and inner edge pixel coordinates of this area are extracted and arranged in clockwise order to obtain the outer contour coordinate data and inner contour coordinate data of the heat-affected zone, which together form the contour coordinate data of the heat-affected zone. In the automotive steel sheet spot welding scenario of this embodiment, the method successfully identified the candidate area for the weld nugget as a circular area with a diameter of approximately 8 mm, and the heat-affected zone as an annular area with an inner diameter of 8 mm and an outer diameter of 20 mm, and accurately extracted the corresponding contour coordinate data and boundary point coordinate data.

[0068] Furthermore, the matching process, which uses a timestamp alignment method to match the collected instantaneous values ​​of welding current, electrode pressure, and welding time with each frame of the real-time image sequence, includes: first, extracting the timestamp of each image frame in the real-time image sequence and the timestamp of each set of current, pressure, and time data in the welding process parameter sequence, with all timestamps starting at zero; then, establishing a timestamp matching mapping table, using the timestamp of the image frame as a reference, searching for the process parameter data with the smallest difference from that timestamp in the welding process parameter sequence, and if the difference is within the 1-millisecond range allowed by the synchronization error, matching that set of process parameter data to the corresponding image frame; for each image frame, arranging the matched instantaneous values ​​of welding current, electrode pressure, and cumulative welding time in a fixed order to generate a three-dimensional welding process parameter time sequence vector, which corresponds one-to-one with the image frame, achieving precise alignment of the time dimension of visual features and process parameter features. In this embodiment, the timestamp of the 100th frame of the real-time image sequence is 500ms. The matched instantaneous value of welding current is 8.2kA, the instantaneous value of electrode pressure is 3.4MPa, and the cumulative value of welding time is 500ms. The generated welding process parameter timing vector is [8.2,3.4,500], thus completing the synchronous matching of the frame image and the process parameters.

[0069] S2.2: Perform feature concatenation and fusion on the contour coordinate data of the heat-affected zone, the boundary point coordinate data of the weld nugget candidate zone, and the welding process parameter time vector to generate a multimodal feature vector;

[0070] Furthermore, vectorizing the contour coordinate data of the heat-affected zone and the boundary point coordinate data of the weld nugget candidate region is a prerequisite for feature cascade fusion. For the outer and inner contour coordinate data of the heat-affected zone, the horizontal and vertical coordinates of all pixels on the contour are concatenated sequentially to form two one-dimensional coordinate vectors. Then, the length of the two vectors is normalized to ensure that the length of the contour coordinate vector of the heat-affected zone remains consistent in different image frames. For the boundary point coordinate data of the weld nugget candidate region, the horizontal and vertical coordinates of the boundary points are concatenated sequentially to form a one-dimensional boundary point coordinate vector, and the length is also normalized. For the welding process parameter time series vector, its three-dimensional numerical form is directly retained, and the values ​​are normalized to map the welding current, electrode pressure, and welding time to the [0,1] interval to eliminate the dimensional differences of different physical quantities. In this embodiment, the normalized outer contour coordinate vector of the heat-affected zone has a length of 200, the inner contour coordinate vector has a length of 200, the boundary point coordinate vector of the weld nugget candidate region has a length of 100, and the welding process parameter timing vector is 3-dimensional. The above four vectors are spliced ​​together in the order of outer contour coordinate vector of heat-affected zone - inner contour coordinate vector of heat-affected zone - boundary point coordinate vector of weld nugget candidate region - welding process parameter timing vector to form a one-dimensional vector with a length of 503. This vector is the multimodal feature vector that integrates spatial visual features and process timing features. Each image frame corresponds to a multimodal feature vector, and the multimodal feature vectors of all image frames together form the multimodal feature set of the welding process.

[0071] S2.3: The multimodal feature vector is input into a pre-trained fully connected neural network for decoding and mapping. The fully connected neural network sequentially includes an input layer, three hidden layers using a linear rectified function as the activation function, and an output layer using an S-shaped growth curve function as the activation function. The output layer outputs a normalized probability distribution, which, after decoding, is represented as a heat-affected zone distribution map containing pixel coordinates, a binary mask image representing the welding spatter area, and a weld nugget forming contour model described by a sequence of boundary point coordinates, which together constitute weld state configuration data. The binary mask image of the welding spatter area is used to calibrate the spatter pixel area during the welding process. The S-shaped growth curve function is prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0072] Furthermore, the structure and parameter settings of the pre-trained fully connected neural network used in this embodiment are as follows: the number of neurons in the input layer is consistent with the length of the multimodal feature vector, set to 503, for receiving the input of the multimodal feature vector; the first hidden layer has 256 neurons, the second hidden layer has 128 neurons, and the third hidden layer has 64 neurons. All three hidden layers use a linear rectified function as the activation function, which can effectively solve the gradient vanishing problem in the neural network training process and improve the efficiency of feature decoding and mapping. Each hidden layer has a Dropout layer with a Dropout probability of 0.2 to prevent model overfitting; the output layer has 1024 neurons and uses an S-shaped growth curve function as the activation function. This function can map the output value of the neurons to the [0,1] interval to form a normalized probability distribution. The output dimension of the output layer is designed according to the feature dimension of the heat-affected zone distribution map, the binary mask image of the welding spatter area, and the weld nugget forming contour model to ensure accurate feature mapping. During the training phase, this fully connected neural network uses standard sample data of the resistance welding process as the training set. The standard sample data includes multimodal feature vectors under different welding process parameters and the corresponding true values ​​of weld state. The weights and biases of the network are iteratively optimized through the backpropagation algorithm until the model's prediction mean square error is less than 1×10⁻⁶. -4 After pre-training, it has the ability to decode and map multimodal feature vectors.

[0073] Furthermore, the process of inputting the multimodal feature vector into a pre-trained fully connected neural network for decoding and mapping includes: firstly, the multimodal feature vector is input into the input layer of the fully connected neural network, and then passed to the first hidden layer. A linear rectifier function performs a nonlinear transformation on the input feature values, retaining positive feature values ​​and setting negative feature values ​​to 0, thus achieving preliminary feature extraction and filtering; the transformed feature values ​​are then passed to the second hidden layer, and again subjected to a nonlinear transformation by a linear rectifier function for further fusion and compression of the features; subsequently, they are passed to the third hidden layer to complete the deep extraction of features, obtaining high-dimensional features that can characterize the core state of the weld; finally, the high-dimensional features are passed to the output layer, and after transformation by an S-shaped growth curve function, a normalized probability distribution vector of length 1024 is output. Each element of this vector is a value between 0 and 1, corresponding to the feature probabilities of the heat-affected zone, the welding spatter zone, and the weld nugget forming zone, respectively.

[0074] Furthermore, the process of decoding the probability distribution to form weld state configuration data includes: firstly, the probability distribution vector is parsed in blocks, and the dimensions are divided according to the fixed segmentation ratio of 512:256:256 of the 1024-dimensional vector of the output layer, dividing the probability distribution vector into three parts, corresponding to the decoding parameters of the heat-affected zone distribution map, the binary mask image of the welding spatter area, and the weld nugget forming contour model, respectively; for the decoding parameters of the heat-affected zone distribution map, the probability of each pixel belonging to the heat-affected zone is determined according to the probability value, and a probability threshold of 0.5 is set. Pixels with probability values ​​greater than the probability threshold are marked as heat-affected zone pixels. The area composed of all heat-affected zone pixels is the heat-affected zone distribution map, which contains the precise horizontal and vertical coordinates of each heat-affected zone pixel, and can directly... The spatial distribution and morphological characteristics of the heat-affected zone are observed. For the decoding parameters of the binary mask image of the welding spatter area, a probability threshold of 0.5 is set. Pixels with a probability value greater than the probability threshold are marked as spatter pixels and assigned a value of 1, while other pixels are assigned a value of 0, forming a binary mask image composed of 0 and 1. This binary mask image can accurately mark the location, range, and distribution density of spatter generated during the welding process, realizing the visualization of the spatter area. For the decoding parameters of the weld nugget forming contour model, the key boundary points of the weld nugget contour are selected according to the probability value. The coordinates of these boundary points are arranged in clockwise order to form a boundary point coordinate sequence. The boundary point coordinate sequence can accurately describe the forming contour of the weld nugget, including the morphological parameters such as the center, radius, and roundness of the weld nugget, thus constituting the weld nugget forming contour model. In the automotive steel sheet spot welding scenario of this embodiment, the heat-affected zone distribution map obtained after decoding is an annular area with an inner diameter of 8mm and an outer diameter of 20mm. The binary mask image of the welding spatter area shows that the spatter pixels are mainly distributed within a 5mm radius around the weld nugget area. The weld nugget forming contour model is a circular contour with a diameter of 7.8mm whose center coincides with the electrode contact point. Together, these three constitute the configuration data that can comprehensively characterize the weld state during the spot welding process.

[0075] It's important to understand that welding spatter is a typical phenomenon in resistance welding. The generation and distribution of spatter are directly related to the matching degree of welding current and electrode pressure. Furthermore, the presence of spatter areas indirectly affects weld nugget formation, the extent of the heat-affected zone (HAZ), and may even be a precursor to over-fusion / incomplete fusion defects. Incorporating the binary mask image of the welding spatter area into the weld state configuration data allows the model to learn the dynamic characteristics of the welding process more comprehensively, avoiding missed or false defect detections due to missing feature dimensions, and improving the model's adaptability to complex welding scenarios. In actual resistance welding production, if the welding current is too high and the electrode pressure is too low, the molten pool metal will overheat and vaporize, generating a large amount of welding spatter. This spatter carries away metal from the molten pool, resulting in incomplete weld nugget formation and easily leading to incomplete fusion defects. If the matching degree of welding current and electrode pressure is at a critical state, a small amount of spatter will occur. The distribution of the spatter area reflects the uneven stress and heating of the molten pool, which may cause the HAZ to shift, creating a hidden danger for over-fusion defects. By incorporating the binary mask image of the welding spatter area into the weld condition configuration data, the deep learning detection model can simultaneously learn the correlation features of the weld nugget, heat-affected zone, and spatter area, establishing a feature mapping relationship between spatter distribution, weld nugget formation, and defect generation. During the detection process, it can predict abnormalities in weld nugget formation based on the location and range of the spatter, and identify potential over-welding or non-fusion defects in advance. At the same time, during the defect area segmentation process, it can effectively distinguish between the spatter interference area and the real defect area, avoiding misjudging the spatter area as an over-welding or non-fusion area, thus improving the accuracy and reliability of defect detection.

[0076] S3: Based on the weld state configuration data, perform iterative optimization on the weight parameters in the preset deep learning detection model until the sum of the area ratios of the over-welded area and the non-fused area in the current detection frame output by the deep learning detection model is less than the preset first threshold, and output the detection model convergence signal.

[0077] Furthermore, the preset deep learning detection model is a convolutional neural network based on an encoder-decoder structure. This convolutional neural network structure has powerful image segmentation and feature extraction capabilities, enabling precise pixel-level segmentation of the normal welding area, over-welded area, non-fusion area, heat-affected zone, and spatter area in the weld region. It is the core model for online detection of defects in resistance welding. In this embodiment, the weld state configuration data of spot welding of automotive steel plates is used as training samples to iteratively optimize the weight parameters of the deep learning detection model. Through continuous forward inference, loss calculation, and gradient update, the defect region segmentation accuracy of the convolutional neural network model is continuously improved until the sum of the area ratios of over-welded areas and non-fusion areas in the detection frames output by the model is less than a preset first threshold, which is set to 0.01 in this embodiment. At this point, the segmentation accuracy and generalization ability of the model meet the requirements for online defect detection, and the model convergence signal is output, completing the iterative optimization of the model.

[0078] The iterative optimization of the weight parameters in the preset deep learning detection model includes:

[0079] S3.1: Initialize a convolutional neural network with an encoder-decoder structure as the deep learning detection model; wherein, the encoder includes four sequentially connected convolutional blocks, each convolutional block containing a convolutional layer, an activation layer using a linear rectified function, and a max pooling layer using a 2×2 window; the decoder includes three sequentially connected deconvolutional blocks, each deconvolutional block containing a deconvolutional layer with a deconvolution stride of 2 and an activation layer using a linear rectified function; the decoder is terminated by an output convolutional layer using a Softmax activation function, wherein the Softmax activation function is prior art in this application and is not an inventive solution of this application, and will not be described in detail here;

[0080] Furthermore, in this embodiment, the specific parameters of each convolutional block of the encoder in the deep learning detection model are set as follows: the first convolutional block uses a 3×3 convolutional kernel with a stride of 1, uniform padding, and 64 output channels. After the convolutional layer completes the initial extraction of input features, the activation layer uses a linear rectified function for nonlinear transformation to improve the model's feature representation capability. Subsequently, the max pooling layer uses a 2×2 window with a stride of 2 to downsample the transformed feature map, reducing the size of the feature map to half of its original size while retaining the key information of the feature map. The second convolutional block also uses a 3×3 convolutional kernel. The first convolutional block uses a 3×3 kernel with a stride of 1, identical padding, and 128 output channels. The activation layer is a linear rectified function, and the max pooling layer uses a 2×2 window with a stride of 2, performing a second downsampling of the feature map. The second convolutional block uses a 3×3 kernel with a stride of 1, identical padding, and 256 output channels. The activation layer is a linear rectified function, and the max pooling layer uses a 2×2 window with a stride of 2, performing a third downsampling. The third convolutional block uses a 3×3 kernel with a stride of 1, identical padding, and 512 output channels. The activation layer is a linear rectified function, and the max pooling layer uses a 2×2 window with a stride of 2, performing a fourth downsampling. Through continuous downsampling across these four convolutional blocks, the encoder progressively compresses the input weld state configuration data feature map into a high-dimensional, small-sized feature vector, achieving accurate extraction of deep weld features. The high-dimensional feature vector contains core associated features of the weld nucleus, heat-affected zone, spatter zone, and defect zone.

[0081] Furthermore, the specific parameters of each deconvolution block in the decoder are set as follows: The first deconvolution block uses a 4×4 convolution kernel, a deconvolution stride of 2, and the same padding method, with 256 output channels. The deconvolution layer upsamples the high-dimensional feature vector output by the encoder, enlarging the size of the feature map to twice its original size and restoring some spatial feature information. The activation layer uses a linear rectified function to perform a non-linear transformation on the deconvolutioned features. The second deconvolution block uses a 4×4 convolution kernel, a deconvolution stride of 2, and the same padding method, with 128 output channels. The activation layer uses a linear rectified function and performs two upsampling operations. The third deconvolution block uses a 4×4 convolution kernel, a deconvolution stride of 2, and the same padding method, with 64 output channels. The activation layer uses a linear rectified function and performs three upsampling operations. The decoder uses three deconvolutional blocks to continuously upsample the high-dimensional feature vector, gradually restoring it to a feature map with the same size as the input feature map. During the restoration process, it progressively recovers the spatial details of the weld area, achieving the fusion of deep features and spatial features. The output convolutional layer at the end of the decoder uses a 1×1 convolutional kernel with a stride of 1 and uniform padding. It has 5 output channels, corresponding to five categories: normal welding area, over-welded area, non-fused area, heat-affected zone, and welding spatter area. The output convolutional layer uses the Softmax activation function, which maps the output value of each pixel to a probability distribution of the five categories. The sum of the probability values ​​is 1. The category of each pixel can be determined by the magnitude of the probability values, achieving pixel-level classification and segmentation of the weld area.

[0082] Furthermore, the deep learning detection model initialization process includes: randomly initializing the weight parameters of all convolutional and deconvolutional layers in the encoder and decoder using the He normal initialization method, and initializing all bias parameters to 0; setting the model's learning rate to 0.001, with a step decay strategy, multiplying the learning rate by 0.9 every 100 iterations; setting the optimizer to an adaptive moment estimation algorithm, with the momentum parameter set to 0.9 and the weight decay coefficient set to 0.0001; dividing the weld state configuration data into a training set, a validation set, and a test set in a 7:2:1 ratio, where the training set is used for iterative optimization of the model, the validation set is used to monitor the model's training process to prevent overfitting, and the test set is used to evaluate the model's final performance. After completing the above settings, the deep learning detection model initialization is complete, and it enters the iterative optimization stage. The He normal initialization method is existing technology in this field and is not an inventive solution of this application, and will not be elaborated upon here.

[0083] S3.2: The heat-affected zone distribution map, the binary mask image of the welding spatter area, and the weld nugget forming contour model in the weld condition configuration data are used as inputs to the deep learning detection model. In a single iteration, the deep learning detection model outputs a predicted defect region segmentation map and calculates the loss value between the predicted defect region segmentation map and the supervision label generated based on the weld condition configuration data. The loss value is calculated by weighted summation of the cross-entropy loss function and the Dessian loss function. The backpropagation algorithm is used to calculate the gradient of each weight parameter of the deep learning detection model based on the loss value. The adaptive moment estimation algorithm is used based on the gradient. Update all weight parameters in the deep learning detection model; repeat the iteration process until the sum of the total area of ​​over-welded regions and the total area of ​​non-fused regions in the current detection frame output by the deep learning detection model is less than a first threshold in proportion to the total image area. The binary mask image of the welding spatter region is used to assist the model in distinguishing between normal molten pool regions and spatter interference regions, thereby improving the model's segmentation accuracy for over-welded and non-fused defect regions. The specific calculation formulas of the cross-entropy loss function, the Dessian loss function, the backpropagation algorithm, and the adaptive moment estimation algorithm are existing technologies in this field and are not inventive solutions of this application, and will not be elaborated here.

[0084] Furthermore, the process of realizing the predicted defect region segmentation map includes: firstly, fusing the heat-affected zone distribution map, the binary mask image of the welding spatter area, and the weld nugget forming contour model, integrating the feature information of the three into a fused feature map of size 1920×1080, which serves as the input to the deep learning detection model; after the fused feature map is input to the encoder, it undergoes convolution, activation, and downsampling operations in four convolutional blocks to extract a high-dimensional deep feature vector; this high-dimensional deep feature vector is input to the decoder, undergoing deconvolution, activation, and upsampling operations in three deconvolutional blocks to gradually recover the spatial feature information. The system obtains a feature map with the same size as the input fused feature map. This feature map is then input to the output convolutional layer, processed by the Softmax activation function, and outputs the probability distribution of each pixel belonging to five categories. The category with the highest probability value for each pixel is taken as the predicted category for that pixel. Pixels are then colored according to category: normal welding areas are blue, over-welded areas are red, non-fused areas are yellow, heat-affected zones are green, and welding spatter areas are black. This results in a visualized predicted defect region segmentation map, which intuitively demonstrates the model's pixel-level segmentation results for each category of the weld area. In the first iteration of this embodiment, the predicted defect region segmentation map output by the deep learning detection model exhibits significant segmentation errors, misclassifying some spatter areas as non-fused areas and some heat-affected zone edges as over-welded areas. The sum of the areas of over-welded and non-fused areas is 0.15, far exceeding the preset first threshold.

[0085] When calculating the loss value between the predicted defect region segmentation map and the supervision label generated based on the weld condition configuration data, the method for generating the supervision label includes:

[0086] S3.2.1: Based on the heat-affected zone distribution map and weld nugget forming contour model in the weld condition configuration data, generate a category label for each pixel by rule judgment;

[0087] S3.2.2: For pixels located inside the boundary defined by the weld nugget forming contour model, if their grayscale value is higher than the preset over-welding grayscale threshold, they are marked as over-welding areas; if their grayscale value is lower than the preset non-fusion grayscale threshold, they are marked as non-fusion areas; otherwise, they are marked as normal welding areas.

[0088] S3.2.3: For pixels located within the area defined by the heat-affected zone distribution map but not inside the weld nugget forming contour model, they are labeled as heat-affected zone categories. For pixels marked by the binary mask image of the welding spatter area, they are separately labeled as spatter area categories to exclude their interference with the determination of over-welding and non-fusion defect areas.

[0089] S3.2.4: The category labels of all pixels constitute the supervision label.

[0090] Furthermore, in this embodiment, the specific generation process of the supervision tag is based on the weld state configuration data of spot welding of automotive steel plates. The preset grayscale threshold for over-welding is 200, the grayscale threshold for non-fusion is 80, and the grayscale value range is 0-255. First, the coordinates of all pixels in the heat-affected zone distribution map, the weld nugget forming contour model, and the binary mask image of the welding spatter area are extracted to establish a pixel coordinate mapping table. For pixels inside the boundary of the weld nugget forming contour model, the grayscale value of the corresponding image frame in the real-time image sequence is extracted one by one. If the grayscale value is greater than 200, it indicates that the area is overheated due to excessive welding heat, and is marked as an over-welded area, with a category number of 1. If the grayscale value is less than 80, it indicates that the area is underheated due to insufficient welding heat, and is marked as a non-fusion area, with a category number of 1. Numbered as 2; if the gray value is between 80 and 200, it indicates that the molten pool metal in this area has melted sufficiently and formed normally, and is labeled as a normal welding area, with category number 0; for pixels within the heat-affected zone distribution map but outside the molten core forming contour model, this area is indirectly affected by welding heat and no metal melting has occurred, and is labeled as a heat-affected zone, with category number 3; for pixels marked as 1 in the binary mask image of the welding spatter area, this area is the area generated by welding spatter, not the weld body area, and is labeled as a spatter area, with category number 4. After classifying all pixels in the fused feature map according to the above rules, a pixel-level category label matrix with the same size as the fused feature map is formed. This matrix is ​​the supervision label and is the true reference standard for judging the accuracy of the prediction results during model training.

[0091] Furthermore, in the process of calculating the loss value, the weighted summation rule of the cross-entropy loss function and the Dess loss function is as follows: the weighting ratio of the cross-entropy loss value and the Dess loss value is set to 0.4:0.6. Then the final loss value is the weighted sum of the cross-entropy loss value and the Dess loss value according to the weighting ratio. This weighting ratio is determined according to the actual needs of resistance welding defect detection. The Dess loss function is more sensitive to the segmentation accuracy of the defect area. Giving it a higher weight enables the model to pay more attention to the segmentation accuracy of over-welded and non-fused defect areas during the training process. The cross-entropy loss function is used to calculate the class probability error between the predicted segmentation map and the supervision label. It can effectively measure the model's prediction accuracy for each pixel's class. For each pixel, the cross-entropy between its predicted class probability and the true class probability is calculated, and then the average of the cross-entropy values ​​of all pixels is used to obtain the overall cross-entropy loss value. The Dess loss function is used to calculate the region overlap error between the predicted defect region segmentation map and the supervision label. It is obtained by calculating the intersection-union ratio (IU) of the predicted region and the true region for each class, and then averaging the IU of all classes. The larger the IU, the smaller the loss value, and the higher the model's region segmentation accuracy. In the first iteration of this embodiment, the calculated cross-entropy loss value is 0.85, the Dess loss value is 0.92, and the total loss value calculated according to the weighted rule is 0.892. This loss value reflects a large error between the model's prediction result and the true label, which needs to be optimized through backpropagation and parameter updates.

[0092] Furthermore, the process of calculating the gradients of each weight parameter of the deep learning detection model based on the loss value using the backpropagation algorithm includes: using the total loss value as the objective function, starting from the output convolutional layer of the deep learning detection model, and following the decoder-encoder order, calculating the partial derivatives of the objective function with respect to each weight parameter and bias parameter. These partial derivatives are the gradients of the parameters; the direction of the gradient indicates the direction in which the parameter needs adjustment, and the magnitude of the gradient indicates the magnitude of the parameter adjustment. During backpropagation, the gradients of the linear rectified function and the softmax activation function are calculated pixel-by-pixel. For the max pooling layer, gradient backpropagation is used, retaining the gradient of the pixel corresponding to the maximum value during downsampling, and setting the gradients of other pixels to 0, ensuring that the gradients can be accurately propagated back to each convolutional layer of the encoder. Simultaneously, a gradient clipping strategy is introduced during gradient calculation, limiting the maximum value of the gradient to 5 to prevent model training instability due to gradient explosion.

[0093] Furthermore, the process of updating all weight parameters in the deep learning detection model using the adaptive moment estimation algorithm based on the gradient includes: the adaptive moment estimation algorithm can dynamically adjust the learning rate for each parameter according to the gradient mean and gradient variance, making the model training process more stable and the convergence speed faster. During training, the adaptive moment estimation algorithm continuously accumulates the first-order moment estimate and second-order moment estimate of each parameter. The first-order moment estimate is the exponential moving average of the gradient, reflecting the gradient change trend of the parameter, and the second-order moment estimate is the exponential moving average of the squared gradient, reflecting the gradient fluctuation of the parameter. Combining the first-order moment estimate, the second-order moment estimate, and the preset learning rate, the update step size of each parameter is calculated. Then, according to the gradient direction, the weight parameters and bias parameters are updated element-wise. The update formula is: the new parameter is the difference between the old parameter and the value of the first variable, where the first variable value is the product of the learning rate and the value of the second variable, and the second variable value is the quotient of the first-order moment estimate and the value of the third variable, where the third variable value is the sum of the square root of the second-order moment estimate and a minimum value, where the minimum value is set to 1×10. -8 To prevent the denominator from being zero, in the first iteration of this embodiment, all weight parameters and bias parameters of the model are updated using an adaptive moment estimation algorithm. After the update, the feature extraction and segmentation capabilities of the model are initially improved.

[0094] In this embodiment, after 500 iterations, the sum of the area ratios of over-welded and non-fused regions in the detection frames output by the model drops to 0.008, which is lower than the preset first threshold of 0.01. At this point, the model's segmentation accuracy and generalization ability meet the requirements, the iterative optimization stops, and the detection model convergence signal is output. Throughout the iterative optimization process, the binary mask image of the welding spatter area serves as an important input feature, enabling the model to clearly identify the spatter interference area. By classifying the spatter area separately during segmentation, the model effectively avoids misclassifying the spatter area as an over-welded or non-fused region, allowing the model to more accurately focus on defect detection in the weld body area and improving the segmentation accuracy of over-welded and non-fused defect areas.

[0095] S4: Based on the received detection model convergence signal, perform real-time weld defect detection on the resistance welding process at a preset frame rate, and collect pixel-level molten pool feature data for each detection frame during the detection process; the pixel-level molten pool feature data includes the gray value of each pixel and an over-welding mark or non-fusion mark determined based on the gray value threshold.

[0096] Furthermore, the preset frame rate is adapted to the welding cycle of the welding process. In this embodiment, the welding cycle of spot welding of automotive steel plates is 200ms. To ensure that the dynamic changes of the molten pool during the welding process can be captured in real time, the preset frame rate for real-time weld defect detection is set to 100 frames / second, that is, one frame of detection image is acquired every 10ms to complete one defect detection and feature data acquisition. After receiving the convergence signal of the detection model, the high-speed industrial camera device continuously captures real-time images of the welding area at the preset frame rate, which are used as detection frames and input to the iteratively optimized deep learning detection model. The model performs real-time forward inference on each detection frame to complete pixel-level defect segmentation and recognition, and simultaneously acquires pixel-level molten pool feature data of each detection frame, realizing online, real-time, and accurate defect detection in the resistance welding process, meeting the online detection needs in industrial production.

[0097] The method involves real-time weld defect detection during resistance welding at a preset frame rate, and collecting pixel-level molten pool feature data for each detection frame during the detection process, including:

[0098] S4.1: When the convergence signal of the detection model is received, the image acquisition device is started to capture real-time images of the welding area at a preset frame rate. For each detection frame image acquired, the deep learning detection model that has been iteratively optimized is called to perform forward inference, and the category label of each pixel in the corresponding detection frame image is output as belonging to the normal welding area, the over-welded area or the non-fused area.

[0099] Furthermore, the linkage control process between the image acquisition device startup and the welding process includes: using the detection model convergence signal as a trigger signal, transmitting it to the control system of the resistance welding equipment and the control module of the high-speed industrial camera device. Upon receiving this signal, the control system sends an acquisition start command to the camera device control module simultaneously with the start of the welding process, ensuring that the camera device operates synchronously with the welding process. The camera device continuously captures real-time images of the welding area at a preset frame rate of 100 frames per second. Each detection frame image undergoes real-time preprocessing, including grayscale conversion, Gaussian filtering, and size normalization, unifying the image size to 1920×1080, consistent with the input size during model training. The preprocessed detection frame images are then transmitted in real-time to the inference end of the deep learning detection model via industrial Ethernet. The inference module calls the iteratively optimized model to perform forward inference on the detection frame images. The inference process is consistent with the forward inference process during model training. After encoder feature extraction, decoder feature recovery, and output convolutional layer class probability calculation, the module outputs a class label for each pixel, classifying it as a normal welded area, an over-welded area, or an unfused area. The class labels for heat-affected zones and spatter areas are masked at this stage, retaining only the three types of labels directly related to defect detection, simplifying feature data acquisition and defect determination. In this embodiment, the model's inference time for each detection frame is controlled within 8ms, less than the 10ms acquisition interval of the detection frames, ensuring real-time defect detection with zero latency.

[0100] S4.2: According to the category label, traverse all pixels in the detection frame image, record the gray value of each pixel, and add an over-soldering label to the pixels classified as over-soldering areas and an un-fused label to the pixels classified as unfused areas.

[0101] Furthermore, the process of traversing pixels and recording grayscale values, and adding labels, includes: using the pixel coordinates of the detection frame image as an index, traversing all pixels in a left-to-right, top-to-bottom order, and establishing a pixel feature information table. This table contains five fields: pixel x-coordinate, y-coordinate, category label, grayscale value, and feature label. For each pixel, firstly, its actual grayscale value in the detection frame image is extracted and filled into the grayscale value field; then, based on its category label, it is labeled in the feature label field. If the category label is an over-soldered area, an over-soldered label is added, with the symbol "H"; if the category label is an unfused area, an unfused label is added, with the symbol "U"; if the category label is a normal soldered area, the feature label field is empty. During the traversal, a line-by-line scanning method is used to ensure that the information of each pixel is accurately recorded without omission or duplication. For a detection frame image with a resolution of 1920×1080, a total of 2,073,600 pixels are contained. The entire traversal and recording process is completed within 2ms, ensuring real-time performance. In this embodiment, within a certain detection frame, 1200 pixels in the fusion core area are marked as over-welded areas with the additional label "H", 800 pixels are marked as unfused areas with the additional label "U", and the remaining pixels are normal welding areas without additional labels. The grayscale value and label of all pixels are accurately recorded in the pixel feature information table.

[0102] S4.3: Summarize the grayscale values ​​of all pixels and their corresponding identifiers to generate pixel-level melt pool feature data.

[0103] Furthermore, the process of generating pixel-level molten pool feature data includes: First, cleaning the pixel feature information table to remove pixel information from non-molten pool areas such as the heat-affected zone and spatter area, retaining only the pixel information within the molten core area. This is because the molten core area is the main location for over-welding and non-fusion defects, and its pixel features can accurately reflect the forming state and defect status of the molten pool. Then, the cleaned pixel information is sorted according to the horizontal and vertical axes to form an ordered pixel feature dataset. Finally, this dataset is packaged into a standard industrial data format, including a data header, data body, and data tail. The data header records the detection frame number, acquisition timestamp, and welding process parameters; the data body contains the grayscale value and corresponding identifier of each pixel; and the data tail records the total number of pixels in the molten pool area, the number of over-welding identifier pixels, and the number of non-fusion identifier pixels. The packaged dataset is the pixel-level molten pool feature data. This data can accurately characterize the grayscale distribution, defect location, and defect range of the molten pool at the pixel level, serving as the core data basis for subsequent defect judgment and welding process parameter adjustment. In this embodiment, the generated pixel-level molten pool feature data accurately records the dynamic changes of the molten pool during spot welding. When the welding current fluctuates, the gray value distribution of the molten pool area changes significantly, and the number of pixels indicating over-welding or non-fusion also changes accordingly, which can reflect the abnormality of molten pool formation in a timely manner.

[0104] The method further includes:

[0105] A1: Based on the pixel-level weld pool feature data, calculate the average gray value of all pixels with over-soldering marks in the current detection frame as the average gray value of the over-soldering area, and calculate the average gray value of all pixels with non-fusion marks as the average gray value of the non-fusion area.

[0106] Furthermore, the process of calculating the average grayscale of the soldered area and the average grayscale of the unfused area includes: First, extracting the grayscale values ​​of all pixels with soldered markings from the pixel-level molten pool feature data to form a set of soldered grayscale values. The arithmetic mean of all values ​​in the set of soldered grayscale values ​​is then calculated, which is the average grayscale of the soldered area. The arithmetic mean is calculated to two decimal places. Next, extracting the grayscale values ​​of all pixels with unfused markings to form a set of unfused grayscale values, and calculating its arithmetic mean using the same method, which is the average grayscale of the unfused area. If there are no pixels with soldered markings in the pixel-level molten pool feature data, the average grayscale of the soldered area is recorded as 0; if there are no pixels with unfused markings, the average grayscale of the unfused area is recorded as 0. In this embodiment, the set of grayscale values ​​for any detection frame that is over-soldered contains 1200 grayscale values, and the average grayscale value of the over-soldered area is calculated to be 225.68; the set of grayscale values ​​for the un-fused area contains 800 grayscale values, and the average grayscale value of the un-fused area is calculated to be 65.32. The two average grayscale values ​​accurately reflect the difference in grayscale characteristics between the over-soldered area and the un-fused area. The over-soldered area exhibits high grayscale characteristics due to excessive heat, while the un-fused area exhibits low grayscale characteristics due to insufficient heat.

[0107] A2: Calculate the ratio of the average grayscale of the welded area to the average grayscale of the unfused area to obtain the real-time grayscale ratio, and determine whether the real-time grayscale ratio is less than a preset second threshold; if the real-time grayscale ratio is less than the second threshold, generate and output the defect detection result of the current detection frame based on the output of the current deep learning detection model and the preset defect judgment rule; if the real-time grayscale ratio is greater than or equal to the second threshold, calculate the welding current compensation amount and the electrode pressure adjustment amount according to the pixel-level molten pool feature data, send the welding current compensation amount to the welding power controller, send the electrode pressure adjustment amount to the electrode pressure servo mechanism, and after the welding process parameters are updated, return to execute the real-time weld defect detection of the resistance welding process at a preset frame rate until the real-time grayscale ratio is less than the second threshold.

[0108] Furthermore, in this embodiment, based on the process characteristics and defect detection requirements of spot welding of automotive steel plates, a second threshold of 2.5 is preset. The calculation accuracy of the real-time grayscale ratio is retained to two decimal places. The calculation method is as follows: the real-time grayscale ratio is the quotient of the average grayscale of the over-welded area and the average grayscale of the non-fused area. If the average grayscale of the non-fused area is 0, the real-time grayscale ratio is recorded as infinity and is determined to be greater than the second threshold. The real-time grayscale ratio can reflect the degree of difference in grayscale characteristics between the over-welded area and the non-fused area. When the ratio is less than 2.5, it indicates that the heat distribution in the molten pool area is relatively uniform, and the characteristics of over-welding and non-fusion defects have tended to stabilize, and defect judgment can be performed. When the ratio is greater than or equal to 2.5, it indicates that the heat distribution in the molten pool area is severely uneven, and the over-welding and non-fusion defects are still in a dynamic development process. At this time, it is necessary to adjust the welding process parameters in time to improve the heating and stress state of the molten pool, so that the defect characteristics tend to stabilize, and then perform defect detection. In this embodiment, the real-time grayscale ratio of any detection frame is 225.68 / 65.32≈3.46, which is greater than the preset second threshold of 2.5. This indicates that the heat distribution of the molten pool is uneven at this time, and it is necessary to calculate the welding current compensation amount and the electrode pressure adjustment amount to adjust the welding process parameters in real time.

[0109] Based on the output of the current deep learning detection model and the preset defect judgment rules, the defect detection result of the current detection frame is generated and output, including:

[0110] A2.1: Call the deep learning detection model to infer the current detection frame and obtain a pixel-level classification result map. In the pixel-level classification result map, identify all connected regions formed by pixels with over-soldering labels and count the area of ​​each connected region. Identify all connected regions formed by pixels with unfused labels and count the area of ​​each connected region.

[0111] Further, the process of identifying connected regions and calculating their areas includes: first, calling a deep learning detection model to perform further reasoning on the current detection frame to obtain a pixel-level classification result image containing normal welding areas, over-welded areas, and non-fused areas. This pixel-level classification result image is a binary classification mask image. Then, a connected region analysis algorithm is used to process the pixel-level classification result image. This algorithm divides adjacent pixels of the same category into a connected region based on the neighborhood relationship of pixels. A neighborhood is defined as an 8-neighborhood, meaning that pixels in the eight directions above, below, left, right, upper left, upper right, lower left, and lower right of a pixel are its neighboring pixels. For over-welded areas, all connected regions composed of pixels with over-welded identifiers are identified, each connected region is assigned a unique number, and the number of pixels contained in each connected region is counted. Based on the pixel resolution of the detection frame image (0.01 mm / pixel in this embodiment), the number of pixels is converted into an actual area value in mm². The same method is used to identify, number, and calculate the area of ​​connected regions in non-fused areas. If there are no pixels with signs of over-soldering or non-fusion in the pixel-level classification result image, it is determined that there is no corresponding connected region, and the area is recorded as 0. In this embodiment, in the pixel-level classification result image of any detection frame, the over-soldering area forms two connected regions with areas of 1.2 mm² and 0.8 mm², respectively, and the non-fusion area forms one connected region with an area of ​​2.5 mm². The position and area of ​​all connected regions are accurately identified and counted. The connected region analysis algorithm is prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0112] A2.2: If the area of ​​any over-welded indicator connected region exceeds the first area threshold, an over-welded defect is determined to exist. If the area of ​​any non-fused indicator connected region exceeds the second area threshold, a non-fused defect is determined to exist. Based on the determination result, a defect detection result report containing the defect type, the contour coordinates of the connected region where the defect is located, and the defect area is generated.

[0113] Furthermore, in this embodiment, based on the quality acceptance standards for spot welding of automotive body steel plates, a first area threshold of 1.0 mm² and a second area threshold of 2.0 mm² are preset. The areas of the connected regions of the statistically obtained over-welded areas are judged one by one. If the area of ​​any connected region is greater than 1.0 mm², the region is determined to have an over-welded defect. Similarly, the areas of the connected regions of the non-fused regions are judged one by one. If the area of ​​any connected region is greater than 2.0 mm², the region is determined to have a non-fused defect. If multiple defects exist, they are judged and recorded separately. The defect detection result report adopts a standardized document format, mainly including three parts: basic detection information, defect judgment results, and detailed defect information. The basic detection information includes the detection frame number, acquisition timestamp, and welding process parameters; the defect judgment results include whether a defect exists and the defect type; and the detailed defect information includes the connected region number, contour coordinates, defect area, and defect location description for each type of defect. In this embodiment, the area of ​​the first connected region of the over-welded area is 1.2 mm², which is greater than the first area threshold of 1.0 mm², indicating the presence of an over-welded defect; the area of ​​the connected region of the non-fused region is 2.5 mm², which is greater than the second area threshold of 2.0 mm², indicating the presence of a non-fused defect; the generated defect detection result report accurately records the contour coordinates and area of ​​the over-welded defect and the contour coordinates and area of ​​the non-fused defect, and the defect location is located at the center of the fusion nugget region.

[0114] A2.3: Record the defect detection results of the current detection frame, including defect type, defect area, and defect location; accumulate and statistically analyze the defect detection results of all detection frames within the same workpiece welding cycle, and calculate the total area ratio of over-welding defects and the total area ratio of non-fusion defects within that cycle; associate and store the accumulated statistical results with the welding process parameters used in the current welding cycle to form a process-defect association database; based on the process-defect association database, use statistical analysis algorithms to determine the sensitive range of process parameters that cause various defects.

[0115] Further, the specific process of cumulatively statistics the defect detection results of all detection frames within the welding cycle of the same workpiece includes: First, assign a unique welding cycle number to each workpiece, record the defect detection results of all detection frames within this welding cycle, classify and summarize them by defect type, and calculate the sum of the areas of all connected regions of the overwelding defect, which is the total area of the overwelding defect, and calculate the sum of the areas of all connected regions of the lack of fusion defect, which is the total area of the lack of fusion defect; Subsequently, calculate the actual total area of the fusion nucleus region. In this embodiment, the standard total area of the fusion nucleus in the spot welding of automotive steel plates is 49 mm²; Calculate the ratios of the total area of the overwelding defect and the total area of the lack of fusion defect to the actual total area of the fusion nucleus region respectively, which are the proportion of the total area of the overwelding defect and the proportion of the total area of the lack of fusion defect. The proportion results are reserved to two decimal places and expressed in percentage form. In this embodiment, within any welding cycle, the total area of the overwelding defect is 3.5 mm², the total area of the lack of fusion defect is 4.2 mm², and the actual total area of the fusion nucleus region is 48 mm². It is calculated that the proportion of the total area of the overwelding defect is 7.29%, and the proportion of the total area of the lack of fusion defect is 8.75%.

[0116] Further, the process of forming the process-defect association database includes: establishing a structured data table of the process-defect association database. The fields of the structured data table include welding cycle number, workpiece information, welding process parameter combination, total area of overwelding defect, proportion of total area of overwelding defect, total area of lack of fusion defect, proportion of total area of lack of fusion defect, and comprehensive defect evaluation. The welding process parameter combination field records the specific values of the welding current, electrode pressure, and welding time used in this welding cycle; The comprehensive defect evaluation field is evaluated according to the proportion of the total area of defects. If the sum of the proportions of the total areas of overwelding and lack of fusion defects is less than 5%, it is evaluated as qualified. If the sum of the proportions is between 5% and 10%, it is evaluated as pending re-inspection. If the sum of the proportions is greater than 10%, it is evaluated as unqualified. Associate the cumulative statistical results of the same welding cycle with the corresponding welding process parameter combination one by one and fill them into the structured data table. As the welding production continues, new records are continuously added to the data table to form an increasingly rich process-defect association database. The process-defect association database can intuitively reflect the association relationship between different welding process parameter combinations and weld defects.

[0117] Furthermore, based on the aforementioned process-defect correlation database, the process of determining the sensitive intervals of process parameters that lead to various defects using statistical analysis algorithms includes: processing the data in the process-defect correlation database using a multivariate statistical analysis algorithm. The multivariate statistical analysis algorithm includes two steps: correlation analysis and interval division. First, correlation analysis is performed to calculate the correlation coefficients between the three process parameters—welding current, electrode pressure, and welding time—and the proportion of the total area of ​​over-welding defects and the proportion of the total area of ​​non-fusion defects, respectively. The range of the correlation coefficient is [-1, 1]. The larger the absolute value, the stronger the correlation between the process parameter and the defect. A positive correlation indicates that the larger the process parameter value, the higher the proportion of defects, while a negative correlation indicates that the larger the process parameter value, the lower the proportion of defects. In this embodiment, correlation analysis revealed that the correlation coefficient between welding current and the proportion of over-welding defects was 0.85, showing a strong positive correlation, while the correlation coefficient with the proportion of non-fusion defects was -0.78, showing a strong negative correlation. The correlation coefficient between electrode pressure and the proportion of over-welding defects was -0.82, showing a strong negative correlation, while the correlation coefficient with the proportion of non-fusion defects was 0.75, showing a strong positive correlation. The correlation coefficient between welding time and the proportion of over-welding defects was 0.65, showing a moderate positive correlation, while the correlation coefficient with the proportion of non-fusion defects was -0.60, showing a moderate negative correlation. Subsequently, interval division was performed. Based on the results of the correlation analysis, using a satisfactory comprehensive defect evaluation as the standard, all records evaluated as satisfactory in the process-defect correlation database were selected. The process parameter values ​​in these records were extracted, and the minimum and maximum values ​​of welding current, electrode pressure, and welding time were calculated. This range represents the satisfactory interval for the corresponding process parameter. Using a dissatisfaction-based comprehensive defect evaluation as the standard, all records evaluated as dissatisfaction were selected, and the process parameter values ​​were extracted to divide the sensitive intervals of process parameters leading to over-welding and non-fusion defects. In this embodiment, the sensitive range of process parameters leading to over-welding defects is determined to be: welding current > 8.5kA, electrode pressure < 3.0MPa, and welding time > 220ms; the sensitive range of process parameters leading to non-fusion defects is: welding current < 7.5kA, electrode pressure > 4.0MPa, and welding time < 180ms. This sensitive range of process parameters can provide a clear reference for setting process parameters in resistance welding production, effectively avoiding weld defects caused by improper parameter settings and improving the stability of welding quality. Correlation analysis is prior art in this field and is not an inventive solution of this application, and will not be elaborated upon here.

[0118] The welding current compensation and electrode pressure adjustment are calculated based on the pixel-level molten pool feature data, including:

[0119] B1: Based on the pixel-level molten pool feature data, the total number of pixels with over-soldering markings and the total number of pixels with non-fusion markings are counted. The difference between the pixels in the region is calculated by the difference. At the same time, the absolute difference between the average gray level of the over-soldering region and the average gray level of the non-fusion region is calculated to obtain the absolute gray level difference.

[0120] Furthermore, the calculation process for the regional pixel difference and the absolute difference in grayscale includes: firstly, extracting the total number of pixels with over-soldering markers from the pixel-level weld pool feature data, denoted as N. H The total number of pixels with unfused markings is denoted as N. U Regional pixel difference N T The calculation method for N is as follows: T =N H -N U This value reflects the difference in pixel count between the soldered and un-fused areas. A positive value indicates that the soldered area has more pixels than the un-fused area, suggesting that the overall heat of the molten pool is too high. A negative value indicates that the un-fused area has more pixels than the soldered area, suggesting that the overall heat of the molten pool is insufficient. Subsequently, the average grayscale G of the soldered area is calculated. H Average gray level G of the unfused region U The absolute difference, the absolute difference in grayscale G T The calculation method is as follows: This value reflects the degree of difference in grayscale characteristics between the welded area and the unfused area; the larger the value, the more uneven the heat distribution of the molten pool. In this embodiment, N H =1200, N U =800, N T =1200-800=400; G H =225.68, G U =65.32, G T =|225.68-65.32|=160.36, the regional pixel difference is positive, indicating that the overall heat of the molten pool is too high. The large absolute difference in grayscale indicates that the heat distribution of the molten pool is seriously uneven, and the welding current and electrode pressure need to be adjusted accordingly.

[0121] B2: The pixel difference in the region and the absolute difference in grayscale are respectively input into a dual-channel proportional-integral controller; the first channel of the dual-channel proportional-integral controller takes the pixel difference in the region as the first input, and obtains a first control quantity through proportional and integral operations, which serves as the welding current compensation quantity; the second channel of the dual-channel proportional-integral controller takes the absolute difference in grayscale as the second input, and obtains a second control quantity through proportional and integral operations, which serves as the electrode pressure adjustment quantity; the proportional operation is to multiply the first input or the second input by a preset proportional coefficient, and the integral operation is to perform discrete-time integration on the first input or the second input and then multiply it by a preset integral coefficient.

[0122] Furthermore, the dual-channel proportional-integral controller used in this embodiment is specifically designed for adjusting resistance welding process parameters. It has two independent control channels, corresponding to the calculation of welding current compensation and electrode pressure adjustment, respectively. The sampling period of the controller is consistent with the acquisition period of the detection frame, set to 10ms, to ensure real-time adjustment of process parameters. The first channel of the controller has a preset proportional coefficient of -0.002 and an integral coefficient of -0.001. Both the proportional and integral coefficients are negative because when the regional pixel difference is positive, the welding current needs to be reduced to cool the molten pool. The second channel has a preset proportional coefficient of 0.0015 and an integral coefficient of 0.0008. Both the proportional and integral coefficients are positive because the larger the absolute difference in grayscale, the greater the electrode pressure needs to be, improving the stress state of the molten pool, promoting uniform melting of the molten metal, and reducing the degree of uneven heat distribution.

[0123] Furthermore, the process of calculating the welding current compensation in the first channel includes: firstly, using the regional pixel difference as the first input, performing a proportional calculation, the result of which is the product of the regional pixel difference and the proportional coefficient; then, performing a discrete-time integration calculation on the regional pixel difference using the rectangular integration method, the result of which is the sum of the current regional pixel difference and the accumulated historical regional pixel differences, multiplied by the integration coefficient; finally, summing the proportional calculation result and the integral calculation result, the sum of which is the first control quantity, i.e., the welding current compensation, in kA, and the result is rounded to three decimal places. During the proportional-integral calculation, the proportional calculation enables rapid adjustment based on the current regional pixel difference, while the integral calculation eliminates static errors, making the welding current adjustment more precise and stable. In this embodiment, the current area pixel difference is 400, and the cumulative value of the historical area pixel difference is 100. The proportional calculation result is: 400×(-0.002)=-0.8kA, the integral calculation result is: (400+100)×(-0.001)=-0.5kA, and the welding current compensation is: -0.8+(-0.5)=-1.3kA. The negative sign indicates that the welding current needs to be reduced, that is, adjusted from the current 8.2kA to 6.9kA.

[0124] Furthermore, the process of calculating the electrode pressure adjustment in the second channel is consistent with the principle of the first channel. The absolute difference of grayscale is used as the second input. First, a proportional operation is performed: the result of the proportional operation is the product of the absolute difference of grayscale and the proportional coefficient. Then, a discrete-time integral operation is performed: the result of the integral operation is the sum of the current absolute difference of grayscale and the accumulated value of the historical absolute difference of grayscale, which is then multiplied by the integral coefficient. Finally, the proportional operation result and the integral operation result are summed, and the sum is the second control quantity, which is the electrode pressure adjustment quantity, in MPa. The result is retained to three decimal places. In this embodiment, the current absolute grayscale difference is 160.36, and the cumulative historical absolute grayscale difference is 40.64. Therefore, the proportional calculation result is: 160.36 × 0.0015 ≈ 0.2405 MPa, the integral calculation result is: (160.36 + 40.64) × 0.0008 = 201 × 0.0008 = 0.1608 MPa, and the electrode pressure adjustment amount is: 0.2405 + 0.1608 ≈ 0.401 MPa. The positive sign indicates that the electrode pressure needs to be increased, that is, adjusted from the current 3.4 MPa to 3.801 MPa.

[0125] Furthermore, after calculating the welding current compensation and electrode pressure adjustment, these values ​​are sent to the welding power controller and electrode pressure servo mechanism respectively via industrial Ethernet. Upon receiving the welding current compensation, the welding power controller immediately dynamically adjusts the output current, adjusting the power output in real time according to the compensation value to ensure the welding current is accurately adjusted to the target value. Upon receiving the electrode pressure adjustment, the electrode pressure servo mechanism controls the air intake and exhaust of the cylinder to change the pressure of the electrode on the workpiece, achieving precise adjustment of the electrode pressure. The entire adjustment process is completed within 5ms, ensuring real-time updates of process parameters.

[0126] Furthermore, after the welding process parameters are updated, the system immediately returns to the real-time weld defect detection step S4. The high-speed industrial camera continues to capture real-time images of the welding area at a preset frame rate of 100 frames per second. The deep learning detection model performs forward inference on the new detection frame, re-acquires pixel-level molten pool feature data, and re-executes the calculations and judgments of steps A1 and A2. In this embodiment, after adjusting the process parameters, the welding current is adjusted to 6.9kA, and the electrode pressure is adjusted to 3.801MPa. The heat distribution in the molten pool area is significantly improved. The recalculated average grayscale of the over-welded area is 190.25, and the average grayscale of the non-fused area is 85.68. The real-time grayscale ratio is 190.25 / 85.68≈2.22, which is less than the preset second threshold of 2.5. At this point, the defect characteristics of the molten pool tend to stabilize, the process parameter adjustment stops, and the system formally enters the defect judgment stage. Based on the output of the deep learning detection model and the preset defect judgment rules, the defect detection result of the current detection frame is generated and output.

[0127] In the actual spot welding production process of automotive body steel plates, the online defect detection method for resistance welding of this invention can achieve real-time detection and dynamic process adjustment throughout the entire process. From the synchronous acquisition of images and process parameters during the welding process, to the construction of multimodal feature fields and the generation of weld state configuration data, to the iterative optimization of the deep learning detection model and real-time defect detection, and finally to the dynamic compensation of process parameters and accurate defect judgment through pixel-level molten pool feature data, a closed-loop online detection and process optimization system is formed. This method overcomes the limitations of traditional resistance welding defect detection, such as offline detection, low efficiency, and poor accuracy, achieving pixel-level real-time defect identification. It can accurately detect typical weld defects such as over-welding and lack of fusion. Simultaneously, by dynamically adjusting the welding current and electrode pressure, it effectively improves the molten pool formation state and reduces defect generation.

[0128] Furthermore, by constructing a process-defect correlation database, the inherent correlation between different combinations of welding process parameters and weld defects can be explored, and the sensitive range of process parameters can be determined, providing data support for the optimization and standardization of resistance welding processes. In the automotive steel sheet spot welding scenario of this embodiment, the application of this method significantly improved the detection accuracy of weld over-welding and incomplete fusion defects, reduced the defect omission rate and false detection rate, and the dynamic adjustment of welding process parameters reduced the weld defect generation rate by more than 60%, greatly improving the stability of resistance welding quality and production efficiency, and meeting the high-precision and high-real-time requirements of online inspection of resistance welding weld quality in high-end equipment manufacturing industries such as automobile manufacturing.

[0129] It should be noted that this embodiment only takes the typical resistance welding application scenario of spot welding of automotive body steel plates as an example to elaborate on the online detection method for weld defects in resistance welding of the present invention. In practical applications, this method can adaptively adjust relevant parameters such as acquisition frame rate, grayscale threshold, area threshold, proportional coefficient, integral coefficient, etc. according to different resistance welding process types, such as seam welding, projection welding, butt welding, etc., as well as different workpiece materials, such as aluminum alloy, stainless steel, copper alloy, etc., or workpiece thickness. At the same time, the deep learning detection model and the fully connected neural network are retrained and optimized in a targeted manner, so as to achieve accurate online detection of weld defects in different resistance welding scenarios.

[0130] For example, in the resistance welding scenario of aluminum alloy plates, due to the good thermal conductivity and low melting point of aluminum alloys, the dynamic changes of the molten pool during welding are faster. The frame rate of the high-speed industrial camera can be increased to 300 frames / second, while the over-welding grayscale threshold can be appropriately reduced, such as setting it to 180, because the grayscale value of the aluminum alloy molten pool when overheated is lower than that of steel. In the resistance welding scenario of stainless steel plates, due to the high resistivity and large coefficient of thermal expansion of stainless steel, spatter is easily generated during welding. When constructing the multimodal feature field, the extraction of spatter area density features can be increased. At the same time, in the training of the deep learning detection model, the sample weight of the spatter area can be increased to further improve the model's segmentation accuracy of the defect area. Regardless of the application scenario, the core technical framework of this invention remains unchanged. It achieves online detection of resistance weld defects through the technical route of multimodal feature fusion, deep learning model optimization, pixel-level feature acquisition, and dynamic adjustment of process parameters, and has strong scenario adaptability and engineering practicality.

[0131] Example 2:

[0132] Please see Figure 3 Another embodiment of the present invention provides an online detection system for weld defects in resistance welding, comprising:

[0133] The data acquisition module 10 is used to synchronously acquire real-time image sequences and welding process parameters of the resistance welding process, and at the same time realize the alignment of the timestamps of the process parameters and image frames to ensure the spatiotemporal consistency of the data.

[0134] The feature construction module 20 is used to perform inter-frame difference, watershed segmentation, and geometric prior recognition on real-time image sequences to extract coordinate data of heat-affected zones and weld nugget candidate regions. The region coordinate data and welding process parameter time-series vectors are fused together to generate multimodal feature vectors, which are then decoded and mapped by a pre-trained fully connected neural network. Finally, the output includes weld state configuration data containing heat-affected zone distribution map, binary mask image of welding spatter area, and weld nugget forming contour model.

[0135] The model iteration optimization module 30 uses a convolutional neural network with an initialized encoder-decoder structure as a deep learning detection model. It generates supervision labels, calculates cross-entropy loss and Dess loss, backpropagates to find gradients, and updates weights using an adaptive moment estimation algorithm. It iteratively optimizes the model weight parameters until the sum of the proportions of the over-welded and unfused areas is lower than a preset first threshold, and outputs a converged high-precision deep learning detection model.

[0136] The defect detection module 40 is used to capture real-time images of the welding area at a preset frame rate after receiving the model convergence signal, call the optimized deep learning detection model to perform forward inference on each detection frame, output pixel-level category labels, then traverse the pixels to record gray values, attach corresponding labels to defect pixels, and summarize to generate pixel-level molten pool feature data to realize real-time pixel-level defect identification in the welding process.

[0137] The parameter control module 50 calculates the real-time grayscale ratio of the welded / unfused areas based on pixel-level molten pool feature data. If the real-time grayscale ratio is greater than or equal to the second threshold, it counts the pixel difference and absolute grayscale difference in the area. The welding current compensation and electrode pressure adjustment are calculated by the dual-channel proportional-integral controller, and the adjustment signals are sent to the welding power controller and electrode pressure servo mechanism to realize dynamic closed-loop control of welding process parameters and reduce the generation of defects in the process.

[0138] The data storage module 60 is used to calculate the area of ​​the welded / unfused connected region based on the pixel-level classification result image of the deep learning detection model when the real-time grayscale ratio is less than the second threshold, determine the defect type according to the preset defect judgment rules, generate a defect detection result report containing defect type, contour coordinates and defect area, and record all defect data in the welding cycle, which are associated with the corresponding welding process parameters and stored to build a process-defect association database.

[0139] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments under the guidance of the present invention without departing from the spirit and scope of the present invention. All of these variations are within the protection scope of the present invention.

Claims

1. A method for on-line detection of a weld defect for resistance welding, characterized by, include: The system acquires a real-time image sequence of the resistance welding process of the workpiece to be inspected, and simultaneously collects the corresponding welding process parameters; the welding process parameters include at least welding current, electrode pressure, and welding time. Based on the real-time image sequence and welding process parameters, a multimodal feature field of the welding process is constructed to obtain weld state configuration data; Based on the weld state configuration data, iterative optimization is performed on the weight parameters in the preset deep learning detection model until the sum of the area ratios of the over-welded area and the non-fused area in the current detection frame output by the deep learning detection model is less than the preset first threshold, and the detection model convergence signal is output. Based on the received convergence signal of the detection model, the resistance welding process is subjected to real-time weld defect detection at a preset frame rate, and pixel-level molten pool feature data of each detection frame is collected during the detection process; the pixel-level molten pool feature data includes the gray value of each pixel and over-welding or non-fusion markers determined based on the gray value threshold.

2. The method for on-line inspection of a resistance welding seam defect according to claim 1, wherein The process of constructing a multimodal feature field of the welding process based on the real-time image sequence and welding process parameters to obtain weld state configuration data includes: Inter-frame difference calculation is performed on consecutive image frames in the real-time image sequence. The dynamic welding area is located based on the pixel grayscale change. The image of the located dynamic welding area is then processed by a watershed image segmentation algorithm based on morphological gradient and preset label to obtain a preliminary segmentation result image containing multiple candidate regions. Then, combined with the geometric prior knowledge that the weld area is distributed in a concentric ring shape, the contour coordinate data of the heat-affected zone and the boundary point coordinate data of the weld nugget candidate region are identified from the preliminary segmentation result image. The instantaneous values ​​of welding current, electrode pressure, and welding time are matched with each frame of the real-time image sequence using the timestamp alignment method to generate a welding process parameter time vector synchronized with each image frame. The contour coordinate data of the heat-affected zone, the boundary point coordinate data of the weld nugget candidate zone, and the time-series vector of the welding process parameters are cascaded and fused to generate a multimodal feature vector.

3. The method for on-line inspection of a resistance welding seam defect according to claim 2, wherein The step of constructing a multimodal feature field of the welding process based on the real-time image sequence and welding process parameters to obtain weld state configuration data also includes: The multimodal feature vectors are input into a pre-trained fully connected neural network for decoding and mapping. The fully connected neural network sequentially includes an input layer, three hidden layers using a linear rectified function as the activation function, and an output layer using an S-shaped growth curve function as the activation function. The output layer outputs a normalized probability distribution. After decoding, the probability distribution is represented as a heat-affected zone distribution map containing pixel coordinates, a binary mask image representing the welding spatter area, and a weld nugget forming contour model described by a sequence of boundary point coordinates, which together constitute the weld state configuration data. The binary mask image of the welding spatter area is used to calibrate the spatter pixel area during the welding process.

4. The method for on-line inspection of a resistance welding seam defect according to claim 1, wherein The iterative optimization of the weight parameters in the preset deep learning detection model includes: Initialize a convolutional neural network with an encoder-decoder structure as a deep learning detection model; The heat-affected zone distribution map, the binary mask image of the welding spatter area, and the weld nugget forming contour model from the weld condition configuration data are used as inputs to the deep learning detection model. In a single iteration, the deep learning detection model outputs a predicted defect region segmentation map and calculates the loss value between the predicted defect region segmentation map and the supervision label generated based on the weld condition configuration data. The loss value is calculated by weighted summation of the cross-entropy loss function and the Dessian loss function. The backpropagation algorithm is used to calculate the gradient of each weight parameter of the deep learning detection model based on the loss value. The adaptive moment estimation algorithm is used to update all weight parameters of the deep learning detection model based on the gradient. The iteration process is repeated until the sum of the total area of ​​the over-welded region pixels and the total area of ​​the non-fused region pixels in the current detection frame output by the deep learning detection model is less than a first threshold in proportion to the total image area.

5. The method for on-line inspection of a resistance welding seam defect according to claim 4, wherein When calculating the loss value between the predicted defect region segmentation map and the supervision label generated based on the weld condition configuration data, the method for generating the supervision label includes: Based on the heat-affected zone distribution map and weld nugget forming contour model in the weld condition configuration data, a category label for each pixel is generated by rule judgment. For pixels located inside the boundary defined by the weld nugget forming contour model, if their grayscale value is higher than the preset over-welding grayscale threshold, they are marked as over-welding areas; if their grayscale value is lower than the preset non-fusion grayscale threshold, they are marked as non-fusion areas; otherwise, they are marked as normal welding areas. For pixels located within the area defined by the heat-affected zone distribution map but not inside the weld nugget forming contour model, they are labeled as heat-affected zone category. For pixels marked by the binary mask image of the welding spatter area, they are separately labeled as spatter area category. The category labels of all pixels constitute the supervision label.

6. The method for on-line inspection of a resistance-welded weld joint defect according to claim 1, wherein The method involves real-time weld defect detection during resistance welding at a preset frame rate, and collecting pixel-level molten pool feature data for each detection frame during the detection process, including: When the convergence signal of the detection model is received, the image acquisition device is started to capture real-time images of the welding area at a preset frame rate. For each detection frame image acquired, the deep learning detection model that has been iteratively optimized is called to perform forward inference, and the category label of each pixel in the corresponding detection frame image is output as belonging to the normal welding area, the over-welded area or the non-fused area. Based on the category label, traverse all pixels in the detection frame image, record the gray value of each pixel, and add an over-soldering label to pixels classified as over-soldering areas and an un-fused label to pixels classified as unfused areas. The grayscale values ​​of all pixels and their corresponding identifiers are aggregated to generate pixel-level melt pool feature data.

7. The online detection method for weld defects in resistance welding as described in claim 6, characterized in that, The method further includes: Based on the pixel-level weld pool feature data, the average gray value of all pixels with over-soldering marks in the current detection frame is calculated as the average gray value of the over-soldering area, and the average gray value of all pixels with non-fusion marks is calculated as the average gray value of the non-fusion area. The ratio of the average grayscale of the welded area to the average grayscale of the unfused area is calculated to obtain the real-time grayscale ratio. It is then determined whether the real-time grayscale ratio is less than a preset second threshold. If the real-time grayscale ratio is less than the second threshold, the defect detection result of the current detection frame is generated and output based on the output of the current deep learning detection model and the preset defect judgment rules. If the real-time grayscale ratio is greater than or equal to the second threshold, the welding current compensation and electrode pressure adjustment are calculated based on the pixel-level molten pool feature data. The welding current compensation is sent to the welding power controller, and the electrode pressure adjustment is sent to the electrode pressure servo mechanism. After the welding process parameters are updated, the process returns to perform real-time weld defect detection at a preset frame rate during the resistance welding process until the real-time grayscale ratio is less than the second threshold.

8. The online detection method for weld defects in resistance welding as described in claim 7, characterized in that, The welding current compensation and electrode pressure adjustment are calculated based on the pixel-level molten pool feature data, including: Based on the pixel-level molten pool feature data, the total number of pixels with over-soldering markers and the total number of pixels with non-fusion markers are counted. The difference in pixel size of the region is calculated by the difference. At the same time, the absolute difference between the average gray level of the over-soldering region and the average gray level of the non-fusion region is calculated to obtain the absolute gray level difference. The pixel difference in the region and the absolute difference in grayscale are respectively input into a dual-channel proportional-integral controller. The first channel of the dual-channel proportional-integral controller takes the pixel difference in the region as the first input and obtains a first control quantity through proportional and integral operations, which serves as the welding current compensation quantity. The second channel of the dual-channel proportional-integral controller takes the absolute difference in grayscale as the second input and obtains a second control quantity through proportional and integral operations, which serves as the electrode pressure adjustment quantity. The proportional operation is to multiply the first input or the second input by a preset proportional coefficient, and the integral operation is to perform discrete-time integration on the first input or the second input and then multiply it by a preset integral coefficient.

9. The online detection method for weld defects in resistance welding as described in claim 7, characterized in that, Based on the output of the current deep learning detection model and the preset defect judgment rules, the defect detection result of the current detection frame is generated and output, including: The deep learning detection model is invoked to infer the current detection frame and obtain a pixel-level classification result map. In the pixel-level classification result map, all connected regions formed by pixels with over-soldering marks are identified and the area of ​​each connected region is calculated. All connected regions formed by pixels with non-fused marks are identified and the area of ​​each connected region is calculated. If the area of ​​any over-welded indicator connected region exceeds the first area threshold, an over-welded defect is determined to exist. If the area of ​​any non-fused indicator connected region exceeds the second area threshold, a non-fused defect is determined to exist. Based on the determination results, a defect detection result report is generated, which includes the defect type, the contour coordinates of the connected region where the defect is located, and the defect area.

10. An online detection system for weld defects in resistance welding, used to implement the online detection method for weld defects in resistance welding as described in any one of claims 1-9, characterized in that, include: The data acquisition module is used to acquire real-time image sequences and welding process parameters of the resistance welding process; The feature construction module is used to perform inter-frame difference, watershed segmentation and geometric prior recognition on real-time image sequences, extract coordinate data of heat-affected areas and weld nugget candidate areas, and generate multimodal feature vectors by feature concatenation with welding process parameter time vectors. After decoding and mapping, the weld state configuration data is output. The model iteration and optimization module is used to iteratively optimize the weight parameters of the deep learning detection model and output a converged deep learning detection model. The defect detection module is used to capture real-time images of the welding area and output pixel-level category labels. It traverses pixels to record grayscale values, adds corresponding labels to defect pixels, and generates pixel-level molten pool feature data. The parameter control module is used to calculate the real-time grayscale ratio between the welded area and the unfused area, and to calculate the welding current compensation and electrode pressure adjustment. The data storage module is used to calculate the area of ​​connected regions with over-welding and non-fusion markings based on the pixel-level classification result image, generate a defect detection result report, record all defect data within the welding cycle, and construct a process-defect association database after storing the data in association with the corresponding welding process parameters.

Images