A multi-modal data fusion-based resistance spot welding quality monitoring method and system

Abstract

This invention discloses a method and system for monitoring the quality of resistance spot welding based on multimodal data fusion. The method includes: acquiring welding process parameters and weld nugget visual images of a resistance spot welding equipment; performing multi-dimensional heterogeneous data collaborative preprocessing on the welding process parameters and the weld nugget visual images; obtaining a two-dimensional spatial attention mask based on the preprocessed welding process parameters; modulating and weighting the deep visual features of the preprocessed weld nugget visual images based on the two-dimensional spatial attention mask to obtain fusion features; predicting quality indicators based on the fusion features to obtain quality prediction results; and executing corresponding alarm or equipment start / stop actions based on control signals generated from the quality prediction results to achieve closed-loop control of resistance spot welding quality. This invention can achieve higher accuracy and stronger robustness in resistance spot welding quality prediction and optimization.

Description

Technical Field

[0001] This invention belongs to the field of non-destructive testing technology for resistance spot welding, and particularly relates to a method and system for monitoring the quality of resistance spot welding based on multimodal data fusion. Background Technology

[0002] Resistance spot welding is a core metal joining process widely used in sheet metal joining in industrial production. Its basic principle involves applying pressure and a large current to stacked metal sheets using electrodes. The Joule heat generated by the contact resistance melts localized areas, forming a weld nugget under pressure and completing the connection. Weld quality is typically evaluated using weld nugget diameter, tensile and shear strength, peel strength, and internal defects, directly impacting the production pass rate and industrial application safety of resistance spot welding equipment. In industrial production, process parameters such as welding current, energizing time, electrode pressure, and electrode wear condition significantly affect weld quality. Traditional quality evaluation methods often rely on destructive mechanical testing or sampling, which are inefficient and costly, failing to meet the online quality control requirements of resistance spot welding equipment in the context of intelligent manufacturing.

[0003] Most existing methods for predicting resistance spot welding quality rely on vision or parameter-based methods for quality inspection. These methods are not deeply adapted to the hardware characteristics and process requirements of the welding equipment, leading to a disconnect between the algorithm and the equipment and hindering closed-loop control. One type of method primarily establishes a prediction model based on welding process parameters. The basic structure of this type typically includes a sensor acquisition module, a data preprocessing module, a feature extraction module, and a machine learning prediction module. Its core principle is to establish a mapping relationship between process parameters and quality results through statistical learning, and to construct a neural network prediction model using time-series data such as welding displacement and force signals. The advantage of this type of method is that it does not require an image acquisition system and has a relatively simple structure. However, its disadvantages include relying solely on process parameters without combining welding process parameters with equipment control logic, difficulty in reflecting the geometric features and surface defects of the weld point after formation, high dependence on sensor accuracy and signal stability, limited model generalization ability when the production environment changes, and inability to adapt to the complex operating conditions of resistance spot welding equipment.

[0004] Another type of existing technology focuses on weld joint quality assessment based on visual images. With the development of deep learning technology, researchers extract geometric features such as weld joint contours and areas through image segmentation, and combine them with fuzzy logic systems or convolutional neural networks to achieve quality assessment. These systems typically include an industrial camera acquisition module, an image preprocessing module, a feature extraction network, and a classification output module. Their advantage lies in their ability to intuitively reflect weld joint appearance defects and weld nugget morphology. However, this type of method usually ignores physical parameter information during the welding process, making judgments based solely on surface images. It is difficult to reflect the internal weld nugget formation mechanism, is sensitive to changes in lighting, image noise, and surface contamination, and lacks the ability to model the causal relationships of the welding process, thus failing to provide effective support for the process parameter control of resistance spot welding equipment.

[0005] In recent years, some studies have attempted to fuse welding parameters with image information to improve prediction performance. These methods typically employ feature concatenation or simple fusion strategies, connecting parameter feature vectors with image convolutional features at high levels, and then outputting the prediction result through a fully connected layer. While this approach improves prediction accuracy to some extent, existing technologies generally lack in-depth modeling mechanisms for how parameters affect image feature representation. Most methods only perform simple concatenation at the decision layer, failing to dynamically modulate the parameter responses to spatial regions or channels during feature extraction. Furthermore, traditional convolutional neural networks extract image features with a fixed structure, without dynamically adjusting the region of interest according to different welding process conditions. This results in insufficient adaptability of the model under complex process variations and makes it difficult to achieve closed-loop control for process optimization based on prediction results.

[0006] Furthermore, existing monitoring schemes based on general algorithms lack customized design for the process characteristics of resistance spot welding equipment (such as weld nugget formation mechanism, welding spatter interference, and process parameter fluctuations), resulting in insufficient model generalization ability and an inability to adapt to complex industrial conditions. In summary, although existing technologies have evolved from single-parameter prediction to visual detection and even multimodal fusion, several significant shortcomings remain: First, most methods fail to establish a deep coupling relationship between parameters and images at the feature level; second, there is a lack of mechanisms to dynamically guide the model to focus on key weld point areas based on process conditions; third, their ability to achieve online optimization and adaptive control is limited; and fourth, the integration of monitoring devices with resistance spot welding equipment is low, preventing the formation of closed-loop control.

[0007] To overcome the above problems, there is an urgent need for an integrated monitoring system that can use process parameters to guide image spatial features, achieve dynamic modulation at the feature layer, and be deeply adapted to resistance spot welding equipment, so as to meet the technological development needs of resistance spot welding equipment and achieve higher accuracy and stronger robustness in resistance spot welding quality prediction and optimization. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention proposes a resistance spot welding quality monitoring method and system based on multimodal data fusion. It aims to solve technical challenges such as low integration between existing monitoring devices and spot welding equipment, insufficient multimodal data fusion, inadequate quality judgment accuracy, and lack of closed-loop control capabilities. The core of this invention is adapted to the online closed-loop control requirements of resistance spot welding equipment.

[0009] To achieve the above objectives, this invention provides a method for monitoring the quality of resistance spot welding based on multimodal data fusion, comprising: The welding process parameters and visual images of the weld nugget of the resistance spot welding equipment are collected, and the welding process parameters and the visual images of the weld nugget are subjected to multi-dimensional heterogeneous data collaborative preprocessing. Based on the pre-processed welding process parameters, obtain the two-dimensional spatial attention mask; Based on the two-dimensional spatial attention mask, the deep visual features of the preprocessed fusion kernel visual image are modulated and weighted to obtain the fused features; Based on the fusion features, quality indicators are predicted to obtain quality prediction results; Based on the control signal generated from the quality prediction results, corresponding alarm or equipment start / stop actions are executed to achieve closed-loop control of resistance spot welding quality.

[0010] Optionally, the acquisition of welding process parameters and visual images of the weld nugget from the resistance spot welding equipment includes: By connecting to the sensor interface of the resistance spot welding equipment, process parameter data such as welding current, welding voltage, welding time, and electrode pressure are collected. Visual images of the weld nugget are captured by a high-speed camera deployed at the welding station, and the images are precisely aligned with the corresponding process parameter data based on the timestamp.

[0011] Optionally, multi-dimensional heterogeneous data collaborative preprocessing of the welding process parameters and the weld nugget visual image includes: The welding process parameters are Z-score standardized and outlier corrected to obtain pre-processed welding process parameters; The melting core visual image is sequentially subjected to layered pixel normalization, multi-scale noise suppression and adaptive size calibration to obtain the preprocessed melting core visual image; A two-dimensional coordinate grid is generated based on the size of the preprocessed weld nugget visual image. The two-dimensional coordinate grid is used to provide spatial coordinate support for the mapping of the welding process parameters to a two-dimensional spatial attention mask.

[0012] Optionally, the two-dimensional spatial attention mask can be obtained based on the pre-processed welding process parameters, including: The preprocessed welding process parameters are combined with the two-dimensional coordinate grid to obtain network input features; The network input features are input into a parameter space guided model based on a sinusoidal representation network. The nonlinear relationship between the welding process parameters and spatial coordinates is captured by a sinusoidal activation function to obtain the hidden layer output features. The output features of the hidden layer are mapped to a two-dimensional spatial attention mask with the same size as the visual image of the weld nugget. The weight value of each pixel in the two-dimensional spatial attention mask represents the degree of influence of the welding process parameters on the weld nugget features at the corresponding position.

[0013] Optionally, the deep visual features of the preprocessed fusion kernel visual image are modulated and weighted according to the two-dimensional spatial attention mask to obtain the fused features, including: The preprocessed visual image of the molten core is input into the backbone network of the model to extract deep visual features. The two-dimensional attention mask is expanded in dimension to obtain the expanded attention mask; The extended attention mask is multiplied pointwise with the deep visual features to achieve feature recalibration, and the recalibrated features are then subjected to global average pooling to obtain the fused features.

[0014] Optionally, quality index prediction is performed based on the fusion features, and the quality prediction results are obtained including: The fused features are dimensionally compressed to obtain a compressed feature vector; A fully connected network is used to map the compressed feature vector into predicted quality index values, which include predicted melt core diameter and predicted tensile strength. Based on the predicted values ​​of the quality indicators, obtain the quality prediction results.

[0015] Optionally, based on the control signal generated from the quality prediction result, corresponding alarm or equipment start / stop actions are executed to achieve closed-loop control of resistance spot welding quality, including: When the quality prediction result indicates that the weld point is qualified, the resistance spot welding equipment is maintained in normal operation and a normal status indication is displayed. When the quality prediction result indicates that the solder joint is close to the quality threshold, the early warning alarm device is triggered to issue a prompt signal; When the quality prediction result indicates that the weld point is unqualified, a stop command is sent to the PLC control system of the resistance spot welding equipment. The control equipment immediately stops the welding operation and activates the emergency alarm device.

[0016] The present invention also provides a resistance spot welding quality monitoring system based on multimodal data fusion, comprising: a data acquisition module, a quality monitoring module, and an alarm execution module; The data acquisition module is used to collect welding process parameters and visual images of the weld nugget of the resistance spot welding equipment in real time. The quality monitoring module is used to perform multi-dimensional heterogeneous data collaborative preprocessing on the welding process parameters and the weld nugget visual image, generate a two-dimensional spatial attention mask based on the preprocessed welding process parameters, modulate and weighted fuse the deep visual features of the preprocessed weld nugget visual image based on the two-dimensional spatial attention mask to obtain fused features, and predict quality indicators based on the fused features to obtain quality prediction results. The alarm execution module is used to receive control signals generated based on the quality prediction results, execute corresponding alarm or equipment start / stop actions, and realize closed-loop control of resistance spot welding quality.

[0017] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for monitoring the quality of resistance spot welding.

[0018] Compared with the prior art, the present invention has the following advantages and technical effects: 1. Deep fusion of multimodal features for high accuracy in complex working conditions: The parameter-driven spatial masking algorithm integrated in this invention uses implicit neural representation technology to transform real-time current and pressure parameters into spatially guided weights. This design is not simply a computer vision application, but rather uses physical parameters to force the model to focus on the core area of ​​the weld nugget during the feature extraction stage, thereby fundamentally suppressing extreme interferences unique to industrial sites, such as welding spatter and intense light flash. Compared to traditional single-modal monitoring or simple feature stitching, this invention, through a dynamic spatial-channel collaborative mechanism guided by physical parameters, can more accurately characterize the high-dimensional nonlinear relationship between process fluctuations and weld quality, significantly improving the accuracy of online non-destructive testing.

[0019] 2. Significantly enhanced real-time closed-loop control capabilities, resulting in substantial reductions in production losses: This invention achieves a complete "sensing-calculation-control" closed-loop process through its execution and alarm modules. Upon detecting a quality anomaly, the system can directly send interrupt or correction commands to the spot welding machine's PLC via the industrial bus, rather than simply displaying data at the backend. This closed-loop control capability, deeply coupled with the welding machine's operating status, allows the equipment to intervene immediately upon detecting the first defective weld, effectively preventing the generation of batches of defective products. Furthermore, the system generates mask heatmaps and weight distributions, providing operators with physically interpretable quantitative decision support for adjusting welding pressure and current, greatly enhancing the intelligent management level of the production line. Attached Figure Description

[0020] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a resistance spot welding quality monitoring method based on multimodal data fusion according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a resistance spot welding quality monitoring system based on multimodal data fusion according to an embodiment of the present invention; Figure 3 This is a flowchart of the quality monitoring module according to an embodiment of the present invention; Figure 4 This is an overall workflow diagram of an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the loss changes during training according to an embodiment of the present invention; Figure 6 This is a feature comparison diagram before and after modulation in an embodiment of the present invention; Figure 7 These are comparison diagrams of the comparison methods in this embodiment of the invention, wherein (a) is a schematic diagram of the prediction results of the model in this embodiment, (b) is a schematic diagram of the prediction results of CrossAtten, (c) is a schematic diagram of the prediction results of CBAM-CMA, and (d) is a schematic diagram of the prediction results of SpatialAttention-FiLM. Figure 8 These are comparison diagrams of ablation experiment results in embodiments of the present invention, wherein (a) is a schematic diagram of the prediction results of the model in this embodiment, (b) is a schematic diagram of the prediction results of the ablation model ImageOnly, (c) is a schematic diagram of the prediction results of the ablation model FiLMOnly, and (d) is a schematic diagram of the prediction results of the ablation model MaskOnly. Figure 9 This is the quality classification result of an embodiment of the present invention. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0023] This embodiment proposes a method for monitoring the quality of resistance spot welding based on multimodal data fusion, such as... Figure 1 As shown, the specific steps include: The welding process parameters and visual images of the weld nugget of the resistance spot welding equipment are collected, and the welding process parameters and the visual images of the weld nugget are subjected to multi-dimensional heterogeneous data collaborative preprocessing. Based on the pre-processed welding process parameters, obtain the two-dimensional spatial attention mask; Based on the two-dimensional spatial attention mask, the deep visual features of the preprocessed fusion kernel visual image are modulated and weighted to obtain the fused features; Based on the fusion features, quality indicators are predicted to obtain quality prediction results; Based on the control signal generated from the quality prediction results, corresponding alarm or equipment start / stop actions are executed to achieve closed-loop control of resistance spot welding quality.

[0024] Furthermore, the acquisition of welding process parameters and visual images of the weld nugget from the resistance spot welding equipment includes: By connecting to the sensor interface of the resistance spot welding equipment, process parameter data such as welding current, welding voltage, welding time, and electrode pressure are collected. Visual images of the weld nugget are captured by a high-speed camera deployed at the welding station, and the images are precisely aligned with the corresponding process parameter data based on the timestamp.

[0025] Furthermore, the multi-dimensional heterogeneous data collaborative preprocessing of the welding process parameters and the weld nugget visual image includes: The welding process parameters are Z-score standardized and outlier corrected to obtain pre-processed welding process parameters; The melting core visual image is sequentially subjected to layered pixel normalization, multi-scale noise suppression and adaptive size calibration to obtain the preprocessed melting core visual image; A two-dimensional coordinate grid is generated based on the size of the preprocessed weld nugget visual image. The two-dimensional coordinate grid is used to provide spatial coordinate support for the mapping of the welding process parameters to a two-dimensional spatial attention mask.

[0026] Furthermore, based on the pre-processed welding process parameters, the two-dimensional spatial attention mask is obtained, including: The preprocessed welding process parameters are combined with the two-dimensional coordinate grid to obtain network input features; The network input features are input into a parameter space guided model based on a sinusoidal representation network. The nonlinear relationship between the welding process parameters and spatial coordinates is captured by a sinusoidal activation function to obtain the hidden layer output features. The output features of the hidden layer are mapped to a two-dimensional spatial attention mask with the same size as the visual image of the weld nugget. The weight value of each pixel in the two-dimensional spatial attention mask represents the degree of influence of the welding process parameters on the weld nugget features at the corresponding position.

[0027] Furthermore, the deep visual features of the preprocessed fusion kernel visual image are modulated and weighted according to the two-dimensional spatial attention mask to obtain the fused features, including: The preprocessed fused kernel visual image is input into the backbone network of the model to extract deep visual features, wherein the model is ResNet18; The two-dimensional attention mask is expanded in dimension to obtain the expanded attention mask; The extended attention mask is multiplied pointwise with the deep visual features to achieve feature recalibration, and the recalibrated features are then subjected to global average pooling to obtain the fused features.

[0028] Furthermore, based on the fusion features, quality index prediction is performed to obtain the quality prediction results, including: The fused features are dimensionally compressed to obtain a compressed feature vector; A fully connected network is used to map the compressed feature vector into predicted quality index values, which include predicted melt core diameter and predicted tensile strength. Based on the predicted values ​​of the quality indicators, obtain the quality prediction results.

[0029] Furthermore, based on the control signal generated from the quality prediction result, corresponding alarm or equipment start / stop actions are executed to achieve closed-loop control of resistance spot welding quality, including: When the quality prediction result indicates that the weld point is qualified, the resistance spot welding equipment is maintained in normal operation and a normal status indication is displayed. When the quality prediction result indicates that the solder joint is close to the quality threshold, the early warning alarm device is triggered to issue a prompt signal; When the quality prediction result indicates that the weld point is unqualified, a stop command is sent to the PLC control system of the resistance spot welding equipment. The control equipment immediately stops the welding operation and activates the emergency alarm device.

[0030] This embodiment also provides a resistance spot welding quality monitoring system based on multimodal data fusion, such as... Figure 2 As shown, it specifically includes: a data acquisition module, a quality monitoring module, and an execution alarm module; The data acquisition module is used to collect welding process parameters and visual images of the weld nugget of the resistance spot welding equipment in real time. The quality monitoring module is used to perform multi-dimensional heterogeneous data collaborative preprocessing on the welding process parameters and the weld nugget visual image, generate a two-dimensional spatial attention mask based on the preprocessed welding process parameters, modulate and weighted fuse the deep visual features of the preprocessed weld nugget visual image based on the two-dimensional spatial attention mask to obtain fused features, and predict quality indicators based on the fused features to obtain quality prediction results. The alarm execution module is used to receive control signals generated based on the quality prediction results, execute corresponding alarm or equipment start / stop actions, and realize closed-loop control of resistance spot welding quality.

[0031] The quality monitoring equipment includes a data acquisition module, a quality monitoring host, and an execution and alarm module. These modules are connected via an industrial bus (such as Profinet or Modbus) and are deeply integrated with the hardware interface and control system of the resistance spot welding equipment, forming an integrated operating system. The specific structure is as follows: S1. Data Acquisition Module: The data acquisition module is a customized hardware module, serving as an extension unit for data acquisition in the resistance spot welding equipment. It directly connects to the sensor interface and external high-speed camera of the resistance spot welding equipment, and is used to synchronously acquire welding process parameters and weld nugget visual images in real time, providing a multimodal data source for quality assessment. During the welding production process, two types of core data are acquired synchronously in real time: welding process parameters and weld nugget visual images, providing a multimodal data source for the intelligent monitoring of the resistance spot welding equipment. Welding process parameters are obtained through the equipment's built-in sensors and data acquisition module, specifically including key process parameters such as welding current, welding voltage, welding time, and electrode pressure, ensuring that each weld point corresponds to a complete set of process parameter data. The weld nugget visual images are acquired by a high-speed camera deployed at the welding station. The camera's acquisition angle is aligned with the weld nugget formation area, and the acquisition timing is precisely aligned with the welding process parameter acquisition timing through timestamps, ensuring a one-to-one correspondence between the process parameters and the corresponding weld nugget image for the same weld point, avoiding prediction deviations caused by data misalignment.

[0032] S2. Quality Monitoring Host (Quality Monitoring Module): As the core processing unit of the entire monitoring equipment, it is compatible with the installation station of the resistance spot welding equipment and can be wall-mounted or integrated. It includes a memory and a processor. The processor uses an industrial-grade multi-core CPU with a main frequency of no less than 2.0GHz, supporting hardware acceleration. The memory includes DDR4 RAM (no less than 8GB) and a solid-state drive (no less than 256GB) to ensure the efficiency of algorithm operation and the reliability of data storage. It is equipped with multiple industrial communication interfaces (Ethernet, RS485) for communication with the data acquisition module, execution and alarm module, and the PLC control system of the resistance spot welding equipment. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the multimodal feature fusion quality monitoring method. The structure of this method is as follows... Figure 3 As shown, the specific implementation steps are as follows: S21. Data Preprocessing: Multi-dimensional heterogeneous data collaborative preprocessing is carried out on the welding process parameters and weld nugget visual images collected in step S1. Differentiated processing logic is designed for the feature differences between the two types of data. At the same time, an adaptive generation mechanism for spatial coordinate grid and a data consistency verification mechanism are introduced. On the basis of data standardization and noise suppression, spatial-numerical dual-dimensional matching of process parameters and weld nugget images is achieved, providing highly consistent and effective basic data for subsequent parameter spatial guidance and feature modulation.

[0033] S22. Forward Inference of the Parameter Space Guidance Module: During the training phase, the welding process parameters preprocessed in step S21 are input into the parameter space guidance module for forward inference. This module is built based on Sinusoidal Representation Networks (SIREN) and is customized for the process parameter characteristics of the welding equipment. It can map low-dimensional process parameters into a two-dimensional spatial attention mask with the same size as the weld nugget image. The core physical meaning of this step is to map low-dimensional energy parameters into a spatial attention mask that reflects the physical morphology of the weld nugget. The weight values ​​in the mask directly correspond to the degree of influence of physical energy on the formation of the weld nugget at that spatial location, thereby realizing the structured transformation of "visual attention points" guided by "physical parameters". This provides a structured parameter guidance basis for subsequent feature fusion, thereby improving the feature recognition accuracy of online non-destructive testing of the equipment.

[0034] S23. Mask-weighted feature fusion: The preprocessed molten core image from step S21 is input into the backbone network of the model to extract the deep visual features of the molten core image. Subsequently, the spatial attention mask output from step S22 is fused with the extracted deep visual features of the image through feature-wise linear modulation (FiLM) and spatial position-weighted fusion operations to achieve deep coupling of multimodal features and provide accurate feature support for the quality judgment of arc welding equipment.

[0035] S24. Online prediction of quality indicators: Input the parameter-visual coupling enhancement feature obtained in step S23 into the quality prediction head of the model to perform real-time prediction of quality indicators of industrial automatic control system.

[0036] S25. Anomaly Judgment and Threshold Decision: The quality prediction results output in step S24 are compared with the preset welding quality standards in real time to provide a basis for judgment on anomaly handling of resistance spot welding equipment.

[0037] S3. Execution and Alarm Module: The execution and alarm module is communicatively connected to the quality monitoring host and also interfaces with the PLC control system and on-site alarm device of the resistance spot welding equipment. It receives control signals from the quality monitoring host and executes corresponding alarm or equipment start / stop actions to achieve closed-loop control. When step S25 determines a serious abnormality, the system should immediately activate the real-time alarm mechanism. This mechanism can directly interface with the PLC control system of the resistance spot welding equipment, controlling the equipment to immediately stop welding operations, achieving closed-loop intelligent control of welding quality. Simultaneously, the system displays real-time information about the current welding station through a visual interface, including the process parameters of the current weld point, weld nugget image, spatial attention mask heatmap, quality prediction value, and judgment result. This allows on-site operators to intuitively understand the welding quality status, quickly locate the cause of abnormalities, and adapt to the on-site operation requirements of intelligent manufacturing.

[0038] Further, in step S1, the collected process parameters are defined as follows: ,in This is the acquisition timestamp. The size of the acquired melt core image is defined as follows: (Where H is the image height, W is the image width, and C is the number of image channels), the acquired image data is uniformly represented as... ,in , , These correspond to the width, height, and channel dimensions of the image, respectively.

[0039] Furthermore, in step S21, the specific data preprocessing procedure is as follows: For melting nucleus images To address the issues of splash noise, uneven illumination, and pixel scale differences in molten core images, a three-step processing strategy is adopted: hierarchical normalization, multi-scale noise suppression, and adaptive size calibration. This strategy preserves the fine spatial features of the molten core region and the heat-affected zone while eliminating irrelevant interference from subsequent feature extraction. Specifically, hierarchical pixel normalization divides the image into candidate molten core regions and background regions. Linear normalization is then applied to the candidate molten core regions, mapping pixel values ​​to the [−1,1] interval to maximize the preservation of the molten core grayscale gradient features. The calculation formula is as follows: ; in The image pixel value is the normalized value. Combining the spatial scale characteristics of the noise, a composite denoising strategy is adopted, integrating foreground-background multi-scale Gaussian filtering with salt-and-pepper noise median filtering. A 3×3 small-kernel Gaussian filter is used in the foreground region of the weld nugget to suppress fine-grained noise generated by welding spatter, while a 7×7 large-kernel Gaussian filter is used in the background region to suppress coarse-grained noise caused by uneven illumination. Simultaneously, a 2×2 median filter is applied to the entire image to remove random salt-and-pepper noise. The filtered image is denoted as [image value]. This effectively achieves precise suppression of noise of different types and scales. (Filtered image) ( (This represents a convolution operation). The size is uniformly adjusted using bilinear interpolation to resize the filtered image to the preset dimensions. Ensure that the dimensions match the input dimensions of the model.

[0040] For welding process parameters, Z-score standardization is used to eliminate differences in parameter dimensions, mapping each parameter value to a standard normal distribution interval with a mean of 0 and a variance of 1. The calculation formula is as follows: ; in This is the mean vector of the process parameter training set. The standard deviation vector of the process parameter training set, and the standardized parameters. This ensures consistency with the parameter preprocessing logic during model training, preventing differences in parameter dimensions from affecting model inference accuracy. Simultaneously, the interquartile range (ICM) method is used to identify outliers in the time-series feature vectors, addressing those exceeding... For outliers within the specified range, adaptive correction is performed using nearest neighbor feature value interpolation based on the normal data distribution of that parameter dimension. This avoids interference from outliers in subsequent model training. The corrected feature vector is denoted as... ; Meanwhile, based on the size of the preprocessed melt core image Generate a two-dimensional coordinate grid consistent with the image resolution. The principle behind generating the coordinate grid is to map the spatial positions of image pixels to standardized coordinates, providing spatial coordinate support for mapping low-dimensional process parameters to two-dimensional attention masks. The specific generation process is as follows: First, obtain the original coordinates of each pixel in the image. ( , Then, it is standardized to the interval [-1, 1] using a linear mapping, and the calculation formula is: ; Finally, a two-dimensional coordinate grid is obtained. ( , Each grid element corresponds to the standardized spatial coordinates of an image pixel.

[0041] Furthermore, in step S22, the core structure of the SIREN network includes an input layer, a hidden layer, and an output layer. The computational logic of each neuron is as follows: First, the input layer receives a standardized low-dimensional welding process parameter vector. (in (for process parameter dimensions), and compare them with the two-dimensional coordinate grid generated in step 21. (in , These represent the height and width of the molten core image, and the spatial coordinates of each element in the coordinate grid corresponding to a pixel in the image. Feature concatenation is performed to obtain the network input features. .

[0042] The hidden layer uses a sinusoidal activation function, and the formula for calculating the neuron output is as follows: ; in The fundamental frequency of the sinusoidal activation function. , The first The weight matrix and bias vector of the hidden layer For the first Input features of the layer. Through iterative calculations of multiple hidden layers, deep fusion of process parameter information and spatial coordinate information is achieved, capturing the nonlinear correlation between the two.

[0043] The output layer uses a linear activation function to map the output features of the hidden layer to a two-dimensional spatial attention mask with the same size as the melting kernel image. The calculation formula is as follows: ; in This is the output of the last hidden layer. , These are the weight matrix and bias vector of the output layer, respectively. This is the Sigmoid activation function, used to normalize the mask values ​​to the [0,1] interval.

[0044] This spatial attention mask Weight value of each pixel ( The weight of the welding process parameters corresponds to the degree of influence of the weld nugget feature at that location. The higher the weight, the greater the influence of the process parameters on the weld nugget feature at that location, which is a key area for subsequent quality judgment. This realizes the transformation of process parameter information into spatial dimension and provides a structured parameter guidance basis for subsequent feature fusion.

[0045] Furthermore, in step S23, the preprocessed melt image from step 21 is mainly processed. The input is fed into the model backbone network, i.e., the feature extraction stage based on ResNet18, to extract deep visual features from the fused kernel image. This feature contains key visual information such as the spatial morphology and grayscale distribution of the melt core, but does not incorporate the influence mechanism of process parameters.

[0046] Subsequently, the spatial attention mask output in step 22 is... Perform dimensional adjustments to align with image features. For dimensional matching, the formula is adjusted as follows: ; in For dimensional expansion operations, the mask is moved from... Expand to This ensures that each feature channel receives the corresponding spatial attention weight.

[0047] Next, a feature mask space weighting operation is performed to apply the expanded attention mask... With deep image features Feature recalibration is achieved by performing point-by-point multiplication, channel-by-channel and spatial-position-by-spatial-position. The calculation formula is as follows: ; in , , These correspond to the height, width, and channel dimensions of the feature, respectively. The core logic of this operation is mask weighting. The higher the value, the more the image features of the corresponding position and channel are enhanced, and vice versa. This highlights the key areas of the melt core that are greatly affected by process parameters, and suppresses the interference of irrelevant areas such as background and splashes.

[0048] Finally, the weighted features Perform global average pooling to further compress feature dimensions and extract global features, resulting in parameter-visual coupling enhanced features. The calculation formula is: ; This feature integrates structured information of process parameters with spatial visual information of melt core images, providing accurate feature input for subsequent quality indicator prediction.

[0049] Furthermore, in step S24, the quality prediction head adopts a hybrid structure of convolutional layers and fully connected layers, and is divided into two stages: feature compression and index prediction.

[0050] In the feature compression stage, a 1×1 convolutional layer is used to couple and enhance the features. Dimensionality compression is performed to reduce computational cost while preserving key feature information. The convolution operation formula is as follows: ; in , These represent the convolution weights and biases, respectively, and the output is the compressed feature. .

[0051] In the indicator prediction stage, a 3-layer fully connected network is used to map the compressed features to the predicted values ​​of each quality indicator. The computation logic of the fully connected layer is as follows: ; ; ; in , , These are the weights of the fully connected layers, , , Each layer is biased. This is the activation function used to introduce nonlinear associations.

[0052] The prediction head outputs two core quality indicators corresponding to the solder joint in real time, specifically including: the predicted weld nugget diameter. (Unit: mm) Predicted tensile strength (Unit: kN)

[0053] Furthermore, in step S25, anomaly detection and threshold decision-making are performed. The core principle is based on the quality prediction results output in step S6. By combining preset welding quality standard thresholds and comparing multiple thresholds, the system can accurately determine the quality of weld points online and distinguish between qualified and abnormal situations.

[0054] Furthermore, step S3 specifically includes an alarm unit and an execution unit: Alarm unit: Includes audible and visual alarm and indicator lights. When an alarm signal is received from the quality monitoring host, the audible and visual alarm will emit a high-decibel sound alarm (volume not less than 85dB) and a flashing light alarm (red light, flashing frequency 2Hz). The indicator lights will display the current equipment status (green - normal, yellow - warning, red - abnormal), reminding on-site operators to handle the situation in a timely manner.

[0055] Execution unit: Includes relay module and signal conversion circuit. When a stop signal is received from the quality monitoring host, the signal conversion circuit converts the control signal into an instruction that the resistance spot welding equipment PLC can recognize, and controls the resistance spot welding equipment to immediately stop the welding operation. When a normal operation signal is received, the equipment continues to run. It supports manual reset function, and the equipment operation status can be reset by pressing a button after troubleshooting.

[0056] This monitoring equipment works in conjunction with resistance spot welding equipment. The overall workflow is as follows: Figure 4 As shown.

[0057] Start-up phase: After the resistance spot welding equipment is started, this monitoring equipment is simultaneously powered on and initialized. The data acquisition module completes sensor calibration and camera self-test. The quality monitoring host loads the algorithm program and completes communication handshake with each module. The indicator lights of the execution and alarm modules show green, indicating that the equipment is ready. Data acquisition phase: When the resistance spot welding equipment starts welding, the process parameter acquisition unit of the data acquisition module acquires welding current, voltage and other parameters in real time, and the visual image acquisition unit simultaneously captures images of the weld nugget formation process. After being synchronized with the timestamp, the images are transmitted to the quality monitoring host. Quality assessment stage: The quality monitoring host runs a multi-modal fusion algorithm to preprocess the received data, fuse features, predict indicators and determine compliance, and generate control signals (normal operation signal, alarm signal, shutdown signal). Closed-loop execution phase: The execution and alarm module receives control signals. If the weld joint is qualified, the resistance spot welding equipment is maintained to operate normally and the status indicator light is green. If a warning occurs (approaching the quality threshold) at the weld joint, the status indicator light is yellow and the buzzer sounds a warning sound. If the weld joint is unqualified, the audible and visual alarm is immediately activated (red light flashing + high-decibel alarm sound), and the resistance spot welding equipment is controlled to stop welding. Reset phase: After the operator has investigated and dealt with the cause of the abnormality (such as adjusting process parameters or replacing electrodes), the equipment is reset by manually pressing the reset button or remotely operating the system. The alarm module is then restored to normal status, and the equipment resumes welding operations.

[0058] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for monitoring the quality of resistance spot welding.

[0059] The following is a detailed description of this embodiment with reference to the accompanying drawings: To verify the effectiveness and superiority of the resistance spot welding equipment welding quality prediction method proposed in this invention, a systematic experimental verification was conducted on a resistance spot welding dataset. This dataset, based on a Chicago Electric 61205 small resistance welding machine, contains 495 valid samples, covering different combinations of resistance spot welding process parameters and corresponding quality states, and can support model training, verification, and performance evaluation.

[0060] At the process parameter level, key process variables affecting welding quality, such as electrode pressure, welding time, electrode angle, electrode force, welding current, and workpiece material thickness, are included and can serve as input data for the parameter space guidance module of this invention. At the image data level, visual images of the resistance spot welding process are collected simultaneously, providing a rich source of visual features for the model's image feature extraction module. At the quality index level, tensile strength and weld nugget diameter are explicitly defined as core evaluation parameters. During the experiment, to ensure the sufficiency of model training and the objectivity of evaluation, the dataset was randomly divided into training and testing sets in a 4:1 ratio.

[0061] To quantitatively evaluate the performance difference between the proposed method and existing comparative models, and to ensure the scientific validity and comparability of the evaluation results, this invention uses Mean Absolute Error (MAE) and Coefficient of Determination (R²) as core performance evaluation indicators. MAE measures the deviation between the model's predicted values ​​and the actual values; a smaller MAE value indicates higher prediction accuracy. R² measures the model's fit to the data's variation patterns; a value closer to 1 indicates a better fit. The specific calculation formulas are as follows: ; ; In the above formula, The number of samples in the test set. For the first Predicted values ​​of quality indicators for each sample For the first The true values ​​of the quality indicators for each sample This is the average of the true values ​​of the quality indicators for all samples in the test set.

[0062] This embodiment has undergone sufficient training, and the loss during training changes as follows: Figure 5 As shown, all have reached a relatively convergent state. Meanwhile, Figure 6It is a feature comparison before and after modulation. Before modulation, the molten core and the background are completely mixed. After modulation, the range of the molten core can be seen more clearly, and the internal details of the molten core are more prominent.

[0063] This invention is compared with three data-driven models, the specific descriptions of which are as follows: CrossAtten: Extracts different features from both the process parameter feature branch and the image feature dual branch, and achieves prediction through cross-attention fusion; CBAM-CMA: Process parameters and images are processed by CNN, and CBAM modules are embedded in each layer of the image branch. CBAM consists of channel attention modules and spatial attention modules. Finally, cross attention is used to achieve alignment and fusion between modalities. SpatialAttention-FiLM: This is a method that combines spatial attention mechanisms with FiLM. The comparison results are shown in Table 1.

[0064] Table 1 Combining the performance data of each model on the test and training sets, it can be seen that although CrossAtten employs dual-branch feature extraction of process parameters and images, and achieves feature fusion through cross-attention, its average MAE on the test set is still 0.0609, and its average R² is only 0.8702. The performance gap between the training and test sets is significant, indicating that its cross-attention fusion strategy fails to fully exploit the complementary information of the two modalities, resulting in weak overall fitting effect and prediction accuracy. SpatialAttention-FiLM, as a method combining spatial attention mechanism and FiLM, can complete basic tensile strength and melt diameter prediction tasks, with an average R² of 0.9047 on the test set. However, its average MAE is still 0.0519, indicating that its combination of spatial attention and FiLM fails to effectively enhance the expression of key features, and there is still considerable room for improvement in accuracy and fitting ability. While CBAM-CMA processes process parameters and images using CNN and embeds CBAM modules consisting of channel attention and spatial attention modules in each layer of the image branch, and finally achieves modal alignment and fusion through cross attention, and performs well in tensile strength prediction, its R² is only 0.9112 in the melt core diameter prediction task, and the average MAE on the test set is 0.0384. This indicates that the CBAM module of this model is not adaptable to different quality indicators, and the modal fusion effect of cross attention is uneven, resulting in performance imbalance between different prediction tasks and the overall accuracy is lower than expected. In contrast, the method proposed in this invention, by fusing complementary information between images and process parameters, achieves an average MAE of only 0.0269 on the test set, which is approximately 56% lower than CrossAtten, approximately 30% lower than CBAM-CMA, and approximately 48% lower than SpatialAttention-FiLM. Simultaneously, it maintains good performance consistency across different prediction tasks such as tensile strength and weld nugget diameter, with an average R² of 0.9732 on the test set and an even higher average R² of 0.9885 on the training set, significantly outperforming other comparative models. This fully demonstrates that the multi-source fusion strategy adopted in this invention, compared to the fusion methods of existing comparative models, can more fully leverage the synergistic effect of process parameters and image data, achieving more accurate and robust performance in predicting resistance spot welding quality indicators.

[0065] Combination Figure 7(a)-(d) further analyze the confidence intervals of each model separately, corresponding to the proposed method, CrossAtten, CBAM-CMA, and SpatialAttention-FiLM, respectively. The confidence interval is mainly reflected by the fluctuation range of each performance index; the smaller the fluctuation range, the stronger the stability of the model prediction. Data shows that although CrossAtten uses a two-branch cross-attention fusion, the average MAE fluctuation on the test set is ±0.0048, and the average R² fluctuation is ±0.0229. The performance fluctuations on both the training and test sets are relatively large, indicating insufficient stability of its cross-attention fusion and weak adaptability to data distribution, resulting in poor prediction stability. SpatialAttention-FiLM combines spatial attention mechanisms with FiLM, but the average MAE fluctuation on the test set is ±0.0039, and the average R² fluctuation is ±0.0143, with overall fluctuation still being significant. Especially in the melting kernel diameter prediction task, the combination of spatial attention and FiLM failed to effectively suppress fluctuations, further affecting the prediction reliability. Although CBAM-CMA embeds CBAM channels and a spatial attention module and employs cross-attention fusion, its stability is uneven. In tensile strength prediction, the R² fluctuation is only ±0.0048, indicating good stability. However, in melt core diameter prediction, the R² fluctuation reaches ±0.0113, suggesting inconsistent feature enhancement effects of its CBAM module for different quality indicators, leading to an overall higher confidence interval. In contrast, the method proposed in this invention has an average MAE fluctuation of ±0.0050 and an average R² fluctuation of ±0.0265 on the test set, while the average MAE fluctuation on the training set is only ±0.0024 and the average R² fluctuation is ±0.0118. Although the R² fluctuation on the test set is slightly higher, considering its optimal accuracy performance, the overall fluctuation is within a reasonable range. Furthermore, the fluctuation trends are consistent in both quality indicator prediction tasks, indicating that the method of this invention has a lower confidence interval and stronger stability, effectively reducing the impact of random interference on the prediction results. The following line graphs of predicted and actual values ​​will further verify this conclusion from a trend matching perspective.

[0066] Simultaneously, the present invention also conducted ablation experiments, comparing it with ImageOnly (using only image data as input, without fusing process parameters and any attention or modulation modules), FiLMOnly (retaining only the FiLM modulation module, removing the parameter space guidance and mask weighted fusion modules), and MaskOnly (retaining only the mask weighted fusion module, removing the FiLM modulation and parameter space guidance modules). The results are shown in Table 2: Table 2 Combination Figure 8In the diagram, (a)-(d) correspond to the proposed method, ImageOnly, FiLMOnly, and MaskOnly, respectively. Based on the test set performance data, the ImageOnly model performed the worst, with an average MAE of 0.1236 and an average R² of only 0.4763. Even on the training set, the average MAE was only 0.0373 and the average R² was 0.9435. The performance difference between the training and test sets was significant, indicating overfitting. This demonstrates that relying solely on single-modal image data cannot fully capture the core influencing factors of welding quality indicators, resulting in extremely poor model generalization ability and difficulty adapting to the complex changes in actual welding scenarios.

[0067] The FiLMOnly and MaskOnly models showed similar performance, both significantly improving upon ImageOnly, but still falling far short of the method proposed in this invention. Specifically, the FiLMOnly model had an average MAE of 0.0606 and an average R² of 0.8781 on the test set, indicating that while retaining only the FiLM modulation module could enhance feature representation to some extent, the lack of synergistic effects from parameter space guidance and mask weighting prevented the full exploitation of complementary information between process parameters and image data, resulting in significant deficiencies in prediction accuracy and fitting performance. The MaskOnly model had an average MAE of 0.0609 and an average R² of 0.8697 on the test set, slightly lower than the FiLMOnly model. This suggests that relying solely on the mask weighting fusion module, lacking the precise feature calibration provided by FiLM modulation and the targeted guidance from parameter space for mask generation, made it difficult to achieve high-precision prediction of quality indicators. Furthermore, its R² of only 0.8305 was particularly weak in the melt core diameter prediction task.

[0068] In contrast, the method proposed in this invention integrates the advantages of each core module. The average MAE on the test set is only 0.0269, a reduction of approximately 78% compared to ImageOnly, approximately 56% compared to FiLMOnly, and approximately 56% compared to MaskOnly. The average R² on the test set reaches 0.9732, an improvement of approximately 104% compared to ImageOnly, approximately 11% compared to FiLMOnly, and approximately 12% compared to MaskOnly. Meanwhile, on the training set, the method of this invention has an average MAE of only 0.0183 and an average R² as high as 0.9885, both significantly outperforming the three ablation comparison models. Furthermore, the performance difference between the training and test sets is small, indicating stronger generalization ability.

[0069] This fully demonstrates that the modules in this invention, such as parameter space guidance, channel modulation, and mask weighted fusion, do not function in isolation, but rather form a synergistic and complementary whole. Retaining only a single module or removing the core module would lead to a significant decrease in model performance. Only by organically combining the modules can the synergistic effect of process parameters and image data be fully utilized to achieve high-precision and robust prediction of tensile strength and weld nugget diameter. This further verifies the rationality and effectiveness of the overall design of the method in this invention, fully adapts to the technological development needs of the intelligent manufacturing equipment industry, and meets the core requirements of non-destructive quality inspection of resistance spot welding equipment.

[0070] At the same time, by Figure 9 As can be seen from the confusion matrix results on the validation set, the model performs excellently in identifying normal samples. 75 real normal samples were accurately predicted as normal, while only 3 normal samples were misclassified as abnormal, resulting in a false alarm rate of only 3.85%, effectively avoiding product loss due to false alarms during production. Simultaneously, among real abnormal samples, 6 were successfully identified as abnormal, while only 3 were misclassified as normal, resulting in a false negative rate of 33.33%. This demonstrates strong anomaly detection capabilities even in imbalanced sample scenarios. Overall, the model achieves a total classification accuracy of 90.11%. The number of correctly classified samples on the diagonal of the confusion matrix (75+6) is significantly higher than the number of misclassified samples off-diagonally (3+3), fully demonstrating that the multi-task learning framework proposed in this invention, combining mask generation and FiLM modulation mechanisms, can effectively learn the mapping relationship between image features and process parameters, accurately distinguishing between normal and abnormal process states. It exhibits significant advantages of high accuracy and low false alarm rate in industrial quality monitoring scenarios, meeting the stringent requirements of actual production for quality early warning.

[0071] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for monitoring the quality of resistance spot welding based on multimodal data fusion, characterized in that, include: The welding process parameters and visual images of the weld nugget of the resistance spot welding equipment are collected, and the welding process parameters and the visual images of the weld nugget are subjected to multi-dimensional heterogeneous data collaborative preprocessing. Based on the pre-processed welding process parameters, obtain the two-dimensional spatial attention mask; Based on the two-dimensional spatial attention mask, the deep visual features of the preprocessed fusion kernel visual image are modulated and weighted to obtain the fused features; Based on the fusion features, quality indicators are predicted to obtain quality prediction results; Based on the control signal generated from the quality prediction results, corresponding alarm or equipment start / stop actions are executed to achieve closed-loop control of resistance spot welding quality.

2. The method for monitoring resistance spot welding quality based on multimodal data fusion according to claim 1, characterized in that, The acquisition of welding process parameters and visual images of the weld nugget from the resistance spot welding equipment includes: By connecting to the sensor interface of the resistance spot welding equipment, process parameter data such as welding current, welding voltage, welding time, and electrode pressure are collected. Visual images of the weld nugget are captured by a high-speed camera deployed at the welding station, and the images are precisely aligned with the corresponding process parameter data based on the timestamp.

3. The method for monitoring resistance spot welding quality based on multimodal data fusion according to claim 1, characterized in that, Multi-dimensional heterogeneous data collaborative preprocessing of the welding process parameters and the weld nugget visual image includes: The welding process parameters are Z-score standardized and outlier corrected to obtain pre-processed welding process parameters; The melting core visual image is sequentially subjected to layered pixel normalization, multi-scale noise suppression and adaptive size calibration to obtain the preprocessed melting core visual image; A two-dimensional coordinate grid is generated based on the size of the preprocessed weld nugget visual image. The two-dimensional coordinate grid is used to provide spatial coordinate support for the mapping of the welding process parameters to a two-dimensional spatial attention mask.

4. The method for monitoring resistance spot welding quality based on multimodal data fusion according to claim 3, characterized in that, Based on the pre-processed welding process parameters, the two-dimensional spatial attention mask is obtained, including: The preprocessed welding process parameters are combined with the two-dimensional coordinate grid to obtain network input features; The network input features are input into a parameter space guided model based on a sinusoidal representation network. The nonlinear relationship between the welding process parameters and spatial coordinates is captured by a sinusoidal activation function to obtain the hidden layer output features. The output features of the hidden layer are mapped to a two-dimensional spatial attention mask with the same size as the visual image of the weld nugget. The weight value of each pixel in the two-dimensional spatial attention mask represents the degree of influence of the welding process parameters on the weld nugget features at the corresponding position.

5. The method for monitoring resistance spot welding quality based on multimodal data fusion according to claim 3, characterized in that, Based on the two-dimensional spatial attention mask, the deep visual features of the preprocessed fusion kernel visual image are modulated and weighted to obtain the fused features, including: The preprocessed visual image of the molten core is input into the backbone network of the model to extract deep visual features. The two-dimensional attention mask is expanded in dimension to obtain the expanded attention mask; The extended attention mask is multiplied pointwise with the deep visual features to achieve feature recalibration, and the recalibrated features are then subjected to global average pooling to obtain the fused features.

6. The method for monitoring resistance spot welding quality based on multimodal data fusion according to claim 1, characterized in that, Based on the fusion features, quality index prediction is performed, and the quality prediction results are obtained, including: The fused features are dimensionally compressed to obtain a compressed feature vector; A fully connected network is used to map the compressed feature vector into predicted quality index values, which include predicted melt core diameter and predicted tensile strength. Based on the predicted values ​​of the quality indicators, obtain the quality prediction results.

7. The method for monitoring resistance spot welding quality based on multimodal data fusion according to claim 1, characterized in that, Based on the control signal generated from the quality prediction results, corresponding alarm or equipment start / stop actions are executed to achieve closed-loop control of resistance spot welding quality, including: When the quality prediction result indicates that the weld point is qualified, the resistance spot welding equipment is maintained in normal operation and a normal status indication is displayed. When the quality prediction result indicates that the solder joint is close to the quality threshold, the early warning alarm device is triggered to issue a prompt signal; When the quality prediction result indicates that the weld point is unqualified, a stop command is sent to the PLC control system of the resistance spot welding equipment. The control equipment immediately stops the welding operation and activates the emergency alarm device.

8. A resistance spot welding quality monitoring system based on multimodal data fusion, used to implement the method as described in any one of claims 1-7, characterized in that, include: Data acquisition module, quality monitoring module, and alarm execution module; The data acquisition module is used to collect welding process parameters and visual images of the weld nugget of the resistance spot welding equipment in real time. The quality monitoring module is used to perform multi-dimensional heterogeneous data collaborative preprocessing on the welding process parameters and the weld nugget visual image, generate a two-dimensional spatial attention mask based on the preprocessed welding process parameters, modulate and weighted fuse the deep visual features of the preprocessed weld nugget visual image based on the two-dimensional spatial attention mask to obtain fused features, and predict quality indicators based on the fused features to obtain quality prediction results. The alarm execution module is used to receive control signals generated based on the quality prediction results, execute corresponding alarm or equipment start / stop actions, and realize closed-loop control of resistance spot welding quality.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 7.

Images