Prefabricated cabin welding seam quality detection method based on self-supervised learning

Abstract

The invention discloses a prefabricated cabin welding seam quality detection method based on self-supervised learning, and the method comprises the following steps: collecting a multi-source welding seam image and process parameters, carrying out the synchronous calibration, and constructing a multi-modal observation matrix; the image is input into a DINOv2 model based on a visual Transform, dense features are obtained, and inter-frame registration and sequence reconstruction are completed; performing difference analysis on adjacent frames, extracting spatial changes and marking potential defects; the dense features and the process parameters of the corresponding time periods are fused, joint features are constructed and classified according to rules, and a preliminary result is output; inputting the defect area and the process parameters into an improved DEER model, extracting an influence path and amplitude, and generating a sensitivity score and a confidence coefficient; final judgment is given in combination with the preliminary result, the sensitivity and the confidence coefficient, and feedback is formed according to judgment and process difference. According to the method, high-precision and explainable detection and process optimization of weld defects are realized, and the method is suitable for online / offline quality control and tracing.

Description

Technical Field

[0001] This invention relates to the fields of industrial quality inspection and computer vision technology, and in particular to a method for inspecting the quality of prefabricated cabin welds based on self-supervised learning. Background Technology

[0002] With the rapid development of prefabricated cabin manufacturing in the fields of rail transit and high-end equipment, the requirements for intelligent and high-precision weld quality inspection are constantly increasing. Existing weld inspection methods mostly rely on manual visual inspection or traditional image algorithms, which are difficult to adapt to the actual needs of complex working conditions and diverse defect types. In image inspection, commonly used edge analysis and texture matching methods are sensitive to changes in illumination and image noise, and are usually based on single-frame image processing, lacking temporal continuity modeling, resulting in insufficient ability to identify the dynamic defect evolution process. At the same time, there is often a lack of precise synchronization between multi-source images and welding process parameters, leading to information alignment errors that affect the stability and accuracy of model inference results.

[0003] Furthermore, current supervised learning-based detection models rely on large amounts of labeled data, resulting in high labeling costs, poor generalization ability, and a lack of interpretable analysis of defect causes. In terms of process parameter modeling, existing methods often ignore the causal relationships and temporal sensitivity between parameters, making it difficult to effectively identify key influence paths and assess the degree of defect correlation.

[0004] Therefore, how to provide a method for inspecting the quality of prefabricated cabin welds based on self-supervised learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a prefabricated cabin weld quality inspection method based on self-supervised learning. This invention fully utilizes DINOv2 self-supervised feature learning based on visual Transformer and introduces an improved DEER model to achieve interpretable defect judgment and closed-loop feedback optimization. It has the advantages of low annotation cost, high detection accuracy, strong interpretability, robustness to noise and working condition changes, and support for online / offline deployment and quality traceability.

[0006] A method for inspecting the quality of weld seams in prefabricated cabins based on self-supervised learning according to an embodiment of the present invention includes the following steps: Multi-source weld images and welding process parameters are collected and synchronously calibrated based on a unified timestamp and spatial location identifier to construct a multi-modal observation matrix for welds. Image data from the multimodal observation matrix of weld seams are input into the DINOv2 self-supervised model based on the visual Transformer structure to perform occlusion restoration, image reconstruction and view comparison tasks, obtain dense visual features of weld seam images, and perform inter-frame registration and sequence reconstruction on multi-source weld seam images to generate a weld seam image sequence. Perform difference analysis on adjacent frames in the weld seam image sequence to extract spatially varying regions, combine temporal order to label potential defect regions, and generate a set of defect candidate regions. The dense visual features are fused with the welding process parameters of the time period corresponding to the defect candidate region to construct a joint feature state set. Each defect candidate region is classified according to the preset anomaly detection rules, and the preliminary defect identification results are output. Input the defect areas and corresponding welding process parameters from the preliminary defect identification results into the improved DEER model, extract the influence path and change magnitude of the process parameters on the prediction results, and generate the process sensitivity score and confidence label for each defect area. By combining the preliminary defect identification results, sensitivity scores, and confidence labels, the final defect judgment result is generated. Feedback information is constructed based on the difference between the final defect judgment result and the process parameters to optimize the parameter configuration of the improved DEER model and the DINOv2 self-supervised model.

[0007] Optionally, the acquisition of multi-source weld images and welding process parameters, synchronized calibration based on a unified timestamp and spatial location identifier, and the construction of a multi-modal weld observation matrix specifically include: Collect multi-source image data to describe the surface structure, temperature distribution, spatial depth and contour features of the weld area. The multi-source image data includes two-dimensional images or three-dimensional dot matrix forms with different physical property channels. A unified timestamp is assigned to each image frame, and the spatial position index corresponding to the image acquisition is calculated based on the trajectory planning information, coordinate reference points and attitude parameters during the welding process. The spatial position index is used to identify the relative position of the image in the weld coordinate system. The system synchronously collects parameter data output by the control system during the welding process. The parameter data includes welding current, welding voltage, welding speed and welding torch trajectory. The image data and parameter data are paired according to a unified timestamp to construct a time-synchronized sample unit. For time-synchronized sample units of the same weld location in multiple image channels, time alignment is performed based on a unified timestamp, and rearrangement is completed according to the spatial location identifier order. Frame data with time differences exceeding a set threshold are removed, and a sequence of image samples rearranged by time and space is generated. Image sample sequences and parameter data are synchronously calibrated, and a correspondence is established based on a unified timestamp and spatial location identifier. A combined data unit containing image data, parameter data, and identifier information is constructed and summarized to form a weld multimodal observation matrix.

[0008] Optionally, the DINOv2 self-supervised model specifically includes: The multi-source weld images in the weld multimodal observation matrix are arranged in a unified timestamp order to form an image input sequence, which is a set of image frames with unified timestamps and spatial location identifiers; The image input sequence is pre-trained using the DINOv2 self-supervised model with a visual Transformer structure. The DINOv2 self-supervised model learns features based on image occlusion reconstruction, image patch restoration, and view comparison tasks. In the image occlusion reconstruction task, a portion of the image is randomly occluded and reconstructed. The occluded area is preferentially set to the weld seam trajectory path and the adjacent area. In the image patch restoration task, the image input sequence is divided into several sub-blocks and their order is shuffled. Image patch restoration training is then performed in the DINOv2 self-supervised model to restore the image order to its original arrangement. In the view comparison task, image frames are selected based on spatial location identifiers to construct a view comparison sample set. When constructing a positive sample pair, image frames with consistent welding process parameters are selected. When constructing a negative sample pair, image frames with differences in at least one of welding current, welding voltage, or welding speed are selected. The feature similarity score of each sample pair is calculated, and the weights of the DINOv2 self-supervised model are updated with the feature similarity score. A cross-frame semantic consistency constraint is introduced. A mapping relationship is established based on the spatial position identifiers of adjacent frames in the image input sequence. Position alignment is performed on the feature vectors generated by adjacent frames, and the semantic difference between the aligned feature vectors is minimized. The dense visual feature representation of each frame in the image input sequence is extracted from the pre-trained DINOv2 self-supervised model. The dense visual feature representation is a set of high-dimensional feature vectors corresponding to each spatial location in the image. Perform inter-frame position alignment processing on consecutive image frames in the image input sequence, including extracting key point features in consecutive frames, calculating the spatial offset between adjacent frames, and performing affine transformation correction on the image frames based on the spatial offset. The image frames are recombined in chronological order to form a weld seam image sequence with a unified timestamp and spatial consistency, and then bound to the dense visual feature representation.

[0009] Optionally, the step of performing difference analysis on adjacent frames in the weld image sequence, extracting spatially varying regions, combining temporal sequence to label potential defect regions, and generating a defect candidate region set specifically includes: Based on a unified timestamp, adjacent image frames in the weld seam image sequence are paired up to construct a set of time-continuous frame pairs. Pixel-level difference calculation is performed on images in a set of consecutive time frames. The pixel-level difference calculation includes channel value difference calculation and image block intensity change evaluation. Local region difference features are extracted by a sliding window method to form a spatial change heat map. Based on the intensity distribution of differences in the spatial change heat map, a multi-threshold segmentation method is used to extract significant change regions. Then, based on the spatial location markers in the continuous time frame pairs, the difference regions are located to the corresponding weld segments to obtain the initial set of change regions. The initial set of changed regions is labeled with time sequence, and the timestamp of the first appearance of each region and the corresponding image frame number are recorded. Changed regions that have the same spatial location and appear repeatedly at different timestamps are merged to generate a candidate set of potential defect regions. The system performs structural reorganization on the set of potential defective regions, constructs the path sequence of the region's evolution over time, and outputs the final aggregation result as a set of defective candidate regions.

[0010] Optionally, the process of fusing dense visual features with welding process parameters corresponding to the time period of the defect candidate region to construct a joint feature state set, classifying each defect candidate region according to a preset anomaly detection rule, and outputting preliminary defect identification results specifically includes: Extract the time period corresponding to each defect candidate region in the weld image sequence, and obtain the dense visual feature sequence within the corresponding time period; Extract the corresponding welding process parameter sequence and align it to each image frame according to the timestamp to form a process parameter vector sequence that corresponds one-to-one with the image frame; The dense visual feature vector of the image frame and the process parameter vector are concatenated along the feature dimension. The Z-score normalization method is used to normalize each dimension of the concatenated feature vector according to the historical mean and standard deviation to generate a joint feature state set. A preset anomaly detection rule set is constructed. The preset anomaly detection rule set is based on the joint feature samples labeled as defects and non-defects in the training samples. The classification boundary is established by fitting the feature distribution boundary and setting the Mahalanobis distance threshold. At the same time, local change rate constraints are introduced to form a composite rule set. Each entry in the joint feature state set is input into a preset anomaly detection rule set for classification, and it is determined whether it belongs to a defect area. The preliminary defect identification result is output, which includes defect markers, time periods, and spatial location identifiers.

[0011] Optionally, the improved DEER model specifically includes: Each defect region in the preliminary defect identification results is paired with the welding process parameters in the corresponding time period to form a defect region-process parameter mapping pair, which is then used as an input sample. An improved DEER model is constructed, which includes a multi-scale dynamic encoder, a causal attribution inference network, a hierarchical path selector, and an adaptive belief fusion device. The multi-scale dynamic encoder adopts a time attention-driven multi-scale residual structure, performs frequency-segmented and layered modeling of process parameters according to time series, sets dynamic weight gating units in each scale branch, automatically adjusts the time window length according to the fluctuation characteristics of the welding stage, calculates the residual mean and residual variance under the current window after each time window sliding cycle, and fuses short-term changes and long-term trends through cross-scale residual connection. The causal attribution reasoning network is based on a graph neural network causal reasoning model. It takes process parameter nodes as input and defect prediction nodes as output. It dynamically generates a directed causal dependency graph through a learnable graph attention layer and introduces a counterfactual channel to perform comparative perturbation simulation on the node state. During the training phase, it performs self-constraint optimization on the edge weights by maximizing the joint objective of prediction consistency and causal separation, forming an interpretable and robust process-defect causal path structure. The hierarchical path selector receives a directed causal dependency graph, identifies a set of reachable paths to the defect with all parameters, and calculates a confidence score for each path based on the activation frequency, average edge weight, and attribution stability in the training set, forming a set of path confidence scores. A dual-gated aggregation mechanism of structure gate and confidence gate is adopted. The structure gate is used to remove paths with stability lower than a preset value, and the confidence gate sorts and truncates the paths according to their confidence scores, outputting a final subset of paths that meet the credibility criteria. The edge weight, confidence score, and activation frequency of each path in the final subset of paths are weighted and aggregated to obtain a path importance score. Normalized weighted fusion is performed on the path set under each defect region to generate a process sensitivity score for the defect region. The adaptive confidence fusion unit is based on the variational Bayesian posterior inference mechanism. It jointly models the path confidence score with the residual variance in the multi-scale dynamic encoder to generate a multi-dimensional confidence distribution. By minimizing the joint loss of path confidence divergence and sample prediction bias, it performs confidence interval estimation and level classification on the output of each defect region and outputs a statistically consistent confidence label. The defect area and welding process parameters are input into the improved DEER model, and then processed sequentially by a multi-scale dynamic encoder, a causal attribution inference network, a hierarchical path selector, and an adaptive confidence fusion unit to output the process sensitivity score and confidence label for each defect area.

[0012] Optionally, the step of combining preliminary defect identification results, sensitivity scores, and confidence labels to generate a final defect judgment result, and constructing feedback information based on the difference between the final defect judgment result and process parameters to optimize the parameter configuration of the improved DEER model and the DINOv2 self-supervised model, specifically including: The preliminary defect identification results, process sensitivity scores, and confidence labels are indexed and aligned according to the defect area number to construct a defect multi-factor fusion table. For each defect area, a fusion judgment is performed, including: when the sensitivity score is higher than the preset association threshold and the confidence label is in the high confidence level range, the defect area is marked as a high confidence process-related defect; when the sensitivity score is low or the confidence label is in the low confidence level range, it is marked as a defect to be confirmed or a non-process-related defect, forming a fusion identification matrix. Based on the fusion recognition matrix and process parameter time series, a defect event triggering window is constructed. Within the time period corresponding to each high-confidence process-related defect area, local fragments of welding process parameters are extracted and compared item by item with the standard process template. The process offset vector is calculated, and the offset difference in all dimensions is statistically analyzed to form a defect-process difference vector set. A feedback sample pool is constructed based on the defect-process difference vector set. Defect regions with significant offsets are selected as key feedback samples to update the parameters of the DINOv2 self-supervised model and the improved DEER model. Set the closed-loop convergence conditions for updating, including the improvement of fusion recognition accuracy, the decrease of model reconstruction residual, and the improvement of sensitivity score stability. When the closed-loop optimization meets all the set conditions, the model update of the current round is completed; otherwise, the remaining samples in the sample pool are fed back to continue iterating, forming an adaptive optimization linkage closed-loop mechanism.

[0013] The beneficial effects of this invention are: This invention addresses the challenges of scarce labeled samples, strong cross-sensor heterogeneity, and spatiotemporal drift and noise interference caused by operating condition fluctuations in prefabricated cabin weld quality inspection by synergistically fusing DINOv2 self-supervised learning with an improved DEER causal attribution model using a visual Transformer architecture. It employs multimodal synchronous calibration and inter-frame registration reconstruction with unified timestamps and spatial location identifiers, combined with occlusion restoration, image reconstruction, and view comparison tasks to obtain dense weld representations with cross-frame semantic consistency. In the temporal change analysis stage, it constructs a heatmap of differences between adjacent frames to stably extract potential defect candidate regions; and in the feature-process fusion stage… The initial defect detection process combines dense visual features with process parameters such as welding current, voltage, speed, and welding torch trajectory over corresponding time periods for joint modeling and rule-based discrimination, outputting preliminary defect results. In the interpretable determination stage, an improved DEER model is introduced to perform causal attribution on the impact path and magnitude of defect prediction, generating process sensitivity scores and confidence labels. Closed-loop feedback based on difference vectors adaptively optimizes DINOv2 and DEER parameters, enabling continuous adaptation and robust upgrades to changing operating conditions. Temporal consistency constraints and multi-source verification suppress common-mode noise and spurious changes, improving the spatial accuracy and temporal stability of defect localization. Ultimately, this achieves high-precision, interpretable, and low-annotation-cost detection of weld defects, supporting online / offline quality control and process traceability, significantly improving defect detection rate, reducing false alarm rate, and shortening detection and review cycles. Attached Figure Description

[0014] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0015] Figure 1 This is a flowchart of a prefabricated cabin weld quality inspection method based on self-supervised learning proposed in this invention; Figure 2 This is a framework diagram of the DINOv2 self-supervised model in the prefabricated cabin weld quality inspection method based on self-supervised learning proposed in this invention; Figure 3 This is a framework diagram of the improved DEER model in the prefabricated cabin weld quality inspection method based on self-supervised learning proposed in this invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0017] refer to Figure 1-3 A method for inspecting the weld quality of prefabricated cabins based on self-supervised learning includes the following steps: Multi-source weld images and welding process parameters are collected and synchronously calibrated based on a unified timestamp and spatial location identifier to construct a multi-modal observation matrix for welds. Image data from the multimodal observation matrix of weld seams are input into the DINOv2 self-supervised model based on the visual Transformer structure to perform occlusion restoration, image reconstruction and view comparison tasks, obtain dense visual features of weld seam images, and perform inter-frame registration and sequence reconstruction on multi-source weld seam images to generate a weld seam image sequence. Perform difference analysis on adjacent frames in the weld seam image sequence to extract spatially varying regions, combine temporal order to label potential defect regions, and generate a set of defect candidate regions. The dense visual features are fused with the welding process parameters of the time period corresponding to the defect candidate region to construct a joint feature state set. Each defect candidate region is classified according to the preset anomaly detection rules, and the preliminary defect identification results are output. Input the defect areas and corresponding welding process parameters from the preliminary defect identification results into the improved DEER model, extract the influence path and change magnitude of the process parameters on the prediction results, and generate the process sensitivity score and confidence label for each defect area. By combining the preliminary defect identification results, sensitivity scores, and confidence labels, the final defect judgment result is generated. Feedback information is constructed based on the difference between the final defect judgment result and the process parameters to optimize the parameter configuration of the improved DEER model and the DINOv2 self-supervised model.

[0018] In this embodiment, the acquisition of multi-source weld images and welding process parameters, synchronous calibration based on a unified timestamp and spatial location identifier, and the construction of a multi-modal weld observation matrix specifically include: Collect multi-source image data to describe the surface structure, temperature distribution, spatial depth and contour features of the weld area. The multi-source image data includes two-dimensional images or three-dimensional dot matrix forms with different physical property channels. A unified timestamp is assigned to each image frame, and the spatial position index corresponding to the image acquisition is calculated based on the trajectory planning information, coordinate reference points and attitude parameters during the welding process. The spatial position index is used to identify the relative position of the image in the weld coordinate system. The system synchronously collects parameter data output by the control system during the welding process. The parameter data includes welding current, welding voltage, welding speed, and welding torch trajectory. The welding current is the actual current sample value output by the welding power source, the welding voltage is the actual voltage sample value output by the welding power source, the welding speed is the ratio of the displacement change of the welding torch in a continuous time period to the time interval, and the welding torch trajectory is the time-position continuous path recorded by the welding control system. The image data and parameter data are paired according to a unified timestamp to construct a time-synchronized sample unit. For time-synchronized sample units of the same weld location in multiple image channels, time alignment is performed based on a unified timestamp, and rearrangement is completed according to the spatial location identifier order. Frame data with time differences exceeding a set threshold are removed, and a sequence of image samples rearranged by time and space is generated. Image sample sequences and parameter data are synchronously calibrated, and a correspondence is established based on a unified timestamp and spatial location identifier. A combined data unit containing image data, parameter data, and identifier information is constructed and summarized to form a weld multimodal observation matrix.

[0019] This implementation method constructs a multimodal observation matrix for welds by unifying the timestamps and spatial location identifiers of multi-source images, thereby achieving precise synchronization and structured fusion of cross-modal data and significantly improving the stability and consistency of subsequent joint analysis of visual and process parameters.

[0020] In this embodiment, the DINOv2 self-supervised model specifically includes: The multi-source weld images in the weld multimodal observation matrix are arranged in a unified timestamp order to form an image input sequence, which is a set of image frames with unified timestamps and spatial location identifiers; The image input sequence is pre-trained using the DINOv2 self-supervised model with a visual Transformer structure. The DINOv2 self-supervised model learns features based on image occlusion reconstruction, image patch restoration, and view comparison tasks. In the image occlusion reconstruction task, a portion of the image is randomly occluded and reconstructed to learn the image context information. The occluded area is preferentially set to the weld trajectory path and the adjacent area to enhance the DINOv2 self-supervised model's ability to learn features of key parts of the weld. In the image block restoration task, the image input sequence is divided into several sub-blocks and then the order is shuffled. Image block restoration training is then performed in the DINOv2 self-supervised model to restore the image order to the original arrangement in order to maintain the local consistency of the weld structure. In the view comparison task, image frames are selected based on spatial location identifiers to construct a view comparison sample set. When constructing a positive sample pair, image frames with consistent welding process parameters are selected. When constructing a negative sample pair, image frames with differences in at least one of welding current, welding voltage, or welding speed are selected. The feature similarity score of each sample pair is calculated, and the weights of the DINOv2 self-supervised model are updated with the feature similarity score to improve the ability to distinguish image differences caused by process changes. A cross-frame semantic consistency constraint is introduced. A mapping relationship is established based on the spatial position identifiers of adjacent frames in the image input sequence. Position alignment is performed on the feature vectors generated by adjacent frames, and the semantic difference between the aligned feature vectors is minimized to improve the stability and continuous expressive ability of the model in weld seam image sequences. The dense visual feature representation of each frame in the image input sequence is extracted from the pre-trained DINOv2 self-supervised model. The dense visual feature representation is a set of high-dimensional feature vectors corresponding to each spatial location in the image. Perform inter-frame position alignment processing on consecutive image frames in the image input sequence, including extracting key point features in consecutive frames, calculating the spatial offset of adjacent frames, and performing affine transformation correction on the image frames based on the spatial offset to keep adjacent frames consistent in spatial position. The image frames are recombined in chronological order to form a weld seam image sequence with a unified timestamp and spatial consistency, and then bound to the dense visual feature representation.

[0021] This implementation introduces the DINOv2 self-supervised model based on the visual Transformer structure. Without requiring manual annotation, it leverages a multi-task training mechanism including occlusion restoration, image patch reconstruction, and view comparison to fully capture minute structural changes and regional semantic features in weld images, significantly improving the modeling capability for complex weld images. Through cross-frame semantic consistency constraints and inter-frame registration methods, it ensures the consistency of image sequences in spatial location and the continuity in the temporal dimension, enhancing the sensitivity of feature representation to the dynamic evolution of the weld. The resulting dense visual features possess advantages such as high spatial resolution and strong semantic expression, providing a solid data foundation for subsequent defect detection and causal analysis, and effectively improving the system's robustness and generalization performance in multi-source image scenarios.

[0022] In this embodiment, the step of performing difference analysis on adjacent frames in the weld image sequence, extracting spatially varying regions, marking potential defect regions in conjunction with temporal sequence, and generating a defect candidate region set specifically includes: Based on a unified timestamp, adjacent image frames in the weld seam image sequence are paired up to construct a set of time-continuous frame pairs. Pixel-level difference calculation is performed on images in a set of consecutive time frames. The pixel-level difference calculation includes channel value difference calculation and image block intensity change evaluation. Local region difference features are extracted by a sliding window method to form a spatial change heat map. Based on the intensity distribution of differences in the spatial change heat map, a multi-threshold segmentation method is used to extract significant change regions. Then, based on the spatial location markers in the continuous time frame pairs, the difference regions are located to the corresponding weld segments to obtain the initial set of change regions. The initial set of changed regions is labeled with time sequence, and the timestamp of the first appearance of each region and the corresponding image frame number are recorded. Changed regions that have the same spatial location and appear repeatedly at different timestamps are merged to generate a candidate set of potential defect regions. The system performs structural reorganization on the set of potential defective regions, constructs the path sequence of the region's evolution over time, and outputs the final aggregation result as a set of defective candidate regions.

[0023] This implementation method constructs the spatial-temporal evolution path of potential defect regions by performing differential analysis and temporal aggregation in the weld seam image sequence, thereby effectively capturing the dynamic defect change process and improving the timeliness and continuous identification capability of defect detection.

[0024] In this embodiment, the process of fusing dense visual features with welding process parameters corresponding to the time period of the defect candidate region to construct a joint feature state set, classifying each defect candidate region according to a preset anomaly detection rule, and outputting preliminary defect identification results specifically includes: Extract the time period corresponding to each defect candidate region in the weld image sequence, and obtain the dense visual feature sequence within the corresponding time period; Extract the corresponding welding process parameter sequence, including welding current, welding voltage, welding speed and welding torch trajectory, and align it to each frame of the image according to the timestamp to form a process parameter vector sequence that corresponds one-to-one with the image frame. The dense visual feature vector of the image frame and the process parameter vector are concatenated along the feature dimension. The Z-score normalization method is used to normalize each dimension of the concatenated feature vector according to the historical mean and standard deviation to generate a joint feature state set. A preset anomaly detection rule set is constructed. The preset anomaly detection rule set is based on the joint feature samples labeled as defects and non-defects in the training samples. The classification boundary is established by fitting the feature distribution boundary and setting the Mahalanobis distance threshold. At the same time, a local change rate constraint is introduced to form a composite rule set. The local change rate constraint is calculated by calculating the change amplitude of the joint feature state of adjacent time slices. Based on the feature fluctuation range of defect and non-defect regions in historical samples, a rate threshold is set and used for anomaly determination. Each entry in the joint feature state set is input into a preset anomaly detection rule set for classification, and it is determined whether it belongs to a defect area. The preliminary defect identification result is output, which includes defect markers, time periods, and spatial location identifiers.

[0025] This implementation method integrates dense visual features with welding process parameters to construct a joint feature state set, and introduces anomaly detection rule set and local change rate constraints to achieve early identification and classification of defect causes, thereby improving the accuracy and robustness of preliminary identification.

[0026] In this embodiment, the improved DEER model specifically includes: Each defect region in the preliminary defect identification results is paired with the welding process parameters in the corresponding time period to form a defect region-process parameter mapping pair, which is then used as an input sample. An improved DEER model is constructed, which includes a multi-scale dynamic encoder, a causal attribution inference network, a hierarchical path selector, and an adaptive belief fusion device. The multi-scale dynamic encoder adopts a time attention-driven multi-scale residual structure to perform frequency-segmented and layered modeling of process parameters according to time series. Dynamic weight gating units are set in each scale branch. The time window length is automatically adjusted according to the fluctuation characteristics of the welding stage. After each time window sliding cycle, the residual mean and residual variance under the current window are calculated. Short-term changes and long-term trends are fused through cross-scale residual connection. Under unsupervised conditions, the correspondence between process disturbances and potential defect responses is adaptively captured. The causal attribution reasoning network is based on a graph neural network causal reasoning model. It takes process parameter nodes as input and defect prediction nodes as output. It dynamically generates a directed causal dependency graph through a learnable graph attention layer and introduces a counterfactual channel to perform comparative perturbation simulation on the node state. During the training phase, it performs self-constraint optimization on the edge weights by maximizing the joint objective of prediction consistency and causal separation, forming an interpretable and robust process-defect causal path structure. The hierarchical path selector receives a directed causal dependency graph, identifies a set of reachable paths to the defect with all parameters, and calculates a confidence score for each path based on the activation frequency, average edge weight, and attribution stability in the training set, forming a set of path confidence scores. A dual-gated aggregation mechanism of structure gate and confidence gate is adopted. The structure gate is used to remove paths with stability lower than a preset value, and the confidence gate sorts and truncates the paths according to their confidence scores, outputting a final subset of paths that meet the credibility criteria. The edge weight, confidence score, and activation frequency of each path in the final subset of paths are weighted and aggregated to obtain a path importance score. Normalized weighted fusion is performed on the path set under each defect region to generate a process sensitivity score for the defect region. The adaptive confidence fusion unit is based on the variational Bayesian posterior inference mechanism. It jointly models the path confidence score with the residual variance in the multi-scale dynamic encoder to generate a multi-dimensional confidence distribution. By minimizing the joint loss of path confidence divergence and sample prediction bias, it performs confidence interval estimation and level classification on the output of each defect region and outputs a statistically consistent confidence label. The defect area and welding process parameters are input into the improved DEER model, and then processed sequentially by a multi-scale dynamic encoder, a causal attribution inference network, a hierarchical path selector, and an adaptive confidence fusion unit to output the process sensitivity score and confidence label for each defect area.

[0027] This implementation systematically solves the challenge of causal modeling between welding process parameters and defect regions by constructing an improved DEER model that includes a multi-scale dynamic encoder, a causal attribution inference network, a hierarchical path selector, and an adaptive confidence fusion unit. The multi-scale dynamic encoder models the temporal characteristics of welding parameters using a time attention mechanism and residual structure, taking into account both short-term anomalies and long-term trend changes. The causal inference network, based on a graph neural network structure, achieves interpretable defect generation path learning, effectively avoiding the fuzzy processing of predictive factors in traditional black-box models. The hierarchical path selector fuses path activation frequency and confidence weights, using a dual-gating strategy to select high-stability paths, enhancing the credibility of attribution. The adaptive confidence fusion unit, based on a Bayesian mechanism, implements confidence level classification, providing reliable region credibility labels. The overall model not only improves the accuracy of process anomaly tracing but also provides causal interpretability for defect determination results, significantly enhancing the system's controllability and intelligence under complex welding conditions.

[0028] In this embodiment, the step of combining preliminary defect identification results, sensitivity scores, and confidence labels to generate a final defect determination result, and constructing feedback information based on the difference between the final defect determination result and process parameters to optimize the parameter configuration of the improved DEER model and the DINOv2 self-supervised model, specifically includes: The preliminary defect identification results, process sensitivity scores, and confidence labels are indexed and aligned according to the defect region number to construct a multi-factor fusion table for defects. For each defect region, a fusion judgment is performed, including: when the sensitivity score is higher than a preset association threshold and the confidence label is in the high confidence level range, the defect region is marked as a high-confidence process-related defect; when the sensitivity score is lower or the confidence label is in the low confidence level range, it is marked as a defect to be confirmed or a non-process-related defect, forming a fusion identification matrix. The high confidence level range refers to the confidence score range in the training set that is higher than the high confidence threshold, set according to the distribution of confidence labels and percentiles. The low confidence level range refers to the confidence score range in the training set that is lower than the low confidence threshold, set according to the distribution of confidence labels and percentiles. Based on the fusion recognition matrix and process parameter time series, a defect event triggering window is constructed. Within the time period corresponding to each high-confidence process-related defect area, local fragments of welding process parameters are extracted and compared item by item with the standard process template. The process offset vector is calculated, and the offset difference in all dimensions is statistically analyzed to form a defect-process difference vector set. A feedback sample pool is constructed based on the defect-process difference vector set. Defect regions with significant offsets are selected as key feedback samples to update the parameters of the DINOv2 self-supervised model and the improved DEER model. Set the closed-loop convergence conditions for updating, including the improvement of fusion recognition accuracy, the decrease of model reconstruction residual, and the improvement of sensitivity score stability. When the closed-loop optimization meets all the set conditions, the model update of the current round is completed; otherwise, the remaining samples in the sample pool are fed back to continue iterating, forming an adaptive optimization linkage closed-loop mechanism.

[0029] This implementation method constructs a fusion identification matrix and a defect-process difference vector set to form a feedback sample pool and execute a model update closed loop, thereby achieving adaptive optimization of the DINOv2 and DEER models and effectively improving the stability and self-evolution capability of the detection system.

[0030] Example 1: To verify the feasibility of this invention in practice, it was applied to the box assembly welding production line of a precast power module manufacturing company. Visible light, infrared, and depth cameras were installed on-site, and data on current, voltage, welding speed, and welding torch trajectory were collected synchronously with the welding machine control system. Multi-channel alignment was achieved using unified timestamps and spatial location identifiers. Without adding additional dense pixel annotations, the DINOv2 visual Transformer structure was used to perform occlusion restoration, image reconstruction, and view comparison, learning the dense features of weld contours, molten pool boundaries, and heat-affected zones. Optical flow and keypoint registration were then used to achieve cross-frame alignment, constructing a continuous weld image sequence. Difference analysis was performed on adjacent frames to obtain a change heatmap and generate candidate regions. These candidate regions were then combined with corresponding time-period process parameters to form joint features, and preliminary defects were obtained according to historical distribution and threshold rules. The preliminary results and their process sequence were input into an improved DEER model, outputting a process sensitivity score and confidence label to form an interpretable final judgment. A feedback sample pool was constructed using the judgment-process difference for small-step online optimization, ensuring the system's robustness with changes in material batches and operating conditions.

[0031] The model deployment utilizes edge GPUs for online inference, maintaining average latency within the production line cycle time. In the self-supervised phase, only unlabeled images are used to build the feature library, with limited annotations used solely for rule and hyperparameter selection. The system is compared with a traditional combined approach under the same week, production line, and batch conditions. The traditional approach consists of infrared temperature rise threshold detection, manual visual inspection, and randomized ultrasonic sampling.

[0032] Table 1 Comparison of Quality Indicators

[0033] To verify the value of interpretability for process optimization, the mean process sensitivity of different defects was statistically analyzed. The higher the value, the more significant the impact of the process variable on this type of defect.

[0034] Table 2. Process sensitivity data of typical defects

[0035] The significant improvement in defect detection rate, as shown in Tables 1 and 2, stems from two main factors. First, the combination of self-supervised features and cross-frame registration suppresses spurious changes, concentrating candidates on genuine geometric and thermal texture anomalies. Second, joint modeling with process parameters provides strong priors, enabling the system to distinguish artifacts such as high-temperature reflections from genuine pores when current fluctuations and speed anomalies occur simultaneously. The significant decrease in false alarm rate is closely related to the confidence screening of causal paths in the improved DEER model. Samples with single-point anomalies but lacking path support are automatically downweighted and placed in the confirmation queue, reducing interference with the work cycle. The reduction in positioning error primarily benefits from the consistency of position after registration; millimeter-level 95th percentile errors facilitate rapid rework and trajectory playback. Online latency is significantly lower than traditional processes, alarms can be completed in a closed loop within the same work step, reducing manual review workload and allowing quality control personnel to shift from repetitive inspections to complex sample judgment and process window optimization.

[0036] In actual production operations, the system binds each frame of image to the tooling coordinates and weld baseline, achieving spatial traceability. After a candidate region is triggered, it automatically backtracks the current, voltage, speed, and welding torch attitude curves of the corresponding time slice to generate a difference vector. If the sensitivity and confidence are high, a strong alarm is issued and the MES is linked to generate a rework order and positioning scale. If the confidence is insufficient, targeted re-inspection or an extended observation window is triggered to avoid mishandling. After the shift, the difference between the final judgment and the standard process template is used as a seed to update a small number of parameters of the feature extractor and DEER, achieving lightweight adaptation.

[0037] In conjunction with the above embodiments, this invention achieves high-precision, low-latency, and highly interpretable detection of prefabricated cabin welds without incurring significant annotation costs. Compared to traditional methods, the detection rate is increased by approximately 11 percentage points, the false alarm rate is halved, the positioning accuracy is improved to the millimeter level, and the rework rate and review time are significantly reduced, forming an efficient closed loop from detection, positioning, rework to review. Simultaneously, the output process sensitivity indicators intuitively show the impact path of parameters on defects, providing a quantitative basis for subsequent process window setting and equipment maintenance, and thus possesses promotional value.

[0038] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for inspecting the quality of welds in prefabricated cabins based on self-supervised learning, characterized in that, Includes the following steps: Multi-source weld images and welding process parameters are collected and synchronously calibrated based on a unified timestamp and spatial location identifier to construct a multi-modal observation matrix for welds. Image data from the multimodal observation matrix of weld seams are input into the DINOv2 self-supervised model based on the visual Transformer structure to perform occlusion restoration, image reconstruction and view comparison tasks, obtain dense visual features of weld seam images, and perform inter-frame registration and sequence reconstruction on multi-source weld seam images to generate a weld seam image sequence. Perform difference analysis on adjacent frames in the weld seam image sequence to extract spatially varying regions, combine temporal order to label potential defect regions, and generate a set of defect candidate regions. The dense visual features are fused with the welding process parameters of the time period corresponding to the defect candidate region to construct a joint feature state set. Each defect candidate region is classified according to the preset anomaly detection rules, and the preliminary defect identification results are output. Input the defect areas and corresponding welding process parameters from the preliminary defect identification results into the improved DEER model, extract the influence path and change magnitude of the process parameters on the prediction results, and generate the process sensitivity score and confidence label for each defect area. By combining the preliminary defect identification results, sensitivity scores, and confidence labels, the final defect judgment result is generated. Feedback information is constructed based on the difference between the final defect judgment result and the process parameters to optimize the parameter configuration of the improved DEER model and the DINOv2 self-supervised model.

2. The method for inspecting the quality of prefabricated cabin welds based on self-supervised learning according to claim 1, characterized in that, The acquisition of multi-source weld images and welding process parameters is synchronized and calibrated based on a unified timestamp and spatial location identifier to construct a multi-modal weld observation matrix, specifically including: Collect multi-source image data to describe the surface structure, temperature distribution, spatial depth and contour features of the weld area. The multi-source image data includes two-dimensional images or three-dimensional dot matrix forms with different physical property channels. A unified timestamp is assigned to each image frame, and the spatial position index corresponding to the image acquisition is calculated based on the trajectory planning information, coordinate reference points and attitude parameters during the welding process. The spatial position index is used to identify the relative position of the image in the weld coordinate system. The system synchronously collects parameter data output by the control system during the welding process. The parameter data includes welding current, welding voltage, welding speed and welding torch trajectory. The image data and parameter data are paired according to a unified timestamp to construct a time-synchronized sample unit. For time-synchronized sample units of the same weld location in multiple image channels, time alignment is performed based on a unified timestamp, and rearrangement is completed according to the spatial location identifier order. Frame data with time differences exceeding a set threshold are removed, and a sequence of image samples rearranged by time and space is generated. Image sample sequences and parameter data are synchronously calibrated, and a correspondence is established based on a unified timestamp and spatial location identifier. A combined data unit containing image data, parameter data, and identifier information is constructed and summarized to form a weld multimodal observation matrix.

3. The method for inspecting the quality of prefabricated cabin welds based on self-supervised learning according to claim 1, characterized in that, The DINOv2 self-supervised model specifically includes: The multi-source weld images in the weld multimodal observation matrix are arranged in a unified timestamp order to form an image input sequence, which is a set of image frames with unified timestamps and spatial location identifiers; The image input sequence is pre-trained using the DINOv2 self-supervised model with a visual Transformer structure. The DINOv2 self-supervised model learns features based on image occlusion reconstruction, image patch restoration, and view comparison tasks. In the image occlusion reconstruction task, a portion of the image is randomly occluded and reconstructed. The occluded area is preferentially set to the weld seam trajectory path and the adjacent area. In the image patch restoration task, the image input sequence is divided into several sub-blocks and their order is shuffled. Image patch restoration training is then performed in the DINOv2 self-supervised model to restore the image order to its original arrangement. In the view comparison task, image frames are selected based on spatial location identifiers to construct a view comparison sample set. When constructing a positive sample pair, image frames with consistent welding process parameters are selected. When constructing a negative sample pair, image frames with differences in at least one of welding current, welding voltage, or welding speed are selected. The feature similarity score of each sample pair is calculated, and the weights of the DINOv2 self-supervised model are updated with the feature similarity score. A cross-frame semantic consistency constraint is introduced. A mapping relationship is established based on the spatial position identifiers of adjacent frames in the image input sequence. Position alignment is performed on the feature vectors generated by adjacent frames, and the semantic difference between the aligned feature vectors is minimized. The dense visual feature representation of each frame in the image input sequence is extracted from the pre-trained DINOv2 self-supervised model. The dense visual feature representation is a set of high-dimensional feature vectors corresponding to each spatial location in the image. Perform inter-frame position alignment processing on consecutive image frames in the image input sequence, including extracting key point features in consecutive frames, calculating the spatial offset between adjacent frames, and performing affine transformation correction on the image frames based on the spatial offset. The image frames are recombined in chronological order to form a weld seam image sequence with a unified timestamp and spatial consistency, and then bound to the dense visual feature representation.

4. The method for inspecting the quality of prefabricated cabin welds based on self-supervised learning according to claim 1, characterized in that, The process of performing difference analysis on adjacent frames in the weld image sequence, extracting spatially varying regions, combining temporal sequence to label potential defect regions, and generating a defect candidate region set specifically includes: Based on a unified timestamp, adjacent image frames in the weld seam image sequence are paired up to construct a set of time-continuous frame pairs. Pixel-level difference calculation is performed on images in a set of consecutive time frames. The pixel-level difference calculation includes channel value difference calculation and image block intensity change evaluation. Local region difference features are extracted by a sliding window method to form a spatial change heat map. Based on the intensity distribution of differences in the spatial change heat map, a multi-threshold segmentation method is used to extract significant change regions. Then, based on the spatial location markers in the continuous time frame pairs, the difference regions are located to the corresponding weld segments to obtain the initial set of change regions. The initial set of changed regions is labeled with time sequence, and the timestamp of the first appearance of each region and the corresponding image frame number are recorded. Changed regions that have the same spatial location and appear repeatedly at different timestamps are merged to generate a candidate set of potential defect regions. The system performs structural reorganization on the set of potential defective regions, constructs the path sequence of the region's evolution over time, and outputs the final aggregation result as a set of defective candidate regions.

5. The method for inspecting the quality of prefabricated cabin welds based on self-supervised learning according to claim 1, characterized in that, The process of fusing dense visual features with welding process parameters corresponding to the time period of the defect candidate region to construct a joint feature state set, classifying each defect candidate region according to a preset anomaly detection rule, and outputting preliminary defect identification results specifically includes: Extract the time period corresponding to each defect candidate region in the weld image sequence, and obtain the dense visual feature sequence within the corresponding time period; Extract the corresponding welding process parameter sequence and align it to each image frame according to the timestamp to form a process parameter vector sequence that corresponds one-to-one with the image frame; The dense visual feature vector of the image frame and the process parameter vector are concatenated along the feature dimension. The Z-score normalization method is used to normalize each dimension of the concatenated feature vector according to the historical mean and standard deviation to generate a joint feature state set. A preset anomaly detection rule set is constructed. The preset anomaly detection rule set is based on the joint feature samples labeled as defects and non-defects in the training samples. The classification boundary is established by fitting the feature distribution boundary and setting the Mahalanobis distance threshold. At the same time, local change rate constraints are introduced to form a composite rule set. Each entry in the joint feature state set is input into a preset anomaly detection rule set for classification, and it is determined whether it belongs to a defect area. The preliminary defect identification result is output, which includes defect markers, time periods, and spatial location identifiers.

6. The method for inspecting the quality of prefabricated cabin welds based on self-supervised learning according to claim 1, characterized in that, The improved DEER model specifically includes: Each defect region in the preliminary defect identification results is paired with the welding process parameters in the corresponding time period to form a defect region-process parameter mapping pair, which is then used as an input sample. An improved DEER model is constructed, which includes a multi-scale dynamic encoder, a causal attribution inference network, a hierarchical path selector, and an adaptive belief fusion device. The multi-scale dynamic encoder adopts a time attention-driven multi-scale residual structure, performs frequency-segmented and layered modeling of process parameters according to time series, sets dynamic weight gating units in each scale branch, automatically adjusts the time window length according to the fluctuation characteristics of the welding stage, calculates the residual mean and residual variance under the current window after each time window sliding cycle, and fuses short-term changes and long-term trends through cross-scale residual connection. The causal attribution reasoning network is based on a graph neural network causal reasoning model. It takes process parameter nodes as input and defect prediction nodes as output. It dynamically generates a directed causal dependency graph through a learnable graph attention layer and introduces a counterfactual channel to perform comparative perturbation simulation on the node state. During the training phase, it performs self-constraint optimization on the edge weights by maximizing the joint objective of prediction consistency and causal separation, forming an interpretable and robust process-defect causal path structure. The hierarchical path selector receives a directed causal dependency graph, identifies a set of reachable paths to the defect with all parameters, and calculates a confidence score for each path based on the activation frequency, average edge weight, and attribution stability in the training set, forming a set of path confidence scores. A dual-gated aggregation mechanism of structure gate and confidence gate is adopted. The structure gate is used to remove paths with stability lower than a preset value, and the confidence gate sorts and truncates the paths according to their confidence scores, outputting a final subset of paths that meet the credibility criteria. The edge weight, confidence score, and activation frequency of each path in the final subset of paths are weighted and aggregated to obtain a path importance score. Normalized weighted fusion is performed on the path set under each defect region to generate a process sensitivity score for the defect region. The adaptive confidence fusion unit is based on the variational Bayesian posterior inference mechanism. It jointly models the path confidence score with the residual variance in the multi-scale dynamic encoder to generate a multi-dimensional confidence distribution. By minimizing the joint loss of path confidence divergence and sample prediction bias, it performs confidence interval estimation and level classification on the output of each defect region and outputs a statistically consistent confidence label. The defect area and welding process parameters are input into the improved DEER model, and then processed sequentially by a multi-scale dynamic encoder, a causal attribution inference network, a hierarchical path selector, and an adaptive confidence fusion unit to output the process sensitivity score and confidence label for each defect area.

7. The method for inspecting the quality of prefabricated cabin welds based on self-supervised learning according to claim 1, characterized in that, The process combines preliminary defect identification results, sensitivity scores, and confidence labels to generate a final defect determination result. Feedback information is then constructed based on the difference between the final defect determination result and process parameters to optimize the parameter configuration of the improved DEER model and the DINOv2 self-supervised model. Specifically, this includes: The preliminary defect identification results, process sensitivity scores, and confidence labels are indexed and aligned according to the defect area number to construct a multi-factor fusion table for defects. For each defect area, a fusion judgment is performed, including: when the sensitivity score is higher than the preset association threshold and the confidence label is in the high confidence level range, the defect area is marked as a high confidence process-related defect; when the sensitivity score is low or the confidence label is in the low confidence level range, it is marked as a defect to be confirmed or a non-process-related defect, forming a fusion identification matrix. Based on the fusion recognition matrix and process parameter time series, a defect event triggering window is constructed. Within the time period corresponding to each high-confidence process-related defect area, local fragments of welding process parameters are extracted and compared item by item with the standard process template. The process offset vector is calculated, and the offset difference in all dimensions is statistically analyzed to form a defect-process difference vector set. A feedback sample pool is constructed based on the defect-process difference vector set. Defect regions with significant offsets are selected as key feedback samples to update the parameters of the DINOv2 self-supervised model and the improved DEER model. Set the closed-loop convergence conditions for updating, including the improvement of fusion recognition accuracy, the decrease of model reconstruction residual, and the improvement of sensitivity score stability. When the closed-loop optimization meets all the set conditions, the model update of the current round is completed; otherwise, the remaining samples in the sample pool are fed back to continue iterating, forming an adaptive optimization linkage closed-loop mechanism.

Images