Additive manufacturing defect detection method of VQ-VCAE and isolated forest based on molten pool video

Abstract

The invention discloses a VQ-VCAE and isolated forest additive manufacturing defect detection method based on a molten pool video. According to the method, video frames in the manufacturing process are collected in real time through a molten pool camera, and a VQ-VAE model fused with a channel attention mechanism is constructed and trained so as to detect and recognize abnormal defects in the manufacturing process. The method comprises the following specific steps: extracting a training set from a defect-free video, generating a reconstructed image and calculating a residual image, identifying an abnormal frame through an incremental mean value and variance updating algorithm, performing pixel-level segmentation on an abnormal region, and processing and refining a segmentation result in combination with morphology. According to the invention, real-time on-line monitoring of electric arc additive manufacturing can be realized, high-precision defect detection and segmentation capabilities are realized, the manufacturing quality consistency and the production efficiency are effectively improved, and technical support is provided for intelligent manufacturing and green manufacturing.

Description

Technical Field

[0001] The present invention belongs to the technical field of arc additive manufacturing, and relates to an online defect monitoring method for arc additive manufacturing, and specifically to an online defect detection method for arc additive manufacturing based on fusion channel attention VQ-VCAE and temporal isolation forest of molten pool video. Background Art

[0002] Arc additive manufacturing is very suitable for manufacturing complex and large parts by depositing materials layer by layer. It has been widely used in aerospace, shipbuilding, automobile and energy fields (Tan C, Li R, Su J, et al. Review on field assisted metal additive manufacturing [J]. International Journal of Machine Tools and Manufacture, 2023, 189: 104032.). Compared with subtractive manufacturing, its rapid melting and cooling triggers complex thermal cycles, which are prone to thermal stress and cause deformation or cracking. The temperature and shape of the molten pool are difficult to control accurately, and pores and inclusions are easily formed. The slight deviations of layer-by-layer stacking will continue to accumulate, affecting the structural accuracy and mechanical properties (Song H, Li C, Fu Y, et al. A two-stage unsupervised approach for surface anomaly detection in wire and arc additive manufacturing [J]. Computers in Industry, 2023, 151: 103994.). Different properties of materials will lead to differences in thermal expansion and fluidity. Improper parameter selection will greatly increase the defect rate (Li W, Zhang H, Wang G, et al. Deep learning based online metallic surface defect detection method for wire and arc additive manufacturing [J]. Robotics and Computer-Integrated Manufacturing, 2023, 80: 102470.). If defects are not discovered in time, subsequent stacking will further deteriorate the quality of the finished product, or even scrap it, increasing the cost of repair (Xia C, Pan Z, Polden J, et al. Modelling and prediction of surface roughness in wire arc additive manufacturing using machine learning [J]. Journal of Intelligent Manufacturing, 2022, 33 (5): 1467-1482.). Therefore, real-time monitoring is crucial. Compared with traditional manufacturing methods, the quality monitoring requirements of arc additive manufacturing are significantly higher.

[0003] For arc additive manufacturing, usually, each layer of welding wire melts and re-solidifies under the action of the arc, and accumulates to form a new layer. The thickness of each layer is usually between 0.5mm and several millimeters (Hajili S. Welding processes for joining dissimilar metals and plastics [J]. School of Industrial and Information Engineering, 2018, 1: 12-14.). After each arc action, the welding gun is moved to the position of the next layer, and new welding wire is added for melting, and then the process is repeated until a stable three-dimensional (3D) structure is completed (Liu J, Xu Y, Ge Y, et al. Wire and arc additive manufacturing of metal components: a review of recent research developments [J]. The International Journal of Advanced Manufacturing Technology, 2020, 111 (1): 149-198.). Each "build" process contains hundreds to thousands of layers of stacking, so each run may take several hours to several days. A "build" can generate one or more complex, durable large parts.

[0004] Considering the above issues, a major challenge in the application of arc additive manufacturing technology, especially in the production of key structural components, is how to effectively identify and monitor defects during the manufacturing process and adjust process parameters in real time to ensure the quality of parts. The dynamic characteristics of the additive manufacturing process make defect detection more difficult, and if these defects are not discovered in time, they will seriously affect the performance and service life of the parts. At present, many studies and reports have emphasized that the combination of online, closed-loop process control systems and sensor technology can improve the quality assurance of additive manufacturing and achieve consistency and reproducibility. These technological innovations provide strong support for the widespread application of arc additive manufacturing and are expected to further promote its application in high-demand application fields (Liu C, Law ACC, Roberson D, et al. Image analysis-based closed loop quality control for additive manufacturing with fusedfilament fabrication [J]. Journal of Manufacturing Systems, 2019, 51: 75-86.).

[0005] As the demand for high-performance, complex structural parts in industries such as aerospace continues to grow, the quality standards of additive manufacturing are also gradually improving. These industries have put forward higher requirements for precise control, structural consistency and performance stability of finished products during the manufacturing process, which are difficult to fully meet with traditional detection methods. To this end, the field of additive manufacturing needs to develop more advanced defect detection methods to identify and locate defects in real time during the manufacturing process, thereby optimizing process parameters and reducing quality fluctuations. Summary of the invention

[0006] The purpose of the present invention is to overcome the problems existing in the above-mentioned technology and provide an online defect detection method for arc additive manufacturing based on fusion channel attention VQ-VCAE and temporal isolation forest based on molten pool video.

[0007] The technical solution of the present invention:

[0008] An arc additive manufacturing online defect detection method based on fusion channel attention VQ-VCAE and temporal isolation forest based on molten pool video includes the following steps:

[0009] (1) Obtain a dataset consisting of video frames captured by a molten pool camera during a defect-free manufacturing process as a training set (Train-frames) for training a VQ-VCAE model fused with channel attention (VQ-VCAE-attention);

[0010] Among them, the VQ-VCAE model integrating channel attention is mainly composed of the VQ-VCAE model, channel attention mechanism and loss function;

[0011] 1) The VQ-VCAE module is based on the vector quantized variational autoencoder (VQ-VAE), and its architecture includes an encoder, a quantization layer, and a decoder. The encoder and the decoder are connected in sequence to form an encoder-decoder structure. In the encoder-decoder structure, the quantization layer is inserted into the latent feature maps of different levels after the encoder convolution processing. Specifically, the quantization layer processes the latent feature map after the output of the latent feature map, maps the continuous latent feature map into a discrete code book vector, and realizes the quantization of the latent feature map. The quantization process can help the model capture discrete patterns in the image while retaining key information and improving the model's expressiveness. In addition, according to the complexity of different tasks, multiple quantization layers are introduced to enhance the feature expression ability, and cross-layer information is retained through jump connections, thereby optimizing information flow and improving overall performance.

[0012] Encoder: The encoder converts the input video frame into multiple levels of latent feature maps through multiple convolution operations;

[0013] Quantization layer: After being processed by the encoder, a latent feature map of a certain level is obtained, whose dimension is C×H×W, where C is the number of channels (i.e. the dimension of each position), H and W represent the height and width of the latent feature map respectively; for any position (h, w), it corresponds to a vector of dimension C, representing the latent feature of that position; define a set codebook, which contains n C-dimensional vectors; each vector in the codebook is a reference point in the latent space, and each C-dimensional vector of the latent feature map is quantized to a discrete vector in the codebook; quantization steps: for the C-dimensional vector of each position in the latent feature map, calculate its Euclidean distance with all vectors in the codebook; select the codebook vector with the smallest distance as the quantized representation of that position, the formula is as follows:

[0014]

[0015] Among them, q(x i,j ) represents the final selected code book vector, which is a vector selected from the code book; z e (x i,j ) represents the C-dimensional vector at position (i, j) in the feature map in the latent space, representing the feature to be quantized; e k ∈E represents the reference vector in the codebook E, where E = {e1, e2, ..., en}; ||z e (x i,j )-e k || 2 Indicates z e (x i,j ) and the reference vector e k The Euclidean distance of the feature map in the latent space is replaced by the corresponding code book vector, and finally a discrete feature map is obtained, which has the same dimension as the original latent feature map. Figure 1 To (still C×H×W);

[0016] When dealing with complex tasks or high-dimensional images, multiple quantization layers can be introduced after the feature maps of different levels of the encoder. These quantization layers are distributed at different locations in the network, allowing the model to quantize on the potential feature maps at each level, thereby effectively capturing multi-level feature information. In order to further optimize the feature flow and information transfer, jump connections are incorporated into the network structure. By applying jump connections between multiple quantization layers, the model can more flexibly integrate feature information at different levels, thereby improving overall performance and enhancing the model's adaptability to complex tasks.

[0017] Decoder: The decoder remaps the quantized feature map back to the original input space; specifically, the decoder gradually restores the quantized feature map to a higher spatial resolution through deconvolution operations, and finally restores it to an output similar to the original input image. During the restoration process, the decoder not only uses the quantized feature map, but also fuses it with the feature map information in the encoder through jump connections. In this way, the decoder can use the different levels of information generated by the encoder and optimize the model through reconstruction errors, making the final output more expressive and accurate.

[0018] 2) Channel attention mechanism: The channel attention mechanism is integrated into the residual block to form an attention residual block (Res-Attention). The attention residual block is mainly composed of a main path and a jump connection. The main path receives the input feature map A1 (with dimensions of C×H×W). After the first convolution, the dimension of the feature map A1 becomes C / 2×H×W, and the feature map A2 is obtained; then, after the second convolution operation, the feature map is restored to C×H×W, and the feature map A3 is obtained; after the introduction of the channel attention mechanism, the feature map A3 (with dimensions of C×H×W) after two convolutions is subjected to a global average pooling operation, and the spatial features of each channel are aggregated into a feature vector of size C×1×1. This feature vector can be understood as the global response of each channel on the entire image, and the importance information of the channel level is preliminarily extracted. Then, the above feature vector is flattened into a C×1 form and input into two fully connected networks for feature processing. The first fully connected network compresses the channel dimension from C to C / 2, reducing the computational complexity and extracting a more compact feature representation; then, the feature dimension is restored to the original number of channels C through the second fully connected network to ensure consistency with the dimension of the input feature map. After the two fully connected networks, the features are normalized by the Sigmoid activation function, and the weight value of each channel is mapped to the range of [0,1] to generate a C×1×1 channel weight map. Finally, the channel weight is expanded to the dimension of C×H×W through the broadcast mechanism, and multiplied with the original input feature map channel by channel to dynamically adjust the importance of the channel features. Through this weighted operation, the model can strengthen the response of key channels and weaken the influence of unimportant or interfering channels, thereby achieving precise focus on significant features. The introduction of the channel attention mechanism enables the attention residual block to not only retain the advantage of the traditional residual block in effectively transferring gradients, but also significantly enhance the feature expression ability, especially in complex scenes. This mechanism can effectively improve the model's sensitivity to abnormal features, reduce the impact of background interference, and provide a more reliable feature expression basis for anomaly detection.

[0019] 3) The loss function used in training is as follows:

[0020] L(x, D(e)) = ||xD(e)|| 2+||sg[E(x)]-e|| 2 +β||sg[e]-E(x)|| 2

[0021] Among them, the first term ||xD(e)|| 2 is the reconstruction loss, which measures the error between the reconstructed image D(e) and the input video frame x. The loss metric is the mean square error (MSE). The second term ||sg[E(x)]-e‖‖ 2 is the codebook loss, which optimizes the codebook e to make it as close as possible to the output features of the encoder E(x). sg[E(x)] represents the output of the encoder E(x), but the gradient is prohibited from flowing to the encoder by stopping the gradient (stop-gradient, sg) operation. The third term β||sg[e]-E(x)|| 2 is the commitment loss, which constrains the encoder E(x) output to be close to the codebook e. sg[e] represents the selection vector in the codebook, which prevents the codebook from being updated by stopping the gradient operation. The hyperparameter β controls the weight of this part of the loss, which determines the degree to which the encoder output is constrained to align with the codebook.

[0022] The codebook loss (the second term in the formula) updates the codebook vector by the exponential moving average (EMA) method. The update formula is as follows:

[0023]

[0024] in, is the cumulative count (or the cumulative number of features) of the i-th item in the codebook (or codebook vector) at the t-th iteration. is the number of codebook vectors in the previous iteration. is the quantized code item e in the current mini-batch i γ is the exponential moving average parameter, which controls the smoothness of the update; is the cumulative feature vector of the i-th codebook entry, which is calculated by smoothing the features matched to this codebook vector. is the cumulative feature vector of the code book vector in the previous iteration. is the weighted sum of the i-th term, which is obtained by multiplying the input features with the one-hot encoding. It is the update result of the i-th code book vector at the t-th iteration.

[0025] (2) The video frames captured by the molten pool camera during the defect-free manufacturing process are used to form a training set (Train-frames) and input into the trained VQ-VCAE-attention model to generate the corresponding reconstructed video frames; the difference between the original video frame and the reconstructed video frame is calculated to generate a residual map; and based on the residual map, the sliding window method (SWMS) is used to gradually calculate the anomaly score and generate the corresponding training set anomaly score sequence (Train-score);

[0026] Calculate the anomaly score based on the local MSE of the sliding window (SWMS): Set a sliding window on the image to slide on the image with a specified size and stride. Every time the window moves to a new position, calculate the MSE of the area, and use the maximum MSE value of all window positions as the anomaly score of the video frame. This method enhances the model's sensitivity to subtle defects through localized calculations, and is suitable for detecting local anomalies with small areas but significant impacts. The calculation formula is as follows:

[0027]

[0028] in, Represents the original image x and the reconstructed image The pixel difference matrix within the sliding window position (i,j). a and b are the height and width of the sliding window. Represents the Frobenius norm of the difference matrix within the window (i.e., the MSE within the window). The max operation selects the maximum value of the MSE in all sliding windows as the anomaly score of the frame.

[0029] The anomaly score sequence (Train-score) generated by the training set is used to train the Isolation Forest Model (IF) to establish a model for detecting anomaly scores. The Isolation Forest Model is trained by the anomaly score sequence (Train-score) of normal video frames in the training set, so that it has the ability to distinguish normal frames from abnormal frames.

[0030] (3) Input the real-time acquired video frames into the trained VQ-VAE-attention model to generate the corresponding reconstructed video frames; calculate the difference between the original video frame and the reconstructed video frame to generate a residual map; and use the sliding window method (SWMS) based on the residual map to gradually calculate the anomaly score. Based on the anomaly score, the anomaly score is first analyzed by the trained isolation forest model (IF) to preliminarily identify the abnormal frames; then, combined with the timing characteristics of welding defects (welding defects usually move in the opposite direction of the welding speed, and it usually takes 10-15 frames from the first appearance of the defect to its complete removal from the molten pool camera shooting range), set the screening conditions, and when the number of consecutive abnormal frames is greater than or equal to 5 frames (not limited to 5 frames), it will eventually be marked as an abnormal point to achieve real-time anomaly detection and marking.

[0031] Beneficial effects of the present invention:

[0032] 1. The present invention can realize online monitoring of arc additive manufacturing, which provides far-reaching practical significance for the development of green manufacturing, intelligent manufacturing and additive manufacturing, while reducing the time and cost investment for subsequent repairs.

[0033] 2. The present invention realizes abnormality detection in the manufacturing process through the video input model collected in real time by the molten pool camera.

[0034] 3. The present invention adopts the arc additive manufacturing online defect detection method based on the fusion channel attention VQ-VCAE of the molten pool video and the time series isolation forest to perform anomaly detection, and compares and verifies the prediction results with the manual detection results. The method is simple and highly reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is the basic architecture of VQ-VCAE model;

[0036] Figure 2 It is the structure of the Res-attention module;

[0037] Figure 3 It is a schematic diagram of the process of the present invention;

[0038] Figure 4 A schematic diagram of the device structure of an arc additive manufacturing defect online detection method using the channel attention fusion VQ-VAE model based on molten pool video provided by the present invention (taking Tig welding as an example);

[0039] Figure 5 Video image samples captured by industrial cameras, (a) video frames without defects, (b) video frames with defects;

[0040] Figure 6 Schematic diagram of the network structure containing two vector quantization modules and integrated skip connections;

[0041] Figure 7 Examples of reconstructed images on the test set;

[0042] Figure 8 Preliminary detection results of abnormal points, where 1 indicates a defect and 0 indicates normal;

[0043] Fig. 9 The outlier detection result after considering the timing characteristics, where 1 indicates a defect and 0 indicates normal.

[0044] In the figure: 1-molten pool camera; 2-welding gun; 3-wire feeding mechanism; 4-weld being printed; 5-weld that has been printed. DETAILED DESCRIPTION

[0045] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0046] See also Figure 3 and Figure 4 As shown, the arc additive anomaly detection method based on unsupervised learning of the present invention comprises the following steps:

[0047] 1. During TIG welding, the normal video of the printing process is captured by an industrial camera and converted into video frames;

[0048] 2. Based on the data set obtained in step 1, train an unsupervised anomaly detection model, which includes VQ-VCAE-attention and isolation forest model;

[0049] 3. Use the anomaly detection model trained in step 2 to infer the real-time video of the welding process and output the anomaly detection results.

[0050] like Figure 4 As shown, the hardware system of the present invention is composed of an industrial camera and a TIG welding system. The industrial camera and the welding gun maintain a fixed relative position and can capture manufacturing video images during the welding process in real time. Figure 3 Video image samples captured by industrial cameras are shown.

[0051] The construction of the dataset is divided into two parts: training set and test set. The training set contains data without abnormalities from the printing process using verified materials to ensure the stability of the printing process during data collection; while the test set contains data that may have abnormalities. These data come from the printing process of unverified materials and may contain unknown defect characteristics. The details of the data collection process are detailed in the table below:

[0052]

[0053] In view of the complexity of the task, we constructed a network structure that contains two vector quantization modules and integrates skip connections, such as Figure 6 All training and testing were done on a computer equipped with a 12th Gen Intel(R) Core(TM) i5-12490F 3.00GHz processor, NVIDIA GEFORCE RTX 4060Ti GPU and 16GB RAM.

[0054] After completing the training of the VQ-VCAE-attention model, the video frames are first reconstructed on the training set, and the residual map between the reconstructed video frame and the original video frame is calculated. Subsequently, the anomaly score sequence of the training set is calculated using the sliding window method. Based on the anomaly score sequence, the Isolation Forest algorithm is trained to identify potential abnormal patterns.

[0055] In the testing phase, the trained VQ-VCAE-attention model is applied to the test set to generate reconstructed images (such as Figure 7 ), and calculate the corresponding residual graph. Then, the anomaly score of the test set is further calculated by the sliding window method. Based on the anomaly score, the trained isolation forest algorithm is used to preliminarily detect the anomalies in the test set. The detection results are shown in Figure 8 shown.

[0056] Combined with the time series characteristics of the anomaly points, the preliminary detection results were further screened, which significantly improved the accuracy and robustness of the detection, and finally the optimized anomaly detection results (such as Fig. 9 The relevant performance indicators are shown in the following table:

[0057]

[0058] The present invention proposes an anomaly detection method based on unsupervised learning, which demonstrates excellent detection performance and application effect in the TIG welding additive manufacturing process.

Claims

1. A method for additive manufacturing defect detection based on VQ-VCAE and isolation forest of melt pool video, characterized in that: The following steps are involved: (1) A dataset consisting of video frames captured by a molten pool camera during defect-free manufacturing is obtained as a training set for training a VQ-VCAE model integrating channel attention. Among them, the VQ-VCAE model integrating channel attention is mainly composed of the VQ-VCAE model, channel attention mechanism and loss function; 1) The VQ-VCAE module is based on a vector quantized variational autoencoder, and its architecture includes an encoder, a quantization layer, and a decoder. The encoder and the decoder are sequentially connected into an encoding-decoding structure, in which the quantization layer is inserted into the latent feature maps of different levels after the encoder convolution processing. Specifically, the quantization layer processes the latent feature map after the output of the latent feature map, maps the continuous latent feature map into a discrete code book vector, and realizes the quantization of the latent feature map. According to the complexity of different tasks, multiple quantization layers are introduced in the latent feature maps of different levels of the encoder to enhance the feature expression ability, and the cross-layer information is retained through jump connections. Encoder: The encoder converts the input video frame into multiple levels of latent feature maps through multiple convolution operations; Quantization layer: After being processed by the encoder, a latent feature map of a certain level is obtained, whose dimension is C×H×W, where C is the number of channels, that is, the dimension of each position, and H and W represent the height and width of the latent feature map respectively; for any position (h, w), it corresponds to a vector of dimension C, representing the latent feature of that position; define a set code book containing n C-dimensional vectors; each vector in the code book is a reference point in the latent space, and each C-dimensional vector of the latent feature map is quantized to a discrete vector in the code book; Quantization steps: for the C-dimensional vector of each position in the latent feature map, calculate its Euclidean distance with all vectors in the code book; select the code book vector with the smallest distance as the quantized representation of that position, the formula is as follows: Among them, q(x i,j ) represents the final selected code book vector, which is a vector selected from the code book; z e (x i,j ) represents the C-dimensional vector at position (i, j) in the feature map in the latent space, representing the feature to be quantized; e k ∈E represents the reference vector in the codebook E, where E = {e1, e2, ..., en}; ||z e (x i,j )-e k || 2 Indicates z e (x i,j ) and the reference vector e k The Euclidean distance of the original latent feature map is obtained by replacing the C-dimensional vectors of all positions in the feature map in the latent space with the corresponding code book vectors. Decoder: The decoder remaps the quantized discrete feature map back to the original input space. Specifically, the decoder gradually restores the quantized discrete feature map to a higher spatial resolution through deconvolution operations, and finally restores it to an output similar to the original input video frame. During the restoration process, the decoder not only uses the quantized discrete feature map, but also fuses it with the potential feature map information in the encoder through jump connections. 2) Channel attention mechanism: The channel attention mechanism is integrated into the residual block to form an attention residual block. The attention residual block is mainly composed of a main path and a skip connection. The main path receives the feature map A1 with an input dimension of C×H×W. After the first convolution, the dimension of the feature map A1 becomes C / 2×H×W, and the feature map A2 is obtained. Subsequently, after the second convolution operation, the feature map A2 is restored to C×H×W, and the feature map A3 is obtained. After the introduction of the channel attention mechanism, the feature map A3 after the two convolutions is subjected to a global average pooling operation. The spatial features of each channel are first aggregated into a feature vector with a size of C×1×1. Then, the above feature vectors are pooled. The feature vector is flattened into a C×1 form and input into two fully connected networks for feature processing; the first fully connected network compresses the channel dimension of the feature vector from C to C / 2; the dimension of the feature vector is then restored to the original number of channels C through the second fully connected network to ensure consistency with the dimension of the input feature map A3; after the two fully connected networks, the feature vector is normalized through the Sigmoid activation function, and the weight value of each channel is mapped to the range of [0,1] to generate a C×1×1 channel weight map; finally, the channel weight map is expanded to the dimension of C×H×W through the broadcast mechanism, and multiplied channel by channel with the original input feature map A3 to dynamically adjust the importance of channel features; 3) The loss function used in training is as follows: L(x,D(e))=||xD(e)|| 2 +||sg[E(x)]-e|| 2 +β||sg[e]-E(x)|| 2 Among them, the first term ||xD(e)|| 2 is the reconstruction loss, which measures the error between the reconstructed image D(e) and the input video frame x. The loss metric is the mean square error. The second term ||sg[E(x)]-e|| 2 is the codebook loss, which optimizes the codebook e to make it as close as possible to the output characteristics of the encoder E(x); sg[E(x)] represents the output of the encoder E(x), and by stopping the gradient operation, the gradient is prohibited from flowing to the encoder; the third term β||sg[e]-E(x)|| 2 is the commitment loss, which constrains the encoder E(x) output to be close to the codebook e; sg[e] represents the selection vector in the codebook, which prevents the codebook from being updated by stopping the gradient operation; the hyperparameter β controls the weight of this part of the loss and determines the degree to which the encoder output is constrained to align with the codebook; The codebook loss updates the codebook vector by exponential moving average method, and the formula is as follows: in, is the cumulative count of the i-th item in the codebook at the t-th iteration or the number of features accumulated by the codebook vector; is the number of codebook vectors in the previous iteration; is the current small batch, quantized to the code book e i The number of vectors; γ is the exponential moving average parameter, which controls the smoothness of the update; is the cumulative feature vector of the i-th codebook item, which is calculated by smoothing the features matched to this codebook vector; is the cumulative feature vector of the code book vector in the previous iteration; is the weighted sum of the i-th term, which is obtained by multiplying the input feature with the one-hot encoding; is the update result of the i-th code book vector at the t-th iteration; (2) The video frames captured by the molten pool camera during the defect-free manufacturing process are used as a training set and input into the trained VQ-VCAE model with fusion channel attention to generate the corresponding reconstructed video frames; the difference between the original video frame and the reconstructed video frame is calculated to generate a residual map; and based on the residual map, the sliding window method is used to gradually calculate the anomaly score and generate the corresponding training set anomaly score sequence; Calculate the anomaly score based on the local MSE of the sliding window: Set a sliding window on the image to slide on the image with a specified size and stride; each time the window moves to a new position, calculate the MSE of the area, and take the maximum MSE value of all window positions as the anomaly score of the video frame; the calculation formula is as follows: in, Represents the original video frame x and the reconstructed video frame The pixel difference matrix within the sliding window position (i, j); a and b are the height and width of the sliding window; The Frobenius norm of the difference matrix in the window is the MSE in the window; the max operation selects the maximum value of the MSE in all sliding windows as the anomaly score of the frame; The isolation forest model is trained using the anomaly score sequence generated by the training set to establish a model for detecting anomaly scores. The isolation forest model is trained using the anomaly score sequence of normal video frames in the training set, so that it has the ability to distinguish normal frames from abnormal frames. (3) The real-time acquired video frames are input into the trained VQ-VCAE model with fused channel attention to generate the corresponding reconstructed video frames; the difference between the original video frame and the reconstructed video frame is calculated to generate a residual map; and the anomaly score is gradually calculated based on the residual map using a sliding window method; based on the anomaly score, the anomaly score is first analyzed using the trained isolation forest model to preliminarily identify the abnormal frames; then, combined with the timing characteristics of welding defects, the screening conditions are set, and when the number of consecutive abnormal frames is greater than or equal to the set threshold, it is finally marked as an abnormal point, thus realizing real-time anomaly detection and marking.

Images