A lightweight welding phase classification method and system based on time sequence mixing

Abstract

The present application belongs to the technical field of welding image processing, and specifically relates to a lightweight welding stage classification method and system based on time sequence mixing, which comprises the following steps: continuously acquiring welding images, and constructing a sliding window containing the current frame and a plurality of historical frames before the current frame; extracting the spatial features of each frame in the sliding window, and performing dynamic highlight channel protection type time sequence mixing at a preset feature position; outputting unnormalized class scores for each time position to obtain a sequence level fusion score corresponding to the sliding window; and selecting the class with the maximum sequence level fusion score as the welding stage of the current frame to be classified. The present application reduces the model parameter quantity and calculation overhead, realizes effective interaction of multi-frame short-time information, and maintains the current position features of the response relatively prominent channels in the candidate time sequence channels, thereby avoiding the use of the same time sequence processing mode for all candidate channels.

Description

Technical Field

[0001] This invention mainly relates to the technical field of welding image processing, specifically a lightweight welding stage classification method and system based on temporal hybridization. Background Technology

[0002] In multi-layer, multi-pass welding processes, the welding typically involves sequential stages such as root pass, fill pass, and cap pass. Each stage differs in terms of molten pool morphology, weld surface characteristics, heat input, and process parameters. Continuous and accurate identification of these welding stages provides reliable data for segmented recording of the welding process, correlation of process parameters, retrieval of abnormal segments, quality analysis, and subsequent control.

[0003] However, the actual welding environment is subject to strong optical interference and dynamic uncertainties. Welding images are easily affected by factors such as strong arc light, metal spatter, smoke and dust obstruction, workpiece surface reflection, exposure changes, and transient fluctuations in the molten pool. The appearance of images at adjacent moments in the same welding stage may change significantly, while different welding stages may have similar local visual features. Therefore, when judging the welding stage based solely on a single frame image, the recognition results are easily interfered with by instantaneous noise and local appearance fluctuations, making it difficult to fully utilize the continuity and evolution of the welding process over time.

[0004] In existing technologies, 3D convolutional neural networks, recurrent neural networks, and complex attention networks can extract temporal information from video sequences to a certain extent. However, these methods typically require the introduction of additional 3D convolutional kernels, recurrent states, attention parameters, or large-scale intermediate features, thereby increasing the number of model parameters, computational complexity, and memory usage. This makes them unsuitable for real-time deployment on industrial equipment with limited computing power, storage space, and power consumption. AlexNet is a representative publicly available convolutional neural network. However, most of its parameters are concentrated in the fully connected classification layer, resulting in a large model file size and parameter storage overhead, making it difficult to meet the requirements of lightweight models and real-time operation for edge devices in welding fields.

[0005] Therefore, there is an urgent need to provide a stage classification scheme for intelligent perception of the welding process, which can reduce the number of model parameters and computational overhead, achieve effective interaction of short-time information across multiple frames, and maintain the current position characteristics of channels with relatively prominent responses in the candidate time-series channels, thereby avoiding the use of the same time-series processing method for all candidate channels. Summary of the Invention

[0006] To address the shortcomings of current technologies, this invention, combining existing technologies and focusing on practical applications, provides a lightweight welding stage classification method and system based on temporal hybridization. This method can reduce the number of model parameters and computational overhead while achieving effective interaction of short-time information across multiple frames. Furthermore, it maintains the current position characteristics of channels with relatively prominent responses among candidate temporal channels, thereby avoiding the use of the same temporal processing method for all candidate channels.

[0007] The technical solution of the present invention is as follows:

[0008] According to one aspect of the present invention, a lightweight welding stage classification method based on time-series hybridization is provided, comprising the following steps:

[0009] S1. Acquire welding images continuously according to the acquisition time, and write them into the image stream in the order of acquisition time;

[0010] S2. Using the current frame to be classified as the end frame, construct a sliding window that includes the current frame and multiple previous historical frames;

[0011] S3. Perform preprocessing on the image, including size unification, channel arrangement, tensor quantization, and normalization;

[0012] S4. Extract the spatial features of each frame within the sliding window using a lightweight spatial feature extraction network that shares parameters at each time position, and perform dynamic highlighting channel protection temporal mixing at preset feature positions.

[0013] S5. Output unnormalized category scores for each time location by sharing the classification head;

[0014] S6. Before probability normalization, calculate the arithmetic mean of the unnormalized category scores for each time position along the time dimension to obtain the sequence-level fusion score corresponding to the sliding window.

[0015] S7. Select the category with the largest sequence-level fusion score as the welding stage of the current frame to be classified;

[0016] S8. Output the welding stage classification result of the current frame, where the output result is root pass, fill pass, or cap pass.

[0017] S9. After outputting the welding stage of the current frame, add the newly acquired welding image to the sliding window and move it out of the earliest frame to form the sliding window for the next moment. Repeat the above steps to continuously output the welding stage results.

[0018] Furthermore, in step S4, the lightweight spatial feature extraction network includes a regular convolutional front end and multiple depthwise separable convolutions. Each depthwise separable convolution first uses channel-wise convolution to extract the spatial features of each input channel, and then uses pointwise convolution to combine the information of different channels. The lightweight spatial feature extraction network does not set a local response normalization layer.

[0019] Furthermore, in step S4, the ordinary convolutional front end includes three ordinary convolutional layers, the depthwise separable convolutions are five, and the preset feature position is located between the fourth and fifth depthwise separable convolutions.

[0020] Furthermore, in step S4, the dynamic highlighting channel protective timing mixing includes: dividing the feature channel into a first candidate group, a second candidate group, and an identity preservation group at a preset feature position; the first candidate group obtains the feature to be mixed from an adjacent earlier time position, and the second candidate group obtains the feature to be mixed from an adjacent later time position.

[0021] Dynamic protection identifiers are generated based on the normalized prominence of the current position response of the candidate channel relative to the average response of the candidate channel. Candidate channels that have reached the protection threshold retain their current position features, while candidate channels that have not reached the protection threshold mix their current position features with adjacent time features in a fixed ratio. The identity preservation group always retains its current position features.

[0022] Furthermore, when an adjacent earlier or later time position exceeds the boundary of the sliding window, the corresponding source features are filled with zero values. For unprotected candidate channels, a portion of the current position features is still retained at the boundary. For protected candidate channels and identity-preserving groups, the output is always equal to the current position input.

[0023] Furthermore, in step S4, the average absolute response of the current position feature map of each candidate channel is statistically analyzed in the spatial dimension to obtain the current position response of the candidate channel. The expression is as follows:

[0024]

[0025] In the formula, This represents the current positional features at time position t, channel c, and spatial position (h, w). Let G(−) represent the set of candidate channels, which is the union of the first candidate channel group G(−) and the second candidate channel group G(+).

[0026] In the candidate channel set Internal Calculate the average to obtain the average response of the candidate channel for the current sample and the current time position. The expression is as follows:

[0027] .

[0028] Furthermore, in step S4, the ratio of the response of a single candidate channel to the average response of the candidate channels is normalized, and a stability constant is added. Obtain the normalized salience of candidate channels The expression is as follows:

[0029]

[0030] Normalized saliency With preset protection threshold Comparison is performed to generate dynamic protection identifiers for each sample, each time location, and each candidate channel. The expression is as follows:

[0031]

[0032] For the candidate channel set When, retain the current location features; when At that time, based on the difference between the current location features and the features of adjacent times, according to the mixing coefficient To update, use the following expression:

[0033]

[0034] In the formula, This indicates the output characteristics of the timing-mixing module. This represents the adjacent time characteristics after boundary complementation.

[0035] Furthermore, in step S6, the unnormalized category score is obtained for each time position through the shared classification head. Then, before normalization, the arithmetic mean is calculated for each category along the time dimension to obtain the sequence-level category score. The expression is as follows:

[0036]

[0037] in, The value is 5, which represents the length of the sliding window.

[0038] Furthermore, in step S8, when outputting the welding stage classification result of the current frame, the normalized confidence score and the current frame timestamp are also output, and the classification result, confidence score, timestamp, image index and model version are written into the process record.

[0039] According to another aspect of the present invention, a lightweight welding stage classification system based on time-series hybridization is provided, comprising:

[0040] The image acquisition module is configured to continuously acquire welding images according to the acquisition time.

[0041] A sliding window construction module is configured to construct a sliding window containing the current frame and multiple previous historical frames, with the current frame to be classified as the ending frame.

[0042] The image preprocessing module is configured to perform preprocessing on images, including size unification, channel arrangement, tensorization, and normalization.

[0043] A shared lightweight spatial feature extraction module is configured to extract spatial features of each frame within the sliding window.

[0044] A dynamic highlight channel-protected timing blending module is configured to perform dynamic highlight channel-protected timing blending at preset feature locations;

[0045] The shared classification module is configured to receive features after temporal mixing and output unnormalized class scores for each time position.

[0046] The time fusion module is configured to calculate the arithmetic mean of the unnormalized category scores for each time position along the time dimension before probability normalization, so as to obtain the sequence-level fusion score corresponding to the sliding window.

[0047] The classification output module is configured to output the classification results of the welding stage in the current frame.

[0048] The process recording and interface module is configured to write classification results, confidence scores, current frame timestamps, image indexes, and model versions into the process record and send them to the display terminal, data interface, or upper-layer system as needed for deployment.

[0049] The beneficial effects of this invention are:

[0050] This invention does not merely change the number of output categories, but rather synergistically modifies the spatial backbone, classification head, and multi-frame temporal processing chain. Depth-separable convolution and channel compression reduce the size of the convolutional backbone; global average pooling and the classification head replace the classifier, eliminating the main source of parameter quantity; and local response normalization is eliminated to simplify the computation chain.

[0051] This invention performs temporal mixing only at a late position and adds input-related salience determination within the candidate channel range. Furthermore, this invention generates dynamic protection flags based on the current input, so that salience candidate channels maintain the current position characteristics, non-salience candidate channels perform fixed-ratio temporal mixing, and channels in the identity preservation group do not participate in the salience benchmark calculation and always maintain the current position characteristics. This dynamic determination and diversion process does not add learnable temporal parameters. Attached Figure Description

[0052] Figure 1 This is a flowchart of the method proposed in this invention.

[0053] Figure 2 This is a schematic diagram of the lightweight spatial backbone, dynamically protruding channel protection type timing hybrid module, and classification head structure of the present invention.

[0054] Figure 3 This is a schematic diagram of the depth-separable convolution module structure of the present invention.

[0055] Figure 4 This is a schematic diagram of the constant-preservation type timing hybrid module structure of the present invention. Detailed Implementation

[0056] The present invention will be further described in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined in this application.

[0057] Example 1

[0058] This embodiment provides a lightweight welding stage classification method based on time-series hybridization. The main process of this method is as follows: Figure 1 As shown, the specific steps are as follows.

[0059] S1. Welding images are continuously acquired according to the acquisition time. The acquisition range includes the electric arc, the molten pool and its surrounding area, and written into the image stream in the order of acquisition time.

[0060] S2. Using the current frame I(t) to be classified as the end frame, select four historical frames and the current frame from the image stream to construct a five-frame sliding window [I(t−4), I(t−3), I(t−2), I(t−1), I(t)]. This window does not contain images after I(t).

[0061] S3. Perform uniform size adjustment, channel arrangement, tensor quantization and normalization on the five frames of images, and rearrange the batch tensor of B×T×3×256×256 to (B×T)×3×256×256 so that each time position shares the same spatial feature extraction network.

[0062] S4. The rearranged five frames are fed into a shared lightweight spatial feature extraction network for processing. At each time point, a three-layer ordinary convolutional front-end and the first four depthwise separable convolutions are shared. After the fourth depthwise separable convolution, the features are restored to B×T×192×H(f)×W(f). Then, the images enter a dynamic prominence channel-protected temporal mixing module. This module performs adjacent time value taking, candidate channel response statistics, normalized prominence calculation, dynamic protection determination, and selective temporal mixing on candidate channels. The specific calculation relationships are explained in the "Key Temporal Mixing Calculation and Formula Explanation" section below. The temporal mixing result is then processed by a fifth depthwise separable convolution, max pooling, global average pooling, and a small classification head with a range of 128→64→3.

[0063] refer to Figure 2 , Figure 3 As shown, for the shared lightweight spatial feature extraction network described above, the ordinary convolutional front-ends are as follows: 3→48, 7×7, stride 2; 48→48, 3×3, stride 1; 48→128, 3×3, stride 2; and 3×3 max pooling is set after the third layer. The ordinary convolutional front-ends are used to preserve the initial encoding capabilities for brightness distribution, edges, melt pool contours, and low-level textures.

[0064] Following the front-end, five depthwise separable convolutions are sequentially applied, with channel mappings of 128→128, 128→128, 128→192, 192→192, and 192→128. Each depthwise separable convolution first uses a 3×3 channel-wise convolution to extract the spatial features of each input channel, then uses a 1×1 pointwise convolution to combine the information from different channels. The complete module is then activated using ReLU. The network does not include local response normalization layers.

[0065] After the fifth deep separable convolution and max pooling, the features at each time point are 128×7×7. Adaptive global average pooling transforms them into 128×1×1, which is then flattened to obtain a 128-dimensional channel description vector. This vector is then passed through a 128→64 fully connected layer, ReLU, a Dropout layer with a 0.5 inactivation probability, and a 64→3 fully connected layer to output three unnormalized class scores.

[0066] refer to Figure 4 As shown, the dynamically highlighted channel-protected temporal mixing module is only set after the fourth depthwise separable convolution and before the fifth depthwise separable convolution. The number of channels C at this position is 192. Channels are divided according to the transfer ratio divisor d = 8. The first candidate group G(−) and the second candidate group G(+) each contain 24 channels, and the two candidate groups together constitute the candidate channel set G(cand). The remaining 144 channels constitute the identity preservation group G(0). G(−) obtains the features to be mixed from the adjacent earlier time position, G(+) obtains the features to be mixed from the adjacent later time position, and G(0) does not change the time position.

[0067] For candidate channels, this embodiment does not directly replace the current position feature with adjacent time features, nor does it uniformly perform fixed-ratio mixing on all candidate channels. Instead, it first determines the channel salience based on the spatial average absolute response of the current position feature map. The average response benchmark is calculated using only 48 candidate channels; the latter 144 identity-preserving channels are not included in the benchmark calculation. When the normalized salience of a candidate channel reaches the protection threshold, the current position feature of that channel is preserved; only candidate channels that do not reach the protection threshold undergo fixed-ratio mixing of the current position feature and adjacent time features. Thus, the difference between this embodiment and the original TSM is that the processing state of candidate channels depends not only on the fixed channel grouping and time value direction, but also on the per-sample, per-time position, and per-candidate channel protection flags generated by the current input.

[0068] When adjacent time positions exceed the five-frame window boundary, the corresponding source features are padded with zero values. For unprotected candidate channels, a portion of the current position features is retained at the boundary; for protected candidate channels and identity-preserving groups, the output is always equal to the current position input. The module only performs tensor rearrangement, channel shifting, spatial average absolute response, normalization, threshold comparison, difference, and scalar multiplication, and does not include learnable parameters or buffers.

[0069] The aforementioned dynamic prominence channel protection temporal mixing process has a clear calculation order, which mainly follows the sequence of "determining adjacent time sources - calculating candidate channel responses - establishing candidate channel benchmarks - calculating normalized prominence - generating protection labels - performing selective temporal mixing - fusing five-frame category scores".

[0070] In order to provide a more detailed explanation of the calculation method for the dynamic protruding channel protection type timing mixing process, this embodiment first defines the features, channel groups and control variables involved in the timing mixing module, as shown in Table 1.

[0071] Table 1. Definitions of Symbols and Variables

[0072]

[0073] The values ​​of key technical parameters are shown in Table 2.

[0074] Table 2 Key Technical Parameters

[0075]

[0076] Before calculating the candidate channel response, first determine the direction of the channel. For candidate channels in G(−), features are obtained from the earlier adjacent time positions; for candidate channels in G(+), features are obtained from the later adjacent time positions; when the corresponding adjacent time positions exceed the five-frame window boundary, Fill in with zero values. Then perform the following calculations in sequence:

[0077] (1) Current position response of candidate channels. The average absolute response of the current position feature map of each candidate channel is statistically analyzed in the spatial dimension to obtain... This is used to characterize the response intensity of the candidate channel at the current sample and current time position:

[0078] (1)

[0079] (2) Candidate channel baseline response. Only within the candidate channel set G(cand) are the baseline responses. Calculate the average to obtain the average response of the candidate channel for the current sample and the current time position. The identity preservation group G(0) does not participate in the calculation of this benchmark:

[0080] (2)

[0081] (3) Normalized saliency. The ratio of the response of a single candidate channel to the average response of the candidate channels is normalized, and a stability constant is added. To prevent the denominator from being zero, we get :

[0082] (3)

[0083] ε is used only to avoid division by zero when the average response of the candidate channel is zero.

[0084] (4) Dynamic protection judgment. Normalized salience will be used to determine... With preset protection threshold Comparison is performed to generate dynamic protection identifiers for each sample, each time location, and each candidate channel. :

[0085] (4)

[0086] (5) Candidate channel selective temporal mixing. For c∈G(cand), when When, retain the current location features; when At that time, based on the difference between the current location features and the features of adjacent times, according to the mixing coefficient Update:

[0087] (5)

[0088] (6) Identity Preservation Channel Processing. For c∈G(0), no time value taking and timing mixing are performed, and the output is always equal to the input at the current position:

[0089] (6)

[0090] (7) Five-frame score fusion. Unnormalized category scores are obtained from each time position through a shared classification head. Then, the arithmetic mean is calculated for each category along the time dimension before applying the softmax algorithm to obtain the sequence-level category score. In this embodiment, the window length T=5:

[0091] (7)

[0092] As can be seen from equations (4) to (6), the protective markings Directly control whether candidate channels participate in timing mixing: when When =1, the timing difference term in equation (5) is turned off, and the output remains intact. ;when When =0, the output is [1− ]× + × For channels in G(0), the current position feature is always maintained according to equation (6). If X_shift is padded with zero values ​​due to window boundaries, unprotected candidate channels are still retained [1− The current position features are multiplied by 1, and the protected candidate channels and channels in G(0) are not affected by boundary zero padding. Equation (7) further aggregates the category scores of the five time positions into the sequence-level classification basis for the current five-frame window.

[0093] S5. Generate three-dimensional unnormalized class scores for each of the five time positions using the shared classification head, i.e., rawlogits before Softmax.

[0094] S6. Before Softmax, calculate the arithmetic mean of the scores of the five groups of unnormalized categories along the time dimension to obtain the sequence-level fusion scores corresponding to the five frame windows.

[0095] Specifically, each time location is processed by a shared classification head to obtain 3D rawlogits. These are then restored to B×T×3, and an arithmetic mean is calculated for each category along the time dimension before probability normalization to form a sequence-level category score. This fusion differs from image pixel averaging, frame-by-frame softmax probability averaging, and frame-by-frame hard class majority voting; after fusion, the maximum score decision is performed, and if necessary, softmax can be applied to the sequence-level category scores to obtain confidence levels.

[0096] S7. Compare the three category components of the sequence-level fusion score, and select the category with the largest fusion score as the welding stage of the current frame I(t) to be classified.

[0097] S8. Output the welding stage classification result of the current frame, which is the root pass weld, fill weld, or cover weld; if necessary, output the confidence score calculated by Softmax and the timestamp of the current frame.

[0098] S9. Write the classification results, confidence level, timestamp, image index, and model version into the process log, and send them to the display terminal, data interface, or upper-level system as needed for deployment.

[0099] S10. After the new image arrives, add the new frame to the sliding window and move it out of the earliest frame to form the history of the next moment - the current five-frame window. Repeat S3 to S9 to continuously output the welding stage results.

[0100] Example 2

[0101] To demonstrate the effectiveness of the method proposed in Example 1, this example uses the following example parameters for verification.

[0102] System network configuration example: The input consists of five 256×256 three-channel welding images, in the time sequence [I(t−4), I(t−3), I(t−2), I(t−1), I(t)]. The five frames share the same set of 218,995 learnable parameters. The model front end uses three layers of ordinary convolutions, and the back end uses five depthwise separable convolutions; the dynamic highlighting channel protection temporal mixing is located between the fourth depthwise separable convolution (192→192) and the fifth depthwise separable convolution (192→128); the classification end uses global average pooling and a 128→64→3 classification head.

[0103] Dynamic temporal hybrid computing example: In this example, C = 192, d = 8, G(−) and G(+) each have 24 channels, G(0) has 144 channels, and G(cand) has a total of 48 channels. =0.5, = , =1.10. G(−) obtains X_shift from the adjacent earlier time position, and G(+) obtains X_shift from the adjacent later time position. A(t,c) is calculated for each candidate channel, and only within the 48 candidate channels. .

[0104] When R(t,c)≥1.10, K(t,c)=1, and the output retains the current position feature; when R(t,c)<1.10, K(t,c)=0, and the output is equivalent to 0.5 times the current position feature plus 0.5 times the adjacent time position feature. For the 144 channels in G(0), the output is exactly the same as the input. For unprotected candidate channels that lack adjacent sources at the beginning and end of a five-frame sequence, after padding with zero values, the output still has 0.5 times the current position feature; for candidate channels that meet the protection conditions, the boundary padding does not change their current position feature.

[0105] Protection threshold These are preset parameters, not learnable model weights. In this embodiment, from a predefined candidate set [1.00, 1.10, 1.20, 1.25, 1.30, 1.40, 1.50], the parameters are selected based on the validation set Macro-F1, validation set Accuracy, and smaller... Determining the order rules for the earliest optimal training cycle =1.10. The test set is not involved in threshold selection. This value is used to illustrate this embodiment and does not represent a universally optimal value under other imaging platforms, welding materials, or process conditions.

[0106] Sample organization and training example: The training set, validation set, and test set are divided into mutually exclusive parent continuous welding process groups. The training set contains 10,160 five-frame sequences and 79 parent groups, the validation set contains 2,134 sequences and 17 parent groups, and the test set contains 1,903 sequences and 15 parent groups. The parent group identifier, target frame path, and pairwise overlap of any constituent frame paths among the three data subsets are all 0.

[0107] In this embodiment, the model is first initialized with a spatially adapted lightweight backbone, and then full training is performed on a five-frame computation graph containing a dynamic protruding channel-protected temporal mixture. The five-frame training uses the Adam optimizer with a fixed learning rate. The weight decay is 0, the training lasts for 15 epochs, and the model parameters and protection threshold are selected based on the validation set metrics. The test set does not participate in the model selection or threshold selection.

[0108] Examples of classification results: Refer to Table 3 for the parameter quantities and classification results under the same test caliber.

[0109] Table 3. Parameter quantity and classification results under the same test caliber.

[0110]

[0111] Compared to the AlexNet-2012 baseline, the technical solution in this embodiment does not merely change the number of output classes, but rather synergistically modifies the spatial backbone, classification head, and multi-frame temporal processing chain. Depthwise separable convolutions and channel compression reduce the size of the convolutional backbone; global average pooling and a 128→64→3 classification head replace the 9216→4096→4096→3 classifier, eliminating the main source of parameter quantity; local response normalization is eliminated to simplify the computation chain.

[0112] Compared to the fixed channel time shifting of the original TSM, the technical solution of this embodiment performs temporal mixing only at a late position and adds input-related salience determination within the range of 48 candidate channels. The candidate channel processing state of the original TSM is determined by fixed channel numbers and predetermined time value directions; this embodiment further generates dynamic protection flags based on the current input, ensuring that salience candidate channels maintain their current position characteristics, non-salience candidate channels undergo fixed-ratio temporal mixing, and the last 144 identity-maintained channels do not participate in the salience benchmark calculation and remain constant. This dynamic determination and splitting process does not add learnable temporal parameters.

[0113] On the same test set with fixed components and no frame overlap, the model in this embodiment and the AlexNet-2012 baseline both correctly classified 1,902 out of 1,903 five-frame sequences, with an accuracy of 99.9475% and a Macro-F1 score of 99.9527%. The number of parameters in this invention is reduced from 56,880,515 to 218,995, a reduction of approximately 99.615%, and the parameter size is reduced to approximately 0.385%, meaning that the same discrete classification results are achieved with approximately 1 / 259.73 of the parameter retention.

[0114] Example 3

[0115] This embodiment also provides a lightweight welding stage classification system based on temporal hybridization, mainly including an image acquisition module, a sliding window construction module, an image preprocessing and tensor organization module, a shared lightweight spatial feature extraction module, a dynamic highlighting channel protection temporal hybridization module, a shared classification module, a temporal fusion module, a classification output module, and a process recording and interface module. The modules are connected in the order of the welding image acquisition time, with the output of the previous module serving as the input of the next module.

[0116] Image acquisition module: Used to continuously acquire welding images from industrial cameras, video stream interfaces or image file interfaces. The acquisition range includes the electric arc, molten pool and its surrounding area, and retains the image acquisition time sequence.

[0117] (1) Sliding window construction module: used to maintain a five-frame window [I(t−4),I(t−3),I(t−2),I(t−1),I(t)] consisting of the current frame to be classified I(t) as the end frame; this window does not contain images after I(t).

[0118] (2) Image preprocessing and tensor organization module: used to perform uniform size adjustment, channel arrangement, tensorization and normalization on five frames of images, and rearrange the input tensor of B×T×3×256×256 to (B×T)×3×256×256 so that the five time positions can share the same set of two-dimensional network parameters.

[0119] (3) Shared lightweight spatial feature extraction module: used to reuse the same set of lightweight two-dimensional convolutional network parameters for each time position. Each frame first goes through three layers of ordinary convolutional front-end and the first four depthwise separable convolutions, and then the features are restored to B×T×192×H(f)×W(f), which is used as the input of the temporal mixing module.

[0120] (4) Dynamic prominence channel protection time-series mixing module: used to divide the channel into a first candidate group G(−), a second candidate group G(+), and an identity preservation group G(0) at a late feature position. The two candidate groups obtain the features to be mixed from the adjacent earlier time position and the adjacent later time position, respectively, and generate dynamic protection labels based on the normalized prominence of the current position response of the candidate channel relative to the average response of the candidate channel; the candidate channel that reaches the protection threshold maintains the current position feature, and the candidate channel that does not reach the protection threshold mixes the current position feature with the adjacent time feature in a fixed ratio, and G(0) always maintains the current position feature.

[0121] (5) Shared classification module: It is used to receive the features after temporal mixing and pass them through the fifth depthwise separable convolution, max pooling, adaptive global average pooling and the small classification head of 128→64→3 in sequence. It outputs the three-dimensional unnormalized class scores for the five time positions, that is, the rawlogits before Softmax.

[0122] (6) Temporal fusion module: used to restore the rawlogits of the five time positions to B×T×3, and calculate the arithmetic mean of each category along the time dimension before Softmax to obtain the sequence-level category scores corresponding to the five frame windows.

[0123] (7) Classification output module: used to compare the three category components of the sequence-level category score, and take the category corresponding to the maximum score as the welding stage of the current frame I(t) to be classified; when necessary, perform Softmax on the sequence-level category score to obtain confidence.

[0124] (8) Process Recording and Interface Module: Used to write classification results, confidence scores, current frame timestamps, image indexes and model versions into the process record, and send them to the display terminal, data interface or upper-level system as needed for deployment.

[0125] The data transmission relationship of the above system during continuous operation is as follows: the welding image enters the sliding window construction module through the image acquisition module to form a historical-current five-frame input; after the image preprocessing and tensor organization module, it is sent to the shared lightweight spatial feature extraction module; at the preset late feature position, it enters the dynamic protrusion channel protection temporal mixing module; its output is sent to the shared classification module to generate rawlogits at each time position; the temporal fusion module performs sequence-level fusion, the classification output module gives the welding stage of the current frame, and the process recording and interface module completes the result saving and sending.

Claims

1. A lightweight welding stage classification method based on temporal hybridization, characterized in that, Includes the following steps: S1. Continuously acquire welding images and write them into the image stream in the order of acquisition time; S2. Using the current frame to be classified as the end frame, construct a sliding window that includes the current frame and multiple previous historical frames; S3. Perform preprocessing on the image, including size unification, channel arrangement, tensor quantization, and normalization; S4. Extract the spatial features of each frame within the sliding window using a lightweight spatial feature extraction network that shares parameters at each time position, and perform dynamic highlighting channel protection temporal mixing at preset feature positions. S5. Output unnormalized category scores for each time location by sharing the classification head; S6. Before probability normalization, calculate the arithmetic mean of the unnormalized category scores for each time position along the time dimension to obtain the sequence-level fusion score corresponding to the sliding window. S7. Select the category with the largest sequence-level fusion score as the welding stage of the current frame to be classified; S8. Output the welding stage classification result of the current frame, where the output result is root pass, fill pass, or cap pass. S9. After outputting the welding stage of the current frame, add the newly acquired welding image to the sliding window and move it out of the earliest frame to form the sliding window for the next moment. Repeat the above steps to continuously output the welding stage results.

2. The lightweight welding stage classification method based on time-series hybridization according to claim 1, characterized in that, In step S4, the lightweight spatial feature extraction network includes a regular convolutional front end and multiple depthwise separable convolutions. Each depthwise separable convolution first uses channel-wise convolution to extract the spatial features of each input channel, and then uses pointwise convolution to combine the information of different channels. The lightweight spatial feature extraction network does not set a local response normalization layer.

3. The lightweight welding stage classification method based on time-series hybridization according to claim 2, characterized in that, In step S4, the ordinary convolutional front end includes three ordinary convolutional layers, the depthwise separable convolutions are five, and the preset feature position is located between the fourth and fifth depthwise separable convolutions.

4. The lightweight welding stage classification method based on time-series hybridization according to claim 1, characterized in that, In step S4, the dynamic highlighting channel protective timing mixing includes: dividing the feature channel into a first candidate group, a second candidate group, and an identity preservation group at a preset feature position; the first candidate group obtains the feature to be mixed from an adjacent earlier time position, and the second candidate group obtains the feature to be mixed from an adjacent later time position. Dynamic protection identifiers are generated based on the normalized prominence of the current position response of the candidate channel relative to the average response of the candidate channel. Candidate channels that have reached the protection threshold retain their current position features, while candidate channels that have not reached the protection threshold mix their current position features with adjacent time features in a fixed ratio. The identity preservation group always retains its current position features.

5. The lightweight welding stage classification method based on time-series hybridization according to claim 4, characterized in that, When an adjacent earlier or later time position exceeds the boundary of the sliding window, the corresponding source features are padded with zero values. For unprotected candidate channels, a portion of the current position features is still retained at the boundary. For protected candidate channels and identity-preserving groups, the output is always equal to the current position input.

6. The lightweight welding stage classification method based on time-series hybridization according to claim 4, characterized in that, In step S4, the average absolute response of the current position feature map of each candidate channel is statistically analyzed in the spatial dimension to obtain the current position response of the candidate channel. The expression is as follows: ， In the formula, This represents the current positional features at time position t, channel c, and spatial position (h, w). Let G(−) represent the set of candidate channels, which is the union of the first candidate channel group G(−) and the second candidate channel group G(+). In the candidate channel set Internal Calculate the average to obtain the average response of the candidate channel for the current sample and the current time position. The expression is as follows: 。 7. The lightweight welding stage classification method based on time-series hybridization according to claim 6, characterized in that, In step S4, the ratio of the response of a single candidate channel to the average response of the candidate channels is normalized, and a stability constant is added. Obtain the normalized salience of candidate channels The expression is as follows: ， Normalized saliency With preset protection threshold Comparison is performed to generate dynamic protection identifiers for each sample, each time location, and each candidate channel. The expression is as follows: ， For the candidate channel set The current location features are retained. when At that time, based on the difference between the current location features and the features of adjacent times, according to the mixing coefficient To update, use the following expression: ， In the formula, This indicates the output characteristics of the timing-mixing module. This represents the adjacent time characteristics after boundary complementation.

8. The lightweight welding stage classification method based on time-series hybridization according to claim 7, characterized in that, In step S6, the unnormalized category score is obtained for each time position through the shared classification head. Then, before normalization, the arithmetic mean is calculated for each category along the time dimension to obtain the sequence-level category score. The expression is as follows: ， in, The value is 5, which represents the length of the sliding window.

9. The lightweight welding stage classification method based on time-series hybridization according to claim 1, characterized in that, In step S8, when outputting the classification result of the welding stage of the current frame, the normalized confidence score and the timestamp of the current frame are also output. The classification result, confidence score, timestamp, image index and model version are written into the process record.

10. A system for implementing the time-series hybrid lightweight welding stage classification method according to any one of claims 1-9, characterized in that, include: The image acquisition module is configured to continuously acquire welding images according to the acquisition time. A sliding window construction module is configured to construct a sliding window containing the current frame and multiple previous historical frames, with the current frame to be classified as the ending frame. The image preprocessing module is configured to perform preprocessing on images, including size unification, channel arrangement, tensorization, and normalization. A shared lightweight spatial feature extraction module is configured to extract spatial features of each frame within the sliding window. A dynamic highlight channel-protected timing blending module is configured to perform dynamic highlight channel-protected timing blending at preset feature locations; The shared classification module is configured to receive features after temporal mixing and output unnormalized class scores for each time position. The time fusion module is configured to calculate the arithmetic mean of the unnormalized category scores for each time position along the time dimension before probability normalization, so as to obtain the sequence-level fusion score corresponding to the sliding window. The classification output module is configured to output the classification results of the welding stage in the current frame. The process recording and interface module is configured to write classification results, confidence scores, current frame timestamps, image indexes, and model versions into the process record and send them to the display terminal, data interface, or upper-layer system as needed for deployment.

Images