Welding quality diagnosis method and platform considering multi-sensing time sequence characteristics

Abstract

The invention discloses a welding quality diagnosis method and platform considering multi-sensing time sequence characteristics, and relates to the technical field of welding quality diagnos.The welding quality diagnosis method comprises the steps that visible light, infrared light and temperature data in the welding process are collected, and first welding process data are obtained; filtering the first welding process data to obtain second welding process data; normalizing the second welding process data to obtain third welding process data; the third welding process data are input into a 3DCNN-TimesNet model; the first k main frequencies of the third welding process data are extracted through fast Fourier transform, and the corresponding period length is calculated; reshaping the one-dimensional time sequence data into k two-dimensional tensors based on the period length; performing feature extraction on each two-dimensional tensor, and fusing features of adjacent time points, adjacent periods and different sensing signals; and feature extraction results are subjected to weighted aggregation according to frequency spectrum amplitudes, and welding quality classification results are output.

Description

Technical Field

[0001] The present invention belongs to the technical field of welding quality diagnosis, and in particular relates to a welding quality diagnosis method and platform considering multi-sensor timing characteristics. Background Art

[0002] Welding, as a fundamental manufacturing process, plays an irreplaceable and critical role in the equipment manufacturing sector. Accurately diagnosing welding quality is not only crucial for improving product quality but also an effective way to increase resource utilization efficiency. Light, sound, and heat signals generated during the welding process are closely related to welding quality. Fusion of multi-sensor data enables a more comprehensive understanding of the welding environment and improves the ability to diagnose welding defects. Furthermore, low-dimensional time series data, compared to image data, is simpler and more suitable for real-time applications. During the welding process, the intertwining of droplet transfer and the periodic changes in keyhole morphology, as well as physical processes such as melting and solidification, often results in multi-periodic characteristics in the signals collected by sensors. However, current research lacks a unified understanding and utilization of these multi-periodic characteristics in the welding process. Furthermore, current research on time-series data-driven welding process quality diagnosis often relies on existing time-series models, lacking in-depth exploration of welding data characteristics. Furthermore, current multi-sensor data fusion is often overly complex, making it unsuitable for real-time applications. Therefore, a welding quality diagnosis method and platform that considers multi-sensor time-series characteristics is urgently needed. Summary of the Invention

[0003] To solve the above technical problems, the present invention proposes a welding quality diagnosis method and platform that takes into account the multi-sensor timing characteristics, thereby realizing the efficient fusion of multi-sensor data. The complementarity of multi-sensor information enhances the analysis capability of complex timing data and improves the judgment capability of the model in welding scenarios.

[0004] In order to achieve the above-mentioned object, the present invention provides a welding quality diagnosis method considering multi-sensor timing characteristics, comprising: Collecting visible light, infrared light and temperature data of the welding process to obtain first welding process data; performing Savitzky-Golay filtering on the first welding process data to obtain second welding process data; performing normalization processing on the second welding process data to obtain third welding process data; The third welding process data is input into the 3DCNN-TimesNet model: Extracting the first k main frequencies of the third welding process data by fast Fourier transform and calculating the corresponding cycle lengths; Reshape the one-dimensional time series data into k two-dimensional tensors based on the period length; Feature extraction is performed on each two-dimensional tensor through 3D convolution, fusing the features of adjacent time points, adjacent cycles, and different sensor signals; The k feature extraction results are weighted and aggregated according to the spectrum amplitude, and the welding quality classification results are output.

[0005] Optionally, the Savitzky-Golay filtering process includes: Set the window width to n =2 m +1 filter window; use k- A first-degree polynomial is fitted to the data within the window; Solving the coefficient matrix by the least squares method ; according to The second welding process data is obtained.

[0006] Optionally, the data reshaping process includes: The second series of samples Reshape into multiple 2D tensors: ; in, Reshape the matrix. Indicates that the second sequence samples are zero-filled in the time dimension to make its length suitable for transformation operation.

[0007] Optionally, the 3D convolution feature extraction process includes: The 3D convolution kernel processes the three dimensions of a 2D tensor simultaneously: The row direction captures the inter-cycle variation characteristics; The column direction captures the changing characteristics within the period; Channel-wise fusion of multi-sensor signal features.

[0008] Optionally, the weighted aggregation process includes: Will Different one-dimensional representations Perform fusion and aggregate the one-dimensional representations based on magnitude: ; in, is the unnormalized amplitude, It is a normalization operation that converts a set of values ​​into a probability distribution. is the normalized amplitude weight, is the variable representation obtained after fusion.

[0009] Optionally, the welding quality classification process includes: Deepen timing information by stacking multiple layers of modules; Introducing trainable weight indices for each weld quality ; Using binary cross entropy loss, we expect the weight index in different paths to be Train separately to obtain the best value for each welding state. Before calculating the loss, the network output is processed by the Sigmoid function. After backpropagation to calculate the gradient, the network parameters are updated. The loss calculation formula is as follows: ; ; in, Indicates the category number of welding quality, represents the output of the model, is the model’s predicted probability for the i-th category, is the true label.

[0010] On the other hand, to achieve the above-mentioned purpose, the present invention also provides a welding quality diagnosis platform considering multi-sensor timing characteristics, comprising: Temperature sensor, photoelectric sensor, industrial computer, edge controller, motion mechanism, laser welding head, fiber laser, protective gas device, IO module; The laser welding head is mounted on the flange port of the motion mechanism. The temperature sensor is mounted on the side of the laser welding head and is used to measure the temperature of the welding area and transmit the collected data to the IO module. The photoelectric sensor is mounted on the coaxial optical port of the laser welding head and is used to capture visible light and infrared light generated during the welding process and transmit the collected data to the industrial computer via a bus. The IO module transmits the data from the temperature sensor to the industrial computer and receives instructions from the industrial computer. The industrial computer sends position data to the motion mechanism, receives the collected data transmitted by the IO module and the photoelectric sensor, pre-processes the data, and transmits the processing results to the edge controller. At the same time, the industrial computer adjusts the power of the fiber laser and the flow rate of the shielding gas device through the IO module. The edge controller judges the current welding quality through the deployed model and feeds back the judgment result to the industrial computer. The motion mechanism receives the position data transmitted by the industrial computer and cooperates with the laser welding head to achieve welding.

[0011] Technical effect of the present invention: The present invention discloses a welding quality diagnosis method and platform that considers multi-sensor timing characteristics. Discrete Fourier transform is used to extract the multi-periodic characteristics of time series data such as temperature and photoelectricity in the welding process, and convolution is used to capture and model the intra-periodic changes and inter-periodic changes of these data, which significantly enhances the ability to capture the time characteristics of the welding process data; 3D convolution is introduced into the TimesNet neural network to achieve efficient fusion of multi-sensor data. The complementarity of multi-sensor information enhances the analysis ability of complex time series data and improves the judgment ability of the model in welding scenarios; considering that different welding qualities have different response degrees to sensors, different fusion weights are set for each sensor to reflect its importance under specific welding quality, thereby improving the judgment ability of the model for different welding qualities. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings: Figure 1 Schematic diagram of a flow chart of a welding quality diagnosis method considering multi-sensor timing characteristics according to an embodiment of the present invention; Figure 2 This is a model structure diagram of an embodiment of the present invention; Figure 3 Schematic diagram of a welding quality diagnosis platform according to an embodiment of the present invention; Figure 4 This is a physical picture of the welding quality diagnosis platform according to an embodiment of the present invention. DETAILED DESCRIPTION

[0013] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0014] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0015] like Figure 1-2 As shown, this embodiment provides a welding quality diagnosis method considering multi-sensor timing characteristics, including: Step S1: Collection and preprocessing of multi-sensor time series data during welding, including the following steps: Step S11: Acquisition of multi-sensor time series data. Specifically, photoelectric sensors and temperature sensors are used to collect visible light, infrared light, and temperature data generated during the laser welding process at a frame rate of 1000 Hz. By varying process parameters such as laser power and welding speed, first welding process data corresponding to various welding quality types is obtained. Five welding quality types are used as examples: incomplete penetration, good penetration, excessive penetration, burn-through, and weld deviation. Due to the acceleration and deceleration of the motion platform, the initial state of the welding process is unstable, so the sensor data in this portion is discarded. Ultimately, the data from the relatively stable processing area within the middle 4000 milliseconds of each welding sample is selected as the data used. Step S12: sensor signal filtering processing, using Savitzky-Golay time domain signal filtering, processing the first welding process data through a filtering window to obtain second welding process data; Specifically, Savitzky-Golay time domain signal filtering includes the following steps: Window selection, set the width of the filter window to n, and use Represents the measurement point, where m is the half-width of the window and x is the position of the specific point within the window; Polynomial Fit, select The first welding process data within the filter window is fitted with a polynomial; Matrix form, defines the first welding process data matrix within the filter window and design matrix , and the relationship is as follows: ; More specific: ; in, is the first welding process data matrix, which is subsequently abbreviated as , is the design matrix, which is subsequently abbreviated as , is the coefficient matrix, which is abbreviated as , is the error matrix; The residual sum of squares is calculated as follows: ; in, is the residual sum of squares, which represents the sum of squares of the differences between the data points and the predicted values; Represents the transpose of a matrix; Solve the coefficients. According to the principle of least squares, to minimize the sum of squares of the residuals, for The partial derivative must be 0, and the coefficient matrix is ​​obtained , the solution formula is as follows: ; Get the second welding process data and use the obtained coefficient matrix The first welding process data within the filter window is smoothed using the following formula: ; in, is the second welding process data matrix obtained within the filter window; Step S13: normalizing the sensor signal, such as using a Min-Max normalization method to normalize the second welding process data, mapping and scaling the second welding process data value to a range of [0, 1], and obtaining third welding process data; Step S14: Constructing a data set. An integer sequence starting from 0 is used as the welding state label. The data set is constructed using the sliding window method. A sliding window with a time step of 80 ms is set. The time series data is divided into multiple samples using a semi-non-overlapping method with a window overlap rate of 50%. The third welding process data in each divided window is regarded as a sequence sample. The sequence samples are matched with the welding state labels to obtain sequence samples corresponding to the quality types of incomplete penetration, good penetration, excessive penetration, burn-through, and weld deviation. All matched sequence samples and labels are divided into training sets, validation sets, and test sets according to a 4:1:1 ratio. Step S2: Propose a 3DCNN-TimesNet model. This model is used to combine the multi-cycle characteristics of welding time series data and the efficient fusion of multi-sensor data for modeling. Specifically, 3D convolution is used to replace the Inception module in the TimesNet neural network to obtain an improved model, which includes the following steps: Step S21: Fast Fourier transform, for length time steps, including A sequence sample of variables, in this case, , project the sequence samples to the deep features through the embedding layer to obtain the second sequence samples with a dimension of , , is the dimension of the new feature obtained after the embedding layer, and the front is selected by fast Fourier transform. The main frequencies can be obtained by sorting the amplitude spectrum. The formula is as follows: ; ; ; in, represents the second sequence of samples, For averaging operation, represents the amplitude calculation, represents the fast Fourier transform, Represents the amplitude corresponding to a series of frequencies, which is obtained by Averaging operation in each dimension Calculated, Before selection The frequencies with the largest amplitudes are obtained to obtain the most significant frequency set , and its corresponding unnormalized amplitude , Indicates the frequency The intensity of the periodic basis function, the corresponding period length is , a total of Cycle length .

[0016] Step S22: Data is periodically reshaped, and the second sequence samples are transformed into Reshape into multiple 2D tensors: ; in, Reshape the matrix. Indicates that the second sequence samples are zero-filled in the time dimension to make its length suitable for transformation operation. It should be noted that Frequency-based After reshaping A two-dimensional tensor whose column direction represents the length of the period The row direction represents the change between cycles, and the channel number It is mapped by the number of sensing signals. Based on the selected frequency and estimated period, the second sequence of samples obtains a set of two-dimensional tensors , this set of tensors corresponds to the derived values ​​of different periods The reshaped two-dimensional tensor contains three types of local features: locality between adjacent time points, locality between adjacent cycles, and locality between different sensor signals. Therefore, these changes can be easily processed by 3D convolution kernels.

[0017] Step S23: Data feature extraction and inverse reshaping: 3D convolution is used to extract features from a set of two-dimensional tensors obtained from the second sequence of samples to achieve the fusion of adjacent time points, adjacent cycles, and different sensor signals. The formula is as follows: ; ; in, Indicates the use of 3D convolution to process two-dimensional tensors, Represents the two-dimensional tensor after 3D convolution processing, is Converting back to one-dimensional space, yes The reverse operation, is the length of the sequence after padding Truncate to original length ,The addition of 3D convolution can model adjacent time points and adjacent cycles in ,the spatial dimension, on the one hand, and fuse multiple sensor signals in ,the channel dimension, on the other hand; Step S24: Adaptive aggregation, Different one-dimensional representations Perform fusion and aggregate the one-dimensional representations based on magnitude: ; in, is the unnormalized amplitude mentioned above, It is a normalization operation that converts a set of values ​​into a probability distribution. is the normalized amplitude weight, is the variable representation obtained after fusion; Since the intra-cycle and inter-cycle variations as well as different sensor signals are already contained in multiple highly structured two-dimensional tensors, this method can fully capture the multi-scale temporal two-dimensional variations and interacting different sensor signals simultaneously.

[0018] Step S25: Classification decision, including: Step S251: modular packaging, in-depth mining of timing information by stacking multiple layers of modules; Step S252: Weight fusion, introducing a trainable weight index for each welding quality For example, for incomplete melting, set the branch weight index to ,The rest are similar, so welding state recognition is transformed into multiple binary classification tasks; Step S253: Loss calculation, using binary cross entropy loss, expecting weight indexes in different paths It can be trained separately to obtain the best value for each welding state. Before calculating the loss, the network output is processed by the Sigmoid function. After back propagation to calculate the gradient, the network parameters are updated. loss The calculation formula is as follows: ; ; in, Indicates the category number of welding quality, represents the output of the model, is the model’s predicted probability for the i-th category, is the true label; Step S3: Welding process quality diagnosis specifically involves collecting visible light, infrared light, and temperature data generated during the welding process in real time through photoelectric sensors and temperature sensors. After filtering and normalization, the data is input into the 3DCNN-TimesNet model to identify the actual welding status. Once a defect is detected, such as incomplete penetration, the system automatically triggers early warning feedback and dynamically increases the laser power to optimize the welding quality.

[0019] In this paper, a welding quality diagnosis method that considers multi-sensor timing characteristics is proposed. Discrete Fourier transform is used to extract the multi-cycle characteristics of time series data such as temperature and photoelectricity during the welding process. Convolution is used to capture and model the intra-cycle and inter-cycle variations of these data, significantly enhancing the ability to capture the temporal characteristics of the welding process data. In this paper, 3D convolution is introduced into the TimesNet neural network to efficiently realize the interactive fusion of multi-sensor data. The complementarity of multiple information enhances the analysis ability of complex time series data and improves the judgment ability of the model in welding scenarios. In the present invention, taking into account that different welding qualities have different response degrees to sensors, different fusion weights are set for each sensor to reflect its importance under specific welding quality, thereby improving the model's judgment ability for different welding qualities.

[0020] like Figure 3-4 As shown, this embodiment also provides a welding quality diagnosis platform that considers multi-sensor timing characteristics, including: a temperature sensor, a photoelectric sensor, an industrial computer, an edge controller, a motion mechanism, a laser welding head, a fiber laser, a protective gas device, and an IO module; The laser welding head is mounted on the flange port of the motion mechanism. A temperature sensor, mounted on the side of the laser welding head, measures the temperature of the welding area and transmits the collected data to the I / O module. A photoelectric sensor, mounted on the coaxial optical port of the laser welding head, captures the visible and infrared light generated during the welding process and transmits the collected data to the industrial computer via a bus. The I / O module transmits data from the temperature sensor to the industrial computer and receives instructions from the industrial computer. The industrial computer sends position data to the motion mechanism, receives and pre-processes the data from the I / O module and photoelectric sensor, and transmits the results to the edge controller. The industrial computer, through the I / O module, adjusts the power of the fiber laser and the flow rate of the shielding gas system. The edge controller uses the deployed model to determine the current weld quality and feeds the results back to the industrial computer. The motion mechanism receives the position data from the industrial computer and cooperates with the laser welding head to perform welding.

[0021] This embodiment provides a welding quality diagnosis method and platform that considers multi-sensor timing characteristics. Discrete Fourier transform is used to extract the multi-periodic characteristics of time series data such as temperature and photoelectricity during the welding process. Convolution is used to capture and model the intra-periodic and inter-periodic changes of these data, significantly enhancing the ability to capture the temporal characteristics of the welding process data. 3D convolution is introduced into the TimesNet neural network to achieve efficient fusion of multi-sensor data. The complementary multi-sensor information enhances the analysis capability of complex time series data and improves the model's judgment capability in welding scenarios. Taking into account that different welding qualities have different response levels to sensors, different fusion weights are set for each sensor to reflect its importance under specific welding qualities, thereby improving the model's judgment capability for different welding qualities.

[0022] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A welding quality diagnosis method considering multi-sensor timing characteristics, characterized in that: include: Collecting visible light, infrared light and temperature data of the welding process to obtain first welding process data; performing Savitzky-Golay filtering on the first welding process data to obtain second welding process data; performing normalization processing on the second welding process data to obtain third welding process data; The third welding process data is input into the 3DCNN-TimesNet model: Extracting the first k main frequencies of the third welding process data by fast Fourier transform and calculating the corresponding cycle lengths; Reshape the one-dimensional time series data into k two-dimensional tensors based on the period length; Feature extraction is performed on each two-dimensional tensor through 3D convolution, fusing the features of adjacent time points, adjacent cycles, and different sensor signals; The k feature extraction results are weighted and aggregated according to the spectrum amplitude, and the welding quality classification results are output.

2. The welding quality diagnosis method considering multi-sensor timing characteristics according to claim 1, characterized in that: The Savitzky-Golay filtering process includes: Set the window width to n =2 m +1 filter window; use k- A first-degree polynomial is fitted to the data within the window; Solving the coefficient matrix by the least squares method ; according to The second welding process data is obtained.

3. The welding quality diagnosis method considering multi-sensor timing characteristics according to claim 1, characterized in that: The data reshaping process includes: The second series of samples Reshape into multiple 2D tensors: ； in, Reshape the matrix. Indicates that the second sequence samples are zero-filled in the time dimension to make its length suitable for transformation operation.

4. The welding quality diagnosis method considering multi-sensor timing characteristics according to claim 1, characterized in that: The 3D convolution feature extraction process includes: The 3D convolution kernel processes the three dimensions of a 2D tensor simultaneously: The row direction captures the inter-cycle variation characteristics; The column direction captures the changing characteristics within the period; Channel-wise fusion of multi-sensor signal features.

5. The welding quality diagnosis method considering multi-sensor timing characteristics according to claim 1, characterized in that: The weighted aggregation process includes: Will Different one-dimensional representations Perform fusion and aggregate the one-dimensional representations based on magnitude: ； in, is the unnormalized amplitude, It is a normalization operation that converts a set of values ​​into a probability distribution. is the normalized amplitude weight, is the variable representation obtained after fusion.

6. The welding quality diagnosis method considering multi-sensor timing characteristics according to claim 1, characterized in that: The welding quality classification process includes: Deepen timing information by stacking multiple layers of modules; Introducing trainable weight indices for each weld quality ; Using binary cross entropy loss, we expect the weight index in different paths to be Train separately to obtain the best value for each welding state. Before calculating the loss, the network output is processed by the Sigmoid function. After backpropagation to calculate the gradient, the network parameters are updated. The loss calculation formula is as follows: ； ； in, Indicates the category number of welding quality, represents the output of the model, is the model’s predicted probability for the i-th category, is the true label.

7. A platform for a welding quality diagnosis method considering multi-sensor timing characteristics according to any one of claims 1 to 6, characterized in that: include: Temperature sensor, photoelectric sensor, industrial computer, edge controller, motion mechanism, laser welding head, fiber laser, protective gas device, IO module; The laser welding head is mounted on the flange port of the motion mechanism. The temperature sensor is mounted on the side of the laser welding head and is used to measure the temperature of the welding area and transmit the collected data to the IO module. The photoelectric sensor is mounted on the coaxial optical port of the laser welding head and is used to capture visible light and infrared light generated during the welding process and transmit the collected data to the industrial computer via a bus. The IO module transmits the data from the temperature sensor to the industrial computer and receives instructions from the industrial computer. The industrial computer sends position data to the motion mechanism, receives the collected data transmitted by the IO module and the photoelectric sensor, pre-processes the data, and transmits the processing results to the edge controller. At the same time, the industrial computer adjusts the power of the fiber laser and the flow rate of the shielding gas device through the IO module. The edge controller judges the current welding quality through the deployed model and feeds back the judgment result to the industrial computer. The motion mechanism receives the position data transmitted by the industrial computer and cooperates with the laser welding head to achieve welding.

Images