A welding quality diagnosis method and platform considering multi-sensing timing characteristics

Abstract

The application discloses a welding quality diagnosis method and platform considering multi-sensing time sequence characteristics, relates to the technical field of welding quality diagnosis, and comprises the following steps: collecting visible light, infrared light and temperature data of a welding process to obtain first welding process data; performing filtering processing on the first welding process data to obtain second welding process data; performing normalization processing on the second welding process data to obtain third welding process data; inputting the third welding process data into a 3DCNN-TimesNet model; extracting the first k main frequencies of the third welding process data through fast Fourier transform and calculating the corresponding period lengths; remodeling one-dimensional time sequence data into k two-dimensional tensors based on the period lengths; performing feature extraction on each two-dimensional tensor, fusing the features of adjacent time points, adjacent periods and different sensing signals; and performing spectrum amplitude weighted aggregation on the feature extraction results to output a welding quality classification result.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of welding quality diagnosis, and particularly relates to a welding quality diagnosis method and platform considering multi-sensor time sequence characteristics. BACKGROUND

[0002] Welding, as a basic process in manufacturing industry, plays an irreplaceable key role in the field of equipment manufacturing. Accurate diagnosis of welding quality is not only crucial for improving product quality, but also an effective way to improve resource utilization efficiency. The signals such as light, sound and heat generated during the welding process are closely related to the welding quality. The fusion of multi-sensor data can more comprehensively perceive the welding environment and improve the diagnosis ability of welding defects. In addition, low-dimensional time sequence data is simpler in form than image data and is more suitable for real-time application. In the welding process, the periodic changes of droplet transfer and keyhole shape, as well as the physical processes of melting and solidification, make the signals collected by the sensor often have multi-periodic characteristics. However, current research still lacks a unified understanding and utilization of the multi-periodic characteristics of the welding process. In addition, for time sequence data-driven welding process quality diagnosis, researchers often use existing time sequence models, lacking in-depth mining of the characteristics of welding data, and the current multi-sensor data fusion is often too complex, which is not conducive to real-time application scenarios. Therefore, it is urgent to propose a welding quality diagnosis method and platform considering multi-sensor time sequence characteristics. SUMMARY

[0003] To solve the above technical problems, the application provides a welding quality diagnosis method and platform considering multi-sensor time sequence characteristics, which realizes efficient fusion of multi-sensor data, and the complementary of multi-sensor information enhances the analysis ability of complex time sequence data and improves the judgment ability of the model in the welding scene.

[0004] In one aspect, to achieve the above object, the application provides a welding quality diagnosis method considering multi-sensor time sequence characteristics, comprising:

[0005] Collecting visible light, infrared light and temperature data of the welding process to obtain first welding process data;

[0006] Performing Savitzky-Golay filtering processing on the first welding process data to obtain second welding process data;

[0007] Performing normalization processing on the second welding process data to obtain third welding process data;

[0008] Inputting the third welding process data into a 3DCNN-TimesNet model:

[0009] Extracting the first k main frequencies of the third welding process data by fast Fourier transform, and calculating the corresponding period length;

[0010] resample the one-dimensional time-series data into k two-dimensional tensors based on the period length;

[0011] extract features of each two-dimensional tensor by 3D convolution, fuse features of adjacent time points, adjacent periods and different sensing signals;

[0012] aggregate the k feature extraction results according to spectral amplitude weighting, and output a welding quality classification result.

[0013] Optionally, the process of the Savitzky-Golay filtering processing includes:

[0014] setting a filter window with a window width of n = 2 m + 1;

[0015] adopting a 1st order polynomial to fit the data in the window; k-

[0016] solving a coefficient matrix by a least square method;

[0017] obtaining second welding process data according to .

[0018] Optionally, the process of the data resampling includes:

[0019] resampling the second sequence samples into a plurality of two-dimensional tensors:

[0020] .

[0021] wherein, represents a shape adjustment of a matrix, represents a zero padding on the time dimension to the second sequence samples, so that the length is adapted to a transform operation for .

[0022] Optionally, the process of the 3D convolution feature extraction includes:

[0023] processing three dimensions of the two-dimensional tensor by a 3D convolution kernel at the same time:

[0024] capturing period-to-period change features in the row direction;

[0025] capturing intra-period change features in the column direction;

[0026] fusing multi-sensing signal features in the channel direction.

[0027] Optionally, the process of the weighted aggregation includes: ​

[0028] Will different one-dimensional representations be fused, the one-dimensional representations are aggregated based on the amplitude:

[0029] ;

[0030] wherein, is the unnormalized amplitude, is a normalization operation that converts a set of values into a probability distribution, is the normalized amplitude weight, is the variable representation obtained after fusion.

[0031] Optionally, the welding quality classification process comprises:

[0032] Deeply mine the timing information by stacking multiple layers of modules;

[0033] Introduce a trainable weight index for each welding quality ;

[0034] Adopt binary cross-entropy loss, and expect the weight index in different paths to be trained separately to obtain the optimal value of each welding state, the network output is processed by a Sigmoid function before calculating the loss, and the network parameters are updated after calculating the gradient in the back propagation, and the loss calculation formula is as follows:

[0035] ;

[0036] ;

[0037] wherein, represents the number of categories of welding quality, represents the output of the model, is the prediction probability of the i-th category of the model, is the true label.

[0038] In another aspect to achieve the above object, the present application also provides a welding quality diagnosis platform considering multi-sensing timing characteristics, comprising:

[0039] Temperature sensor, photoelectric sensor, industrial computer, edge controller, motion mechanism, laser welding head, fiber laser, protective gas device, IO module;

[0040] The laser welding head is installed on the flange port of the motion mechanism, the temperature sensor is installed on the side of the laser welding head for measuring the temperature of the welding area and transmitting the collected data to the IO module, the photoelectric sensor is installed on the coaxial optical port of the laser welding head for capturing visible light and infrared light generated in the welding process and transmitting the collected data to the industrial computer through the bus, the IO module transmits the data from the temperature sensor to the industrial computer and receives the instructions from the industrial computer, the industrial computer issues position data to the motion mechanism, receives the collected data transmitted by the IO module and the photoelectric sensor, and pre-processes the collected data, and transmits the processing result to the edge controller, meanwhile, the industrial computer adjusts the power of the fiber laser and the flow of the protective gas device through the IO module, the edge controller judges the current welding quality through the deployed model and feeds back the judgment result to the industrial computer, and the motion mechanism receives the position data transmitted by the industrial computer and cooperates with the laser welding head to realize welding.

[0041] The present application discloses a welding quality diagnosis method and platform considering multi-sensor time sequence characteristics, uses discrete Fourier transform to extract the multi-period characteristics of temperature, photoelectric and other time sequence data in the welding process, captures the changes within and between the periods of these data through convolution, significantly enhances the capture ability of time characteristics in the welding process data, introduces 3D convolution into TimesNet neural network for efficient fusion of multi-sensor data, and the complementary of multi-sensor information enhances the analysis ability of complex time sequence data and improves the judgment ability of the model in the welding scene; considering that different welding qualities have different response degrees to sensors, different fusion weights are set for each sensor to reflect its importance under a specific welding quality, thereby improving the judgment ability of the model for different welding qualities. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application, and their

[0043] Figure 1 A flowchart of a welding quality diagnosis method considering multi-sensor time sequence characteristics according to an embodiment of the present application;

[0044] Figure 2 A model structure diagram according to an embodiment of the present application;

[0045] Figure 3 A welding quality diagnosis platform diagram according to an embodiment of the present application;

[0046] Figure 4A welding quality diagnosis platform physical diagram of an embodiment of the present application. DETAILED DESCRIPTION

[0047] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0048] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0049] As shown in Figures 1-2 The welding quality diagnosis method considering multi-sensor timing characteristics provided in the embodiment includes the following steps:

[0050] Step S1: Acquisition and preprocessing of multi-sensor timing data of the welding process, including the following steps:

[0051] Step S11: Acquisition of multi-sensor timing data, specifically, through photoelectric sensors and temperature sensors, visible light, infrared light and temperature data generated during the laser welding process are collected at a frame rate of 1000 Hz, by changing process parameters such as laser power, welding speed, etc., first welding process data corresponding to multiple welding quality types are obtained, here taking five welding qualities as an example, which are respectively non-penetration, good, over-penetration, burn-through and welding deviation, due to the acceleration and deceleration movement process of the movement platform, the initial stage of the welding process is unstable, so the sensor data of this part is discarded, and finally the stable processing area data of 4000 milliseconds in the middle of each welding sample is selected as the use data;

[0052] Step S12: Sensor signal filtering, Savitzky-Golay time domain signal filtering is adopted, the first welding process data is processed through the filter window to obtain the second welding process data;

[0053] Specifically, the Savitzky-Golay time domain signal filtering includes the following steps:

[0054] Window selection, set the width of the filter window as n, and represent the measurement point, where m is the half width of the window, and x is the position of the specific point within the window range;

[0055] Polynomial fitting, select a polynomial of degree p to fit the first welding process data within the filter window;

[0056] Matrix form, define the first welding process data matrix within the filter window​​ and design matrix , the relationship is as follows:

[0057] ;

[0058] More specifically:

[0059] ;

[0060] wherein, is the first welding process data matrix, hereinafter abbreviated as , is the design matrix, hereinafter abbreviated as , is the coefficient matrix, hereinafter abbreviated as , is the error matrix;

[0061] Residual sum of squares calculation, the formula is as follows:

[0062] ;

[0063] wherein, is the residual sum of squares, indicating the sum of squares of the difference between the data points and the predicted value; represents the transpose of the matrix;

[0064] Solving the coefficients, according to the principle of least squares, to make the residual sum of squares minimum, then The partial derivative of is 0, the coefficient matrix is obtained, the solving formula is as follows:

[0065] ;

[0066] Obtain the second welding process data, using the obtained coefficient matrix Smooth the first welding process data in the filter window, the formula is as follows:

[0067] ;

[0068] wherein, is the second welding process data matrix obtained in the filter window;

[0069] Step S13: sensor signal normalization processing, such as using the Min-Max normalization method to normalize the second welding process data, mapping and scaling the second welding process data value to the range of [0, 1], to obtain the third welding process data;

[0070] Step S14: Construction of the data set, using an integer sequence starting from 0 as the welding state label, using a sliding window method to construct the data set, setting the time step of the sliding window to 80 ms, using a semi-non-overlapping method to divide the time series data into multiple samples, the window overlap rate is 50%, the third welding process data in each window obtained by division is regarded as a sequence sample, the sequence sample is matched with the welding state label, and sequence samples corresponding to the non-penetration, good, over-penetration, burn-through and welding deviation quality types are obtained, and all matched sequence samples-labels are divided according to the ratio of 4:1:1 to obtain the training set, the verification set and the test set;

[0071] Step S2: Propose a 3DCNN-TimesNet model, which uses the model to model the multi-cycle characteristics of welding time series data and the efficient fusion of multi-sensing data. Specifically, it replaces the Inception module in the TimesNet neural network with a 3D convolution to obtain an improved model, including the following steps:

[0072] Step S21: Fast Fourier transform, for a sequence sample with a length of time steps and variables, in this case, , the sequence sample is projected to deep features through an embedding layer to obtain a second sequence sample with a dimension of , , is the dimension of the new features obtained after the embedding layer, and the first main frequencies are selected through the fast Fourier transform. This process can be achieved by sorting the amplitude spectrum, as follows:

[0073] ;

[0074] ;

[0075] ;

[0076] wherein represents the second sequence sample, is the average operation, represents the amplitude calculation, represents the fast Fourier transform, represents a series of frequencies corresponding to the amplitude, which is calculated by averaging the dimensions , is the first largest amplitude frequency, which obtains the most significant frequency set , and its corresponding non-normalized amplitude , represents the intensity of the periodic basis function with frequency , whose corresponding period length is , and there are period lengths .

[0077] Step S22: Data periodic reshaping, the second sequence samples are reshaped into a set of two-dimensional tensors by the following formula:

[0078] ;

[0079] where represents the shape of the reshaped matrix, represents zero padding of the second sequence samples in the time dimension to adapt its length to the transform operation of . It should be noted that represents the th two-dimensional tensor after reshaping based on the frequency , whose column direction represents the change within the period with length , and the row direction represents the change between periods, while the number of channels is mapped by the number of sensing signals, based on the selected frequency and the estimated period, the second sequence samples get a set of two-dimensional tensors , which correspond to different temporal two-dimensional changes derived from different periods . The reshaped two-dimensional tensors contain three types of local features: one is the locality between adjacent time points; two is the locality between adjacent periods; three is the locality between different sensing signals, therefore, these changes can be conveniently processed by 3D convolution kernels.

[0080] Step S23: Data feature extraction and inverse reshaping, a set of two-dimensional tensors are extracted by using 3D convolution on the second sequence samples to realize the fusion of adjacent time points, adjacent periods, and different sensing signals, the formula is as follows:

[0081] ;

[0082] ;

[0083] where represents the use of 3D convolution to process two-dimensional tensors, represents the two-dimensional tensors after 3D convolution processing, is converted from back to one-dimensional space, is the inverse operation of , is the length of the sequence after padding Truncated to original length The addition of 3D convolution can model adjacent time points and adjacent cycles in the spatial dimension, and fuse multiple sensor signals in the channel dimension;

[0084] Step S24: adaptive aggregation, fusing different one-dimensional representations , aggregating one-dimensional representations based on amplitude:

[0085] ;

[0086] wherein, is the aforementioned non-normalized amplitude, is a normalization operation that converts a set of values into a probability distribution, is the normalized amplitude weight, is the variable representation obtained after fusion;

[0087] Since the intra-cycle, inter-cycle changes and different sensor signals are already included in multiple highly structured two-dimensional tensors, this method can simultaneously fully capture multi-scale time two-dimensional changes and interact different sensor signals.

[0088] Step S25: classification decision, including:

[0089] Step S251: modular packaging, deepening the timing information by stacking multiple layers of modules;

[0090] Step S252: weight fusion, introducing a trainable weight index for each welding quality , such as for incomplete penetration, setting the branch weight index to , and the rest are similar, so the welding state recognition is converted into multiple binary classification tasks;

[0091] Step S253: loss calculation, using binary cross entropy loss, expecting the weight index in different paths to be trained separately to obtain the optimal value of each welding state, before calculating the loss, the network output is processed by the Sigmoid function, and after calculating the gradient by back propagation, the network parameters are updated, and the loss loss The calculation formula is as follows:

[0092] ;

[0093] ;

[0094] wherein, represents the number of welding quality categories, represents the output of the model, is the predicted probability of the model for the i-th class, is the real label;

[0095] Step S3: The welding process quality diagnosis is specifically that visible light, infrared light and temperature data generated in the welding process are collected in real time through photoelectric sensors and temperature sensors, and after filtering, normalization and the like, are input into the 3DCNN-TimesNet model, the actual welding state is identified, once a defect such as incomplete penetration is detected, the system automatically triggers an early warning feedback in time, and the laser power is dynamically increased to optimize the welding quality.

[0096] In the present application, a welding quality diagnosis method considering multi-sensing time sequence characteristics is proposed, the multi-period characteristics of temperature, photoelectric and other time sequence data in the welding process are extracted by using discrete Fourier transform, the period-to-period change and the period-to-period change of these data are captured by convolution, and the time feature capturing capability of the welding process data is significantly enhanced;

[0097] In the present application, 3D convolution is introduced into the TimesNet neural network, the interactive fusion of multi-sensing data is efficiently realized, the complementary of multi-information enhances the analysis capability of complex time sequence data, and the judgment capability of the model in the welding scene is improved.

[0098] In the present application, considering that different welding qualities have different response degrees to sensors, different fusion weights are set for each sensor to reflect its importance under a specific welding quality, so as to improve the judgment capability of the model for different welding qualities.

[0099] As shown in Figures 3-4 , the present embodiment also provides a welding quality diagnosis platform considering multi-sensing time sequence characteristics, comprising: a temperature sensor, a photoelectric sensor, an industrial computer, an edge controller, a motion mechanism, a laser welding head, a fiber laser, a shielding gas device, and an IO module.

[0100] The laser welding head is installed on the flange port of the motion mechanism, the temperature sensor is installed on the side of the laser welding head for measuring the temperature of the welding area, and the collected data is transmitted to the IO module, the photoelectric sensor is installed on the coaxial optical port of the laser welding head for capturing visible light and infrared light generated in the welding process, and the collected data is transmitted to the industrial computer through the bus, the IO module transmits the data from the temperature sensor to the industrial computer, and simultaneously receives the instructions from the industrial computer, the industrial computer issues position data to the motion mechanism, receives the collected data transmitted by the IO module and the photoelectric sensor, and pre-processes the processing results, and transmits the processing results to the edge controller, and simultaneously, the industrial computer adjusts the power of the fiber laser and the flow of the protective gas device through the IO module. The edge controller judges the current welding quality through the deployed model, and feeds back the judgment result to the industrial computer. The motion mechanism receives the position data transmitted by the industrial computer, and cooperates with the laser welding head to realize welding.

[0101] The embodiment provides a welding quality diagnosis method and platform considering multi-sensing time sequence characteristics, utilizes discrete Fourier transform to extract multi-period characteristics of temperature, photoelectric and other time sequence data in a welding process, captures period changes and inter-period changes of the data through convolution, and significantly enhances the capturing capability of time characteristics in welding process data; 3D convolution is introduced into a TimesNet neural network, and is used for realizing efficient fusion of multi-sensing data; and complementary of multi-sensing information enhances the analysis capability of complex time sequence data, and improves the judgment capability of the model in a welding scene; considering that different welding qualities have different response degrees to sensors, different fusion weights are set for each sensor to reflect the importance of the sensor under a specific welding quality, so that the judgment capability of the model for different welding qualities is improved.

[0102] The above is only a preferred specific embodiment of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the application, which should be covered in the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A welding quality diagnosis method considering multi-sensing timing characteristics, characterized by, The method comprises the following steps: Collecting visible light, infrared light and temperature data of the welding process to obtain first welding process data; Performing Savitzky-Golay filtering processing on the first welding process data to obtain second welding process data; Performing normalization processing on the second welding process data to obtain third welding process data; Inputting the third welding process data into a 3DCNN-TimesNet model, wherein the 3DCNN-TimesNet model is an improved model obtained by replacing an Inception module in a TimesNet neural network with a 3D convolution, and the specific process is as follows: Extracting the first k main frequencies of the third welding process data through fast Fourier transform and calculating the corresponding period lengths; Reshaping one-dimensional time series data into k two-dimensional tensors based on the period lengths; Extracting features of each two-dimensional tensor through 3D convolution, fusing features of adjacent time points, adjacent periods and different sensing signals; Aggregating the k feature extraction results according to spectral amplitude weighting to output a welding quality classification result; The process of data reshaping comprises: Fast Fourier Transform, for a sequence of samples of length containing variables at each time step, projects the sequence of samples to deep features by an embedding layer to obtain a second sequence of samples; reshapes the second sequence of samples into multiple two-dimensional tensors: ； wherein, represents reshaping the matrix, represents zero-padding the second sequence of samples in the time dimension to adapt its length to the transform operation of ; The process of 3D convolution feature extraction comprises: Processing three dimensions of a two-dimensional tensor through a 3D convolution kernel at the same time: Capturing period-to-period change characteristics in the row direction; Capturing period-to-period change characteristics in the column direction; Fusing multi-sensing signal features in the channel direction; The process of weighted aggregation comprises: The different one-dimensional representations are fused, the one-dimensional representations are aggregated based on the amplitudes: ； wherein, is the unnormalized amplitude, is the normalization operation that converts a set of values into a probability distribution, is the normalized amplitude weight, is the resulting variable representation after fusion. The process of welding quality classification comprises: Deeply mining time series information through stacking multiple layers of modules; Introducing trainable weight indices for each welding quality ; Adopting binary cross-entropy loss, expecting the weight index in different paths Single training to obtain the best value of each welding state, the network output is processed by the Sigmoid function before calculating the loss, and the network parameters are updated after calculating the gradient by back propagation, and the loss calculation formula is as follows: ； ； wherein, a class number representing a quality of welding, an output of the model, a predicted probability of the model for the i-th class, is a true label.

2. The welding quality diagnostic method considering multi-sensing timing characteristics according to claim 1, wherein The process of Savitzky-Golay filtering processing comprises: The window width is set to n = 2 m + 1 filter window; use k- A first-order polynomial is used to fit the data within the window; Solving the coefficient matrix by least squares ; According to Second welding process data is obtained.

3. A platform for a welding quality diagnostic method considering multi-sensing timing characteristics according to any one of claims 1-2, characterized in that, The method comprises the following steps: A temperature sensor, a photoelectric sensor, an industrial computer, an edge controller, a motion mechanism, a laser welding head, a fiber laser, a shielding gas device and an IO module; The laser welding head is installed on a flange port of the motion mechanism, the temperature sensor is installed on the side of the laser welding head to measure the temperature of the welding area and transmit the collected data to the IO module, the photoelectric sensor is installed on the coaxial optical port of the laser welding head to capture visible light and infrared light generated during the welding process and transmit the collected data to the industrial computer through a bus, the IO module transmits data from the temperature sensor to the industrial computer and receives instructions from the industrial computer, the industrial computer sends position data to the motion mechanism, receives collected data transmitted by the IO module and the photoelectric sensor, and performs preprocessing on the data, and transmits the processing result to the edge controller, meanwhile, the industrial computer adjusts the power of the fiber laser and the flow of the shielding gas device through the IO module, the edge controller judges the current welding quality through the deployed model and feeds back the judgment result to the industrial computer, and the motion mechanism receives the position data transmitted by the industrial computer to cooperate with the laser welding head to realize welding.

Images