A voiceprint feature-based electric welding equipment fault diagnosis method and system

Abstract

The present application relates to a kind of based on voiceprint feature electric welding equipment fault diagnosis method and system.Method includes the multi-channel original sound signal received by the microphone array deployed on electric welding equipment;Multi-channel original sound signal is handled with the collaborative filtering method of weight vector self-adaptive adjustment to obtain filtered multi-channel sound signal;The feature vector corresponding to each other in different time scales, different frequency sub-band under filtered multi-channel sound signal is extracted, and feature fusion processing is carried out, to obtain multi-scale fusion feature;Global voiceprint feature is obtained based on the feature dimension reduction processing of multi-scale fusion feature based on attention mechanism;The feature extraction model of extended feature trained based on BYOL model training framework is used to carry out feature extraction to multi-channel original sound signal, to obtain extended voiceprint feature;Based on global voiceprint feature, extended voiceprint feature, obtain equipment fault information.The anti-interference capability of voiceprint feature can be improved, and the foundation is laid for electric welding equipment fault diagnosis.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of fault identification technology, specifically to a fault diagnosis method and system for welding equipment based on voiceprint features. Background Technology

[0002] With the continuous improvement of intelligent manufacturing and industrial automation, welding equipment is being used more and more widely in industrial production. However, the welding operation environment is complex and changeable, accompanied by strong noise, light, electromagnetic and other interferences during operation, which poses a great challenge to the real-time perception of equipment operating status and fault diagnosis. Especially in enclosed or semi-enclosed workshops, where space is limited, noise is high and equipment is dense, existing fault detection technologies often cannot meet the requirements of high precision, low cost and easy deployment.

[0003] Currently, traditional fault diagnosis methods for welding equipment mainly rely on mechanical vibration monitoring or current parameter monitoring. These methods typically require the installation of dedicated hardware sensors on the equipment, such as accelerometers and current transformers. This not only increases system costs and deployment difficulty but also makes them susceptible to interference from the industrial environment. Furthermore, these sensors are prone to wear and tear, require significant maintenance, and have poor long-term reliability. In addition, these methods have limited ability to detect subtle abnormal changes during equipment operation, leading to missed detections or false diagnoses, and thus failing to meet the need for early fault warnings for welding equipment.

[0004] In comparison, using acoustic signatures for fault diagnosis offers significant advantages. First, acoustic signature acquisition equipment is simple in structure, low in cost, and flexible in deployment, making it easy to install on welding equipment of various models and operating conditions. Second, acoustic signals are highly sensitive to the operating status of equipment; different operating states and fault modes are often accompanied by unique acoustic signature changes. By analyzing acoustic signals, the operating status and potential faults of equipment can be quickly and accurately identified. Furthermore, acoustic signals can reveal subtle abnormal fluctuations during equipment operation, enabling early detection and warning of faults, thereby reducing downtime losses and improving equipment reliability and production efficiency. However, current methods for fault diagnosis using acoustic signatures still suffer from the problem of weak anti-interference capabilities of acoustic signature features.

[0005] Therefore, it is particularly important to develop a method that can effectively extract voiceprint features, thereby laying the foundation for subsequent fault diagnosis of welding equipment based on voiceprint features. Summary of the Invention

[0006] This specification provides a method and system for fault diagnosis of welding equipment based on voiceprint features, which can improve the anti-interference capability of voiceprint features and lay the foundation for subsequent fault diagnosis of welding equipment based on voiceprint features.

[0007] The technical solution is as follows:

[0008] Firstly, embodiments of this specification provide a fault diagnosis method for welding equipment based on acoustic signature features, including:

[0009] Acquire multi-channel raw acoustic signals received by a microphone array deployed on welding equipment.

[0010] A collaborative filtering method with adaptive adjustment of weight vectors is used to perform collaborative filtering on the original multi-channel acoustic signal to obtain a filtered multi-channel acoustic signal;

[0011] The feature vectors corresponding to different time scales and different frequency sub-bands in the filtered multi-channel acoustic signal are extracted and feature fusion processing is performed to obtain multi-scale fused features.

[0012] The attention mechanism is used to perform feature dimensionality reduction on multi-scale fusion features to obtain global voiceprint features;

[0013] The extended feature extraction model trained based on the BYOL model training framework extracts features from multi-channel raw acoustic signals to obtain extended voiceprint features.

[0014] Based on global and extended voiceprint features, equipment fault information is obtained.

[0015] As a preferred embodiment, the method of adaptively adjusting the weight vector to perform collaborative filtering on the multi-channel original acoustic signal to obtain a filtered multi-channel acoustic signal includes:

[0016] Obtain the initial weight vector and the ideal output signal;

[0017] Based on the current weight vector, the multi-channel raw acoustic signal is subjected to collaborative filtering to obtain the current output signal;

[0018] Based on the current output signal and the ideal output signal, obtain the current output error data;

[0019] Update the current weight vector based on the current output error data;

[0020] Repeat the above steps until the number of iterations reaches the preset number or the current output error data meets the first preset requirement, and then regard the current weight vector as the preferred weight vector;

[0021] The original multi-channel acoustic signal is processed by collaborative filtering based on the optimized weight vector to obtain the filtered multi-channel acoustic signal.

[0022] As a preferred embodiment, the step of extracting the feature vectors corresponding to different time scales and different frequency sub-bands in the filtered multi-channel acoustic signal and performing feature fusion processing to obtain multi-scale fused features includes:

[0023] Short-time Fourier transform was performed on the filtered multi-channel acoustic signal using different time window scales to obtain the time spectrum corresponding to each time window scale.

[0024] Amplitude operations are performed on the time-frequency spectra corresponding to different time window scales to obtain the time-frequency energy matrices corresponding to different time window scales.

[0025] For each time-frequency energy matrix, it is divided into a predetermined number of frequency sub-bands along the frequency dimension and expanded into feature vectors to obtain the feature vectors corresponding to different time window scales and different frequency sub-bands.

[0026] Feature fusion processing is performed on the feature vectors corresponding to different time window scales and different frequency subbands to obtain multi-scale fused features.

[0027] As a preferred embodiment, the method for obtaining the extended feature extraction model includes:

[0028] Acquire a dataset containing multiple data samples, each of which includes a corresponding multi-channel acoustic signal;

[0029] Select a data sample from the dataset and perform a short-time Fourier transform on the multi-channel acoustic signal corresponding to the selected data sample to obtain the complex amplitude values ​​of the multi-channel acoustic signal at different time and frequency dimensions.

[0030] Based on the complex amplitude values ​​corresponding to the multi-channel acoustic signals at different time and frequency dimensions, the voiceprint image is obtained.

[0031] Random frequency band masking and random time domain cropping are performed on the voiceprint images respectively to obtain the first enhanced time-frequency image and the second enhanced time-frequency image;

[0032] The first enhanced time-frequency plot is input into the current online network in the BYOL model training framework to obtain the first feature vector;

[0033] The second enhanced time-frequency plot is input into the current target network in the BYOL model training framework to obtain the second feature vector;

[0034] Based on the first feature vector and the second feature vector, the current loss data is obtained, and the parameters of the current online network in the BYOL model training framework are updated based on the current loss data.

[0035] Based on the latest parameters of the current online network in the BYOL model training framework and the exponential moving average method, the parameters of the current target network in the BYOL model training framework are updated.

[0036] Repeat the above steps until the number of iterations reaches the preset number or the current loss data meets the second preset requirement, and use the current online network as the extended feature extraction model.

[0037] As a preferred embodiment, the voiceprint image is subjected to random frequency band masking to obtain a first enhanced time-frequency map, including:

[0038] Randomly select a segment of frequency window in the frequency dimension;

[0039] The signal in the voiceprint image within the cropped frequency window is set to zero to obtain the first enhanced time-frequency map.

[0040] As a preferred embodiment, the voiceprint image is subjected to random time-domain cropping to obtain a second enhanced time-frequency map, including:

[0041] Randomly select a segment of time window in the time dimension;

[0042] Signals outside the capture time window of the voiceprint image are discarded to obtain the second enhanced time-frequency map.

[0043] As a preferred embodiment, the step of obtaining device fault information based on global voiceprint features and extended voiceprint features includes:

[0044] The coarse classification results of equipment faults are obtained based on global voiceprint features, extended voiceprint features, and a hybrid deep learning model that includes a convolutional neural network, a long short-term memory network, and a classifier connected in sequence.

[0045] When the coarse classification result of equipment faults does not meet the preset coarse classification requirements, the detailed fault types of the equipment are obtained based on global voiceprint features, extended voiceprint features, variational autoencoder model and Bayesian inference model.

[0046] As a preferred embodiment, the method for obtaining coarse classification results of device faults based on global voiceprint features, extended voiceprint features, and a hybrid deep learning model comprising a convolutional neural network, a long short-term memory network, and a classifier connected in sequence includes:

[0047] The global voiceprint features and extended voiceprint features are respectively input into the hybrid deep learning model to obtain the first coarse classification result of equipment faults and the second coarse classification result of equipment faults;

[0048] Based on the coarse classification results of the first and second equipment faults, the coarse classification results of equipment faults are obtained.

[0049] As a preferred embodiment, the step of obtaining the detailed fault types of the device based on global voiceprint features, extended voiceprint features, variational autoencoder model, and Bayesian inference model includes:

[0050] The global voiceprint features and extended voiceprint features are respectively input into the variational autoencoder model to obtain the latent variables corresponding to the global voiceprint features and extended voiceprint features, as well as the reconstructed samples corresponding to the global voiceprint features and extended voiceprint features.

[0051] Obtain the probability density of the global voiceprint features and the extended voiceprint features under the reconstructed sample, and then obtain the device status anomaly score;

[0052] When the equipment status abnormality score is greater than the preset threshold, the posterior probability of each sub-fault type is obtained based on the Bayesian inference model and the latent variables corresponding to the global voiceprint features and the extended voiceprint features.

[0053] The specific fault types of the equipment are determined based on the posterior probability of each sub-fault type.

[0054] Secondly, embodiments of this specification provide a fault diagnosis system for welding equipment based on voiceprint features, comprising:

[0055] The first acquisition module acquires the multi-channel raw acoustic signals received by the microphone array deployed on the welding equipment;

[0056] The filtering module uses a collaborative filtering method with adaptive adjustment of weight vectors to perform collaborative filtering on the multi-channel original acoustic signals to obtain filtered multi-channel acoustic signals.

[0057] The fusion module extracts the feature vectors corresponding to different time scales and frequency sub-bands in the filtered multi-channel acoustic signal, and performs feature fusion processing to obtain multi-scale fused features.

[0058] The dimensionality reduction module performs feature dimensionality reduction processing on multi-scale fused features based on the attention mechanism to obtain global voiceprint features;

[0059] The extended module uses an extended feature extraction model trained based on the BYOL model training framework to extract features from multi-channel raw acoustic signals to obtain extended voiceprint features.

[0060] The diagnostic module outputs device fault information based on global voiceprint features, extended voiceprint features, and a fault diagnosis model.

[0061] Thirdly, embodiments of this specification provide an electronic device, including a processor and a memory; the processor is connected to the memory; the memory is used to store executable program code; the processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to perform the steps described in the first aspect of the above embodiments.

[0062] Fourthly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps described in the first aspect of the above embodiments.

[0063] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0064] Global voiceprint features obtained by sequentially performing collaborative filtering, multi-scale feature fusion, and feature dimensionality reduction on multi-channel raw acoustic signals are combined with an extended feature extraction model trained based on the BYOL model training framework to extract features from the multi-channel raw acoustic signals, resulting in extended voiceprint features for equipment fault diagnosis. This improves the anti-interference capability of voiceprint features, thus laying the foundation for subsequent fault diagnosis of welding equipment based on voiceprint features. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 A schematic flowchart of a fault diagnosis method for welding equipment based on voiceprint features, according to some embodiments of the present disclosure, is shown.

[0067] Figure 2 A flowchart illustrating a collaborative filtering processing method according to some embodiments of this disclosure is shown.

[0068] Figure 3 A flowchart illustrating a multi-scale fusion method according to some embodiments of the present disclosure is shown.

[0069] Figure 4 The illustration shows a flowchart of some embodiments of the present disclosure for obtaining device fault information based on global voiceprint features and extended voiceprint features.

[0070] Figure 5 A flowchart illustrating the process of obtaining coarse classification results of device faults according to some embodiments of this disclosure is shown.

[0071] Figure 6 A flowchart illustrating the process of obtaining detailed fault types of a device according to some embodiments of this disclosure is shown.

[0072] Figure 7 A schematic diagram of the structure of a welding equipment fault diagnosis system based on voiceprint features, according to some embodiments of the present disclosure, is shown.

[0073] Figure 8 A schematic block diagram of an electronic device according to some embodiments of the present disclosure is shown. Detailed Implementation

[0074] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0075] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0076] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this specification. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0077] Figure 1 A flowchart illustrating a fault diagnosis method for welding equipment based on voiceprint features, according to some embodiments of this disclosure, is shown. It should be understood that the numbers in the flowchart do not indicate the order in which these steps are performed; some or all of these steps can be performed in parallel, or their order can be interchanged, and this disclosure does not limit this. Furthermore, Figure 1 The methods described may also include additional steps not shown and / or the steps shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0078] like Figure 1 As shown, diagnostic methods may include at least:

[0079] Step 102: Acquire the multi-channel raw acoustic signals received by the microphone array deployed on the welding equipment.

[0080] A voiceprint tag terminal is installed on the welding equipment. This voiceprint tag terminal integrates... A microphone array consisting of several high-sensitivity microphones, equipped with a local data processing and wireless communication module, is used to collect multi-channel acoustic signals generated by the welding equipment in real time during operation, and upload them to a central server for analysis and decision-making via wireless communication.

[0081] When deploying voiceprint tag terminals, appropriate installation locations should be selected based on the structure of the welding equipment and the characteristics of the workspace. This ensures the microphone array can fully cover the key sound sources emitted during welding operation, while minimizing the impact of environmental reflections and obstructions. To improve signal acquisition quality and stability, the microphone elements within the microphone array should be designed to be evenly and symmetrically arranged to achieve better spatial resolution and anti-interference capabilities.

[0082] At some point Voiceprint tag terminal through Multi-channel raw audio signal received by one microphone Represented as:

[0083] (1)

[0084] in, For the number of microphones, This indicates the conjugate transpose. Indicates the first The sound signal received by each microphone.

[0085] Step 104: Use a collaborative filtering method with adaptive weight vector adjustment to perform collaborative filtering on the multi-channel original acoustic signal to obtain a filtered multi-channel acoustic signal.

[0086] Step 106: Extract the feature vectors corresponding to different time scales and different frequency sub-bands in the filtered multi-channel acoustic signal, and perform feature fusion processing to obtain multi-scale fused features.

[0087] Step 108: Perform feature dimensionality reduction on the multi-scale fusion features based on the attention mechanism to obtain global voiceprint features.

[0088] Step 110: The extended feature extraction model trained based on the BYOL model training framework is used to extract features from the multi-channel original acoustic signal to obtain extended voiceprint features.

[0089] Step 112: Obtain device fault information based on global voiceprint features and extended voiceprint features.

[0090] Global voiceprint features obtained by sequentially performing collaborative filtering, multi-scale feature fusion, and feature dimensionality reduction on multi-channel raw acoustic signals are combined with an extended feature extraction model trained based on the BYOL model training framework to extract features from the multi-channel raw acoustic signals, resulting in extended voiceprint features for equipment fault diagnosis. This improves the anti-interference capability of voiceprint features, thus laying the foundation for subsequent fault diagnosis of welding equipment based on voiceprint features.

[0091] Figure 2 A flowchart illustrating a collaborative filtering processing method according to some embodiments of this disclosure is shown.

[0092] To ensure stable and high-fidelity signal data acquisition in the complex acoustic environment of welding operations, it is necessary to perform multi-channel collaborative preprocessing on the raw received signals. Due to the presence of high-intensity background noise, mechanical impact noise, and structural reflections in the welding environment, directly using the raw multi-channel acoustic signals for diagnosis would severely impact system performance. Therefore, this system introduces the Covariance Matrix Adaptive Filtering (CMAF) method, which dynamically and collaboratively processes the raw multi-channel acoustic signals based on the minimum mean square error criterion to improve the overall signal-to-noise ratio (SNR) and enhance the stability of subsequent feature extraction. The specific steps are as follows:

[0093] Step 202: Obtain the initial weight vector and the ideal output signal.

[0094] Step 204: Perform collaborative filtering on the multi-channel original acoustic signal based on the current weight vector to obtain the current output signal.

[0095] Step 206: Based on the current output signal and the ideal output signal, obtain the current output error data.

[0096] Step 208: Update the current weight vector based on the current output error data.

[0097] Step 210: Repeat the above corresponding steps (i.e., steps 204 to 208 above) until the number of iterations reaches the preset number or the current output error data meets the first preset requirement, and regard the current weight vector as the preferred weight vector.

[0098] Step 212: Perform collaborative filtering on the original multi-channel acoustic signal based on the optimized weight vector to obtain the filtered multi-channel acoustic signal.

[0099] It is understandable that in many embodiments of this specification, when obtaining multi-channel raw acoustic signals... Then, a weight vector is introduced. It is used to weight and combine multi-channel raw acoustic signals to enhance the signal and effectively suppress interference noise. The dimension is The column vector of the complex field, after filtering, is represented as:

[0100] (2)

[0101] in, for The output signal at time, for The weight vector at each time step; This is the complex conjugate transpose.

[0102] To minimize the difference between the output signal and the ideal output signal The error between them is introduced into the objective cost function using the Minimum Mean Square Error (MMSE) criterion:

[0103] (3)

[0104] in, This represents the mathematical expectation, specifically minimizing the mean square error between the output signal and the ideal output signal. (Note: For simplicity, the entire process of calculating the mean square error is not presented as a formula here, but rather expressed through...) To represent, where express To calculate the mean squared error (MSE), we first need to calculate the variance at each time step, and then take the mean. During the training phase... Estimated from pre-collected clean samples or reference channels.

[0105] To minimize the objective cost function, the weight vector update is derived using the standard gradient descent method. The weight adjustment formula is as follows:

[0106] (4)

[0107] in, The learning rate controls the speed and stability of weight vector updates; To find the derivative of the gradient with respect to the objective cost function, This represents the updated weight vector.

[0108] It can converge after several iterations to obtain the optimal weight vector. Furthermore, the final filtered multi-channel acoustic signal for:

[0109] (5)

[0110] Through the above adaptive collaborative preprocessing method, the processed acoustic signal has a significantly higher signal-to-noise ratio (SNR) than the original signal, effectively reducing the impact of background noise and environmental interference on subsequent diagnostic tasks, making the acoustic signal analysis of the system more accurate, stable and reliable in complex industrial environments.

[0111] Figure 3 A flowchart illustrating a multi-scale fusion method according to some embodiments of the present disclosure is shown.

[0112] After deploying the voiceprint tag terminal and performing adaptive covariance matrix filtering preprocessing on the multi-channel raw acoustic signal, the system obtained a filtered multi-channel acoustic signal with enhanced signal-to-noise ratio. To further improve the expressive power of voiceprint features and the ability to identify complex fault modes, a multi-scale feature fusion mechanism was introduced. This mechanism aims to extract high-dimensional voiceprint features across multiple scales and frequency bands from the signal and focus on key features through an attention mechanism, thereby achieving dimensionality reduction and enhanced representation capabilities. The specific implementation steps are as follows:

[0113] Step 302: Perform short-time Fourier transform on the filtered multi-channel acoustic signal using different time window scales to obtain the time spectrum corresponding to each time window scale.

[0114] respectively Short-Time Fourier Transform (STFT) was performed on the filtered multi-channel acoustic signal at different time window scales, and the time-frequency spectrum at different time window scales was calculated:

[0115] (6)

[0116] in, Indicates the first Time spectrum at a time window scale This represents the filtered output signal obtained after CMAF preprocessing. For the first The time window function corresponding to each time window scale.

[0117] Step 304: Perform amplitude operations on the time spectrum corresponding to different time window scales to obtain the time-frequency energy matrix corresponding to different time window scales.

[0118] right The amplitude operation yields the time-frequency energy matrix. :

[0119] (7)

[0120] in, Indicates the number of time frames. For frequency dimension, The dimension is A vector in the real number field.

[0121] Step 306: For each time-frequency energy matrix, divide it into a preset number of frequency sub-bands along the frequency dimension and expand them into feature vectors to obtain the feature vectors corresponding to each of the different time window scales and different frequency sub-bands.

[0122] Time-frequency energy matrix Divided along the frequency dimension Each frequency sub-band is expanded into a feature vector. :

[0123] (8)

[0124] in, For the first The time window scale, the first The feature vectors corresponding to each frequency sub-band Indicates the flattening operation. , This is the start and end index for the frequency subband. Indicates the first Time-frequency energy matrix at a time window scale Perform subband slicing along the frequency dimension; the symbol : indicates that all time frames are selected. Represents the first division along the frequency dimension The range of frequency sub-bands, i.e., from the index arrive The list. Represents the expanded eigenvectors It is a real vector with dimension . , Indicates the first The time window scale, the first The vector dimension after flattening each frequency subband.

[0125] Step 308: Perform feature fusion processing on the feature vectors corresponding to multiple time window scales and different frequency sub-bands to obtain multi-scale fused features. .

[0126] (9)

[0127] in, The total number of time window scales. The total number of frequency sub-bands represents the total number of frequency sub-bands. The concatenated feature vector contains information across multiple scales and frequency bands, and has high expressive power. This indicates that the multi-scale fused feature obtained after concatenation is a real-valued vector with dimension 1. This dimension is the result of concatenating the feature vectors corresponding to multiple different time window scales and different frequency subbands. .

[0128] In some embodiments of this specification, the step of performing feature dimensionality reduction processing on multi-scale fusion features based on an attention mechanism to obtain global voiceprint features includes the following steps:

[0129] Since the concatenated multi-scale fused features may contain redundant information, an attention mechanism is introduced for feature enhancement and dimensionality reduction. Let the multi-scale fused feature matrix be... for:

[0130] (10)

[0131] in, for A matrix consisting of feature blocks, where each row is a scale-subband feature vector. The vector dimension for each feature block must be uniform. Represented as a matrix over the real number field, with dimension . OK, List.

[0132] Define the attention weight vector:

[0133] (11)

[0134] in, The weight coefficients for each feature block; Calculate the weight vector for attention; For bias terms, Ensure weight normalization. Represents the real number field A dimensional column vector space.

[0135] Finally, the dimensionality-reduced fusion features are obtained by weighted summation:

[0136] (12)

[0137] in, It is a global voiceprint feature representation after weighted dimensionality reduction using an attention mechanism; It is the first The attention coefficient of each feature block represents the importance of that feature block to the global representation; It is the first Eigenvectors at each scale and sub-band.

[0138] Through the above steps, the multi-scale feature fusion mechanism makes full use of the rich features of different time scales and frequency sub-bands, and automatically focuses on the key regions most relevant to the current state through the attention mechanism, thereby obtaining voiceprint features with reasonable dimensions and rich semantics, laying a solid foundation for subsequent equipment fault diagnosis.

[0139] Given that acoustic fingerprint data from welding equipment is typically large in scale but lacks effective annotation, traditional supervised learning methods are difficult to apply. Therefore, this paper introduces a self-supervised feature extraction method that does not require a large number of negative samples, namely BYOL (Bootstrap Your Own Latent). Compared to methods such as SimCLR, which rely on a large number of negative samples, BYOL can achieve stable and scalable feature extraction without contrast loss, making it particularly suitable for the reality of "abundant positive samples and scarce negative samples" in the welding industry. This method effectively avoids reliance on manual labels and maintains good generalization ability under different equipment and operating conditions. Understandably, the final method uses an online network within the BYOL model training framework for extended acoustic fingerprint feature extraction. The acquisition method of the extended feature extraction model includes:

[0140] Step 1102: Obtain a dataset containing multiple data samples, each dataset containing a corresponding multi-channel acoustic signal.

[0141] Step 1104: Select a data sample from the dataset and perform a short-time Fourier transform on the multi-channel acoustic signal corresponding to the selected data sample to obtain the complex amplitude values ​​of the multi-channel acoustic signal at different time and frequency dimensions.

[0142] The multi-channel acoustic signal is transformed using short-time Fourier transform. Convert it to its corresponding time-frequency representation to obtain the local characteristics of the signal in the time and frequency dimensions:

[0143] (13)

[0144] in, Representing time in the time-frequency graph ,frequency The corresponding complex spectral values, Indicates time Multi-channel acoustic signal samples at different times. This is a time window function used for local signal extraction.

[0145] Step 1106: Obtain the voiceprint image based on the complex amplitude values ​​corresponding to the multi-channel acoustic signals in different time and frequency dimensions.

[0146] (14)

[0147] in, This indicates the number of time frames, which is the number of time windows in which the signal is divided. Frequency resolution, i.e., the number of frequency domain sampling points in each frame. This represents a matrix over the real number field with dimensions . OK, Column. This two-dimensional matrix. It can be viewed as a "grayscale voiceprint image", that is, with frequency as the vertical axis, time as the horizontal axis, and energy intensity as the grayscale value.

[0148] Step 1108: Perform random frequency band masking and random time domain cropping on the voiceprint image respectively to obtain the first enhanced time-frequency image and the second enhanced time-frequency image.

[0149] Step 1110: Input the first enhanced time-frequency map into the current online network in the BYOL model training framework to obtain the first feature vector.

[0150] Step 1112: Input the second enhanced time-frequency map into the current target network in the BYOL model training framework to obtain the second feature vector.

[0151] Understandable.

[0152] The BYOL model has two network branches: 1. Online Network: Parameters are... The first type of network generates feature representations of the input through forward propagation; the second type is the target network, which has the same structure as the online network, but its parameter updates are relatively lagging, using exponential moving average (EMA).

[0153] Voiceprint images Two different data augmentation operations, random frequency band masking and random temporal pruning, are performed to obtain two different but semantically consistent views. , The details are as follows:

[0154] The random frequency band masking process is as follows:

[0155] A frequency window is randomly selected along the frequency dimension, and the signal in the speaker image within this window is further set to zero to obtain the first enhanced time-frequency map. The starting point of the selected frequency window is... , length is The enhanced first time-frequency plot is obtained:

[0156] (15)

[0157] in, , , and These are the minimum and maximum values ​​of the shielding bandwidth, respectively.

[0158] Random temporal clipping processing:

[0159] A random time window is selected in the time dimension, and signals outside the selected time window from the voiceprint image are discarded to obtain the second enhanced time-frequency map. The starting point of the selected time window is... , length is The enhanced second time-frequency plot is obtained:

[0160] (16)

[0161] in, , , and These represent the minimum and maximum values ​​of the clipping window length, respectively.

[0162] Through the two data enhancement methods mentioned above, the voiceprint image Enhanced into two views respectively , It can effectively help the BYOL model extract stable voiceprint features that are more robust and insensitive to frequency masking and time segment loss.

[0163] Furthermore, it can be understood that the two views obtained through two different data augmentation methods... , The inputs are fed into two separate networks, and features are extracted using the online network and the target network respectively.

[0164] (17)

[0165] (18)

[0166] in, This refers to the encoder network of the online network. The encoder network representing the target network is represented by a ResNet structure to extract semantic features from the time-frequency graph. This represents an additional non-linear projection head that further maps the features output by the encoder to a lower-dimensional space to improve the model's generalization ability. Represents the first eigenvector. This represents the second eigenvector.

[0167] Step 1114: Based on the first feature vector and the second feature vector, obtain the current loss data, and update the parameters of the current online network in the BYOL model training framework based on the current loss data.

[0168] Step 1116: Based on the latest parameters of the current online network in the BYOL model training framework and the exponential moving average method, update the parameters of the current target network in the BYOL model training framework.

[0169] Step 1118: Repeat the above steps until the number of iterations reaches the preset number or the current loss data meets the second preset requirement, and use the current online network as the extended feature extraction model.

[0170] BYOL employs a non-contrast loss mechanism, learning noise-invariant and scene-robust feature representations by minimizing the mean squared error (MSE) loss between the features of two views. Specifically, it learns feature representations with no contrast and no loss by minimizing the mean squared error (MSE) loss between the features of two views. With the second eigenvector Then, the MSE loss between them is calculated to guide feature alignment learning:

[0171] (19)

[0172] in, This represents the loss function of the BYOL model. Denotes the square norm 2. express Feature vector, The length of the feature vector, This represents the gradient truncation operation, i.e., the target network output. Used only as a training reference and not involved in backpropagation, the online network gradually learns the representation of the target network by minimizing this loss, thereby obtaining a stable and highly generalizable voiceprint feature representation.

[0173] Online network parameter updates:

[0174] All parameters of the online network are obtained through... Update the algorithm using backpropagation and gradient descent, specifically:

[0175] (20)

[0176] in, For learning rate, Indicates the parameters of the online network. gradient, This indicates the updated online network parameters.

[0177] Target network parameter update: To maintain the stability of feature alignment targets, target network parameters are updated. Instead of optimizing through loss backpropagation, updated online network parameters are used. Update using the Exponential Moving Average (EMA):

[0178] (twenty one)

[0179] in, Indicates the EMA smoothing factor. The parameters representing the target network, This represents the parameters of the updated target network.

[0180] Through the above joint mechanism, a stable learning signal is constructed using two semantically consistent but structurally perturbation-different views in the absence of negative samples. This allows the online network to gradually approach the feature representation capability of the target network. At the same time, the target network evolves smoothly through the EMA mechanism, providing a dynamically consistent representation target for the training process. This enables robust extraction of welding acoustic features under different equipment and working conditions.

[0181] The above method requires no manual labeling, making it suitable for the practical problem of difficult data labeling in welding equipment. The time-frequency graph learns invariant and robust representations through data augmentation (band masking, time-domain clipping), automatically learning salient voiceprint patterns in complex background noise, thus enhancing the ability to identify fault precursors. Furthermore, the trained online network can be directly used for voiceprint feature extraction tasks on different devices or under different operating modes.

[0182] Figure 4 The illustration shows a flowchart of some embodiments of the present disclosure for obtaining device fault information based on global voiceprint features and extended voiceprint features.

[0183] To achieve high-precision fault detection of welding equipment under complex operating conditions, this invention employs a hybrid deep learning model that integrates Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). Based on the machine acoustic signature features extracted in the previous step, it identifies and diagnoses potential anomalies. This method fully integrates acoustic spatial structure features with the temporal evolution features of the equipment state, overcoming the shortcomings of traditional methods in terms of sensitivity to complex background interference and insufficient response to temporal fluctuations, thus improving the accuracy and robustness of fault identification.

[0184] Furthermore, in the actual operating environment of welding equipment, potential abnormal events may occur in unstructured, low-frequency, and sudden forms, making it difficult for traditional rule-based or threshold-based discrimination models to cover all abnormal patterns. This invention also introduces an anomaly classification mechanism that combines a variational autoencoder (VAE) with Bayesian inference. This method utilizes the probability distribution information of a large amount of normal voiceprint data to establish a generative model of normal operating modes, thereby achieving highly sensitive detection and fine classification of abnormal events.

[0185] In some embodiments of this specification, obtaining device fault information based on global voiceprint features and extended voiceprint features includes:

[0186] Step 402: Obtain coarse classification results of equipment faults based on global voiceprint features, extended voiceprint features, and a hybrid deep learning model containing sequentially connected convolutional neural networks, long short-term memory networks, and classifiers;

[0187] Step 404: When the coarse classification result of the equipment fault does not meet the preset coarse classification requirements, the detailed fault type of the equipment is obtained based on global voiceprint features, extended voiceprint features, variational autoencoder model and Bayesian inference model.

[0188] The approach first uses a hybrid deep learning model to coarsely classify equipment faults. Only when the coarse classification result does not meet the preset coarse classification requirements is the fault further refined based on an anomaly classification mechanism that combines variational autoencoders and Bayesian inference. This not only improves the accuracy, robustness, and high sensitivity of fault identification to anomaly detection, but also reduces computational load to some extent.

[0189] Among them, for the hybrid deep learning model, the goal of its convolutional neural network is to extract spatial local features such as welding arc pulses, periodic mechanical impacts, and high-frequency interference signals in the spectrum.

[0190] CNN No. In the layer, the first The activation value of each channel output can be represented as:

[0191] (twenty two)

[0192] in, Indicates CNN in the Layer, First Spatial location of each channel The activation value on and This represents the offset of the convolution kernel in the spatial dimension, i.e., the row and column indices in the two-dimensional spatial window, respectively. For the first The weights of the convolutional kernels in the CNN are... In the layer, it is used to connect the input channel. To output channel The convolution kernel is located in The weighting parameters of the location. This represents the pixel value of the input feature map, i.e., the CNN's [pixel value]. The first in the layer Each channel, in spatial location The eigenvalues ​​on. For the corresponding bias term, it is the CNN's first... The first in the layer The scalar bias value corresponding to each output channel is added to the convolution result after the convolution operation is completed, further adjusting the output level of the activation value. The activation function is used to introduce nonlinearity. Through layer-by-layer convolutional operations in a CNN, key spatial features are extracted from the acoustic signature, highlighting fault characteristic patterns, such as abnormal discharge from a welding torch or vibration characteristics of mechanical components.

[0193] Specifically, for the hybrid deep learning model, its recurrent neural network module, in order to further characterize the dynamic evolution characteristics of voiceprint over time during device operation, outputs the spatial feature map from the last layer of the CNN. As time series input, it is fed into a Long Short-Term Memory (LSTM) network to learn long-term temporal dependencies. The specific representations of each section are as follows:

[0194] Input Gate:

[0195] (twenty three)

[0196] Forget Gate:

[0197] (twenty four)

[0198] Cell Update:

[0199] (25)

[0200] Output Gate:

[0201] (26)

[0202] Cell State:

[0203] (27)

[0204] Hidden State:

[0205] (28)

[0206] in, express The hidden state at any given moment express The hidden state at any given moment Indicates the input gate. Represents the Gate of Oblivion This indicates that the memory unit has been updated. Indicates the output gate. express The state of the memory unit at any given moment. express The state of the memory unit at any given moment. This represents the sigmoid activation function. This represents the hyperbolic tangent activation function. This represents Hadamard element-wise multiplication. , , This represents the input weights, recursive weights, and bias terms for each gate. LSTM modeling can effectively capture long-term temporal dependencies such as welding arc extinction and restart cycles, abnormal fluctuations in wire feed motor operation, and power supply instability.

[0207] The last hidden state of the LSTM module As the final feature input for fault classification, the device status is determined through a fully connected classifier (Softmax layer):

[0208] (29)

[0209] in, This indicates the coarse classification results of equipment failures. , Represents the real number field 3D vector space, The number of coarse categories for equipment failure (e.g., normal, minor abnormality, serious failure). , These are the weight matrix and bias vector of the classifier, respectively. Understandably, the final classification result can be directly used for real-time equipment status monitoring and fault alarms, or it can be used for the preliminary coarse classification step as described in the embodiments of this specification.

[0210] Figure 5 A flowchart illustrating the process of obtaining coarse classification results of device faults according to some embodiments of this disclosure is shown.

[0211] The coarse classification result of equipment faults obtained by the hybrid deep learning model based on global voiceprint features, extended voiceprint features, and a classifier consisting of sequentially connected convolutional neural networks, long short-term memory networks, and classifiers includes:

[0212] Step 502: Input the global voiceprint features and extended voiceprint features into the hybrid deep learning model to obtain the first coarse classification result of equipment faults and the second coarse classification result of equipment faults.

[0213] Step 504: Based on the first coarse classification sub-result of equipment faults and the second coarse classification sub-result of equipment faults, obtain the coarse classification result of equipment faults.

[0214] Understandably, since this application extracts global voiceprint features and extended voiceprint features during the aforementioned voiceprint feature extraction, the global voiceprint features and extended voiceprint features can be input into the hybrid deep learning model to obtain the first coarse classification result of equipment fault and the second coarse classification result of equipment fault. If only one of them has a coarse classification result of slight abnormality or serious fault, the coarse classification result of equipment fault is determined to be a fault result.

[0215] Furthermore, the specific implementation process for obtaining the detailed fault types of the device based on global voiceprint features, extended voiceprint features, variational autoencoder model, and Bayesian inference model is as follows:

[0216] A variational autoencoder model is trained using a large amount of normal voiceprint data to learn its latent spatial distribution. Let the input be the voiceprint time-frequency feature vector. , Represents the real number field A dimensional vector space, where This represents the total dimension of the voiceprint feature vector. A variational autoencoder model is constructed to learn the intrinsic generation mechanism of normal voiceprint data. The variational autoencoder model mainly consists of two parts: an encoder and a decoder.

[0217] encoder : Input voiceprint features Encoding as latent variables The distribution is as follows:

[0218] (30)

[0219] in, For encoder parameters, , Let these be the mean vector and variance vector of the latent distribution output by the encoder, respectively. Indicates a normal distribution. This represents the identity matrix, used to construct a diagonal covariance structure where each dimension is independent and has consistent variance.

[0220] decoder From latent variables Reconstructing the voiceprint features of the input Record reconstructed samples The model is as follows:

[0221] (31)

[0222] in, For decoder parameters, The decoder is represented by parameters. latent variables The reconstruction function maps the echo signature feature space, and the output is the reconstructed sample. , The variance parameter represents the reconstructed distribution and is used to control the degree of fluctuation between the reconstructed samples and the input samples.

[0223] Model optimization is performed by maximizing the Evidence Lower Bound (ELBO), specifically defined as:

[0224] (32)

[0225] in, This represents the input voiceprint features. Representing latent variables, express The prior distribution is a standard Gaussian distribution, which provides a regularization constraint for the latent space of VAE. The Kullback-Leibler divergence is used to constrain the encoder distribution to approximate the prior distribution. This optimization objective trades off data reconstruction accuracy from latent space regularization, enabling the model to capture the backbone structure of the voiceprint distribution rather than noise details.

[0226] After the model training is completed, the acoustic signature signals of the newly observed welding equipment will be analyzed. Anomaly detection is performed. The posterior distribution of the latent variables is extracted using the encoder.

[0227] (33)

[0228] Reconstructed samples are generated using the decoder. And calculate the original input. The probability density under this reconstructed sample is used as an anomaly score:

[0229] (34)

[0230] The score reflects the degree of fit of the newly observed acoustic signature of the welding equipment to the normal mode. Samples that are significantly lower than the reconstruction score threshold of the normal training samples can be considered as abnormal events (i.e., fault states).

[0231] Furthermore, to achieve fine-grained classification of detected abnormal events, this method introduces a Bayesian inference mechanism, utilizing latent variables. The spatial distribution characteristics are used for abnormal pattern clustering. The specific implementation process is as follows:

[0232] First, construct a probabilistic model for anomaly pattern classification, such as a Gaussian Mixture Model (GMM):

[0233] (35)

[0234] in, For the first Fault modes; , The first Mean and covariance of Gaussian distribution corresponding to anomaly patterns; The total number of predefined failure modes.

[0235] Then, the posterior probability of each failure mode is calculated according to Bayes' theorem:

[0236] (36)

[0237] in, It is the first One fault mode The prior probability represents the likelihood of the failure mode occurring before any observed data is available.

[0238] Ultimately, the classification result is obtained through Maximum A Posteriori (MAP) decision-making:

[0239] (37)

[0240] in, The predicted failure mode category is used to achieve a fine classification of abnormal events (such as welding torch jamming, power instability, contact tip wear, wire feed motor abnormality, etc.) by maximizing the posterior probability.

[0241] therefore, Figure 6 A flowchart illustrating the process of obtaining detailed fault types of a device according to some embodiments of this disclosure is shown.

[0242] The method for obtaining detailed fault types of the device based on global voiceprint features, extended voiceprint features, variational autoencoder model, and Bayesian inference model includes:

[0243] Step 602: Input the global voiceprint features and extended voiceprint features into the variational autoencoder model respectively to obtain the latent variables corresponding to the global voiceprint features and extended voiceprint features, as well as the reconstructed samples corresponding to the global voiceprint features and extended voiceprint features.

[0244] Step 604: Obtain the probability density of the global voiceprint features and the extended voiceprint features under the reconstructed samples, and then obtain the device status anomaly score;

[0245] Step 606: When the equipment status abnormality score is greater than the preset threshold, the posterior probability of each subdivided fault type is obtained based on the Bayesian inference model and the latent variables corresponding to the global voiceprint features and the extended voiceprint features.

[0246] Step 608: Determine the subdivided fault types of the equipment based on the posterior probability of each subdivided fault type.

[0247] As can be understood, in the embodiments of this specification, a coarse classification stage is first performed before determining the detailed fault types of the equipment, followed by an equipment status anomaly scoring stage. Only when neither of these two stages meets the preset requirements is the detailed fault type determination stage performed. Calculating the posterior probability corresponding to each fault mode according to Bayes' theorem involves a large amount of data computation. Therefore, by first performing a preliminary judgment through the coarse classification stage and the equipment status anomaly scoring stage, the drawback of the large amount of data computation caused by directly performing the detailed fault type determination stage is avoided.

[0248] Furthermore, it is understandable that since this application extracts both global and extended voiceprint features during the aforementioned voiceprint feature extraction, a first-state anomaly score and a second-state anomaly score can be obtained based on the global and extended voiceprint features respectively. The anomaly score with the more obvious anomaly can then be used as the device state anomaly score. Further, a Bayesian inference model can be used to obtain the posterior probabilities of each sub-fault type corresponding to the global and extended voiceprint features for their respective latent variables. Finally, based on the posterior probabilities of all sub-fault types (understandably, there should be two posterior probabilities for the same sub-fault type), the device's sub-fault type is determined.

[0249] In industrial environments where welding equipment operates for extended periods, voiceprint data exhibits distribution drift over time. For example, factors such as equipment aging, environmental changes (e.g., temperature and humidity fluctuations, power supply variations), and material variations alter the statistical properties of voiceprint features. This change leads to a gradual degradation of the original model's performance, impacting the accuracy of fault diagnosis. To address this issue, this invention proposes an incremental online learning mechanism to adaptively update the model and improve its long-term stability. This mechanism employs Exponential Moving Average (EMA) to update model parameters, dynamically integrating new and old data to optimize the model in a low-intervention, controllable manner.

[0250] Online learning objectives:

[0251] During the online phase, whenever new voiceprint data is collected... The system will obtain a set of updated model parameters for the current batch through rapid fine-tuning. To avoid rapidly forgetting historical knowledge, a slow update strategy is adopted, that is, smoothly integrating old and new parameters. The specific update formula is as follows:

[0252] (38)

[0253] in, For the first Historical model parameters at the next iteration; Estimate the new parameters obtained during training within the current sampling window; The learning rate controls the strength of the impact of new data on model updates; This is the model parameter after integrating the old and new weights. This method effectively balances the adaptability of new data with the preservation of historical knowledge, avoiding "catastrophic forgetting."

[0254] Embedded parameter update method:

[0255] To avoid placing an excessive computational burden on the device due to large-scale parameter updates, online learning employs embedded local parameter updates, progressively updating only the key parts of the model. This includes updating only the encoder parameters in critical sub-modules such as the VAE. Decoder parameters The mean of Bayesian GMM Covariance Perform a gradual update. The specific formula is as follows:

[0256] Encoder parameter update:

[0257] (39)

[0258] Decoder parameter update:

[0259] (40)

[0260] in, Indicates the first This update uses fine-tuned estimates of the VAE encoder parameters. Indicates the first This update uses fine-tuned estimates of the VAE decoder parameters. Indicates the first This update uses fine-tuned estimates of the VAE encoder parameters. Indicates the first The next update fine-tunes the obtained VAE decoder parameter estimates.

[0261] Update the mean and covariance for each class of outlier patterns in the GMM:

[0262] (41)

[0263] (42)

[0264] in, This indicates the number calculated based on the latest sample. The mean vector estimate of the anomaly pattern in the latent space. For the first The covariance matrix estimates of anomaly patterns are obtained using Bayesian posterior updates on a small sample size. These estimates are then used to dynamically adjust the distribution location and shape of various anomaly patterns in the GMM. , All are update rates. Indicates the updated (number) During the nth iteration) Mean vector of anomaly patterns Indicates the updated (number) During the nth iteration) Covariance matrix of anomaly patterns Indicates before the update (the first) During the nth iteration) Mean vector of anomaly patterns Indicates before the update (the first) During the nth iteration) Covariance matrix of anomaly patterns.

[0265] Determine the update trigger mechanism:

[0266] To avoid frequent update operations that could affect system stability, this invention introduces an update triggering mechanism. The system will only trigger an online learning update when any of the following conditions are met:

[0267] Reconstruction error continues to exceed the threshold:

[0268] If in the current time period Internal, original voiceprint characteristics With reconstruction features The average value of the squared norm error is greater than the set threshold. If the model is no longer able to accurately fit the data, an update is triggered.

[0269] (43)

[0270] in, Specifically, this means calculating the time period. Internal original voiceprint features With reconstruction features The average value of the L2 squared error (Note: For simplicity, the entire process of calculating the average value of the L2 squared error is not presented as a formula here, but rather through...) To represent, where Indicates time period Find the L2 square error at any given time point within the time interval. Internal original voiceprint features With reconstruction features The average value of the L2 squared error requires calculating the L2 squared error at each time step.

[0271] The downward trend in abnormal scores is obvious:

[0272] recent Among the anomaly detection samples, the average anomaly score is lower than the set threshold. :

[0273] (44)

[0274] Time interval trigger (periodic update): The model is automatically refreshed every fixed time interval.

[0275] Small batch update strategy

[0276] Each update uses only the most recently collected small batch of data, such as training every 64 data points, to reduce resource consumption, and a maximum parameter drift constraint is set. To prevent the model from mutating or diverging, the specific update strategy is as follows:

[0277] (45)

[0278] in, Describes the 2-norm. This represents the parameters before the model update. This represents the parameters after the model update.

[0279] Through the above operations, a slow online learning mechanism suitable for long-term use in the welding industry is introduced, offering the following advantages: exponential moving average suppresses sudden disturbances; continuous integration of new data distributions alleviates data drift caused by equipment aging; local parameter updates result in low power consumption and embedded operation; criterion-driven updates improve system security and control, avoiding meaningless updates. This mechanism can be seamlessly integrated into the overall system architecture of this invention, providing a powerful long-term self-evolution capability for the voiceprint-driven welding equipment condition diagnosis system.

[0280] Figure 7 A schematic diagram of a welding equipment fault diagnosis system 700 based on voiceprint features according to some embodiments of this disclosure is shown. The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments.

[0281] like Figure 7 As shown, a diagnostic system may include at least:

[0282] The first acquisition module 701 acquires the multi-channel raw acoustic signals received by the microphone array deployed on the welding equipment;

[0283] The filtering module 702 uses a collaborative filtering method with adaptive adjustment of weight vectors to perform collaborative filtering on the multi-channel original acoustic signals to obtain filtered multi-channel acoustic signals.

[0284] The fusion module 703 extracts the feature vectors corresponding to different time scales and different frequency sub-bands in the filtered multi-channel acoustic signal, and performs feature fusion processing to obtain multi-scale fused features.

[0285] Dimensionality reduction module 704 performs feature dimensionality reduction on multi-scale fused features based on an attention mechanism to obtain global voiceprint features;

[0286] Extension module 705 extracts features from multi-channel raw acoustic signals using an extended feature extraction model trained based on the BYOL model training framework to obtain extended voiceprint features.

[0287] The diagnostic module 706 outputs equipment fault information based on global voiceprint features, extended voiceprint features, and a fault diagnosis model.

[0288] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0289] Figure 8 A block diagram of an electronic device 800 that can implement various embodiments of the present disclosure is shown. For example... Figure 8 As shown, the electronic device 800 includes a processor 810, a disk drive 820, an input / output interface 830, a network interface 840, and a memory 850. The processor 810, disk drive 820, input / output interface 830, network interface 840, and memory 850 can communicate with each other via a communication bus 860.

[0290] The processor 810 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0291] The memory 850 can be implemented in the form of ROM (Read Only Memory), RAM (Read Access Memory), static memory, dynamic storage devices, etc. The memory 850 can store the operating system 851 for controlling the operation of the electronic device 800, and the basic input / output system (BIOS) 852 for controlling the low-level operations of the electronic device 800. Additionally, it can store a web browser 853, a data storage management system 854, etc. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 850 and is called and executed by the processor 810.

[0292] The input / output interface 830 is used to connect input / output modules to enable information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0293] The network interface 840 is used to connect the communication module (not shown in the figure) to enable communication and interaction between the device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0294] Bus 860 includes a pathway for transmitting information between various components of the device, such as processor 810, disk drive 820, input / input interface 830, network interface 840, and memory 850.

[0295] It should be noted that although the above-described device only shows the processor 810, disk drive 820, input / output interface 830, network interface 840, memory 850, bus 860, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the method of this application, and does not necessarily include all the components shown in the figures.

[0296] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0297] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0298] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A voiceprint feature-based electric welding equipment fault diagnosis method, characterized in that, The method comprises the following steps: acquiring a multi-channel original sound signal received by a microphone array arranged on a welding device; performing collaborative filtering processing on the multi-channel original sound signal by using a weight vector adaptive adjustment collaborative filtering method to obtain a filtered multi-channel sound signal; extracting feature vectors corresponding to different time scales and different frequency subbands in the filtered multi-channel sound signal, and performing feature fusion processing to obtain multi-scale fusion features; performing feature dimension reduction processing on the multi-scale fusion features based on an attention mechanism to obtain global voiceprint features; extracting features from the multi-channel original sound signal by using an extended feature extraction model trained based on a BYOL model training framework to obtain extended voiceprint features; acquiring device fault information based on the global voiceprint features and the extended voiceprint features.

2. The voiceprint feature-based electric welding equipment fault diagnosis method according to claim 1, characterized in that, The method of performing collaborative filtering processing on the multi-channel original sound signal by using the weight vector adaptive adjustment collaborative filtering method to obtain the filtered multi-channel sound signal comprises the following steps: acquiring an initial weight vector and an ideal output signal; performing collaborative filtering processing on the multi-channel original sound signal based on a current weight vector to obtain a current output signal; acquiring current output error data based on the current output signal and the ideal output signal; updating the current weight vector based on the current output error data; repeating the above steps until the number of iterations reaches a preset number or the current output error data meets a first preset requirement, and regarding the current weight vector as an optimal weight vector; performing collaborative filtering processing on the multi-channel original sound signal based on the optimal weight vector to obtain the filtered multi-channel sound signal. 3.The arc welding equipment fault diagnosis method based on voiceprint features of claim 1, characterized in that, The method of extracting feature vectors corresponding to different time scales and different frequency subbands in the filtered multi-channel sound signal and performing feature fusion processing to obtain multi-scale fusion features comprises the following steps: performing short-time Fourier transform processing on the filtered multi-channel sound signal by using different time window scales to obtain time-frequency spectra corresponding to the different time window scales, respectively; performing amplitude value operation on the time-frequency spectra corresponding to the different time window scales to obtain time-frequency energy matrices corresponding to the different time window scales, respectively; for each time-frequency energy matrix, dividing it into a preset number of frequency subbands along the frequency dimension and expanding it into a feature vector to obtain feature vectors corresponding to different time window scales and different frequency subbands, respectively; performing feature fusion processing on the feature vectors corresponding to different time window scales and different frequency subbands to obtain multi-scale fusion features.

4. The voiceprint feature-based electric welding equipment fault diagnosis method according to claim 1, characterized in that, The method of obtaining the extended feature extraction model comprises the following steps: acquiring a data set comprising a plurality of data samples, each data sample comprising corresponding multi-channel sound signals; selecting a data sample from the data set, and performing short-time Fourier transform on the multi-channel sound signals corresponding to the selected data sample to obtain complex amplitudes corresponding to the multi-channel sound signals in different time dimensions and different frequency dimensions, respectively; acquiring a voiceprint image based on the complex amplitudes corresponding to the multi-channel sound signals in different time dimensions and different frequency dimensions, respectively; performing random frequency band masking processing and random time domain clipping processing on the voiceprint image to obtain a first enhanced time-frequency image and a second enhanced time-frequency image, respectively; inputting the first enhanced time-frequency graph into a current online network in a BYOL model training framework to obtain a first feature vector; inputting the second enhanced time-frequency graph into a current target network in the BYOL model training framework to obtain a second feature vector; obtaining current loss data based on the first feature vector and the second feature vector, and performing parameter updating on the current online network in the BYOL model training framework based on the current loss data; performing parameter updating on the current target network in the BYOL model training framework based on the latest parameters of the current online network in the BYOL model training framework and an exponential moving average method; repeating the above corresponding steps until the number of iterations reaches a preset number or the current loss data meets a second preset requirement, and taking the current online network as an extended feature extraction model.

5. The voiceprint feature-based electric welding equipment fault diagnosis method according to claim 4, characterized in that, performing random band masking processing on the voiceprint image to obtain the first enhanced time-frequency graph, including: randomly intercepting an intercepted frequency window in the frequency dimension; zeroing the signal of the voiceprint image within the intercepted frequency window to obtain the first enhanced time-frequency graph.

6. The voiceprint feature-based electric welding equipment fault diagnosis method according to claim 4, characterized in that, performing random time domain clipping processing on the voiceprint image to obtain the second enhanced time-frequency graph, including: randomly intercepting an intercepted time window in the time dimension; discarding the signal of the voiceprint image outside the intercepted time window to obtain the second enhanced time-frequency graph.

7. The voiceprint feature-based electric welding equipment fault diagnosis method according to claim 1, characterized in that, The device fault information is obtained based on the global voiceprint feature and the extended voiceprint feature, including: obtaining a device fault coarse classification result based on the global voiceprint feature, the extended voiceprint feature, and a hybrid deep learning model comprising a convolutional neural network, a long short-term memory network, and a classifier connected in sequence; when the device fault coarse classification result does not meet a preset coarse classification requirement, obtaining a fine fault type of the device based on the global voiceprint feature, the extended voiceprint feature, a variational autoencoder model, and a Bayesian inference model.

8. The voiceprint feature-based electric welding equipment fault diagnosis method according to claim 7, characterized in that, The device fault coarse classification result is obtained based on the global voiceprint feature, the extended voiceprint feature, and the hybrid deep learning model comprising the convolutional neural network, the long short-term memory network, and the classifier connected in sequence, including: inputting the global voiceprint feature and the extended voiceprint feature into the hybrid deep learning model respectively to obtain a first device fault coarse classification sub-result and a second device fault coarse classification sub-result; obtaining the device fault coarse classification result based on the first device fault coarse classification sub-result and the second device fault coarse classification sub-result.

9. The voiceprint feature-based electric welding equipment fault diagnosis method according to claim 8, characterized in that, The fine fault type of the device is obtained based on the global voiceprint feature, the extended voiceprint feature, the variational autoencoder model, and the Bayesian inference model, including: inputting the global voiceprint feature and the extended voiceprint feature into the variational autoencoder model respectively to obtain latent variables corresponding to the global voiceprint feature and the extended voiceprint feature respectively and reconstruction samples corresponding to the global voiceprint feature and the extended voiceprint feature respectively; obtaining probability densities of the global voiceprint feature and the extended voiceprint feature under the reconstruction samples to obtain a device state anomaly score; when the device state anomaly score is greater than a preset threshold, obtaining posterior probabilities of each fine fault type based on the Bayesian inference model and the latent variables corresponding to the global voiceprint feature and the extended voiceprint feature respectively. A sub-fault type of the device is determined based on a posterior probability of each sub-fault type.

10. A voiceprint feature-based electric welding equipment fault diagnosis system based on the voiceprint feature-based electric welding equipment fault diagnosis method of any one of claims 1-9, characterized in that, The method comprises the steps of: a first acquisition module acquires a multi-channel original sound signal received by a microphone array deployed on the electric welding device; a filtering module performs collaborative filtering processing on the multi-channel original sound signal by using a weight vector self-adaptive adjustment collaborative filtering method to obtain a filtered multi-channel sound signal; a fusion module extracts respective corresponding feature vectors of the filtered multi-channel sound signal under different time scales and different frequency sub-bands, and performs feature fusion processing to obtain multi-scale fusion features; a dimension reduction module performs feature dimension reduction processing on the multi-scale fusion features based on an attention mechanism to obtain global voiceprint features; an expansion module performs feature extraction on the multi-channel original sound signal by using an expansion feature extraction model trained based on a BYOL model training framework to obtain expansion voiceprint features; a diagnosis module outputs device fault information based on the global voiceprint features, the expansion voiceprint features, and a fault diagnosis model.

Images