Industrial cross-domain voiceprint fault diagnosis method based on characteristic decomposition and reconstruction

Abstract

The invention discloses an industrial cross-domain voiceprint fault diagnosis method based on characteristic decomposition and reconstruction, and the method comprises the steps: obtaining a sound data signal of industrial equipment, and sequentially carrying out the linear normalization processing, Fourier transform, Mel filter conversion and logarithmic operation, and obtaining a logarithmic Mel spectrogram; and carrying out characteristic decomposition on the obtained logarithm Mel spectrogram to obtain a harmonic component spectrogram and an impact component spectrogram. According to the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction, feature decomposition is carried out on an obtained logarithm Mel spectrogram, deep feature extraction is carried out on two decomposed component spectrograms through a self-adaptive double-path feature fusion module, feature information is enhanced through an SE module, scaling is carried out by introducing a weight, and the fault diagnosis accuracy of the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction is improved. The sound fault diagnosis method has good robustness and regularization effects, and solves the problem that in an industrial scene, a sound fault diagnosis model cannot provide strong generalization ability for unseen data distribution.

Description

Technical Field

[0001] The present invention belongs to the technical field of voiceprint fault diagnosis, and specifically relates to an industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction. Background Art

[0002] Fault diagnosis technology refers to the use of technical means to determine whether a fault has occurred based on the operating status of industrial equipment, so as to determine the location, type and cause of the fault. With the advancement of science and technology, mechanical equipment is becoming more and more large-scale, precise and systematic. Therefore, the research on mechanical fault diagnosis technology is particularly important. Traditional fault detection methods often rely on the experience of professionals or direct contact detection of equipment, which is not only time-consuming and labor-intensive, but also has certain safety risks. At present, mechanical fault diagnosis technology mainly analyzes signals, such as vibration signals, sound signals, force signals, etc. Vibration signals are strongly correlated with faults, and are limited to contact measurement methods in fault diagnosis, which is not a good choice in some application scenarios. Sound signal-based detection systems are widely used in industrial equipment fault diagnosis due to their advantages of low cost, fast installation speed and non-contact measurement, such as voiceprint fault diagnosis of rolling bearings and acoustic machine status detection. Compared with regular and structured sounds, industrial equipment fault sound signals do not have static time patterns like rhythm and melody, nor do they have specific semantic sequences.

[0003] In recent years, with the rise of industrial big data, data-driven methods for fault diagnosis of industrial equipment have become a hot research field. Among them, fault detection methods based on deep learning have attracted widespread attention due to their high accuracy and other advantages. Most methods are based on the premise that the training data and test data obey the same feature distribution law, and the trained models have achieved good results. However, in actual industry, affected by the operating environment and operating conditions, the sound signal usually changes due to factors such as the operating speed of the equipment, environmental noise, and mechanical load, resulting in acoustic characteristics differences between the training data and the test data, and domain shift. At the same time, the model may still be biased due to false correlations in the features. The existing technology generally uses source domain data to learn invariant features in different domains, and generalizes the trained model to the unknown target domain, thereby realizing cross-domain fault diagnosis. Assuming that different domains share certain "stable" or "domain-invariant" features, the goal is to learn these domain-invariant features. However, due to the complexity of the sound signal, it may still not provide strong generalization capabilities for unseen distributions. At the same time, the model may still be biased due to false correlations in the features. Summary of the invention

[0004] The purpose of the present invention is to solve the problems raised in the background technology and to propose an industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] The present invention proposes an industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction, which includes acquiring the sound data signal of the industrial equipment, and sequentially performing linear normalization processing, Fourier transform, Mel filter conversion and logarithmic operation to obtain a logarithmic Mel spectrum graph;

[0007] Perform feature decomposition on the obtained logarithmic Mel spectrum to obtain a harmonic component spectrum and an impact component spectrum;

[0008] The harmonic component spectrum and the impact component spectrum are used as inputs of an adaptive two-way feature fusion module, wherein the adaptive two-way feature fusion module includes a first branch and a second branch, wherein the first branch includes three convolutional layers and an attention mechanism layer connected in sequence, and the second branch includes two convolutional layers and an attention mechanism layer connected in sequence, wherein the harmonic component spectrum is processed by the first branch to obtain a scaled harmonic component spectrum, and the impact component spectrum is processed by the second branch to obtain a scaled impact component spectrum, and the scaled harmonic component spectrum and the scaled impact component spectrum are fused to obtain a fused feature spectrum;

[0009] The fused feature spectrum is input into the trained fault diagnosis model to obtain the type of the sound data signal.

[0010] Preferably, the Fourier transform process comprises:

[0011] The linearly normalized sound data signal is framed according to a preset length to obtain sound data signal frames of preset frame length, and then Hanning windows are sequentially added to the framed sound data signal frames;

[0012] The short-time Fourier transform is performed on each sound data signal frame after adding the Hanning window to obtain the transform coefficient, and the corresponding formula is as follows:

[0013]

[0014] Wherein, X() represents the Fourier transform coefficient, m represents the starting position of the Hanning window in the sound data signal frame, ω represents the frequency, x(n) represents the part covered by the Hanning window in the sound data signal frame, n represents the index value of the time series, which ranges from 0 to N-1, N represents the width of the Hanning window, j is an imaginary unit, and W() represents the window function;

[0015] Then, according to each transformation coefficient, the time is used as the horizontal axis and the frequency is used as the vertical axis to plot on the coordinate system to obtain the corresponding power spectrum. Then, the power spectra of the transformation coefficients of all the sound data signal frames constitute a power spectrum diagram.

[0016] Preferably, the power spectrum is converted through a Mel filter to obtain a Mel spectrum, and then the Mel spectrum is logarithmically operated to obtain a logarithmic Mel spectrum X LMel .

[0017] Preferably, the step of performing feature decomposition on the obtained logarithmic Mel-color spectrum to obtain a harmonic component spectrum and an impact component spectrum includes:

[0018] The obtained logarithmic Mel spectrum is subjected to median filtering in the horizontal and vertical directions respectively, and the harmonic amplitude spectrum and the impulse amplitude spectrum are obtained in turn, and the corresponding formulas are as follows:

[0019]

[0020] in, represents the harmonic amplitude spectrum, represents the impact amplitude spectrum, median() represents the median filter, l h Indicates the time range for median filtering in the horizontal direction, l p Indicates the frequency range of median filtering in the vertical direction, X LMel (ml h ,ω),....,X LMel (m+l h ,ω) means taking the starting position m of the current Hanning window as the center, calculate ±l h The median value of the harmonic amplitude spectrum within the range, X LMel (m,ω-l p ),....,X LMel (m,ω+l h ) means taking the current frequency ω as the center and calculating ±l p The median value of the shock amplitude spectrum within the range;

[0021] Introduce separation factor β, β ≥ 1, when When , the logarithmic Mel spectrum graph X LMel is part of the harmonic component, When , the logarithmic Mel spectrum graph X LMel It is part of the impact component;

[0022] The binary mask of the harmonic amplitude spectrum and the binary mask of the impulse amplitude spectrum are defined as follows:

[0023]

[0024] Among them, M h () represents the binary mask of the harmonic amplitude spectrum, M p () represents the binary mask of the impact amplitude spectrum, ∈ is the preset value;

[0025] Apply the obtained binary mask to the logarithmic Mel spectrum to obtain the harmonic component spectrum and the impulse component spectrum, and the formula is as follows:

[0026]

[0027] Among them, X h () represents the harmonic component spectrum, X p () represents the impact component spectrum.

[0028] Preferably, in the first branch, the convolution kernel size of the first convolution layer is 1×1, and the step size is 1, and the convolution kernel size of the second convolution layer and the third convolution layer is 3×3, and the step size is 1;

[0029] In the second branch, the convolution kernel size of the first convolution layer is 1×1, and the step size is 1, and the convolution kernel size of the second convolution layer is 3×3, and the step size is 1;

[0030] The attention mechanism layer adopts the SE module.

[0031] Preferably, in the fusion process, learnable weights ω1 and ω2 are introduced, and the formula for the fusion process is as follows:

[0032] X h&p =ω1Y h +ω2Y p ;

[0033] Among them, X h&p represents the fused feature spectrum, ω1 represents the weight of the scaled harmonic component spectrum, ω2 represents the weight of the scaled impact component spectrum, and ω1+ω2=1.

[0034] Preferably, the types of the sound data signal include normal and abnormal, and the normal type is the sound data signal of the normal operation of the industrial equipment, and the abnormal type includes the sound data signal of different types of faults occurring in the industrial equipment.

[0035] Preferably, the fault diagnosis model adopts a Whitening-Net network, including two layer normalization modules, three instance normalization modules, a Flatten layer, a first linear layer, an activation layer (LeakyReLU) and a second linear layer connected in sequence, the layer normalization module includes a convolutional layer, a layer normalization layer, an activation layer (LeakyReLU) and a maximum pooling layer connected in sequence, and the instance normalization module includes a convolutional layer, an instance normalization layer, an activation layer (LeakyReLU) and a maximum pooling layer connected in sequence.

[0036] Preferably, when training the fault diagnosis model, a label is set for each fault type, and the training data set used is a homologous voiceprint signal group, wherein the homologous voiceprint signal group includes a preset number of subsets, each subset being a collection of sound data signals in different source domains under the same label, wherein different source domains are different industrial equipment;

[0037] The calculation formula of the total loss Loss of the fault diagnosis model is as follows:

[0038]

[0039] in,

[0040]

[0041] in, represents the classification loss, represents causal loss, λ is a hyperparameter to maintain the balance between classification loss and causal loss, and its initial value is 1. represents the output of the instance normalization module, T Flatten () indicates that it has been processed by the Flatten layer, mean() indicates the calculation of the average value, m indicates the fault type, y m Indicates the label, represents the data of the same source voiceprint signal group, and x represents the data set of all fault types m.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] This industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction performs feature decomposition on the obtained logarithmic Mel-spectrogram, and performs deep feature extraction on the two component spectra after decomposition through an adaptive dual-path feature fusion module. It strengthens the feature information through the SE module and introduces weights for scaling, which plays a good robustness and regularization role. It solves the problem that the sound fault diagnosis model in industrial scenarios cannot provide strong generalization ability for unseen data distributions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction of the present invention;

[0045] Figure 2 Schematic diagram of the adaptive dual-path feature fusion module of the present invention;

[0046] Figure 3 It is a structural schematic diagram of the fault diagnosis model of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] In one embodiment, Figure 1-Figure 3 As shown, an industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction is provided, including:

[0049] Step 1, obtaining the sound data signal of the industrial equipment, and sequentially performing linear normalization processing, Fourier transform, Mel filter conversion and logarithmic operation to obtain a logarithmic Mel spectrum graph;

[0050] The types of the sound data signals of the industrial equipment include normal and abnormal, and the normal ones are the sound data signals of the industrial equipment working normally, and the abnormal ones include the sound data signals of different types of faults of the industrial equipment;

[0051] Step 1.1: Linearly normalize the acquired sound data signals of industrial equipment so that the data of each dimension is mapped to a unified metric space. The formula used is as follows:

[0052]

[0053] Among them, x0 represents the sound data signal of the industrial equipment, x max Indicates the maximum amplitude of the sound data signal, x min Indicates the minimum amplitude value in the sound data signal, x nom Represents the sound data signal after linear normalization processing.

[0054] Step 1.2: The Fourier transform process includes:

[0055] The linearly normalized sound data signal is framed according to a preset length (the preset length is not limited and can be set according to actual needs) to obtain sound data signal frames with preset frame lengths, and then Hanning windows are added to the framed sound data signal frames in sequence;

[0056] The short-time Fourier transform is performed on each sound data signal frame after adding the Hanning window to obtain the transform coefficient, and the corresponding formula is as follows:

[0057]

[0058] Wherein, X() represents the Fourier transform coefficient, m represents the starting position of the Hanning window in the sound data signal frame, ω represents the frequency, x(n) represents the part covered by the Hanning window in the sound data signal frame, n represents the index value of the time series, which ranges from 0 to N-1, N represents the width of the Hanning window, j is an imaginary unit, and W() represents the window function;

[0059] Then, according to each transformation coefficient, the time is used as the horizontal axis and the frequency ω is used as the vertical axis. The corresponding power spectrum is obtained by plotting on the coordinate system. The power spectrum of the transformation coefficients of all sound data signal frames constitutes a power spectrum diagram (specifically, the sound data signal frame is a time domain signal. After short-time Fourier transform, the time domain signal is converted into a frequency domain signal. The converted frequency domain signal is plotted on the coordinate system with time as the horizontal axis and frequency as the vertical axis to obtain the corresponding power spectrum. The power spectrum of the transformation system of all sound data signal frames constitutes a power spectrum diagram).

[0060] Step 1.3: Convert the power spectrum through a Mel filter to obtain a Mel spectrum, and then perform a logarithmic operation on the Mel spectrum to obtain a logarithmic Mel spectrum X. LMel ;

[0061] Among them, in the Mel filter conversion process, the linear frequency in the power spectrum is converted into Mel frequency, and the power spectrum of the converted Mel frequency is the Mel spectrum, and the conversion formula is as follows:

[0062]

[0063] Wherein, Mel(f) represents the Mel frequency, f represents the linear frequency, and 700 represents the reference frequency value.

[0064] Step 2: Perform feature decomposition on the obtained logarithmic Mel spectrum to obtain a harmonic component spectrum and an impact component spectrum, including:

[0065] The obtained logarithmic Mel spectrum is subjected to median filtering in the horizontal and vertical directions respectively, and the harmonic amplitude spectrum and the impulse amplitude spectrum are obtained in turn, and the corresponding formulas are as follows:

[0066]

[0067] in, represents the harmonic amplitude spectrum, represents the impact amplitude spectrum, median() represents the median filter, l h Indicates the time range for median filtering in the horizontal direction, l p Indicates the frequency range of median filtering in the vertical direction, X LMel (ml h ,ω),....,X LMel (m+lh ,ω) means taking the starting position m of the current Hanning window as the center, calculate ±l h The median value of the harmonic amplitude spectrum within the range, X LMel (m,ω-l p ),....,X LMel (m,ω+l h ) means taking the current frequency ω as the center and calculating ±l p The median value of the shock amplitude spectrum within the range;

[0068] Introduce separation factor β, β ≥ 1, when When , the logarithmic Mel spectrum graph X LMel is part of the harmonic component, When , the logarithmic Mel spectrum graph X LMel It is part of the impulse component (specifically, the composition of an audio signal can be determined by analyzing the spectrum. Harmonic components refer to signal components whose frequencies are integer multiples of the reference frequency and usually show periodic changes. Impulse components refer to mutation signals that occur in a short period of time and are not periodic).

[0069] The binary mask of the harmonic amplitude spectrum and the binary mask of the impulse amplitude spectrum are defined as follows:

[0070]

[0071] Among them, M h () represents the binary mask of the harmonic amplitude spectrum, M p () represents the binary mask of the impact amplitude spectrum, ∈ is a preset value (a very small constant greater than 0 to avoid being divided by 0);

[0072] Apply the obtained binary mask to the logarithmic Mel spectrum to obtain the harmonic component spectrum and the impulse component spectrum, and the formula is as follows:

[0073]

[0074] Among them, X h () represents the harmonic component spectrum, X p () represents the impact component spectrum.

[0075] Among them, the harmonic component spectrum can clearly illustrate the frequency distribution and frequency band activity of the sound data; the impact component spectrum can reflect the difference between the time frame part and other noise frame parts.

[0076] Step 3: Figure 2As shown, the harmonic component spectrum and the impact component spectrum are used as the input of the adaptive two-way feature fusion module, and the adaptive two-way feature fusion module includes a first branch and a second branch. The first branch includes three convolutional layers and attention mechanism layers connected in sequence (the three convolutional layers and the attention mechanism layers are connected in sequence from data input to output), and the second branch includes two convolutional layers and attention mechanism layers connected in sequence (the three convolutional layers and the attention mechanism layers are connected in sequence from data input to output). The harmonic component spectrum is processed by the first branch to obtain a scaled harmonic component spectrum, and the impact component spectrum is processed by the second branch to obtain a scaled impact component spectrum. The scaled harmonic component spectrum and the scaled impact component spectrum are fused to obtain a fused feature spectrum.

[0077] Among them, in the first branch, the convolution kernel size of the first convolution layer is 1×1, and the step size is 1, and the convolution kernel size of the second and third convolution layers is 3×3, and the step size is 1;

[0078] In the second branch, the convolution kernel size of the first convolution layer is 1×1, and the stride is 1, and the convolution kernel size of the second convolution layer is 3×3, and the stride is 1;

[0079] The attention mechanism layer adopts SE module.

[0080] It should be noted that the two first convolutional layers compress the channels of the harmonic component spectrum and the impact component spectrum respectively to reduce the possibility of overfitting due to excessive parameters and facilitate subsequent feature splicing;

[0081] Then, in the first branch, the second and third convolutional layers are used to further mine the frequency distribution information contained in the harmonic classification spectrogram;

[0082] Since the shock component spectrum has a more intuitive vertical structure, a smaller receptive field is used to mine the detailed information of the shock component, and the second convolutional layer is used in the second branch to extract features from the shock component spectrum;

[0083] The attention mechanism can effectively enhance the feature extraction ability of deep neural networks, among which the SE module can selectively emphasize information-rich channels and suppress channels with less effect. Therefore, before feature fusion, the SE module is introduced to strengthen the channel features of the convolutional features. The SE module is mainly composed of global average pooling, two fully connected layers, ReLU activation function, and Sigmoid activation function. Specifically, the SE module first compresses the input feature map through global average pooling to extract the global statistical information of each channel. Then, the dependencies between channels are learned through two fully connected layers and ReLU activation function. Finally, the weight of each channel is output through the Sigmoid activation function, and these weights are multiplied with the original feature map to obtain a feature map with channel attention.

[0084] During the fusion process, learnable weights ω1 and ω2 are introduced, and the fusion process formula is as follows:

[0085] X h&p =ω1Y h +ω2Y p ;

[0086] Among them, X h&p represents the fused feature spectrum, ω1 represents the weight of the scaled harmonic component spectrum, ω2 represents the weight of the scaled impact component spectrum, and ω1+ω2=1.

[0087] Step 4: Input the fused feature spectrum into the trained fault diagnosis model to obtain the type of the sound data signal.

[0088] Among them, Figure 3As shown in FIG. 1 , the fault diagnosis model adopts a Whitening-Net network, including two layer normalization modules, three instance normalization modules, a Flatten layer, a first linear layer, an activation layer (LeakyReLU) and a second linear layer connected in sequence (from data input to output direction). The layer normalization module includes a sequentially connected convolution layer (the convolution kernel size is 3×3 and the step size is 2), a layer normalization layer (the number of channels of the layer normalization layer of the first layer normalization module is 32, and the number of channels of the layer normalization layer of the second layer normalization module is 64), an activation layer (using the LeakyReLU activation function) and a maximum pooling layer (the convolution kernel size is 3×3 and the step size is 1). The module includes sequentially connected convolutional layers (the convolution kernel size of the convolution layer of the first instance normalization module is 3×3 and the stride is 2, and the convolution kernel size of the convolution layer of the second and third instance normalization modules is 1×1 and the stride is 2), instance normalization layers (the number of channels of the instance normalization layer of the first instance normalization module is 128, and the number of channels of the instance normalization layers of the second and third instance normalization modules are 64 and 32 respectively), activation layers (using LeakyReLU activation function) and maximum pooling layers (the convolution kernel size of the maximum pooling layer of the first instance normalization module is 3×3 and the stride is 1, and the convolution kernel size of the maximum pooling layer of the second and third instance normalization modules is 3×3 and the stride is 1).

[0089] When training the fault diagnosis model, a label is set for each fault type, and the training data set used is a homologous voiceprint signal group, which includes a preset number of subsets, each subset is a collection of sound data signals under different source domains under the same label (i.e., a collection of sound data signals from different industrial equipment under the same fault type), where different source domains are different industrial equipment;

[0090] The calculation formula of the total loss Loss of the fault diagnosis model is as follows:

[0091]

[0092] in,

[0093]

[0094] in, represents the classification loss, represents causal loss, λ is a hyperparameter to maintain the balance between classification loss and causal loss, and its initial value is 1. represents the output of the instance normalization module, T Flatten () indicates that it has been processed by the Flatten layer, mean() indicates the calculation of the average value, m indicates the fault type, y m Indicates the label, represents the data of the same source voiceprint signal group, and x represents the data set of all fault types m.

[0095] In another embodiment, the effectiveness of the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction proposed in this application in cross-domain fault diagnosis is verified:

[0096] During the verification process, the common feature extraction methods Log-Mel spectrum graph, MFCC and the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction proposed in this application were used for comparison. All verification schemes used the same fault diagnosis model Whitening-Net, and the data set used the drone fault sound data set.

[0097] It should be noted that the dataset collects fault sound data of three different types of drones, represented by A, B, and C respectively. During the training process, the number of training iterations was set to 200 rounds, the batchsize of the training set and the test set was set to 32, the network optimizer was Nadam, and the learning rate was 0.001. The verification follows the principle of using one domain as the target domain (i.e., selecting a dataset from the A, B, and C datasets as the test set) and the remaining two domains as the source domains (i.e., the remaining two datasets as training sets). All tests were repeated ten times to reduce contingency, and the accuracy (ACC) and F1 score of the accuracy were recorded. As shown in Table 1, the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction proposed in this application performs well in three cases (i.e., A and B as training sets, C as test set, represented by AB→C in the table below; A and C as training sets, B as The accuracy and F1 score of the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction proposed in this application are 64.41%, 75.96%, and 60.12% respectively in the three cases; the F1 scores are 0.598, 0.743, and 0.601 respectively, as shown in Table 1:

[0098] Table 1

[0099]

[0100] During the verification process, the following comparative experiments are carried out on the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction proposed in this application to verify the influence of each module in the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction proposed in this application on the cross-domain diagnosis effect of the voiceprint (using the accuracy ACC as the effect of sound diagnosis): Serial number (1) in Table 2 indicates that only HPSS decomposition (that is, the content of step 2) is used, and the adaptive two-way feature fusion module is not used; Serial number (2) indicates the use of HPSS decomposition and adaptive two-way feature fusion module (wherein the adaptive two-way feature fusion module is represented by ADFF, and the ADFF in serial number (2) does not include the SE module); Serial number (3) indicates the use of HPSS decomposition and a complete adaptive two-way feature fusion module (that is, the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction proposed in this application). Among them, under the condition of using only the HPSS decomposition of (1), this paper adopts two combination methods to combine the characteristic spectra after HPSS decomposition: a. directly splicing the component spectra after HPSS decomposition (linearly adding the two component spectra), and then inputting them into the fault diagnosis model; b. inputting the component spectra after HPSS decomposition into the fault diagnosis model respectively. The comparative test results are shown in Table 2:

[0101] Table 2

[0102]

[0103] According to Table 2, verification shows that the proposed industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction has different degrees of improvement in improving cross-domain diagnosis. Compared with using the features after HPSS decomposition directly as the input of the fault diagnosis model, the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction proposed in this application has a certain improvement in accuracy. When using B and C as training sets and A as a test set, the accuracy of the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction proposed in this application has a more obvious improvement, indicating that the use of the ADFF module can improve the accuracy of the diagnosis results. The introduction of the SE module screens the importance of each channel feature and adds it to ADFF as an auxiliary means to improve the diagnosis performance.

[0104] During the verification process, the following comparative experiments were conducted on the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction to verify the influence of different weights (ω1 and ω2) in the proposed industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction on the sound diagnosis effect: the weight values ​​are ω1=ω2=1; ω1=ω2=0.5; ω1=1, ω2=0; ω1=0, ω2=1; the influence of different weight values ​​on the sound diagnosis effect was verified (the accuracy ACC was used as the effect of sound diagnosis). The results are shown in Table 3:

[0105] Table 3

[0106]

[0107] According to Table 3, compared with the weights of the harmonic component spectrogram and the impact component spectrogram, the ADFF module dynamically assigns corresponding weights according to the importance of the features, which performs better in the diagnostic accuracy results. Among them, the experimental results of ω1=1, ω2=1 are better than the experimental results of the other three values ​​(ω1=ω2=0.5; ω1=1, ω2=0; ω1=0, ω2=1), which verifies the effectiveness of the adaptive dual-path feature fusion module in the industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction in this application.

[0108] This industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction performs feature decomposition on the obtained logarithmic Mel-spectrogram, and performs deep feature extraction on the two component spectra after decomposition through an adaptive dual-path feature fusion module. It strengthens the feature information through the SE module and introduces weights for scaling, which plays a good robustness and regularization role. It solves the problem that the sound fault diagnosis model in industrial scenarios cannot provide strong generalization ability for unseen data distributions.

[0109] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction, characterized by: The industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction includes: Acquire the sound data signal of the industrial equipment, and perform linear normalization, Fourier transform, Mel filter conversion and logarithmic operation in sequence to obtain a logarithmic Mel spectrum; Perform feature decomposition on the obtained logarithmic Mel spectrum to obtain a harmonic component spectrum and an impact component spectrum; The harmonic component spectrum and the impact component spectrum are used as inputs of an adaptive two-way feature fusion module, wherein the adaptive two-way feature fusion module includes a first branch and a second branch, wherein the first branch includes three convolutional layers and an attention mechanism layer connected in sequence, and the second branch includes two convolutional layers and an attention mechanism layer connected in sequence, wherein the harmonic component spectrum is processed by the first branch to obtain a scaled harmonic component spectrum, and the impact component spectrum is processed by the second branch to obtain a scaled impact component spectrum, and the scaled harmonic component spectrum and the scaled impact component spectrum are fused to obtain a fused feature spectrum; The fused feature spectrum is input into the trained fault diagnosis model to obtain the type of the sound data signal.

2. The industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction as claimed in claim 1 is characterized by: The Fourier transform process includes: The linearly normalized sound data signal is framed according to a preset length to obtain sound data signal frames of preset frame length, and then Hanning windows are sequentially added to the framed sound data signal frames; The short-time Fourier transform is performed on each sound data signal frame after adding the Hanning window to obtain the transform coefficient, and the corresponding formula is as follows: Wherein, X() represents the Fourier transform coefficient, m represents the starting position of the Hanning window in the sound data signal frame, ω represents the frequency, x(n) represents the part covered by the Hanning window in the sound data signal frame, n represents the index value of the time series, which ranges from 0 to N-1, N represents the width of the Hanning window, j is an imaginary unit, and W() represents the window function; Then, according to each transformation coefficient, the time is used as the horizontal axis and the frequency is used as the vertical axis to plot on the coordinate system to obtain the corresponding power spectrum. Then, the power spectra of the transformation coefficients of all the sound data signal frames constitute a power spectrum diagram.

3. The industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction as claimed in claim 2 is characterized by: The power spectrum is converted through a Mel filter to obtain a Mel spectrum, and then the Mel spectrum is logarithmically operated to obtain a logarithmic Mel spectrum X LMel .

4. The industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction as claimed in claim 3 is characterized by: The feature decomposition of the obtained logarithmic Mel spectrum to obtain a harmonic component spectrum and an impact component spectrum includes: The obtained logarithmic Mel spectrum is subjected to median filtering in the horizontal and vertical directions respectively, and the harmonic amplitude spectrum and the impulse amplitude spectrum are obtained in turn, and the corresponding formulas are as follows: in, represents the harmonic amplitude spectrum, represents the impact amplitude spectrum, median() represents the median filter, Indicates the time range for median filtering in the horizontal direction. Indicates the frequency range of median filtering in the vertical direction, Indicates that the starting position m of the current Hanning window is the center and the calculation The median value of the harmonic amplitude spectrum within the range, It means that the current frequency ω is taken as the center and the calculation The median value of the shock amplitude spectrum within the range; Introduce separation factor β, β ≥ 1, when When , the logarithmic Mel spectrum graph X LMel is part of the harmonic component, When , the logarithmic Mel spectrum graph X LMel It is part of the impact component; The binary mask of the harmonic amplitude spectrum and the binary mask of the impulse amplitude spectrum are defined as follows: Among them, M h () represents the binary mask of the harmonic amplitude spectrum, M p () represents the binary mask of the impact amplitude spectrum, ∈ is the preset value; Apply the obtained binary mask to the logarithmic Mel spectrum to obtain the harmonic component spectrum and the impulse component spectrum, and the formula is as follows: Among them, X h () represents the harmonic component spectrum, X p () represents the impact component spectrum.

5. The industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction as claimed in claim 1 is characterized by: In the first branch, the convolution kernel size of the first convolution layer is 1×1, and the step size is 1, and the convolution kernel size of the second and third convolution layers is 3×3, and the step size is 1; In the second branch, the convolution kernel size of the first convolution layer is 1×1, and the step size is 1, and the convolution kernel size of the second convolution layer is 3×3, and the step size is 1; The attention mechanism layer adopts the SE module.

6. The industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction as claimed in claim 1 is characterized by: During the fusion process, learnable weights ω1 and ω2 are introduced, and the formula for the fusion process is as follows: X h&p =ω1Y h +ω2Y p ; Among them, X h&p represents the fused feature spectrum, ω1 represents the weight of the scaled harmonic component spectrum, ω2 represents the weight of the scaled impact component spectrum, and ω1+ω2=1.

7. The industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction as claimed in claim 1 is characterized by: The types of the sound data signals include normal and abnormal, and the normal ones are sound data signals of normal operation of the industrial equipment, and the abnormal ones include sound data signals of different types of faults of the industrial equipment.

8. The industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction as claimed in claim 1 is characterized by: The fault diagnosis model adopts a Whitening-Net network, including two layer normalization modules, three instance normalization modules, a Flatten layer, a first linear layer, an activation layer (LeakyReLU) and a second linear layer connected in sequence. The layer normalization module includes a convolutional layer, a layer normalization layer, an activation layer (LeakyReLU) and a maximum pooling layer connected in sequence. The instance normalization module includes a convolutional layer, an instance normalization layer, an activation layer (LeakyReLU) and a maximum pooling layer connected in sequence.

9. The industrial cross-domain voiceprint fault diagnosis method based on feature decomposition and reconstruction as claimed in claim 1 is characterized by: When training the fault diagnosis model, a label is set for each fault type, and the training data set used is a homologous voiceprint signal group, which includes a preset number of subsets, each subset is a collection of sound data signals in different source domains under the same label, where different source domains are different industrial equipment; The calculation formula of the total loss Loss of the fault diagnosis model is as follows: in, in, represents the classification loss, represents causal loss, λ is a hyperparameter to maintain the balance between classification loss and causal loss, and its initial value is 1. represents the output of the instance normalization module, T Flatten () indicates that it has been processed by the Flatten layer, mean() indicates the calculation of the average value, m indicates the fault type, y m Indicates the label, represents the data of the same source voiceprint signal group, and x represents the data set of all fault types m.

Images