Audio-based fault diagnosis method, device and electronic equipment

Abstract

The present application relates to the technical field of fault diagnosis, and provides a fault diagnosis method and device based on audio and electronic equipment, the fault diagnosis method based on audio comprises: feature extraction is carried out on the audio file to be diagnosed, and audio features to be diagnosed are obtained;Based on the audio features to be diagnosed and the target piecewise linear function, the first sound feature is obtained, and the parameters of the target piecewise linear function are determined based on the attention conversion of the audio features to be diagnosed;Based on the first sound feature, the fault diagnosis result is obtained.The diagnosis method of the present application dynamically generates the target piecewise linear function with non-fixed parameter characteristics capable of responding to the noise signal in the audio file to be tested through the attention mechanism, can effectively remove noise, improve the accuracy of the fault diagnosis result, and has high universality and flexibility.

Description

Technical Field

[0001] This invention relates to the field of fault diagnosis technology, and in particular to audio-based fault diagnosis methods, apparatus and electronic devices. Background Technology

[0002] Using sound signals to detect faults in mechanical parts is a common method. Current technology often employs soft thresholding to analyze and reduce noise in sound signals; however, the parameters in this method need to be manually set and are fixed, resulting in poor adaptability. Summary of the Invention

[0003] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes an audio-based fault diagnosis method to improve the accuracy of fault diagnosis results, and has high universality and flexibility.

[0004] The present invention also proposes an audio-based fault diagnosis device.

[0005] The present invention also proposes an electronic device.

[0006] The present invention also proposes a non-transitory computer-readable storage medium.

[0007] The present invention also proposes a computer program product.

[0008] According to an embodiment of the first aspect of the present invention, an audio-based fault diagnosis method includes:

[0009] Feature extraction is performed on the audio file to be diagnosed to obtain the audio features to be diagnosed;

[0010] Based on the audio features to be diagnosed and the target piecewise linear function, a first audio feature is obtained, and the parameters of the target piecewise linear function are determined by attention transformation based on the audio features to be diagnosed.

[0011] Based on the first audio feature, the fault diagnosis result is obtained.

[0012] According to the audio-based fault diagnosis method of the present invention, a target piecewise linear function with non-fixed parameter characteristics that can respond to noise signals in the audio file under test is dynamically generated through an attention mechanism. This allows for the generation of different thresholds and slopes based on different noise signals, thereby improving the response of the target piecewise linear function to external noise signals and effectively removing noise. In addition, by preserving the gradient within the threshold, useful information of the noise signal within the threshold can be extracted to improve the accuracy of the noise reduction results, thereby improving the accuracy of the fault diagnosis results. This method has universality and flexibility.

[0013] According to an embodiment of the present invention, the parameters of the target piecewise linear function are determined by the following method:

[0014] Feature extraction is performed on the audio features to be diagnosed to obtain multiple sets of second audio features;

[0015] Attention transformation is performed on the multiple sets of second audio features to obtain the parameters of the target piecewise linear function.

[0016] According to the audio-based fault diagnosis method of the present invention, the target piecewise linear function threshold and the slope within the threshold are automatically derived by the network based on the attention mechanism. This allows the target piecewise linear function threshold and the slope within the threshold to be dynamically adjusted with changes in noise signals, thereby making the entire network more robust and more resistant to the influence of noise, thus greatly improving the accuracy of network judgment.

[0017] According to an embodiment of the present invention, the attention transformation of the plurality of sets of second audio features to obtain the parameters of the target piecewise linear function includes:

[0018] The two sets of second audio features are subjected to absolute value, global average pooling, fully connected, batch regularization, deep residual, fully connected processing and mapping processing respectively to obtain the threshold of the target piecewise linear function and the first slope of the target piecewise linear function within the threshold.

[0019] The other two groups of the second audio features are processed by absolute value, global average pooling, fully connected, batch regularization, deep residual and fully connected processing respectively to obtain the second slope and the third slope of the target piecewise linear function outside the threshold.

[0020] According to an embodiment of the present invention, obtaining the first audio feature based on the audio feature to be diagnosed and the target piecewise linear function includes:

[0021] Feature extraction is performed on the audio features to be diagnosed to obtain multiple sets of second audio features;

[0022] The third audio feature is obtained by extracting features from the multiple sets of second audio features;

[0023] Based on the third audio feature and the target piecewise linear function, the residual value is obtained;

[0024] The first audio feature is obtained by summing the audio feature to be diagnosed and the residual value.

[0025] According to an embodiment of the present invention, the step of extracting features from the audio features to be diagnosed to obtain multiple sets of second audio features includes: sequentially performing batch regularization, depth residual and multi-kernel convolution processing on the audio features to be diagnosed to obtain multiple sets of second audio features.

[0026] According to an embodiment of the present invention, the step of extracting features from the plurality of sets of second audio features to obtain third audio features includes: performing batch regularization, depth residual and single-kernel convolution processing on the plurality of sets of second audio features to obtain third audio features.

[0027] According to an embodiment of the present invention, obtaining the fault diagnosis result based on the first audio feature includes:

[0028] The first audio feature is sequentially processed by batch regularization, deep residual, global average pooling and matrix transformation to obtain the fourth audio feature.

[0029] Based on the fourth audio feature, the fault diagnosis result is obtained.

[0030] According to a second aspect of the present invention, an audio-based fault diagnosis apparatus includes:

[0031] The first processing module is used to extract features from the audio file to be diagnosed, thereby obtaining the audio features to be diagnosed.

[0032] The second processing module is used to obtain a first audio feature based on the audio feature to be diagnosed and the target piecewise linear function, wherein the parameters of the target piecewise linear function are determined by attention transformation based on the audio feature to be diagnosed.

[0033] The third processing module is used to obtain fault diagnosis results based on the first audio features.

[0034] According to the audio-based fault diagnosis device of the present invention, a target piecewise linear function with non-fixed parameter characteristics that can respond to noise signals in the audio file under test is dynamically generated through an attention mechanism. This allows for the generation of different thresholds and slopes based on different noise signals, thereby improving the response of the target piecewise linear function to external noise signals and effectively removing noise. In addition, by preserving the gradient within the threshold, useful information of the noise signal within the threshold can be extracted to improve the accuracy of the noise reduction results, thereby improving the accuracy of the fault diagnosis results. This device is universal and flexible.

[0035] An electronic device according to a third aspect of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described audio-based fault diagnosis methods.

[0036] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium is provided thereon storing a computer program that, when executed by a processor, implements the steps of any of the above-described audio-based fault diagnosis methods.

[0037] A computer program product according to a fifth aspect of the present invention includes a computer program that, when executed by a processor, implements the steps of any of the above-described audio-based fault diagnosis methods.

[0038] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0039] By dynamically generating a target piecewise linear function with non-fixed parameter characteristics that responds to noise signals in the audio file under test through an attention mechanism, different thresholds and slopes can be generated according to different noise signals, thereby improving the response of the target piecewise linear function to external noise signals and effectively removing noise. In addition, by preserving the gradient within the threshold, useful information of the noise signal within the threshold can be extracted to improve the accuracy of the noise reduction results, thereby improving the accuracy of fault diagnosis results, and has universality and flexibility.

[0040] Furthermore, by automatically deriving the target piecewise linear function threshold and the slope within the threshold using the network based on the attention mechanism, the target piecewise linear function threshold and the slope within the threshold can be dynamically adjusted as the noise signal changes, thereby making the entire network more robust and more resistant to the influence of noise signals.

[0041] Furthermore, as the sample data gradually expands, the final diagnostic results will become more accurate.

[0042] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is one of the flowcharts of the audio-based fault diagnosis method provided in the embodiments of the present invention;

[0045] Figure 2This is the second schematic flowchart of the audio-based fault diagnosis method provided in the embodiments of the present invention;

[0046] Figure 3 This is the third flowchart of the audio-based fault diagnosis method provided in this embodiment of the invention;

[0047] Figure 4 This is one of the target piecewise linear function graphs of the audio-based fault diagnosis method provided in this embodiment of the invention;

[0048] Figure 5 This is the second piecewise linear function graph of the target segmented function provided in the embodiment of the audio-based fault diagnosis method of the present invention;

[0049] Figure 6 This is the third piecewise linear function graph of the target segmented function of the audio-based fault diagnosis method provided in this embodiment of the invention;

[0050] Figure 7 This is a schematic diagram of the structure of the audio-based fault diagnosis device provided in an embodiment of the present invention;

[0051] Figure 8 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0052] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0053] In the description of the embodiments of the present invention, the terms "first", "second", and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0054] Those skilled in the art can understand the specific meaning of the above terms in the embodiments of the present invention based on the specific circumstances.

[0055] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0056] The following is combined Figures 1-6 This invention describes an audio-based fault diagnosis method according to an embodiment of the present invention.

[0057] It should be noted that this fault diagnosis method can be used for both household appliances and commercial appliances.

[0058] Household appliances include, but are not limited to: kitchen appliances, cleaning appliances, and refrigeration appliances.

[0059] For example, kitchen appliances can include range hoods and gas stoves.

[0060] It is understood that the entity executing this audio-based fault diagnosis method can be an electrical appliance, a processor or processing device that is connected in communication with the electrical appliance, or an operating terminal of a quality inspector or maintenance personnel. The operating terminal includes, but is not limited to, the mobile phone, tablet computer, or computer of the quality inspector or maintenance personnel.

[0061] like Figure 1 As shown, the audio-based fault diagnosis method includes steps 110, 120 and 130.

[0062] Step 110: Extract features from the audio file to be diagnosed to obtain the audio features to be diagnosed;

[0063] In this step, the audio file to be diagnosed is a sound signal containing noise, which includes the signals used to determine the fault and other redundant signals. For example, in the case of using the sound of a bearing rotating to determine a bearing fault, the audio file to be diagnosed is the sound information of the bearing rotating, and the collected sound of the bearing rotating contains a large amount of noise and redundant signals.

[0064] The audio file to be diagnosed can be captured using a microphone or similar device.

[0065] By extracting the semantic information of the audio file to be diagnosed, the audio features to be diagnosed can be obtained for determining whether there is a fault. It can be understood that the audio features to be diagnosed contain noise signals.

[0066] In actual implementation, convolutional networks can be used to extract features from the audio files to be diagnosed, resulting in audio features of dimension C×W×1.

[0067] Where C is the channel, i.e. the number of convolutional kernels in the convolutional layer; W is the width of the audio feature to be diagnosed; and 1 is the height of the audio feature to be diagnosed.

[0068] It should be noted that the feature height in the noise signal is 1.

[0069] Depending on the deep learning development framework, the arrangement of channels may vary. Some frameworks may use a W×1×C structure. In this embodiment, the channel is placed in the first dimension by default.

[0070] Step 120: Based on the target piecewise linear function and the audio features to be diagnosed, the first audio feature is obtained. The parameters of the target piecewise linear function are determined by attention transformation based on the audio features to be diagnosed.

[0071] The target piecewise linear function is used to denoise the audio features to be diagnosed.

[0072] like Figure 6 As shown, the target piecewise linear function is a piecewise function with the positive or negative value of the threshold τ as the dividing point. The expressions corresponding to the threshold τ and the threshold τ are both linear functions, and the slope of the target piecewise linear function is not fixed.

[0073] It should be noted that the threshold and slope of the target piecewise linear function are correlated with the noise signal in the audio features to be diagnosed.

[0074] The slopes of the linear functions corresponding to each interval of the target piecewise linear function can be the same or different.

[0075] The input value x and the output value y of the target piecewise linear function are both real numbers, where the value of y is the residual value corresponding to the value of x.

[0076] The first audio feature is the sound feature after noise reduction of the audio feature to be diagnosed.

[0077] In this step, based on the audio features to be diagnosed and the target piecewise linear function, the residual value corresponding to the audio features to be diagnosed can be obtained.

[0078] Based on the audio features to be diagnosed and the residual values, the first audio features can be obtained.

[0079] During the research and development process, the inventors discovered that in the existing soft thresholding function, the slope number is 0 within the threshold range, and the slope outside the threshold remains a fixed value of 1. This soft thresholding function cannot guarantee that the network can effectively respond to changes in external noise, and it also completely discards the effective data in the noise within the threshold range, resulting in poor noise reduction effect.

[0080] In this invention, the target piecewise linear function threshold and the slope within the threshold are automatically derived by the network using an attention mechanism. This allows the target piecewise linear function threshold and the slope within the threshold to be dynamically adjusted based on noise signals, thereby making the entire network more robust and more resistant to the influence of noise signals, and greatly improving the accuracy of the network's judgment.

[0081] The implementation method of step 120 is explained in detail below.

[0082] like Figure 2 As shown, in some embodiments, step 120 includes:

[0083] Step 121: Extract features from the audio features to be diagnosed to obtain multiple sets of second audio features.

[0084] In this step, the second audio feature is a feature that has a higher level of speech information than the audio feature to be diagnosed.

[0085] The multiple sets of second audio features are at least four sets of second audio features. Each set of second audio features is identical.

[0086] The multiple sets of second audio features obtained through this step can be used to calculate the parameters for generating the target piecewise linear function.

[0087] In actual execution, feature extraction of the audio features to be diagnosed may include: performing batch normalization, depthwise residual (ReLU), and multi-kernel convolution (Con) processing on the audio features to be diagnosed. These operations are referred to as CBN operations in subsequent embodiments.

[0088] Among them, the batch regularization is used to normalize the audio features to be diagnosed.

[0089] Convolution processing is used to increase the number of channels of the output features. For example, the number of convolutions can be set to 4C, thereby making the number of channels of the output features four times the original, which means obtaining four sets of second audio features with dimensions of 4C×W×1.

[0090] Step 122: Extract features from multiple sets of second audio features to obtain third audio features.

[0091] In this step, the third audio feature is a feature that has a higher level of speech information than the second audio feature.

[0092] The third audio feature is a feature with the same dimension as the audio feature to be diagnosed.

[0093] The third audio feature obtained through this step can be used as the input value x of the target piecewise linear function to obtain the residual value y corresponding to the third audio feature.

[0094] In actual implementation, feature extraction is performed on multiple sets of second audio features to obtain third audio features. This may include: batch regularization, depth residual and single kernel convolution processing on multiple sets of second audio features to obtain third audio features.

[0095] In this embodiment, a CBN operation is performed on multiple sets of second audio features to extract higher-level semantic information and restore the dimension of the output feature channels.

[0096] For example, the number of convolutions can be set to C, thereby changing the dimension of the output feature channels back to the third audio feature of C×W×1.

[0097] Step 123: Based on the target piecewise linear function and the third audio feature, obtain the residual value.

[0098] The residual value is the error value between the audio feature to be diagnosed and the first audio feature caused by the influence of noise signal.

[0099] In this step, the third audio feature obtained in step 122 is used as the input to the target piecewise linear function to obtain the residual value corresponding to the third audio feature as the independent variable.

[0100] Based on this residual value, the audio features to be diagnosed can be denoised, thereby suppressing the noise signal and avoiding the influence of the noise signal on the detection results.

[0101] It should be noted that the dimension of this residual value is the same as the dimension of the audio feature to be diagnosed, which is C×W×1.

[0102] Step 124: Sum the residual value and the audio feature to be diagnosed to obtain the first audio feature.

[0103] In this step, the residual value obtained in step 123 and the audio feature to be diagnosed obtained in step 110 are added together to effectively reduce the noise of the audio feature to be diagnosed, thereby obtaining the first audio feature with noise signal interference removed.

[0104] The dimension of the first audio feature is the same as the dimension of the residual value and the dimension of the audio feature to be diagnosed, which is C×W×1.

[0105] The following specific examples illustrate how the target piecewise linear function is generated.

[0106] Continue to refer to Figure 2 In some embodiments, the steps for generating the target piecewise linear function include:

[0107] Step 101: Extract features from the audio features to be diagnosed to obtain multiple sets of second audio features.

[0108] In this step, the second audio feature is a feature that has a higher level of speech information than the audio feature to be diagnosed.

[0109] The multiple sets of second audio features are at least four sets of second audio features. Each set of second audio features is identical.

[0110] The method for generating the second audio feature is the same as in the above embodiments, and will not be repeated here.

[0111] Step 101 yields a second audio feature with dimensions 4C×W×1.

[0112] Step 102: Perform attention transformation on multiple sets of second audio features to obtain the parameters of the target piecewise linear function.

[0113] The parameters of the target piecewise linear function include the boundary points of the target piecewise linear function and the slope values ​​of the expressions within each segment.

[0114] In this embodiment, the boundary point of the target piecewise linear function is the positive or negative value of the threshold τ. Based on the threshold τ, the target piecewise function can be divided into three intervals: x < -τ, -τ ≤ x ≤ τ, and x > τ. Among them, -τ ≤ x ≤ τ is the interval within the threshold, and x < -τ and x > τ are the two intervals outside the threshold, respectively.

[0115] Each interval has a corresponding slope in its function expression. The slope corresponding to the interval x < -τ is the second slope a, the slope corresponding to the interval -τ ≤ x ≤ τ is the first slope b, and the slope corresponding to the interval x > τ is the third slope c.

[0116] It should be noted that the values ​​of a, b, and c are not fixed, and the values ​​of a, b, and c can be the same or different.

[0117] In actual execution, the strings of multiple sets of second audio features with dimensions of 4C×W×1 obtained in step 101 can be split into two parts, each with a dimension of 2C×W×1.

[0118] Attention transformation is performed on each of the 2C×W×1 dimensional second audio features. The threshold τ and the first slope b within the threshold interval of the target piecewise linear function are obtained through the first set of 2C×W×1 dimensional second audio features.

[0119] The second slope a and the third slope c outside the threshold interval of the target piecewise linear function are obtained by using another set of second audio features of 2C×W×1 dimensions.

[0120] It should be noted that in the process of calculating the threshold τ and the first slope b within the threshold interval, the final result needs to be mapped to ensure that the values ​​of τ and b are within the range of 0-1.

[0121] The implementation of step 102 will be described below through specific embodiments.

[0122] Continue to refer to Figure 2 In some embodiments, step 102 includes:

[0123] The two sets of second audio features are subjected to absolute value, global average pooling, fully connected, batch regularization, deep residual, fully connected processing and mapping processing respectively to obtain the first slope of the target piecewise linear function within the threshold and the threshold of the target piecewise linear function.

[0124] The other two sets of second audio features are processed by absolute value, global average pooling, fully connected, batch regularization, deep residual and fully connected methods respectively to obtain the third slope and the second slope of the target piecewise linear function outside the threshold.

[0125] Firstly, the method for generating the first slope of the target piecewise linear function within the threshold and the threshold of the target piecewise linear function.

[0126] In this embodiment, absolute value and global average pooling (GAP) operations are first performed on the two sets of 2C×W×1 dimensional second audio features to obtain two corresponding 2×1×1 outputs.

[0127] Then, the second audio feature with a 2×1×1 dimension is subjected to fully connected (FC), batch normalization, ReLU and fully connected (FC) operations;

[0128] Then, the values ​​output through the above steps are input into the sigmoid function. The sigmoid function is used to map the values ​​to obtain the first slope of the target piecewise linear function within the threshold and the threshold of the target piecewise linear function.

[0129] The sigmoid function is the activation function of the neural network, used for the output of hidden layer neurons. Its value range is (0, 1), and it is used to map a real number to the interval (0, 1).

[0130] It should be noted that when operating on the second audio feature of 2×1×1 dimension, there are 2C channels of features. One C channel of features can be output as the threshold τ of the target piecewise linear function, and the other C channel of features can be output as the first slope b of the target piecewise linear function within the threshold, thereby distinguishing the output results.

[0131] Secondly, the method for generating the third and second slopes of the target piecewise linear function outside the threshold.

[0132] The two sets of 2C×W×1 second audio features are subjected to absolute value and global average pooling (GAP) operations to obtain two corresponding 2×1×1 outputs.

[0133] Then, fully connected (FC), batch normalization, ReLU, and fully connected (FC) operations are performed on the second audio feature of 2×1×1 dimension; the third slope and the second slope of the target piecewise linear function outside the threshold are obtained directly.

[0134] It should be noted that when operating on the second audio feature of 2×1×1 dimension, there are 2C channels of features. One C channel feature can be output as the third slope c of the target piecewise linear function outside the threshold, and the other C channel feature can be output as the second slope a of the target piecewise linear function outside the threshold, thereby distinguishing the output results.

[0135] After obtaining the boundary point τ and the values ​​of the slopes a, b, and c within each interval, the target piecewise linear function can be determined based on τ and the values ​​of a, b, and c.

[0136] It should be noted that the target piecewise linear function generated through the above steps has non-fixed boundary points and slopes. It is related to the noise signal in the audio file under test, and its value can be dynamically adjusted based on the noise signal, so as to respond well to external noise signals.

[0137] The target piecewise linear function includes multiple sub-linear segments, and the slope value of each sub-linear segment can be flexibly designed according to different application scenarios.

[0138] like Figures 4-6 As shown, depending on the noise conditions, the target piecewise linear function can include: a piecewise linear function with no gradient within the threshold and the same gradient outside the threshold; a piecewise linear function with no gradient within the threshold and different gradients outside the threshold; a piecewise linear function with a gradient within the threshold and the same gradient outside the threshold; a piecewise linear function with a gradient within the threshold and different gradients outside the threshold; and other piecewise linear functions with arbitrary slope values.

[0139] like Figure 4 This illustrates a case where there is no gradient within a threshold, such as... Figure 5 This illustrates a case where there is a gradient within a threshold and the gradient is the same outside the threshold, such as... Figure 6 This illustrates a case where there is a gradient within a threshold and a different gradient outside the threshold.

[0140] According to this embodiment, a target piecewise linear function is obtained through an attention mechanism to replace the soft threshold function. Different thresholds and slopes of the target piecewise linear function can be set based on different noise signals, thereby better removing redundant noise signals in the sound features to be tested, retaining useful information in the noise signals in the region, improving the accuracy of the noise reduction results, and having a wide range of application scenarios.

[0141] Step 130: Based on the first audio feature, obtain the fault diagnosis result.

[0142] In this step, the fault diagnosis results include two cases: faulty and no fault.

[0143] The first audio feature is the feature after noise reduction of the sound feature to be detected.

[0144] By identifying the first audio feature, it can be determined whether the object under test is faulty.

[0145] According to the above embodiments of this application, before fault diagnosis, the target piecewise linear function can effectively reduce the noise of the audio features to be detected while retaining the effective data part of the noise signal, thereby eliminating the interference of redundant noise in the sound features to be detected on the fault diagnosis results and helping to improve the accuracy of the diagnosis results.

[0146] like Figure 3 As shown, in some embodiments, step 130 includes:

[0147] The first audio feature is processed sequentially by batch regularization, deep residual, global average pooling and matrix transformation to obtain the fourth audio feature.

[0148] Based on the fourth audio feature, the fault diagnosis result is obtained.

[0149] The first audio feature is a 1×1×1 dimensional feature.

[0150] In this embodiment, performing batch normalization, ReLU, and global average pooling (GAP) operations on the first audio features transforms the output into an N×1×1 dimension.

[0151] N is a hyperparameter used to represent the channel dimension. N can take any positive integer value.

[0152] The value of N can be user-defined, such as N being 3 or 6.

[0153] The obtained N×1×1 features are reshaped into a vector of length N.

[0154] Reshape is a function that transforms a matrix into a matrix of a specific dimension. In this step, features of dimension N×1×1 are subjected to matrix transformation to obtain an N-dimensional vector of length N.

[0155] The N-dimensional vector obtained through the above steps is subjected to a fully connected (FC) operation, followed by a softmax operation, and the fault classification result is directly output.

[0156] The Softmax function is used to output the classification result, that is, to output either a faulty or no-fault state.

[0157] The audio-based fault diagnosis method provided by the present invention dynamically generates a target piecewise linear function with non-fixed parameter characteristics that can respond to noise signals in the audio file under test through an attention mechanism. This allows for the generation of different thresholds and slopes based on different noise signals, thereby improving the response of the target piecewise linear function to external noise signals and effectively removing noise. In addition, by preserving the gradient within the threshold, useful information of the noise signal within the threshold can be extracted to improve the accuracy of the noise reduction results, thereby improving the accuracy of the fault diagnosis results. This method has high universality and flexibility.

[0158] In actual implementation, the above steps can be completed using a fault diagnosis model.

[0159] For example, the audio file to be diagnosed is input into the feature extraction layer of the fault diagnosis model, and the audio features to be diagnosed are output.

[0160] In this step, the feature extraction layer of the fault diagnosis model extracts features from the audio file to be diagnosed, and then outputs the audio features to be diagnosed.

[0161] The audio features to be diagnosed obtained through the feature extraction layer are used as input to the noise reduction layer of the fault diagnosis model.

[0162] In practice, the audio features to be diagnosed can be stored on a local server or cloud server as sample data for model training, and can be retrieved when needed.

[0163] After completing the above steps, the audio features to be diagnosed are input into the noise reduction processing layer of the fault diagnosis model. The noise reduction processing layer of the fault diagnosis model generates a target piecewise linear function based on the audio features to be diagnosed, and outputs the first audio feature based on the target piecewise linear function and the audio features to be diagnosed.

[0164] In this step, it should be noted that the noise reduction processing layer of the fault diagnosis model includes at least one sub-processing layer, for example, it may include 3 or 4 sub-processing layers, such as... Figure 3 The APU module shown is a sub-processing layer.

[0165] The functions and execution steps of each sub-processing layer are the same.

[0166] The following example illustrates this step by sequentially inputting the audio features to be diagnosed into three noise reduction sub-processing layers.

[0167] The audio features to be diagnosed obtained through the feature extraction layer are used as the input to the noise reduction processing layer of the fault diagnosis model. The first sub-processing layer performs the first noise reduction processing on the audio features to be diagnosed, and obtains the audio features to be diagnosed after one noise reduction processing.

[0168] The processing steps of the first sub-processing layer are as follows:

[0169] The audio features to be diagnosed are processed by batch normalization, deep residual (ReLU) and multi-kernel convolution (Con) to obtain multiple sets of second audio features.

[0170] Based on at least two sets of second audio features, absolute value, global average pooling, fully connected, batch regularization, deep residual, fully connected processing and mapping processing are performed respectively to generate the threshold τ of the target piecewise linear function and the first slope b within the threshold τ.

[0171] Based on two other sets of second audio features, absolute value, global average pooling, fully connected, batch regularization, deep residual, and fully connected processing are performed respectively to generate a second slope a and a third slope c outside the threshold τ.

[0172] Based on the values ​​of τ and a, b, and c, a target piecewise linear function corresponding to the audio features to be diagnosed is generated.

[0173] The residual value is generated based on the target piecewise linear function, and then added to the audio feature to be diagnosed to obtain the audio feature to be diagnosed after the first noise reduction process.

[0174] The audio features to be diagnosed after the first noise reduction process are used as the input values ​​of the second sub-processing layer. The second sub-processing layer performs a second noise reduction process on the audio features to be diagnosed after the first noise reduction process, resulting in audio features to be diagnosed after the second noise reduction process.

[0175] The processing steps of the second sub-processing layer are similar to those of the first sub-processing layer, and will not be described in detail here.

[0176] The target piecewise linear function obtained through this step is the target piecewise linear function under the second sub-processing layer, which corresponds to the audio features to be diagnosed after one noise reduction process.

[0177] The audio features to be diagnosed after the second noise reduction process are used as the input values ​​of the third sub-processing layer. The third sub-processing layer performs a third noise reduction process on the audio features to be diagnosed after the second noise reduction process, resulting in the audio features to be diagnosed after three noise reduction processes, which is the final first audio feature.

[0178] The processing steps of the third sub-processing layer are similar to those of the first sub-processing layer, and will not be described in detail here.

[0179] The target piecewise linear function obtained through this step is the target piecewise linear function under the third sub-processing layer, which corresponds to the audio features to be diagnosed after secondary noise reduction processing.

[0180] The first audio feature obtained through the noise reduction layer is used as the input to the recognition layer of the fault diagnosis model.

[0181] In actual execution, the target piecewise linear function and the first audio feature can be stored on a local server or cloud server as sample data for model training, and can be retrieved when needed.

[0182] After completing the above steps, the first audio feature is input into the recognition layer of the fault diagnosis model, and the fault diagnosis result is output.

[0183] In this step, the fault diagnosis result can be obtained by performing batch regularization, deep residual, global average pooling, matrix transformation, fully connected processing, and softmax operation on the first audio features.

[0184] In actual implementation, the fault diagnosis results can be stored on a local server or cloud server as sample data for model training, and can be retrieved when needed.

[0185] Understandably, in actual implementation, the above-mentioned fault diagnosis model needs to be trained first. The training process of the fault diagnosis model will be explained below through specific examples.

[0186] In some embodiments, the feature extraction layer of the fault diagnosis model is trained by using the audio file to be diagnosed as a sample and the audio features corresponding to the audio file to be diagnosed as sample labels.

[0187] The noise reduction layer of the fault diagnosis model is trained by using the audio features to be diagnosed as samples and the sample target piecewise linear function corresponding to the audio features to be diagnosed as sample labels.

[0188] The noise reduction and recognition layer of the fault diagnosis model is trained by using the first audio feature of the sample as the sample and the fault diagnosis result of the sample corresponding to the first audio feature of the sample as the sample label.

[0189] Among them, the audio file to be diagnosed, the audio features to be diagnosed, the piecewise linear function of the target sample, and the fault diagnosis results of the sample are all data obtained in advance.

[0190] In actual operation, sound data of mechanical rotation can be recorded, and the recorded sound data can be classified into two categories based on whether the machine is faulty.

[0191] The sound data is randomly input into the fault diagnosis model in batches for training.

[0192] During training, an iteration cycle or classification accuracy threshold can be preset. When this iteration cycle or classification accuracy threshold is reached, training is terminated and the fault diagnosis model is saved, thus completing the training of the fault diagnosis model.

[0193] Understandably, each audio feature to be diagnosed, the target piecewise linear function, and the fault diagnosis result generated by the fault diagnosis model can be used as sample data for the next model training process. As the sample data gradually expands, the final output diagnosis result will become more accurate.

[0194] In practical applications, the fault diagnosis model is deployed in mechanical or bearing testing equipment. When the machine is started, the fault diagnosis model is loaded and the sound recording device is turned on, continuously inputting the recorded sound into the fault diagnosis model.

[0195] The output of the fault diagnosis model can be used to determine whether the current equipment is faulty.

[0196] The audio-based fault diagnosis method provided by the present invention dynamically generates a target piecewise linear function with non-fixed parameter characteristics that can respond to noise signals in the audio file under test through an attention mechanism. This allows for the generation of different thresholds and slopes based on different noise signals, thereby improving the response of the target piecewise linear function to external noise signals and effectively removing noise. In addition, by preserving the gradient within the threshold, useful information of the noise signal within the threshold can be extracted to improve the accuracy of the noise reduction results, thereby improving the accuracy of the fault diagnosis results. This method has universality and flexibility.

[0197] The audio-based fault diagnosis device provided in the embodiments of the present invention will be described below. The audio-based fault diagnosis device described below and the audio-based fault diagnosis method described above can be referred to in correspondence.

[0198] like Figure 7 As shown, the audio-based fault diagnosis device includes: a first processing module 710, a second processing module 720, and a third processing module 730.

[0199] The first processing module 710 is used to extract features from the audio file to be diagnosed, and obtain the audio features to be diagnosed.

[0200] The second processing module 720 is used to obtain the first audio feature based on the target piecewise linear function and the audio feature to be diagnosed, wherein the parameters of the target piecewise linear function are determined by attention transformation based on the audio feature to be diagnosed.

[0201] The third processing module 730 is used to obtain fault diagnosis results based on the first audio features.

[0202] In some embodiments, the parameters of the target piecewise linear function are determined by the following method:

[0203] Feature extraction is performed on the audio features to be diagnosed to obtain multiple sets of second audio features;

[0204] Attention transformation is performed on multiple sets of second audio features to obtain the parameters of the target piecewise linear function.

[0205] In some embodiments, attention transformation is performed on multiple sets of second audio features to obtain the parameters of the target piecewise linear function, including:

[0206] The two sets of second audio features are subjected to absolute value, global average pooling, fully connected, batch regularization, deep residual, fully connected processing and mapping processing respectively to obtain the first slope of the target piecewise linear function within the threshold and the threshold of the target piecewise linear function.

[0207] The other two sets of second audio features are processed by absolute value, global average pooling, fully connected, batch regularization, deep residual and fully connected methods respectively to obtain the third slope and the second slope of the target piecewise linear function outside the threshold.

[0208] In some embodiments, the second processing module 720 is further configured to:

[0209] Feature extraction is performed on the audio features to be diagnosed to obtain multiple sets of second audio features;

[0210] The third audio feature is obtained by extracting features from multiple sets of second audio features;

[0211] The residual value is obtained based on the target piecewise linear function and the third audio feature;

[0212] The first audio feature is obtained by summing the residual value and the audio feature to be diagnosed.

[0213] In some embodiments, the second processing module 720 is further configured to:

[0214] The audio features to be diagnosed are sequentially processed by batch regularization, depth residual and multi-kernel convolution to obtain multiple sets of second audio features.

[0215] In some embodiments, the second processing module 720 is further configured to:

[0216] The third audio features are obtained by batch regularization, depth residual and single kernel convolution processing on multiple sets of second audio features.

[0217] In some embodiments, the third processing module 730 is further configured to:

[0218] The first audio feature is processed sequentially by batch regularization, deep residual, global average pooling and matrix transformation to obtain the fourth audio feature.

[0219] Based on the fourth audio feature, the fault diagnosis result is obtained.

[0220] The audio-based fault diagnosis device provided in this embodiment of the invention dynamically generates a target piecewise linear function with non-fixed parameter characteristics that can respond to noise signals in the audio file under test through an attention mechanism. This allows for the generation of different thresholds and slopes based on different noise signals, thereby improving the response of the target piecewise linear function to external noise signals and effectively removing noise. In addition, by preserving the gradient within the threshold, useful information of the noise signal within the threshold can be extracted to improve the accuracy of the noise reduction results, thereby improving the accuracy of the fault diagnosis results. This device is universal and flexible.

[0221] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute an audio-based fault diagnosis method. This method includes extracting features from the audio file to be diagnosed to obtain audio features to be diagnosed; obtaining a first audio feature based on the audio features to be diagnosed and a target piecewise linear function, wherein the parameters of the target piecewise linear function are determined by attention transformation based on the audio features to be diagnosed; and obtaining a fault diagnosis result based on the first audio feature.

[0222] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0223] Furthermore, this invention discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the audio-based fault diagnosis method provided in the above-described method embodiments. The method includes extracting features from an audio file to be diagnosed to obtain audio features to be diagnosed; obtaining a first audio feature based on the audio features to be diagnosed and a target piecewise linear function, wherein the parameters of the target piecewise linear function are determined by attention transformation based on the audio features to be diagnosed; and obtaining a fault diagnosis result based on the first audio feature.

[0224] On the other hand, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the audio-based fault diagnosis method provided in the above embodiments. The method includes extracting features from an audio file to be diagnosed to obtain audio features to be diagnosed; obtaining a first audio feature based on the audio features to be diagnosed and a target piecewise linear function, wherein the parameters of the target piecewise linear function are determined by attention transformation based on the audio features to be diagnosed; and obtaining a fault diagnosis result based on the first audio feature.

[0225] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0226] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0227] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0228] The above embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Although the invention has been described in detail with reference to the embodiments, those skilled in the art should understand that various combinations, modifications, or equivalent substitutions of the technical solutions of the invention do not depart from the spirit and scope of the invention and should be covered within the scope of the claims of the invention.

Claims

1. An audio-based fault diagnosis method, characterized in that, include: Feature extraction is performed on the audio file to be diagnosed to obtain the audio features to be diagnosed; Based on the audio features to be diagnosed and the target piecewise linear function, a first audio feature is obtained, and the parameters of the target piecewise linear function are determined by attention transformation based on the audio features to be diagnosed. Based on the first audio feature, the fault diagnosis result is obtained; The parameters of the target piecewise linear function are determined by the following method: Feature extraction is performed on the audio features to be diagnosed to obtain multiple sets of second audio features; Attention transformation is performed on the multiple sets of second audio features to obtain the parameters of the target piecewise linear function; The attention transformation of the multiple sets of second audio features to obtain the parameters of the target piecewise linear function includes: The two sets of second audio features are subjected to absolute value, global average pooling, fully connected, batch regularization, deep residual, fully connected processing and mapping processing respectively to obtain the threshold of the target piecewise linear function and the first slope of the target piecewise linear function within the threshold. The other two groups of the second audio features are processed by absolute value, global average pooling, fully connected, batch regularization, deep residual and fully connected processing respectively to obtain the second slope and the third slope of the target piecewise linear function outside the threshold.

2. The audio-based fault diagnosis method according to claim 1, characterized in that, The process of obtaining the first audio feature based on the audio features to be diagnosed and the target piecewise linear function includes: Feature extraction is performed on the audio features to be diagnosed to obtain multiple sets of second audio features; The third audio feature is obtained by extracting features from the multiple sets of second audio features; Based on the third audio feature and the target piecewise linear function, the residual value is obtained; The first audio feature is obtained by summing the audio feature to be diagnosed and the residual value.

3. The audio-based fault diagnosis method according to claim 2, characterized in that, The process of extracting features from the audio features to be diagnosed yields multiple sets of second audio features, including: The audio features to be diagnosed are subjected to batch regularization, depth residual and multi-kernel convolution to obtain multiple sets of second audio features.

4. The audio-based fault diagnosis method according to claim 2, characterized in that, The step of extracting features from the multiple sets of second audio features to obtain third audio features includes: The multiple sets of second audio features are subjected to batch regularization, depth residual, and single-kernel convolution to obtain the third audio features.

5. The audio-based fault diagnosis method according to any one of claims 1-4, characterized in that, The process of obtaining fault diagnosis results based on the first audio feature includes: The first audio feature is processed by batch regularization, deep residual, global average pooling and matrix transformation to obtain the fourth audio feature. Based on the fourth audio feature, the fault diagnosis result is obtained.

6. An audio-based fault diagnosis device, characterized in that, include: The first processing module is used to extract features from the audio file to be diagnosed, thereby obtaining the audio features to be diagnosed. The second processing module is used to obtain a first audio feature based on the audio feature to be diagnosed and the target piecewise linear function, wherein the parameters of the target piecewise linear function are determined by attention transformation based on the audio feature to be diagnosed. The third processing module is used to obtain fault diagnosis results based on the first audio features; The parameters of the target piecewise linear function are determined by the following method: Feature extraction is performed on the audio features to be diagnosed to obtain multiple sets of second audio features; Attention transformation is performed on the multiple sets of second audio features to obtain the parameters of the target piecewise linear function; The attention transformation of the multiple sets of second audio features to obtain the parameters of the target piecewise linear function includes: The two sets of second audio features are subjected to absolute value, global average pooling, fully connected, batch regularization, deep residual, fully connected processing and mapping processing respectively to obtain the threshold of the target piecewise linear function and the first slope of the target piecewise linear function within the threshold. The other two groups of the second audio features are processed by absolute value, global average pooling, fully connected, batch regularization, deep residual and fully connected processing respectively to obtain the second slope and the third slope of the target piecewise linear function outside the threshold.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the audio-based fault diagnosis method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the audio-based fault diagnosis method as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the audio-based fault diagnosis method as described in any one of claims 1 to 5.

Images