Coal mine audio data fault diagnosis method based on transfer learning

Abstract

The invention relates to a coal mine audio data fault diagnosis method based on transfer learning. The method comprises the following steps: performing MFCC feature extraction on original coal mine equipment audio signals under different working conditions; establishing a cross-domain diagnosis network model; the feature extraction layer extracts universal features among different domains, and calculates loss values of a cross entropy loss function and a multi-kernel joint distribution difference function; updating model parameters according to the optimal gradient direction; repeating the network updating strategy until an expected diagnosis accuracy rate is reached or a set number of iterations is completed; and outputting a fault diagnosis result of the target domain test set. According to the method, multiple linear mapping is carried out on the features containing category information, meanwhile, the distribution difference of different working condition samples is considered, iterative training is carried out to reduce the maximum mean value difference between the domains so as to realize unsupervised feature matching, the improved migration model can better identify the audio fault category of the target domain, and the robustness of the target domain is improved. The problem of unsupervised audio data fault diagnosis is effectively solved.

Description

Technical Field

[0001] The present invention belongs to the field of coal mine intelligence, and in particular relates to a coal mine audio data fault diagnosis method based on transfer learning. Background Art

[0002] With the rapid development of industrial big data and artificial intelligence technology, the prediction and health management (PHM) of industrial equipment has become increasingly important in modern manufacturing, and the application of intelligent maintenance systems has become more and more common. The underground production and operation environment of coal enterprises is harsh, and their key equipment is often in a state of multiple collisions, easy wear, and instability. Long-term operation of mining equipment is prone to defects such as equipment wear and fracture, pitting and spalling, bearing damage, etc., causing inestimable losses to coal mine production. How to quickly and accurately diagnose equipment status is an inevitable requirement for the modernization and intelligent development of coal enterprises.

[0003] In traditional audio fault diagnosis research, it is mainly divided into the extraction and classification of audio data features, feature extraction through spectrum analysis and time-frequency domain transform analysis, and fault identification using classifiers such as K-nearest neighbor algorithm, neural network, support vector machine, random forest, etc. However, these methods are too dependent on expert knowledge and manual experience, and it is difficult to extract effective fault features from the original signal. The obtained diagnostic model has a simple structure, and its learning ability is related to the accuracy and completeness of the sample data. When the diagnostic accuracy reaches a certain level, it is difficult to make further improvements. Summary of the invention

[0004] Based on the above defects, the present invention proposes a coal mine audio data fault diagnosis method based on transfer learning. First, the Mel-frequency cepstral coefficient (MFCC) feature extraction is performed on the audio signal of coal mine equipment to obtain a two-dimensional coefficient map of fault features. Secondly, a transfer learning fault diagnosis model is constructed, and the improved maximum mean difference algorithm is used as the metric standard. The joint distribution distance is calculated with the help of pseudo labels to reduce the data distribution difference. Finally, the model is applied to the underground space-constrained laboratory for verification. The results show that the proposed method can realize fault diagnosis under unlabeled conditions, and compared with different transfer learning algorithms, which reflects the feasibility and superiority of the present invention.

[0005] The present invention aims to solve the problem of inconsistent feature distribution of audio signals collected by coal mine equipment under variable working conditions and unlabeled samples to be diagnosed. It makes full use of the processing ability of Mel frequency cepstrum coefficients for audio signals and the excellent performance of transfer learning in cross-domain fault diagnosis. The feature map is used as the model input, and the distribution difference between different working condition data is reduced by adapting the marginal distribution and conditional distribution. It can give early warning when coal mine equipment fails to ensure the safety of underground workers and coal mine production, and has good engineering application prospects.

[0006] Currently, the training and updating of machine learning and deep learning models are highly dependent on data annotation. Although a large amount of data is available, most of this data is in unprocessed raw form, and the data that has been correctly manually annotated is extremely limited. The annotation process is time-consuming and costly. Most studies assume that training samples and test samples have the same feature space, probability distribution, and sufficient labels. However, in practical applications, due to environmental changes and system equipment reorganization, system conditions are prone to sudden changes, resulting in data distribution mismatch, which in turn affects the performance of fault diagnosis models. Although diagnostic models can be constructed with the help of knowledge from related data sets, when there are differences in distribution, traditional machine learning and deep learning methods cannot be directly applied, which brings challenges to model training and updating.

[0007] The present invention is achieved through the following technical solutions:

[0008] The present invention provides a coal mine audio data fault diagnosis method based on transfer learning, comprising the following steps:

[0009] (1) MFCC feature extraction is performed on the original audio signals of coal mining equipment in different working conditions, which are converted into two-dimensional feature maps and then normalized. The data set is divided into a training set and a test set according to a fixed ratio;

[0010] (2) Initialize model parameters, set the number of iterations, learning rate, and batch size hyperparameters, and establish a cross-domain diagnostic network model;

[0011] (3) The training set is input into the network model in batches, the feature extraction layer extracts common features between different domains, and the cross entropy loss function and the multi-core joint distribution difference function loss value are calculated;

[0012] (4) Use the back propagation algorithm to perform differential operations and update the model parameters according to the optimal gradient direction;

[0013] (5) Repeat the network update strategy of step (3) and step (4) until the expected diagnostic accuracy is achieved or the set number of iterations is completed;

[0014] (6) Verify the model performance and output the fault diagnosis results of the target domain test set.

[0015] Among them, in step (1), MFCC feature extraction specifically includes the following steps: first, the original audio signal is preprocessed by framing and windowing; then the processed data is subjected to fast Fourier transform (FFT) to obtain an energy spectrum, and filtered through multiple filters to calculate the logarithmic energy; finally, discrete cosine transform (DCT) is applied to generate MFCC features.

[0016] Furthermore, the original audio signal is preprocessed using Mel frequency cepstral coefficients. Mel frequency cepstral coefficients are a widely used feature extraction method commonly used in audio data analysis. They are mainly based on the auditory characteristics of the human ear, combined with auditory perception and speech generation mechanisms, and can effectively filter irrelevant audio information and extract useful signal features. Mel frequency cepstral coefficients are extracted through the Mel scale frequency domain. The Mel scale describes the nonlinear characteristics of frequency, and its relationship with frequency f can be approximately expressed by the following formula:

[0017] Mel(f)=2595×log 10 (1+f / 700) (Formula 1)

[0018] In the formula, f represents frequency, Mel(f) represents Mel frequency, and the Mel frequency cepstral coefficient has a nonlinear correspondence with the frequency. The Mel frequency cepstral coefficient has a nonlinear correspondence with the frequency. Its powerful nonlinear transformation makes the Mel frequency cepstral coefficient have stronger noise resistance than other audio features, so it shows good application prospects in audio feature extraction tasks.

[0019] Among them, in step (3), the loss function contains two optimization objectives during the model training process:

[0020] By minimizing the loss function J of the audio fault diagnosis model on the source domain dataset c , guide the model to output accurate category prediction labels as much as possible, and set the multi-classification cross entropy as the loss function J c , the formula is

[0021]

[0022] Where m is a batch data size, n is the number of categories, y is the independent encoding of the true label, and a is the output prediction probability of a certain category;

[0023] During the training process, the adaptive layer minimizes the multi-core joint distribution difference function J of the source domain and target domain datasets. m , then the total loss function J of the model is

[0024] J=J c +αJ m (Formula 8)

[0025] Where α is a balancing hyperparameter, and pseudo labels need to be used to calculate J m .

[0026] Furthermore, the multi-core joint distribution difference function is an improvement based on the maximum mean difference algorithm.

[0027] The maximum mean discrepancy (MMD) algorithm is a commonly used transfer learning adaptation method, which is used to measure the non-parametric distance of different distribution differences. s , D t For example, the MMD expression is

[0028]

[0029] Where D s is the source domain, D t is the target domain, x s and x t Yes D s , D t The sample instance in is the i-th sample in the source domain, is the corresponding fault type label, n s is the amount of fault sample data in the source domain, n t is the amount of fault sample data in the target domain. The mean difference between the two distributions in the Hilbert space H is used as the data distribution distance in the two domains. φ(·) is a nonlinear mapping function in RKHS. The dot product operation of φ(·) is replaced by a kernel function. Generally, a Gaussian kernel function is used to map the original features to a multidimensional space. The MMD distance of different data distributions is calculated in the same feature space. The mathematical expression is:

[0030] k(x s ,x t ) = exp(-||x s -x t || 2 / 2σ 2 ) (Formula 3)

[0031] Where σ represents the kernel width, k(·) is the kernel function, and the kernel function is used to perform nonlinear mapping in the form of inner product. The expression can be rewritten as

[0032]

[0033] The MMD method is based on a single kernel transformation. In actual modeling applications, different kernel parameters directly lead to data features being projected into different dimensional spaces. The multi-kernel joint distribution difference function (MK-MMD) obtains the optimal kernel through a linear combination of multiple kernels. u to calculate the optimal estimate in high-dimensional space.

[0034]

[0035] where β uis the weighted parameter of different cores, ensuring that the generated multi-core k is unique. By combining different cores, multi-core k can improve the mapping effect of the maximum mean difference algorithm, thereby achieving optimal and reasonable core selection.

[0036] In a specific layer of the model, since the feature representation of the data and its corresponding categories are unevenly mixed, the above MMD measurement method only attempts to align the marginal distribution, and it is difficult to obtain a measurement mapping related only to a single distribution, which may cause the loss of fine-grained feature information of each category. Inspired by the joint adaptive network, in order to enhance the relevance of sample data with the same label in different fields, the product difference between the source domain and target domain sample labels and the corresponding sample features is calculated to achieve this. The improved inter-domain distribution difference multi-core joint maximum mean measurement MKJMMD square distance expression is defined as

[0037]

[0038] Where z sl , z tl Represent the activation values ​​of layer l in the source domain and target domain respectively, representing the characteristic part of the sample data, Represents l in the source domain and target domain respectively c The activation value of the layer reflects the part of the sample data that is related to the label.

[0039] The beneficial effects of the present invention are:

[0040] (1) Aiming at the problem that the characteristic distribution of audio signals collected from coal mine equipment under variable working conditions is inconsistent and the samples to be diagnosed do not contain labels, a coal mine audio data fault diagnosis method based on transfer learning is proposed, which makes full use of the MFCC's processing ability for non-stationary signals and the excellent performance of transfer learning in cross-domain fault diagnosis;

[0041] (2) Taking the audio signal feature map as the model input, on the basis of traditional supervised transfer learning, multiple linear mappings are performed on the features containing category information. Meanwhile, considering the distribution differences of samples under different working conditions, iterative training is performed to reduce the maximum mean difference between domains, thereby achieving unsupervised feature matching. The improved transfer model can better identify the audio fault category in the target domain and effectively solve the problem of unsupervised audio data fault diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of the coal mine audio data fault diagnosis method based on transfer learning of the present invention;

[0043] Figure 2 A schematic diagram of a coal mine audio fault diagnosis model of the present invention;

[0044] Figure 3This is the accuracy change trend in the migration task 1-2 of the present invention. DETAILED DESCRIPTION

[0045] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0046] like Figure 1 As shown, the present invention provides a coal mine audio data fault diagnosis method based on transfer learning, comprising the following steps:

[0047] (1) MFCC feature extraction is performed on the original audio signals of coal mining equipment in different working conditions, which are converted into two-dimensional feature maps and then normalized. The data set is divided into a training set and a test set according to a fixed ratio, where the training set includes a source domain training set and a target domain training set, and the test set includes a target domain test set;

[0048] (2) Initialize model parameters, set the number of iterations, learning rate, and batch size hyperparameters, and establish a cross-domain diagnostic network model;

[0049] (3) The training set is input into the network model in batches, the feature extraction layer extracts common features between different domains, and the cross entropy loss function and the multi-core joint distribution difference function loss value are calculated;

[0050] (4) Use the back propagation algorithm to perform differential operations and update the model parameters according to the optimal gradient direction;

[0051] (5) Repeat the network update strategy of step (3) and step (4) until the expected diagnostic accuracy is achieved or the set number of iterations is completed;

[0052] (6) Verify the model performance and output the fault diagnosis results of the target domain test set.

[0053] Specifically, in step (1), MFCC feature extraction specifically includes the following steps: first, the original audio signal is preprocessed by framing and windowing; then, the processed data is fast Fourier transformed to obtain an energy spectrum, and filtered through multiple filters to calculate the logarithmic energy; finally, discrete cosine transform is applied to generate MFCC features;

[0054] The original audio signal is preprocessed using Mel frequency cepstral coefficients. Mel frequency cepstral coefficients are a widely used feature extraction method commonly used in audio data analysis. They are mainly based on the auditory characteristics of the human ear, combined with auditory perception and speech generation mechanisms, and can effectively filter irrelevant audio information and extract useful signal features. Mel frequency cepstral coefficients are extracted in the Mel scale frequency domain. The Mel scale describes the nonlinear characteristics of frequency, and its relationship with frequency f can be approximately expressed by the following formula:

[0055] Mel(f)=2595×log 10 (1+f / 700) (Formula 1)

[0056] In the formula, f represents frequency, Mel(f) represents Mel frequency, and the Mel frequency cepstral coefficient has a nonlinear correspondence with the frequency. The Mel frequency cepstral coefficient has a nonlinear correspondence with the frequency. Its powerful nonlinear transformation makes the Mel frequency cepstral coefficient have stronger noise resistance than other audio features, so it shows good application prospects in audio feature extraction tasks.

[0057] Specifically, in step (2), the quantitative study on transferability features indicates that the common features of the source domain and the target domain are mainly extracted in the general feature extraction layer, while the feature differences between different domains are mainly reflected in the cross-domain adaptation layer. Based on this theory, the method model proposed in the present invention is as follows: Figure 2 As shown in the figure, the two network parameters of the source domain and the target domain are shared in the data preprocessing part. The device audio signal collected by the camera is used to generate two-dimensional feature map data through MFCC. Secondly, in order to simplify the complex network parameter adjustment process, the general feature extraction layer uses the classic Lenet-5 model to learn the transfer knowledge. The overall model structure only includes convolutional layers (Conv), maximum pooling layers (Max Pooling) and fully connected layers (Dense). Two fully connected layers are added after Lenet-5 as bottleneck layers, which are set as cross-domain adaptive layers. Finally, the audio category prediction label is output through the softmax activation function. Combined with the true label Y, the cross-domain loss and cross entropy loss are calculated to perform migration feature learning. The detailed parameters of the entire network model are shown in Table 1 below:

[0058] Table 1 Model parameters

[0059]

[0060] Specifically, in step (3), the model of the present invention realizes the extraction of common features through deep feature representation learning and multi-core joint maximum mean difference. During the model training process, the loss function contains two optimization objectives: the loss function J c, multi-kernel joint distribution difference function J m ,

[0061] By minimizing the loss function J of the audio fault diagnosis model on the source domain dataset c , guide the model to output accurate category prediction labels as much as possible, and set the multi-classification cross entropy as the loss function J c , the formula is

[0062]

[0063] Where m is a batch data size, n is the number of categories, y is the independent encoding of the true label, and a is the output prediction probability of a certain category;

[0064] During the training process, the adaptive layer minimizes the multi-core joint distribution difference function J of the source domain and target domain datasets. m , then the total loss function J of the model is

[0065] J=J c +αJ m (Formula 8)

[0066] Where α is a balancing hyperparameter, and pseudo labels need to be used to calculate J m Due to the poor ability to extract common features between domains and low classification accuracy in the early stage of model training, the model training process of the present invention adopts a sequential increase method to multiply the balance hyperparameter from 0 to 1 incrementally. And e varies from 0 to 1 with the number of model iterations. As the number of model iterations increases and the feature extraction capability improves, the balancing hyperparameter gradually increases as a way to enhance the loss weight of the inter-domain distribution difference.

[0067] Furthermore, the multi-core joint distribution difference function is an improvement based on the maximum mean difference algorithm.

[0068] The maximum mean discrepancy (MMD) algorithm is a commonly used transfer learning adaptation method, which is used to measure the non-parametric distance of different distribution differences. s , D t For example, the MMD expression is

[0069]

[0070] Where D s is the source domain, D t is the target domain, x s and x t Yes D s , D t The sample instance in is the i-th sample in the source domain, is the corresponding fault type label, n s is the amount of fault sample data in the source domain, n t is the amount of fault sample data in the target domain. The mean difference between the two distributions in the Hilbert space H is used as the data distribution distance in the two domains. φ(·) is a nonlinear mapping function in RKHS. The dot product operation of φ(·) is replaced by a kernel function. Generally, a Gaussian kernel function is used to map the original features to a multidimensional space. The MMD distance of different data distributions is calculated in the same feature space. The mathematical expression is:

[0071]

[0072] Where σ represents the kernel width, k(·) is the kernel function, and the kernel function is used to perform nonlinear mapping in the form of inner product. The expression can be rewritten as

[0073]

[0074] The MMD method is based on a single kernel transformation. In actual modeling applications, different kernel parameters directly lead to data features being projected into different dimensional spaces. The multi-kernel joint distribution difference function (MK-MMD) obtains the optimal kernel through a linear combination of multiple kernels. u to calculate the optimal estimate in high-dimensional space.

[0075]

[0076] where β u is the weighted parameter of different cores, ensuring that the generated multi-core k is unique. By combining different cores, multi-core k can improve the mapping effect of the maximum mean difference algorithm, thereby achieving optimal and reasonable core selection.

[0077] In a specific layer of the model, since the feature representation of the data and its corresponding categories are unevenly mixed, the above MMD measurement method only attempts to align the marginal distribution, and it is difficult to obtain a measurement mapping related only to a single distribution, which may cause the loss of fine-grained feature information of each category. Inspired by the joint adaptive network, in order to enhance the relevance of sample data with the same label in different fields, the product difference between the source domain and target domain sample labels and the corresponding sample features is calculated to achieve this. The improved inter-domain distribution difference multi-core joint maximum mean measurement MKJMMD square distance expression is defined as

[0078]

[0079] Where z sl , z tl Represent the activation values ​​of layer l in the source domain and target domain respectively, representing the characteristic part of the sample data, Represents l in the source domain and target domain respectively c The activation value of the layer reflects the part of the sample data that is related to the label.

[0080] This algorithm considers both the feature space and label space of the sample, matches the global distribution and the local distribution between classes by aligning the marginal distribution and conditional distribution between domains, and unlike the above-mentioned MMD algorithm, which imposes a uniform weight on the kernel function, the improved multi-core joint maximum mean metric MKJMMD algorithm uses non-uniform weights, which reflects the influence of the sample label layer and promotes the information fusion of samples of the same class. l This method is ultimately simplified to the initial MK-MMD algorithm. It should be noted here that the source domain sample data uses uniquely encoded labels for cross entropy loss calculation, because the target domain data is unlabeled. When calculating the above distribution difference, the target domain uses the predicted output of the model classifier as the label of each target domain data. Although there may be cases of incorrectly predicted labels, with the continuous iteration of the classifier, most samples will be labeled correctly.

[0081] The network framework calls the Pytorch library file to simplify model construction, and uses the Tesla v100 configuration for model parameter training. The batch size of the training data is set to 64, and the Adam optimization algorithm is selected to iterate the model parameters for 300 epochs. The learning rate and other hyperparameters adopt a step-by-step reduction strategy, and the initial learning rate parameter is set to 0.001. Considering that the features extracted by the network model in the early iteration process are unstable during the entire training process, the entire migration model is trained on the source domain data for a certain number of rounds before cross-domain transfer learning is performed, so that the cross-domain model has a certain diagnostic ability for audio features and improves the accuracy of predicted labels.

[0082] The experimental data is the audio data set of belt conveyor equipment provided by the underground confined space laboratory, and the actual working conditions are simulated with different speed data. The audio data set of coal mine equipment is collected by sampling the KBA18F camera pickup under different load conditions (500r / min, 700r / min and 900r / min). The data set contains 4 different operating state samples, including normal state (n), roller inner ring fault (ib), roller outer ring fault (ob) and rolling element fault (tb).

[0083] Table 2 shows the divided coal mining equipment audio cross-domain learning task dataset, and different working conditions are regarded as different transfer learning tasks. For example, task 0-1 indicates the position data with a speed equal to 500rpm in the source domain and the position with a speed of 700rpm in the target domain.

[0084] Table 2 Audio data table of underground confined space laboratory migration task

[0085]

[0086] Specifically, in step (6), model-based transfer learning is one of the more popular cross-domain diagnosis methods. It uses labeled data in the target domain to fine-tune the pre-trained network parameters, and then uses the fine-tuned model to directly test new data in the target domain. However, for unsupervised transfer learning fault diagnosis methods, labels in the target domain are not available, and all fine-tuning-based models do not have real labels for loss calculation, so it is impossible to perform differential operations to optimize the gradient direction to update the model network parameters. The present invention sets the method of using only sample labels in the source domain to train the model without using additional samples in the target domain as the baseline model (Base), and uses test samples in the target domain to evaluate the fault diagnosis performance of the model. The source domain and the target domain share the same model structure and parameter settings.

[0087] Five unsupervised cross-domain algorithms are selected to compare and verify the fault diagnosis results of the model proposed in this invention. Considering the fairness of the comparison and verification algorithm, the diagnosis model parameter settings and hardware facilities are configured the same. Table 3 shows the average accuracy of the diagnosis model.

[0088] Table 3 Diagnostic accuracy of different algorithms

[0089]

[0090] For the audio data of the underground confined space laboratory, all the transfer learning fault methods compared by the present invention can improve the accuracy of the Base method, except for the CORAL method. The CORAL algorithm can only improve the accuracy in some single cross-domain tasks, and the average recognition accuracy is 58.71%, which is lower than the Base method. The accuracy of each transfer task is unstable, and the model as a whole has a negative transfer phenomenon. For the AdaBN algorithm, although the average accuracy is slightly higher than that of the Base, its improvement is much smaller than that of other methods. Among them, the MK-MMD and DANN algorithms align the overall data distribution and reduce the difference in edge distribution. The fault diagnosis accuracy is 94.42% and 91.22%, and the cross-domain fault diagnosis performance is better than that of the AdaBN algorithm.

[0091] Figure 3 This is the accuracy trend of the algorithm MKJMMD in the migration task 1-2. According to the model training settings, only the source domain data is used to calculate the cross entropy loss J in the first 50 epochs. c, and the joint distribution difference calculation is not performed on the target domain, so as to improve the probability of the model correctly predicting the output. It can be seen from the figure that before cross-domain transfer learning, the model can achieve good diagnostic effects on both the source domain audio data training set and the test set, and the fault diagnosis rate of the target domain data test set is also close to 0.8, indicating that the model can initially extract the audio features of the data. After 50 epochs, the cross-domain distribution difference J is calculated. d , the common features between domains are extracted, and the accuracy of the target domain is greatly improved, which verifies the effectiveness of the algorithm of the present invention for cross-domain fault diagnosis tasks.

[0092] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A coal mine audio data fault diagnosis method based on transfer learning, characterized in that: The steps include: (1) MFCC feature extraction is performed on the original audio signals of coal mining equipment under different working conditions, which are converted into two-dimensional feature maps and then normalized. The data set is divided into a training set and a test set according to a fixed ratio; (2) Initialize model parameters, set the number of iterations, learning rate, and batch size hyperparameters, and establish a cross-domain diagnostic network model; (3) The training set is input into the network model in batches, the feature extraction layer extracts common features between different domains, and the cross entropy loss function and the multi-core joint distribution difference function loss value are calculated; (4) Use the back propagation algorithm to perform differential operations and update the model parameters according to the optimal gradient direction; (5) Repeat the network update strategy of step (3) and step (4) until the expected diagnostic accuracy is achieved or the set number of iterations is completed; (6) Verify the model performance and output the fault diagnosis results of the target domain test set.

2. The coal mine audio data fault diagnosis method based on transfer learning according to claim 1 is characterized in that: In step (1), MFCC feature extraction specifically includes the following steps: first, the original audio signal is subjected to framing and windowing preprocessing operations; then, the processed data is subjected to fast Fourier transform to obtain an energy spectrum, and is filtered through multiple filters to calculate logarithmic energy; finally, discrete cosine transform is applied to generate MFCC features.

3. The coal mine audio data fault diagnosis method based on transfer learning according to claim 2 is characterized in that: Preprocessing the original audio signal specifically includes: The original audio signal is extracted through the Mel frequency cepstrum coefficient, and its relationship with the frequency f can be approximately expressed by the following formula: Mel(f)=2595×log 10 (1+f / 700) (Formula 1) Where f represents frequency, Mel(f) represents Mel frequency, and the Mel frequency cepstrum coefficient has a nonlinear corresponding relationship with the frequency.

4. The coal mine audio data fault diagnosis method based on transfer learning according to claim 2 is characterized in that: In step (3), the loss function contains two optimization objectives during model training: By minimizing the loss function J of the audio fault diagnosis model on the source domain dataset c , guide the model to output accurate category prediction labels as much as possible, and set the multi-classification cross entropy as the loss function J c , the formula is Where m is a batch data size, n is the number of categories, y is the independent encoding of the true label, and a is the output prediction probability of a certain category; During the training process, the adaptive layer minimizes the multi-core joint distribution difference function J of the source domain and target domain datasets. m , then the total loss function J of the model is J=J c +αJ m (Formula 8) Where α is a balancing hyperparameter, and pseudo labels need to be used to calculate J m .

5. The coal mine audio data fault diagnosis method based on transfer learning according to claim 4 is characterized in that: The multi-core joint distribution difference function is an improvement based on the maximum mean difference algorithm. The maximum mean difference algorithm is used to measure the non-parametric distance of differences between different distributions. The maximum mean difference algorithm expression is: Where D s is the source domain, D t is the target domain, x s and x t Yes D s , D t The sample instance in is the i-th sample in the source domain, is the corresponding fault type label, n s is the amount of fault sample data in the source domain, n t is the amount of fault sample data in the target domain. The mean difference between the two distributions in the Hilbert space H is used as the data distribution distance in the two domains. φ(·) is a nonlinear mapping function in RKHS. The dot product operation of φ(·) is replaced by a kernel function. Generally, a Gaussian kernel function is used to map the original features to a multidimensional space. The MMD distance of different data distributions is calculated in the same feature space. The mathematical expression is: k(x s ,x t ) = exp(-||x s -x t || 2 / 2σ 2 ) (Formula 3) Where σ represents the kernel width, k(·) is the kernel function, and the kernel function is used to perform nonlinear mapping in the form of inner product. The expression can be rewritten as The multi-core joint distribution difference function is to obtain the optimal kernel through the linear combination of multiple kernels, using m kernels k u to calculate the optimal estimate in high-dimensional space. where β u is the weighted parameter of different cores, ensuring that the generated multi-core k is unique. By combining different cores, multi-core k can improve the mapping effect of the maximum mean difference algorithm, thereby achieving optimal and reasonable core selection; This is achieved by calculating the product difference between the source domain and target domain sample labels and the corresponding sample features. The improved inter-domain distribution difference multi-kernel joint maximum mean metric MKJMMD square distance expression is defined as Where z sl , z tl Represent the activation values ​​of layer l in the source domain and target domain respectively, representing the characteristic part of the sample data, Represents l in the source domain and target domain respectively c The activation value of the layer reflects the part of the sample data that is related to the label.

Images