Online Fault Detection Method and System for Mining Submersible Pumps Based on Audio Signals
By converting audio signals into Gammatone spectrograms and combining them with the MobileNetV3 network framework, the problems of strong dependence on feature extraction and insufficient data in fault diagnosis of mining submersible pumps are solved, and highly accurate and adaptive online fault detection is achieved.
Patent Information
- Application Number
- CN202410394634.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-02
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-04-02
AI Technical Summary
Existing technologies for fault diagnosis of mining submersible pumps suffer from strong dependence on feature extraction and uncertainty in model generalization ability. Audio signal detection lacks datasets and has low accuracy under small sample conditions, resulting in poor fault diagnosis.
Audio signals are converted into Gammatone spectrograms, and fault diagnosis is performed by combining clustering and classification branches using the MobileNetV3 network framework. K-means clustering is used to generate pseudo-labels for training, and the network parameters are optimized to adapt to new fault conditions.
It improves the real-time performance and accuracy of fault diagnosis, realizes non-contact and non-invasive measurement, has strong adaptability, and is suitable for intelligent operation and maintenance of mining submersible pumps.
Smart Images

Figure CN118442297B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of fault detection, pattern recognition and image processing technology, and in particular to an online fault detection method and system for mining submersible pumps based on audio signals. Background Technology
[0002] Submersible pumps play a crucial role in the mining industry, primarily used for drainage and slag removal, providing cooling water, and transporting slurry. They effectively pump out accumulated water from mines and mining areas, ensuring a safe and dry underground working environment. Simultaneously, by transporting groundwater, they provide cooling for process equipment, preventing overheating. Furthermore, submersible pumps can be used to transport slurry, improving ore transport efficiency. In emergencies, they can also serve as emergency drainage tools, ensuring the normal operation and safe production of the mine.
[0003] Technological advancements have driven the development of submersible pumps for mining, propelling them towards large-scale, precision, and systematization. While the coordinated operation of multiple mechanical devices improves mining production efficiency, it also brings potential safety hazards. If a submersible pump malfunctions due to the combined effects of various factors and is not resolved promptly, it could lead to the shutdown of the mining production line, causing significant economic losses to the company, or even serious safety accidents such as explosions or leaks.
[0004] Therefore, in the field of mining submersible pumps, in-depth research into mechanical fault diagnosis technology has become particularly crucial. Various intelligent fault detection methods have been widely applied in the data acquisition of mining submersible pumps, providing key support for improving equipment reliability and reducing accident risks. These technologies not only effectively monitor the status of mining submersible pumps but also provide powerful tools for preventing potential faults, thus providing a strong guarantee for the continuous and stable production of mining enterprises. Against this backdrop, through advanced technical means, this invention enables a more comprehensive understanding and management of the operating status of mining submersible pumps, ensuring the continuous and efficient operation of mining production.
[0005] Typically, conventional methods involve extracting feature information from the vibration signals of mining submersible pumps and then combining this with machine learning techniques for pattern recognition. Although these traditional methods for diagnosing mining submersible pump faults have achieved some success, their reliance on feature extraction and the uncertainty of model generalization capabilities limit their practical application.
[0006] Audio signal detection, as an emerging fault diagnosis method, has the advantages of non-contact measurement and fewer required sensors, and is less expensive than vibration signal detection. However, obtaining audio data for faults in mining submersible pumps is relatively difficult, and publicly available datasets are lacking. Under small sample conditions, directly using deep learning algorithms may lead to model overfitting, resulting in poor performance and low accuracy in real-world tests. Summary of the Invention
[0007] To address the shortcomings of existing technologies, this invention provides an online fault detection method and system for mining submersible pumps based on audio signals. This method converts pump audio signals into Gammatone spectrograms, establishes a fault dataset, and transforms the fault diagnosis problem into image classification. Using a MobileNetV3-based network framework, the network output is divided into clustering and classification branches. The clustering branch inputs CNN features into a K-means clustering model to generate "pseudo-labels" for classifier training. Through classifier training and backpropagation, network parameters are optimized, enabling the fault diagnosis model to adapt to handling new, untrained fault conditions. This invention aims to improve the real-time performance and accuracy of fault diagnosis, making the system more intelligent and adaptable, and providing more reliable support for the operation and maintenance of mining submersible pumps.
[0008] On the one hand, an online fault detection method for mining submersible pumps based on audio signals is provided;
[0009] An online fault detection method for mining submersible pumps based on audio signals includes:
[0010] The audio signal of a mining submersible pump is acquired, and the acquired audio signal is segmented to obtain several audio segments. Each audio segment is converted into a spectrogram to construct a first submersible pump audio fault dataset. The first submersible pump audio fault dataset includes: several audio segments, each audio segment having a corresponding spectrogram.
[0011] The first submersible pump audio fault dataset is input into the first fault detection model to train the first fault detection model, resulting in the trained first fault detection model. The trained first fault detection model is then tested. If the test fails, the data samples whose classification accuracy during training is lower than a first set threshold are collected to obtain the second submersible pump audio fault dataset. The second submersible pump audio fault dataset includes several audio segments, each with a corresponding spectrogram. The second submersible pump audio fault dataset is then input into the first fault detection model to train the first fault detection model, resulting in the trained second fault detection model.
[0012] The parameters of the first fault detection model after training are averaged with the parameters of the second fault detection model after training, and the average value is used as the parameters of the first fault detection model to obtain the corrected first fault detection model.
[0013] The audio signal of the mining submersible pump to be tested is acquired and input into the corrected first fault detection model to obtain the fault detection category.
[0014] On the other hand, an online fault detection system for mining submersible pumps based on audio signals was provided;
[0015] An online fault detection system for mining submersible pumps based on audio signals includes:
[0016] The dataset construction module is configured to: acquire audio signals from mining submersible pumps, segment the acquired audio signals to obtain several audio segments, convert each audio segment into a spectrogram, and construct a first submersible pump audio fault dataset; the first submersible pump audio fault dataset includes: several audio segments, each audio segment having a corresponding spectrogram;
[0017] The first training module is configured to: input the first submersible pump audio fault dataset into the first fault detection model, train the first fault detection model, and obtain the trained first fault detection model; test the trained first fault detection model, and if the test fails, summarize the data samples whose classification accuracy is lower than a first set threshold during the training process to obtain the second submersible pump audio fault dataset; the second submersible pump audio fault dataset includes: several audio segments, each audio segment having a corresponding spectrogram;
[0018] The second training module is configured to: input the second submersible pump audio fault dataset into the first fault detection model, train the first fault detection model, and obtain the trained second fault detection model;
[0019] The correction module is configured to: average the parameters of the trained first fault detection model with the parameters of the trained second fault detection model, and use the average value as the parameters of the first fault detection model to obtain the corrected first fault detection model.
[0020] The classification module is configured to: acquire the audio signal of the mining submersible pump to be detected, input the audio signal of the mining submersible pump to be detected into the corrected first fault detection model, and obtain the fault detection category.
[0021] Furthermore, an electronic device is also provided, including:
[0022] Memory, used for non-transitory storage of computer-readable instructions; and
[0023] Processor, for executing the computer-readable instructions,
[0024] When the computer-readable instructions are executed by the processor, they perform the method described in the first aspect above.
[0025] In another aspect, a storage medium is also provided for non-transitory storage of computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the instructions of the method described in the first aspect are executed.
[0026] In another aspect, a computer program product is also provided, including a computer program that, when run on one or more processors, is used to implement the method described in the first aspect above.
[0027] The above technical solution has the following advantages or beneficial effects:
[0028] This invention employs an audio signal detection method, enabling non-contact and non-invasive measurement without disassembling the device;
[0029] We employ an unsupervised learning approach, doing almost no work to adapt it for end-to-end training on visual features on large-scale datasets;
[0030] This invention combines clustering learning and deep learning. The clustering branch and the classification branch share network parameters to perform iterative classification of data, which is more accurate than the classification by deep transfer learning through image pre-training.
[0031] This invention uses the lightweight MobileNetV3 network, which can prevent model overfitting. Its installation and diagnostic process does not affect the normal operation of the device. The device can continue to be diagnosed uninterrupted during operation. Moreover, its network structure is customizable, which makes it very flexible and adaptable to various application scenarios.
[0032] This invention introduces a Hard Sample (HS) learning strategy, which can quickly and adaptively learn and adjust the structure and parameters of the model to output the fault category. Attached Figure Description
[0033] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0034] Figure 1 This is a flowchart of the online fault detection method for intelligent mining submersible pumps based on audio signals in an embodiment of the present invention;
[0035] Figure 2 This is a schematic diagram of the audio spectrogram generation method in an embodiment of the present invention;
[0036] Figure 3 This is a clustering network model diagram in an embodiment of the present invention. Detailed Implementation
[0037] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.
[0038] Example 1
[0039] This embodiment provides an online fault detection method for mining submersible pumps based on audio signals;
[0040] like Figure 1 As shown, the online fault detection method for mining submersible pumps based on audio signals includes:
[0041] S101: Acquire the audio signal of the mining submersible pump, segment the acquired audio signal to obtain several audio segments, convert each audio segment into a spectrogram, and construct a first submersible pump audio fault dataset; the first submersible pump audio fault dataset includes: several audio segments, each audio segment having a corresponding spectrogram;
[0042] S102: Input the first submersible pump audio fault dataset into the first fault detection model, train the first fault detection model to obtain the trained first fault detection model; test the trained first fault detection model, and if the test fails, summarize the data samples whose classification accuracy is lower than the first set threshold during the training process to obtain the second submersible pump audio fault dataset; the second submersible pump audio fault dataset includes: several audio segments, each audio segment has a corresponding spectrogram;
[0043] S103: Input the second submersible pump audio fault dataset into the first fault detection model, train the first fault detection model, and obtain the trained second fault detection model;
[0044] S104: Average the parameters of the first fault detection model after training with the parameters of the second fault detection model after training, and use the average value as the parameters of the first fault detection model to obtain the corrected first fault detection model.
[0045] S105: Acquire the audio signal of the mining submersible pump to be detected, input the audio signal of the mining submersible pump to be detected into the corrected first fault detection model, and obtain the fault detection category.
[0046] Furthermore, S101: acquiring the audio signal of the mining submersible pump is achieved by using a sound sensor to acquire the audio signal of the mining submersible pump under different conditions; the different conditions include: the mining submersible pump in a healthy state and the mining submersible pump in a fault state.
[0047] For example, since submersible pumps are unlikely to experience many failures in the short term during production operation, it is necessary to simulate actual operating failure conditions by conducting destructive tests on the test submersible pump. A smart mining submersible pump was selected for fault detection, using a microphone as the sensor with a sampling frequency of 48kHz. The fixed-position measuring microphone was placed at a predetermined location during normal operation and various fault operation states, and the audio signals were collected and analyzed by a computer.
[0048] Furthermore, in S101: the acquired audio signal is segmented to obtain several audio segments, which is done according to a set duration, and each audio segment includes two audio cycles.
[0049] For example, given the relatively long length of the audio signals acquired in each state, these signals must be segmented. Simultaneously, considering the influence of the submersible pump's dynamic characteristics, each segmented signal should contain at least two complete reciprocating cycles. Therefore, this invention segments the audio signal into 0.5-second samples based on the number of reciprocating cycles of the submersible pump. Once the signal segmentation is complete, a Gammatone cepstral transform is performed to extract the corresponding Gammatone spectrogram, which is then used as a feature sample. The complete signal processing flow is as follows: Figure 2 As shown.
[0050] Further, step S101: converting each audio segment into a spectrogram specifically includes:
[0051] Each audio segment is processed sequentially with pre-emphasis, frame segmentation, windowing, filtering, and Fourier transform to obtain a spectrogram.
[0052] Furthermore, the pre-emphasis processing, through a high-pass filter, improves the signal-to-noise ratio by reducing the low-frequency components and enhancing the high-frequency components. Typically, pre-emphasis is achieved by applying a first-order difference equation.
[0053] Furthermore, the framing process involves dividing the pre-emphasized audio signal into several frames using overlapping sliding windows. Adjacent frames overlap to preserve contextual information between frames.
[0054] Furthermore, the windowing process involves applying a window function to each frame to reduce spectral leakage. Common window functions include the Hanning window and the Hamming window. After windowing, each sample in the frame is multiplied by the value of the window function.
[0055] Furthermore, the filtering process is implemented using a Gammatone filter. A Gammatone filter consists of a set of bandpass filters, each corresponding to a frequency band at the audible frequency range. The frequency response of each filter resembles a gamma distribution, and this filter bank can simulate the human ear's ability to distinguish frequencies of sound over a wide frequency range.
[0056] The impulse response h(t) of the Gammatone filter is as follows:
[0057] h(t) = ct n-1 e -2πbt cos(2πf0t+φ), t>0; (1)
[0058] Where c is the proportionality coefficient; n is the number of filter stages; b is the attenuation rate; f0 is the center frequency; φ is the phase. To simulate the characteristics of human hearing, n is usually set to 4 and φ to 0. The center frequency f0 of each filter is determined according to the range of human hearing frequency and the number of filters. The number of filters is selected as 24, and t represents the impulse response time.
[0059] Furthermore, the Fourier transform processing refers to processing using a Short-time Fourier Transform (STFT) to generate the corresponding Gammatone spectrogram STFT(ω,τ):
[0060]
[0061] Where ω is the normalized angular frequency, τ is the time delay coefficient, a Hanning window of length 128 is used as the window function ψ(t), x(t) represents the output of the audio signal after passing through the Gammatone filter, which is used as the input of the Fourier transform, ψ(t-τ) represents the convolution operation, and t represents the Fourier transform time.
[0062] It should be understood that the submersible pump audio signals acquired under each fault condition are segmented into multiple segments, and these segments are then converted into Gammatone spectrograms. In this way, the present invention transforms the fault diagnosis problem into an image classification problem and constructs an audio fault dataset for mining submersible pumps.
[0063] For each fault condition, the audio signal of the submersible pump is cropped into multiple segments, and the audio signal of each segment of the cropped submersible pump is transformed by Gammatone spectrum transformation to construct an audio fault dataset of mining submersible pumps. At this point, the fault diagnosis problem is transformed into an image classification problem.
[0064] A small sample dataset of audio faults in mining submersible pumps was constructed. Based on this, data augmentation was performed using the collected submersible pump audio signals. The audio signals collected under various fault conditions were segmented into multiple signal samples, ensuring that each segment contained at least two reciprocating motions of the submersible pump, so that the signal samples could better reflect the dynamic characteristics of the submersible pump. By performing Gammatone spectrogram transformation on the audio signals of each segment, a spectrogram of the submersible pump was established.
[0065] Then, the training set D needs to be partitioned from the original dataset. train Validation set D val and test set D test First, to verify the model's cross-task fault identification capability later, several types of data are reserved as D. test The remaining data is divided into D in a 6:4 ratio. train and D val .
[0066] Further, S102: as Figure 3 As shown, the first submersible pump audio fault dataset is input into the first fault detection model to train the first fault detection model, resulting in the trained first fault detection model; wherein, the first fault detection model includes:
[0067] A convolutional neural network (CNN) is provided, wherein the input of the CNN is used to input a spectrogram, the output of the CNN is connected to the input of a clustering layer, and the output of the CNN is also connected to the input of a classification layer. The clustering layer is used to output classification pseudo-labels corresponding to audio frequency bands.
[0068] During training, the first branch is trained first. The first branch consists of a convolutional neural network and a clustering layer connected in sequence. When the loss function value of the clustering layer no longer decreases, the classification pseudo-labels output by the clustering layer are used as the output values of the classification layer.
[0069] The second branch is then trained, which includes a convolutional neural network and a classification layer connected in sequence. Training stops when the loss function value of the first fault detection model no longer decreases, or when the number of iterations exceeds a set number, and the trained first fault detection model is obtained.
[0070] For example, the convolutional neural network is implemented using MobileNetV3, which is used to map spectrograms to vector space. MobileNetV3 is a lightweight network that can perform real-time feature extraction on mobile devices and can be adjusted according to specific tasks and hardware requirements.
[0071] The clustering layer is implemented using the K-means algorithm, which iteratively clusters the features and uses the subsequent clustering assignment results as "pseudo-labels" to optimize the network loss. The clustering loss and the "pseudo-labels" predicted by the convolutional network loss function are alternately optimized to update the parameters of the convolutional network.
[0072] This invention combines clustering learning and deep learning. The clustering branch and the classification branch share network parameters to perform iterative classification of data, which is more accurate than the classification by deep transitive learning.
[0073] First, we will use a large number of deep networks to pre-train the deep neural network as a method for feature extraction, but this method is not suitable for deep neural networks in the case of small samples.
[0074] Among numerous deep neural networks, MobileNetV3 achieves the optimal number of convolutional kernels and channels through the NetAdapt algorithm. It inherits the depthwise separable convolutions of MobileNetV1 and the residual structure with linear bottlenecks from MobileNetV2. Furthermore, MobileNetV3 introduces a SE channel attention structure and uses a new activation function, h-swish(x), instead of ReLU6. Additionally, MobileNetV3 modifies the back-end output header of MobileNetV2. This makes MobileNetV3 powerful, with fewer parameters, lower computational cost, and shorter inference time. Therefore, MobileNetV3 is more suitable for scenarios with limited storage and power consumption, and is particularly well-suited for real-time detection tasks.
[0075] The clustering model uses K-means to iteratively group features and uses the subsequent cluster assignments as "pseudo-labels" to update the network parameters;
[0076] Optimizing clustering loss: Alternating between updating the parameters of the convolutional network using "pseudo-labels" generated by clustering features and these pseudo-labels predicted by the network loss equation; k-means takes a set of vectors as input, in this example, the features f(x) generated by convnet. n They are then clustered into k distinct groups based on geometric criteria.
[0077] Furthermore, the loss function of the clustering layer... c By letting loss c By finding the minimum value, we obtain the predicted cluster center matrix C and the cluster assignment y for each image n. n :
[0078]
[0079] in, Represents a d*k dimensional vector set, loss c Let C represent the clustering loss, C represent the predicted cluster center matrix, and N represent the average distance from the n points to the centroid. This indicates that the distance from the point to the centroid is minimized, y n ∈{0,1} k x represents n Belongs to y n time y n Take 1 otherwise, multiply by c, then calculate the distance between x and y, f. θ (x n ) represents the feature output by the convolutional neural network, x n y represents the vector used to generate features. n The predicted cluster assignment y represents n , This represents the Euclidean distance, used to calculate the distance between each point and the centroid and assign it to the cluster corresponding to the nearest centroid. The predicted cluster assignment y represents n The transpose of 1 k Let k represent a vector with k-dimensional elements all equal to 1, and "such that" means that the formula satisfies Requirements.
[0080] The clustering model alternately uses formula (3) to cluster features to generate pseudo-labels or updates the network parameters by predicting these pseudo-labels using a loss function.
[0081] The loss function of the first fault detection model is defined as follows:
[0082]
[0083] Among them, g W For a parameterized classifier, l is the multinomial log loss, and the gradient is computed using mini-batch stochastic gradient descent and backpropagation to minimize the loss function, x n Given a training set X{x1,x2,...,x...} n N images of f θ (x n ) represents a convolutional neural network, θ is the parameter set; the image vector obtained by applying the convolutional neural network is called a feature or representation; g W (f θ (x n )) represents the parameterized classifier g w Based on the features, predict the correct label, W represents the parameters of the classifier, y n The label to be predicted, l(g) W (fθ (x n )),y n () represents the logistic function, and N represents the N input images. Indicates command Take the minimum value.
[0084] Further, in S102: the trained first fault detection model is tested. If the test fails, the data samples with classification accuracy lower than a first set threshold during training are aggregated to obtain a second submersible pump audio fault dataset, including:
[0085] The data samples with a classification accuracy of less than 10% during the training of the first fault detection model were collected to obtain the second submersible pump audio fault dataset.
[0086] Further, in step S103: the second submersible pump audio fault dataset is input into the first fault detection model, the first fault detection model is trained, and the trained second fault detection model is obtained.
[0087] The second submersible pump audio fault dataset is input into the first fault detection model to train the first fault detection model. Training is stopped when the loss function value of the first fault detection model no longer decreases, or when the number of iterations exceeds the set number, and the trained second fault detection model is obtained.
[0088] Further, in S104: the parameters of the trained first fault detection model and the parameters of the trained second fault detection model are averaged, and the average value is used as the parameters of the first fault detection model to obtain the corrected first fault detection model, wherein the parameter values of the trained first fault detection model and the parameter values of the trained second fault detection model are averaged, and the average value is used as the parameter values of the first fault detection model.
[0089] It should be understood that steps S103-S104 above employ a hard sample (HS) learning strategy. During real-time monitoring, hard samples are resampled based on previously failed tasks with low validation accuracy that occurred during training, and then the fault category is output. Based on the actual needs of submersible pump fault diagnosis, a method for adaptive testing of new types of faults is presented. The HS learning strategy reorganizes categories with low identification accuracy in each task (also known as hard samples) into new categories, thereby improving the classification accuracy of high-difficulty tasks.
[0090] A pre-trained submersible pump acoustic fault classification model was copied and clustered together with existing new categories, allowing the parameters of the two models to be shared throughout the process. At the start of the experiment, the acquired submersible pump audio signal was first decomposed into multiple frames, and then the Gammatone spectrum of the signal was obtained. Based on the supervised learning verification method, the spectrum of each frame was input into the network model for classification.
[0091] Furthermore, the testing of the trained first fault detection model specifically includes:
[0092] If p max ≥acc th If the classification result meets the requirements, the first fault detection model after training is considered to have passed the test.
[0093] If p max <acc th If the classification result does not meet the requirements, the first fault detection model after training will fail the test.
[0094] Where, num i num represents the number of test samples that are identified as belonging to a certain category. n p represents the total number of frames in the test sample. max acc is the maximum probability of the test sample belonging to its class. th For the set decision threshold, the requirement is: acc th <acc min To ensure that test signals of unknown fault types are not misclassified as known fault types, acc min To verify the accuracy of the lowest class in the set.
[0095] Further, in step S105: the audio signal of the mining submersible pump to be detected is acquired, and the audio signal of the mining submersible pump to be detected is input into the corrected first fault detection model to obtain the fault detection category, specifically including:
[0096] The audio signal of the mining submersible pump to be tested is segmented to obtain several audio segments to be tested;
[0097] For each audio segment to be detected, pre-emphasis processing, frame segmentation processing, windowing processing, filtering processing, and Fourier transform processing are performed sequentially to obtain the spectrogram to be detected.
[0098] The spectrogram to be detected is input into the corrected first fault detection model to obtain the fault detection category.
[0099] Example 2
[0100] This embodiment provides an online fault detection system for mining submersible pumps based on audio signals;
[0101] An online fault detection system for mining submersible pumps based on audio signals includes:
[0102] The dataset construction module is configured to: acquire audio signals from mining submersible pumps, segment the acquired audio signals to obtain several audio segments, convert each audio segment into a spectrogram, and construct a first submersible pump audio fault dataset; the first submersible pump audio fault dataset includes: several audio segments, each audio segment having a corresponding spectrogram;
[0103] The first training module is configured to: input the first submersible pump audio fault dataset into the first fault detection model, train the first fault detection model, and obtain the trained first fault detection model; test the trained first fault detection model, and if the test fails, summarize the data samples whose classification accuracy is lower than a first set threshold during the training process to obtain the second submersible pump audio fault dataset; the second submersible pump audio fault dataset includes: several audio segments, each audio segment having a corresponding spectrogram;
[0104] The second training module is configured to: input the second submersible pump audio fault dataset into the first fault detection model, train the first fault detection model, and obtain the trained second fault detection model;
[0105] The correction module is configured to: average the parameters of the trained first fault detection model with the parameters of the trained second fault detection model, and use the average value as the parameters of the first fault detection model to obtain the corrected first fault detection model.
[0106] The classification module is configured to: acquire the audio signal of the mining submersible pump to be detected, input the audio signal of the mining submersible pump to be detected into the corrected first fault detection model, and obtain the fault detection category.
[0107] It should be noted that the dataset construction module, first training module, second training module, correction module, and classification module mentioned above correspond to steps S101 to S105 in Embodiment 1. The examples and application scenarios implemented by these modules and their corresponding steps are the same, but they are not limited to the content disclosed in Embodiment 1. It should be noted that these modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.
[0108] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0109] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.
[0110] Example 3
[0111] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein, the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory to cause the electronic device to perform the method described in Embodiment 1.
[0112] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.
[0113] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.
[0114] In the implementation process, each step of the above method can be completed by the integrated logic circuits in the processor hardware or by software instructions.
[0115] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.
[0116] Those skilled in the art will recognize that the units and algorithm steps described in connection with the various examples of this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.
[0117] Example 4
[0118] This embodiment also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the method described in Embodiment 1.
[0119] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An online fault detection method for mining submersible pumps based on audio signals, characterized in that, include: The audio signal of the mining submersible pump is acquired, and the acquired audio signal is segmented to obtain several audio segments. Each audio segment is converted into a spectrogram to construct the first submersible pump audio fault dataset. The first submersible pump audio fault dataset includes: several audio segments, each audio segment having a corresponding spectrogram; The first submersible pump audio fault dataset is input into the first fault detection model to train the first fault detection model and obtain the trained first fault detection model. The trained first fault detection model is tested. If the test fails, the data samples with classification accuracy lower than the first set threshold during the training process are summarized to obtain the second submersible pump audio fault dataset. The second submersible pump audio fault dataset includes: several audio segments, each audio segment has a corresponding spectrogram. The second submersible pump audio fault dataset is input into the first fault detection model, and the first fault detection model is trained to obtain the trained second fault detection model. The parameters of the first fault detection model after training are averaged with the parameters of the second fault detection model after training, and the average value is used as the parameters of the first fault detection model to obtain the corrected first fault detection model. The audio signal of the mining submersible pump to be tested is acquired and input into the corrected first fault detection model to obtain the fault detection category.
2. The online fault detection method for mining submersible pumps based on audio signals as described in claim 1, characterized in that, Each audio segment is converted into a spectrogram, specifically including: Each audio segment is processed sequentially with pre-emphasis, frame segmentation, windowing, filtering, and Fourier transform to obtain a spectrogram.
3. The online fault detection method for mining submersible pumps based on audio signals as described in claim 2, characterized in that, The filtering process is implemented using a Gammatone filter, and the impulse response h(t) of the Gammatone filter is as follows: h(t)=ct n-1 e -2πbt cos(2πf0t+φ),t>0; Where c is the proportionality coefficient; n is the number of filter stages; b is the attenuation rate; f0 is the center frequency; φ is the phase; the center frequency f0 of each filter is determined according to the range of human hearing frequency and the number of filters; t represents the impulse response time.
4. The online fault detection method for mining submersible pumps based on audio signals as described in claim 1, characterized in that, The audio fault dataset of the first submersible pump is input into the first fault detection model to train the first fault detection model, thereby obtaining the trained first fault detection model; wherein, the first fault detection model includes: A convolutional neural network (CNN) is provided, wherein the input of the CNN is used to input a spectrogram, the output of the CNN is connected to the input of a clustering layer, and the output of the CNN is also connected to the input of a classification layer. The clustering layer is used to output classification pseudo-labels corresponding to audio frequency bands. During training, the first branch is trained first. The first branch consists of a convolutional neural network and a clustering layer connected in sequence. When the loss function value of the clustering layer no longer decreases, the classification pseudo-labels output by the clustering layer are used as the output values of the classification layer. The second branch is then trained, which includes a convolutional neural network and a classification layer connected in sequence. Training stops when the loss function value of the first fault detection model no longer decreases, or when the number of iterations exceeds a set number, and the trained first fault detection model is obtained.
5. The online fault detection method for mining submersible pumps based on audio signals as described in claim 4, characterized in that, loss function of clustering layer c By letting loss c By finding the minimum value, we obtain the predicted cluster center matrix C and the cluster assignment y for each image n. n : in, Represents a d×k dimensional vector set, loss c Let C represent the clustering loss, C represent the predicted cluster center matrix, and N represent the average distance from the n points to the centroid. This indicates that the distance from the point to the centroid is minimized, y n ∈{0,1} k x represents n Belongs to y n At that time, y n Select 1, otherwise y n Take 0, y n After multiplying by C, calculate x. n With y n The distance, f θ (x n ) represents the feature output by the convolutional neural network, x n y represents the vector used to generate features. n The predicted cluster assignment y represents n , This represents the Euclidean distance, used to calculate the distance between each point and the centroid and assign it to the cluster corresponding to the nearest centroid. The predicted cluster assignment y represents n The transpose of 1 k Let k represent a vector with k-dimensional elements all equal to 1, and "such that" means that the formula satisfies Requirements.
6. The online fault detection method for mining submersible pumps based on audio signals as described in claim 4, characterized in that, The loss function of the first fault detection model is defined as: Among them, g W For a parameterized classifier, l is the multinomial log loss, and the gradient is computed using mini-batch stochastic gradient descent and backpropagation to minimize the loss function, x n Given a training set X{x1,x2,...,x...} n N images of f θ (x n ) represents a convolutional neural network, which is a set of parameters; the image vector obtained by applying the convolutional neural network is called a feature or representation; g W (f θ (x n )) represents the parameterized classifier g w Based on the features, predict the correct label, W represents the parameters of the classifier, y n The label to be predicted, l(g) W (f θ (x n )),y n () represents the logistic function, and N represents the N input images. Indicates command Take the minimum value.
7. The online fault detection method for mining submersible pumps based on audio signals as described in claim 1, characterized in that, The first fault detection model after training was tested, specifically including: If p max ≥acc th If the classification result meets the requirements, the first fault detection model after training is considered to have passed the test. If p max <acc th If the classification result does not meet the requirements, the first fault detection model after training will fail the test. Where, num i num represents the number of test samples that are identified as belonging to a certain class. n p represents the total number of frames in the test sample. max To determine the maximum probability that a test sample belongs to a certain class, acc th For the set decision threshold, the requirement is: acc th <acc mi To ensure that test signals of unknown fault types are not misclassified as known fault types, acc min To verify the accuracy of the lowest class in the set.
8. An electronic device, characterized in that it comprises: Memory is used to store computer-readable instructions in a non-transitory manner. as well as Processor, for executing the computer-readable instructions, When the computer-readable instructions are executed by the processor, they perform the method described in any one of claims 1-6.
9. A storage medium characterized by being non-transitory. The system stores computer-readable instructions, wherein, when the non-transitory computer-readable instructions are executed by a computer, the instructions of the method according to any one of claims 1-6 are executed.
10. A computer program product, characterized in that, Includes a computer program, which, when run on one or more processors, is used to implement the method described in any one of claims 1-6.
Citation Information
Patent Citations
Method and equipment for identifying elevator entering behavior of electric vehicle based on YOLO-MobileNet
CN116721373A
Method and system for warning about upcoming anomalies in the drilling process
RU2772851C1
