Industrial equipment voiceprint monitoring system and method for equipment defect analysis

Abstract

The invention discloses an industrial equipment voiceprint monitoring system and method for equipment defect analysis, and belongs to the technical field of voiceprint recognition. The problems that existing manual monitoring is high in subjectivity, and monitoring time cannot be flexibly adjusted are solved, and real-time monitoring and fault diagnosis of sound signals of industrial equipment are achieved through cooperative work of the voiceprint monitoring unit, the feature extraction module and the voiceprint recognition module; meanwhile, voiceprint features of the equipment can be effectively captured, the fault type can be automatically recognized, and powerful support is provided for intelligent maintenance of the industrial equipment; through dynamic acquisition frequency adjustment based on equipment operation duration, data acquisition efficiency can be optimized, and redundant data can be reduced; and meanwhile, by combining the time threshold and the linear interpolation, different operation states of the industrial equipment can be flexibly adapted, so that the method is suitable for real-time monitoring and fault diagnosis of the industrial equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to an industrial equipment voiceprint monitoring system and method for equipment defect analysis. Background Art

[0002] Industrial equipment plays a vital role in modern production, and its operating status is directly related to production efficiency, safety, and cost control. However, over the long term, equipment may develop various defects or failures due to wear, aging, or external factors. Voiceprint monitoring technology can effectively identify the operating status and potential defects of equipment by analyzing the sound signals generated during operation. With the advancement of intelligent manufacturing, voiceprint monitoring technology is developing towards intelligent, automated, and high-precision features.

[0003] However, traditional equipment monitoring methods mainly rely on manual inspections or fixed sensors, which have the following limitations:

[0004] Manual inspection: Relies on staff experience, is highly subjective, and cannot achieve real-time monitoring, making it easy to miss potential problems;

[0005] Fixed sensors: They can only collect data at specific locations, and cannot flexibly adjust monitoring points and monitoring times, making it difficult to fully cover the operating status of the equipment.

[0006] Therefore, the existing needs are not met, so we propose an industrial equipment voiceprint monitoring system and method for equipment defect analysis. Summary of the Invention

[0007] The purpose of the present invention is to provide an industrial equipment voiceprint monitoring system and method for equipment defect analysis. Through the coordinated work of a voiceprint monitoring unit, a feature extraction module and a voiceprint recognition module, real-time monitoring and fault diagnosis of industrial equipment sound signals can be achieved. At the same time, it can effectively capture the voiceprint characteristics of the equipment and automatically identify the fault type, providing strong support for the intelligent maintenance of industrial equipment. By dynamically adjusting the acquisition frequency based on the equipment's operating time, data acquisition efficiency can be optimized and redundant data can be reduced. At the same time, by combining time thresholds and linear interpolation, it can flexibly adapt to different operating states of industrial equipment, so as to be suitable for real-time monitoring and fault diagnosis of industrial equipment, solving the problems raised in the above-mentioned background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] An industrial equipment voiceprint monitoring system for equipment defect analysis, the system comprising: a voiceprint monitoring unit and a voiceprint analysis unit, the voiceprint analysis unit comprising: a feature extraction module and a voiceprint recognition module;

[0010] The voiceprint monitoring unit is configured to use a microphone array to regularly acquire sound signals generated by industrial equipment during operation. The sound signals include: sound signals generated by industrial equipment in normal working conditions and abnormal working conditions. To ensure a sufficiently high sampling rate, the sound signal is usually 44.1kHz or higher to ensure that all relevant frequency components can be captured;

[0011] A feature extraction module is configured to extract spectral features of industrial equipment sound signals based on Mel-frequency cepstral coefficients to describe the unique properties of the sound; analyze the linear prediction features of the sound signals using linear prediction coefficients to reveal the periodicity of the sound and obtain the voiceprint features of the industrial equipment sound signals;

[0012] The voiceprint recognition module is configured to build and train a convolutional neural network model based on historical signal data, voiceprint features, and voiceprint defect templates, so that the trained convolutional neural network model automatically extracts the voiceprint features of the sound signals of industrial equipment and outputs the fault type of the industrial equipment.

[0013] Furthermore, the voiceprint monitoring unit includes:

[0014] A timing feedback module is configured to pre-set the microphone data collection frequency based on the operating time of the industrial equipment;

[0015] The voiceprint processing module is configured to remove background noise based on a low-pass filter, enhance high-frequency components based on a pre-emphasis filter, and divide the sound signal into short time frames to process non-stationary signals; a window function is applied to each frame to reduce boundary effects.

[0016] Furthermore, based on the operating time of the industrial equipment, the data collection frequency of the microphone is pre-set, including the following steps:

[0017] According to the operating time of industrial equipment, industrial equipment is divided into the startup stage, transition stage, stable stage and abnormality detection stage;

[0018] Start-up phase: 0≤t <T1;

[0019] Transition stage: T1≤t <T2;

[0020] Stable stage: t≥T2;

[0021] Where t is the device operation time; T1 and T2 are the preset time thresholds;

[0022] Set different collection frequencies based on different operating stages:

[0023] Startup phase: high frequency acquisition;

[0024] Transition stage: medium frequency acquisition;

[0025] Stable stage: low frequency acquisition;

[0026] Anomaly detection stage: high-frequency acquisition;

[0027] Based on the running time of different stages, the linear interpolation method is used to gradually reduce the acquisition frequency in the transition stage. The calculation formula is as follows:

[0028]

[0029] Among them, f (t) It represents the acquisition frequency in the transition phase; f1 represents the acquisition frequency in the startup phase; and f2 represents the acquisition frequency in the stable phase.

[0030] Furthermore, the process of setting the time threshold T1 in the startup phase includes the following steps:

[0031] Extracting nominal values ​​corresponding to main operating parameters of the industrial equipment, wherein the main operating parameters are determined according to the specific equipment type of the industrial equipment, including but not limited to speed, temperature, pressure, etc.;

[0032] Real-time monitoring of the values ​​of the main operating parameters of the industrial equipment after startup;

[0033] When the main operating parameters include only one operating parameter, the corresponding typical startup time in the equipment manual is extracted as the time threshold T1 corresponding to the startup phase;

[0034] When the main operating parameter includes multiple operating parameters, the values ​​of all the main operating parameters are monitored in real time when the industrial equipment is started;

[0035] When the value of the first main operating parameter reaches 30% of the nominal value, obtaining the time period between the moment when the value of the first main operating parameter reaches 30% of the nominal value and the moment when the industrial equipment is started as the first time period data;

[0036] When the value of the last main operating parameter reaches 30% of the nominal value, obtaining the time period between the moment when the value of the last main operating parameter reaches 30% of the nominal value and the moment when the industrial equipment is turned on as the second time period data;

[0037] Detecting the percentage of the nominal value reached by other main operating parameters when the value of the last main operating parameter reaches 30% of the nominal value, as parameter percentage data;

[0038] The time threshold T1 of the startup phase is set using the first time period data, the second time period data and the parameter percentage data.

[0039] Furthermore, setting the time threshold T1 of the startup phase using the first time period data, the second time period data and the parameter percentage data includes the following steps:

[0040] Retrieve data from the first time period and data from the second time period;

[0041] Obtaining a time difference using data from the first time period and data from the second time period;

[0042] Retrieve the corresponding typical startup time from the device manual;

[0043] Retrieve parameter percentage data corresponding to the percentage of the nominal value reached by other main operating parameters when the value of the last main operating parameter reaches 30% of the nominal value;

[0044] The typical startup time is adjusted using the time difference between the first time period data and the second time period data in combination with the parameter percentage data, wherein the adjusted startup time is the time threshold T1 of the startup phase;

[0045] The time threshold T1 of the startup phase is obtained by the following formula:

[0046]

[0047] Among them, T1 represents the time threshold of the startup phase; T x represents the typical startup time; n represents the number of main operating parameters other than the last one that reaches 30% of the nominal value; P i represents the percentage of the i-th other main parameter reaching the nominal value at the moment when the value of the last main operating parameter reaches 30% of the nominal value; P max Indicates that the values ​​of n other main parameters reach the maximum percentage of the nominal value when the value of the last main operating parameter reaches 30% of the nominal value; T c Indicates the time difference between the data of the first time period and the data of the second time period; T 02 Indicates the data of the second time period.

[0048] Furthermore, the feature extraction module includes:

[0049] an audio conversion module configured to convert a sound signal of the industrial equipment into a spectrogram based on a short-time Fourier transform method;

[0050] a pitch extraction module configured to extract voiceprint features related to pitch for use in analyzing pitch features of sound signals from industrial equipment;

[0051] The feature fusion module is configured to fuse the Mel spectrum features, linear prediction features and pitch features to comprehensively describe the spectrum, periodicity and voiceprint characteristics of the sound signal of industrial equipment.

[0052] Furthermore, the voiceprint recognition module includes:

[0053] a signal collection module configured to collect historical sound signals of industrial equipment in different states, including: a normal operating state and a faulty operating state;

[0054] A template construction module is configured to construct voiceprint feature templates of different fault types based on historical sound signals and their voiceprint features;

[0055] The voiceprint matching module is configured to match the voiceprint features of the historical sound signal with the voiceprint feature template as data samples for the convolutional neural network model.

[0056] Furthermore, the voiceprint recognition module further includes:

[0057] A sample processing module is configured to convert the voiceprint features in the data sample into a two-dimensional matrix, where each row represents the voiceprint features of a frame of signal and each column represents a feature dimension, as the input set of the convolutional neural network model; and divide the input set into a training set, a validation set, and a test set;

[0058] The model training module is configured to transmit the voiceprint feature matrix to the input layer and use multiple convolution kernels to extract local features; use the ReLU activation function to introduce nonlinearity and use the maximum pooling layer to reduce the feature dimension; flatten the output of the convolution layer and connect it to one or more fully connected layers; use the Softmax activation function to output the probability distribution of the fault type and select the fault category with the highest probability as the prediction result;

[0059] The model practice module is configured to deploy the trained convolutional neural network model to the industrial equipment monitoring system to analyze equipment sound signals in real time and detect fault types.

[0060] A method for industrial equipment voiceprint monitoring for equipment defect analysis comprises the following steps:

[0061] Timed collection: Based on the operating time of industrial equipment, the equipment is divided into the startup phase, transition phase, stable phase, and abnormality detection phase, and the microphone data collection frequency is set based on different phases;

[0062] Data acquisition and processing: Use microphone arrays to collect sound signals from industrial equipment during operation, perform denoising, framing, and windowing on the sound signals, and label the device status of each sound signal segment.

[0063] Feature extraction: extract Mel spectrum features to describe the spectral characteristics of the sound signal; extract linear prediction features to analyze the periodic characteristics of the sound signal; extract pitch features to supplement the description of the periodic changes of the sound signal;

[0064] Voiceprint matching: Match the voiceprint features of historical sound signals with voiceprint feature templates as data samples for the convolutional neural network model;

[0065] Model construction: Convert the voiceprint features into a two-dimensional matrix and construct a convolutional neural network model, including convolutional layers, pooling layers, fully connected layers, and output layers. Use the training set to train the convolutional neural network model, adjust the hyperparameters on the validation set, and evaluate the model performance on the test set.

[0066] Model application: The trained convolutional neural network model is deployed in the industrial equipment monitoring system to collect industrial equipment sound signals in real time and determine the operating status of the industrial equipment.

[0067] Furthermore, extracting Mel spectrum features includes the following steps:

[0068] Based on the processed sound signal, a Mel filter bank is designed, which is a set of triangular filters on the Mel frequency scale; usually 20-40 filters are used to cover the entire frequency range;

[0069] Pass the voiceprint spectrum through the Mel filter bank and calculate the output energy of each filter;

[0070] The logarithm of the output energy of each filter is taken to obtain the logarithmic energy spectrum; this is to simulate the nonlinear perception of sound intensity by the human ear;

[0071] Perform DCT discrete cosine transform on the logarithmic energy spectrum to obtain cepstral coefficients. Usually only the first 12-13 MFCC coefficients are retained, which not only reduces the data dimension but also contains the main spectrum envelope information.

[0072] The first-order and second-order differences of the MFCC Mel-frequency cepstral coefficients are calculated to capture dynamic features and describe the changes of features over time, thereby obtaining the spectral characteristics of the industrial equipment sound signal.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] 1. In the present invention, the voiceprint monitoring unit, feature extraction module and voiceprint recognition module work together to achieve real-time monitoring of industrial equipment sound signals and fault diagnosis. At the same time, it can effectively capture the voiceprint characteristics of the equipment and automatically identify the fault type, providing strong support for the intelligent maintenance of industrial equipment.

[0075] 2. In the present invention, by dynamically adjusting the acquisition frequency based on the equipment operating time, the data acquisition efficiency can be optimized and redundant data can be reduced; at the same time, by combining time thresholds and linear interpolation, it can flexibly adapt to the different operating states of industrial equipment, so as to be suitable for real-time monitoring and fault diagnosis of industrial equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is a flow chart of the industrial equipment voiceprint monitoring system for equipment defect analysis of the present invention. DETAILED DESCRIPTION

[0077] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0078] To address the technical issues that traditional equipment monitoring methods rely on manual inspections or fixed sensors, manual inspections: relying on the experience of staff, are highly subjective, and cannot achieve real-time monitoring, which can easily miss potential problems; fixed sensors: can only collect data at specific locations, cannot flexibly adjust monitoring points and monitoring time, and cannot fully cover the operating status of the equipment, please refer to Figure 1 , this embodiment provides the following technical solutions:

[0079] An industrial equipment voiceprint monitoring system for equipment defect analysis, the system comprising: a voiceprint monitoring unit and a voiceprint analysis unit, the voiceprint analysis unit comprising: a feature extraction module and a voiceprint recognition module;

[0080] The voiceprint monitoring unit is configured to use a microphone array to periodically acquire sound signals generated by industrial equipment during operation. The sound signals include: sound signals generated by the industrial equipment in normal working conditions and abnormal working conditions. To ensure a sufficiently high sampling rate, the sound signals are usually 44.1kHz or higher to ensure that all relevant frequency components can be captured. The voiceprint monitoring unit includes:

[0081] The timing feedback module is configured to pre-set a data collection frequency of the microphone based on the operating time of the industrial equipment. The pre-setting of the data collection frequency of the microphone based on the operating time of the industrial equipment includes the following steps:

[0082] According to the operating time of industrial equipment, industrial equipment is divided into the startup stage, transition stage, stable stage and abnormality detection stage;

[0083] Start-up phase: 0≤t <T1;

[0084] Transition stage: T1 ≤ t < T2;

[0085] Stable stage: t ≥ T2;

[0086] where t is the running duration of the device; T1 and T2 are preset time thresholds;

[0087] Set different acquisition frequencies based on different running stages:

[0088] Startup stage: High-frequency acquisition; for example: once per second (1 Hz);<00002​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

[0100] Stable phase: t≥30 minutes, acquisition frequency f2=1 / 60Hz;

[0101] When the device runs for 15 minutes, the acquisition frequency is calculated as follows:

[0102]

[0103] The collected sound signal data is then stored in a database or file, the sound signal is analyzed, and voiceprint features such as MFCC Mel spectrum features, LPC linear prediction features, etc. are extracted for industrial equipment status monitoring and fault diagnosis.

[0104] The voiceprint processing module is configured to remove background noise based on a low-pass filter, enhance high-frequency components based on a pre-emphasis filter, and divide the sound signal into short time frames, usually 20-40ms, to process non-stationary signals; apply a window function to each frame, such as a Hamming window, to reduce boundary effects; specifically, since the collected sound signal may contain noise, preprocessing operations such as filtering and noise reduction are required to improve the accuracy of subsequent feature extraction.

[0105] The beneficial effects achieved by the above content are: by dynamically adjusting the acquisition frequency based on the equipment operating time, the data acquisition efficiency can be optimized and redundant data can be reduced; at the same time, by combining time thresholds and linear interpolation, it can flexibly adapt to the different operating states of industrial equipment, so as to be suitable for real-time monitoring and fault diagnosis of industrial equipment.

[0106] Meanwhile, the process of setting the time threshold T1 in the startup phase includes the following steps:

[0107] Extracting nominal values ​​corresponding to main operating parameters of the industrial equipment, wherein the main operating parameters are determined according to the specific equipment type of the industrial equipment, including but not limited to speed, temperature, pressure, etc.;

[0108] Real-time monitoring of the values ​​of the main operating parameters of the industrial equipment after startup;

[0109] When the main operating parameters include only one operating parameter, the corresponding typical startup time in the equipment manual is extracted as the time threshold T1 corresponding to the startup phase;

[0110] When the main operating parameter includes multiple operating parameters, the values ​​of all the main operating parameters are monitored in real time when the industrial equipment is started;

[0111] When the value of the first main operating parameter reaches 30% of the nominal value, obtaining the time period between the moment when the value of the first main operating parameter reaches 30% of the nominal value and the moment when the industrial equipment is started as the first time period data;

[0112] When the value of the last main operating parameter reaches 30% of the nominal value, obtaining the time period between the moment when the value of the last main operating parameter reaches 30% of the nominal value and the moment when the industrial equipment is turned on as the second time period data;

[0113] Detecting the percentage of the nominal value reached by other main operating parameters when the value of the last main operating parameter reaches 30% of the nominal value, as parameter percentage data;

[0114] The time threshold T1 of the startup phase is set using the first time period data, the second time period data and the parameter percentage data.

[0115] The technical solution described above comprehensively considers the equipment's actual operating characteristics by extracting the nominal values ​​of key operating parameters from industrial equipment and monitoring these values ​​in real time after the equipment starts. For equipment with multiple key operating parameters, this approach not only examines the time it takes for the first and last parameters to reach a certain percentage (30%) of their nominal values, but also measures the percentage of other parameters reaching their nominal values ​​during this time. This allows for a comprehensive assessment of the equipment's startup process from multiple dimensions, ensuring that the set startup phase time threshold, T1, more accurately reflects the equipment's actual startup time, avoiding errors caused by single-parameter judgment or empirical settings.

[0116] Key operating parameters, such as speed, temperature, and pressure, are determined based on the specific type of industrial equipment. This covers a wide range of possible parameter types and can adapt to the characteristics of different types of industrial equipment. Regardless of the type of equipment, time thresholds can be set according to the corresponding process, enhancing the solution's versatility and adaptability, ensuring that different equipment can achieve reasonable startup time limits. Furthermore, this approach effectively aligns the startup time thresholds with the actual conditions of industrial equipment. During the equipment startup process, this threshold is used to determine whether the equipment has started normally, allowing for timely detection of potential anomalies, such as parameters taking too long or too short to reach their nominal values ​​or mismatched parameter changes. This helps operations and maintenance personnel take timely action to ensure stable equipment operation and reduce the likelihood of equipment failure. Furthermore, accurate startup time thresholds ensure that equipment completes startup and enters normal operation within an appropriate timeframe, avoiding production delays or equipment damage caused by improper startup time settings. Furthermore, timely detection and resolution of startup anomalies also helps improve production safety, reduce the risk of safety incidents caused by equipment failure, and ensure smooth production. Setting startup time thresholds collects extensive parameter data from the equipment startup process. This data can be accumulated as historical records, providing valuable reference for subsequent equipment performance analysis, optimization and improvement. Through in-depth analysis of this data, we can further understand the operating patterns and characteristics of the equipment, providing support for equipment upgrades and the application of new technologies.

[0117] Specifically, using the first time period data, the second time period data and the parameter percentage data to set the time threshold T1 of the startup phase includes the following steps:

[0118] Retrieve data from the first time period and data from the second time period;

[0119] Obtaining a time difference using data from the first time period and data from the second time period;

[0120] Retrieve the corresponding typical startup time from the device manual;

[0121] Retrieve parameter percentage data corresponding to the percentage of the nominal value reached by other main operating parameters when the value of the last main operating parameter reaches 30% of the nominal value;

[0122] The typical startup time is adjusted using the time difference between the first time period data and the second time period data in combination with the parameter percentage data, wherein the adjusted startup time is the time threshold T1 of the startup phase;

[0123] The time threshold T1 of the startup phase is obtained by the following formula:

[0124]

[0125] Among them, T1 represents the time threshold of the startup phase; T x represents the typical startup time; n represents the number of main operating parameters other than the last one that reaches 30% of the nominal value; P i represents the percentage of the i-th other main parameter reaching the nominal value at the moment when the value of the last main operating parameter reaches 30% of the nominal value; P max Indicates that the values ​​of n other main parameters reach the maximum percentage of the nominal value when the value of the last main operating parameter reaches 30% of the nominal value. c Indicates the time difference between the data of the first time period and the data of the second time period; T 02 Indicates the data of the second time period.

[0126] The technical effect of the above technical solution is as follows: the solution comprehensively considers multiple key factors in the equipment startup process, such as the data of the first time period, the data of the second time period, and the percentage of each main parameter reaching the nominal value. By calculating the time difference and adjusting the typical startup time in combination with the parameter percentage data, compared with simply relying on the typical startup time in the equipment manual, it can more accurately reflect the time required for the actual startup of the equipment, thereby improving the accuracy of the time threshold setting in the startup phase. The startup characteristics of different devices are different. This solution determines the time threshold based on the parameter changes during the actual startup process of the equipment, and can adapt to the different parameter performances of various industrial equipment during the startup process. Even if the speed of change of each parameter of the equipment at startup and the degree to which it reaches the nominal value are different, the startup time threshold that meets the characteristics of the equipment itself can be obtained through comprehensive processing of these data, thereby enhancing the adaptability of the solution to different equipment. At the same time, the above technical solution can effectively improve the accuracy of the time threshold setting in the startup phase.

[0127] The above technical solution introduces the number n of other main operating parameters except the last one that reaches 30% of the nominal value, the percentage Pi of each parameter reaching the nominal value at a specific moment, and the maximum percentage Pmax. This allows the calculation of the time threshold to fully consider the changes and differences of different parameters during the startup process, and can more carefully reflect the comprehensive status of each parameter when the device is started, avoiding the problem of inaccurate time threshold caused by ignoring the differences between parameters. At the same time, by This part of the calculation combines the time difference between the first time period data and the second time period data (reflected in T c With T 02) and parameter percentage data, the typical startup time is dynamically adjusted. When the percentage of each parameter reaching the nominal value is generally high or the time difference is small, the adjusted time threshold is relatively small; otherwise it is large. This dynamic adjustment mechanism can flexibly determine the time threshold according to the actual startup situation of the equipment, further improving the matching degree between the time threshold and the equipment startup process. The above technical solution makes full use of the various data collected during the equipment startup process, and organically combines time information and parameter status information. Not only the time sequence is considered (through the first and second time period data), but also the status of the parameters at a specific moment (parameter percentage data) is considered, so that the calculated startup phase time threshold can fully reflect the actual situation of the equipment startup, and provide a more valuable reference for equipment operation monitoring and management.

[0128] The feature extraction module is configured to extract the spectral features of the industrial equipment sound signal based on the Mel-frequency cepstral coefficients to describe the unique attributes of the sound; use the linear prediction coefficient to analyze the linear prediction features of the sound signal, reveal the periodicity of the sound, and obtain the voiceprint features of the industrial equipment sound signal; specifically, by constructing a linear prediction model to estimate the resonance peak and spectral envelope of the sound signal, the periodic characteristics of the sound signal are analyzed, thereby effectively capturing the harmonic structure of the sound signal, so as to facilitate the analysis of the periodic changes in the sound signal generated by the industrial equipment during operation.

[0129] Feature extraction module, including:

[0130] The audio conversion module is configured to convert the sound signals of industrial equipment into a spectrogram based on the Short-Time Fourier Transform (STFT). Specifically, the STFT method converts the sound signals of industrial equipment from the time domain to the frequency domain, and represents the conversion result as a two-dimensional matrix with time on the horizontal axis and frequency on the vertical axis. Each point in the matrix represents the signal strength at a corresponding time and frequency. Visualization tools such as Matplotlib can be used to plot the matrix as a spectrogram, visually demonstrating how the frequency components of the signal change over time. The resulting spectrogram can be used to analyze the operating status of industrial equipment. For example, in a normal state, the spectrogram displays stable frequency components. In an abnormal state, the spectrogram displays unusual frequency peaks or changes in frequency components, indicating a possible equipment failure, thereby significantly improving the accuracy and efficiency of equipment monitoring.

[0131] The pitch extraction module is configured to extract voiceprint features related to pitch for use in analyzing the pitch features of sound signals from industrial equipment. Specifically, the peak position of the periodic signal is found by calculating the autocorrelation function of the signal, thereby determining the fundamental frequency (F0). The signal is Fourier transformed, its logarithm is taken, and then an inverse Fourier transform is performed to obtain the cepstrum. The harmonic components are found by analyzing the spectrum of the signal, thereby determining the fundamental frequency. The extracted pitch features can reflect the operating status of the equipment, for example: normal state: the pitch features are stable, and the fundamental frequency fluctuates within a certain range; abnormal state: the pitch features show significant changes, such as frequency offset or abnormal frequency components. By analyzing the changes in pitch features, abnormal conditions of the equipment can be discovered in a timely manner, providing a basis for fault diagnosis.

[0132] The feature fusion module is configured to fuse Mel spectrum features, linear prediction features, and pitch features to comprehensively describe the spectrum, periodicity, and voiceprint characteristics of industrial equipment sound signals. Specifically, by integrating Mel spectrum features, linear prediction features, and pitch features, it can more comprehensively analyze the unique properties of industrial equipment sound signals, provide strong support for industrial equipment status monitoring and fault diagnosis, and thus improve the accuracy and robustness of industrial equipment sound recognition.

[0133] The voiceprint recognition module is configured to build and train a convolutional neural network model based on historical signal data, voiceprint features, and voiceprint defect templates, so that the trained convolutional neural network model automatically extracts the voiceprint features of industrial equipment sound signals and outputs the fault type of the industrial equipment. The voiceprint recognition module includes:

[0134] a signal collection module configured to collect historical sound signals of industrial equipment in different states, including: a normal operating state and a faulty operating state;

[0135] The template construction module is configured to construct voiceprint feature templates of different fault types based on historical sound signals and their voiceprint features; specifically, by preprocessing the collected historical sound signals and extracting voiceprint features, clustering analysis is performed on the extracted voiceprint features, and similar features are classified to form feature sets of different fault types; for example, the feature set of bearing faults may include high-frequency noise and periodic anomalies; based on the clustering results, a voiceprint feature template is generated for each fault type, and the template can be expressed as the average value or typical feature distribution of the feature vector; the model with abnormal historical sound signals is matched with the fault type voiceprint feature template to identify the fault type of the equipment, so that the model can automatically identify whether there is an equipment fault and its fault type in the historical sound signal.

[0136] a voiceprint matching module configured to match the voiceprint features of historical sound signals with voiceprint feature templates as data samples for the convolutional neural network model;

[0137] The sample processing module is configured to convert the voiceprint features in the data sample into a two-dimensional matrix, where each row represents the voiceprint features of a frame of signal and each column represents a feature dimension, such as Mel-spectrogram features, as the input set of the convolutional neural network model; and divide the input set into a training set, a validation set, and a test set, such as 70% training set, 15% validation set, and 15% test set;

[0138] The model training module is configured to transmit the voiceprint feature matrix to the input layer. The voiceprint feature matrix is ​​in two-dimensional form and represents the spectrum or time-frequency characteristics of the sound signal; multiple convolution kernels are used to extract local features. The convolution kernel setting is: for example, 32 3x3 convolution kernels are used to perform convolution operations on the input matrix to extract local features; ReLU activation function: the ReLU activation function is applied after the convolution operation to introduce nonlinearity and enhance the expressiveness of the model; the ReLU activation function is used to introduce nonlinearity and the maximum pooling layer is used to reduce the feature dimension, such as using a 2x2 pooling window to downsample the output of the convolution layer to reduce the amount of calculation and improve the robustness of the model; the output of the convolution layer is flattened and connected to one or more fully connected layers to further extract high-level features; the Softmax activation function is used to output the probability distribution of the fault type and the fault category with the highest probability is selected as the prediction result; the voiceprint feature matrix is ​​processed by a convolutional neural network to effectively extract local features and realize the classification of fault types, providing an efficient and accurate solution for fault diagnosis of industrial equipment.

[0139] The model practice module is configured to deploy the trained convolutional neural network model to the industrial equipment monitoring system to analyze equipment sound signals in real time and detect fault types.

[0140] In order to better demonstrate the operation process of an industrial equipment voiceprint monitoring system for equipment defect analysis, the present invention provides an industrial equipment voiceprint monitoring method for equipment defect analysis, comprising the following steps:

[0141] Timed collection: Based on the operating time of industrial equipment, the equipment is divided into the startup phase, transition phase, stable phase, and abnormality detection phase, and the microphone data collection frequency is set based on different phases;

[0142] Data acquisition and processing: Use microphone arrays to collect sound signals from industrial equipment during operation, perform denoising, framing, and windowing on the sound signals, and label the device status of each sound signal segment.

[0143] Feature extraction: extract Mel spectrum features to describe the spectral characteristics of the sound signal; extract linear prediction features to analyze the periodic characteristics of the sound signal; extract pitch features to supplement the description of the periodic changes of the sound signal;

[0144] Voiceprint matching: Match the voiceprint features of historical sound signals with voiceprint feature templates as data samples for the convolutional neural network model;

[0145] Model building: Convert voiceprint features into a two-dimensional matrix and construct a convolutional neural network model, including convolutional layers, pooling layers, fully connected layers, and output layers. Train the convolutional neural network model using the training set, adjust hyperparameters on the validation set, and evaluate the model's performance on the test set, calculating metrics such as accuracy, recall, and F1 score.

[0146] Model application: The trained convolutional neural network model is deployed in the industrial equipment monitoring system to collect industrial equipment sound signals in real time and determine the operating status of the industrial equipment.

[0147] Extracting Mel spectrum features includes the following steps:

[0148] Based on the processed sound signal, a Mel filter bank is designed, which is a set of triangular filters on the Mel frequency scale; usually 20-40 filters are used to cover the entire frequency range;

[0149] Pass the voiceprint spectrum through the Mel filter bank and calculate the output energy of each filter;

[0150] The logarithm of the output energy of each filter is taken to obtain the logarithmic energy spectrum; this is to simulate the nonlinear perception of sound intensity by the human ear;

[0151] Perform DCT discrete cosine transform on the logarithmic energy spectrum to obtain cepstral coefficients; usually only the first 12-13 MFCC coefficients are retained, which not only reduces the data dimension but also contains the main spectrum envelope information;

[0152] The first-order and second-order differences of the MFCC Mel-frequency cepstral coefficients are calculated to capture dynamic features to describe the changes in features over time and obtain the spectral characteristics of the sound signals of industrial equipment. At the same time, the MFCC Mel-frequency cepstral features can be used to detect abnormal vibrations or changes in resonant frequency of industrial equipment, thereby diagnosing the type of fault in industrial equipment. For example, bearing faults, gear wear and other faults can cause changes in the spectral characteristics of the sound signal. Therefore, the MFCC Mel-frequency cepstral features can also be used as the input set of the convolutional neural network model to train the convolutional neural network to automatically identify the fault type of industrial equipment.

[0153] The beneficial effects achieved by the above content are: through the collaborative work of the voiceprint monitoring unit, feature extraction module and voiceprint recognition module, real-time monitoring and fault diagnosis of industrial equipment sound signals can be achieved; at the same time, it can effectively capture the voiceprint characteristics of the equipment and automatically identify the fault type, providing strong support for the intelligent maintenance of industrial equipment.

[0154] Working Principle: The operating process of industrial equipment is divided into four stages based on its operating duration. The microphone's data acquisition frequency is dynamically adjusted based on the characteristics of each stage to ensure that the collected sound signals fully reflect the equipment's operating status. A microphone array is used to collect sound signals from the equipment during operation, which are then pre-processed for noise reduction and various voiceprint features are extracted from the pre-processed sound signals. The voiceprint features of historical sound signals are matched with voiceprint feature templates, and a convolutional neural network model is constructed and trained based on these data samples. The trained convolutional neural network model is deployed in the industrial equipment monitoring system to collect sound signals from industrial equipment in real time and determine the equipment's operating status. Through real-time monitoring and intelligent analysis, the system can promptly detect equipment anomalies and provide decision support for equipment maintenance and troubleshooting.

[0155] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0156] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An industrial equipment voiceprint monitoring system for equipment defect analysis, characterized in that: The system includes: a voiceprint monitoring unit and a voiceprint analysis unit, and the voiceprint analysis unit includes: a feature extraction module and a voiceprint recognition module; The voiceprint monitoring unit is configured to use a microphone array to periodically acquire sound signals generated by industrial equipment during operation, and to set a data collection frequency based on a threshold value of equipment operation time. The process of setting the threshold value of the equipment operation time includes: based on the nominal value of the equipment operation parameter, monitoring the parameter value after startup; if there is only one parameter, setting the time threshold based on the typical startup time in the equipment manual; if there are multiple parameters, monitoring the specified time period and percentage data for each parameter to reach the nominal value, and setting the time threshold based on this comprehensive data; a feature extraction module configured to extract spectral features of the sound signal based on the Mel-frequency cepstral coefficients, analyze the linear prediction features of the sound signal using the linear prediction coefficients, and obtain the voiceprint features of the sound signal; The voiceprint recognition module is configured to build and train a convolutional neural network model based on historical signal data, voiceprint features, and voiceprint defect templates, so that the convolutional neural network model automatically extracts the voiceprint features of the sound signal and outputs the fault type of the industrial equipment.

2. The industrial equipment voiceprint monitoring system for equipment defect analysis according to claim 1, characterized in that: The voiceprint monitoring unit includes: A timing feedback module is configured to pre-set the microphone data collection frequency based on the operating time of the industrial equipment; The voiceprint processing module is configured to remove background noise based on a low-pass filter, enhance high-frequency components based on a pre-emphasis filter, and divide the sound signal into short time frames to process non-stationary signals; a window function is applied to each frame to reduce boundary effects.

3. The industrial equipment voiceprint monitoring system for equipment defect analysis according to claim 2, characterized in that: Pre-setting the microphone data collection frequency based on the operating time of the industrial equipment includes the following steps: According to the operating time of industrial equipment, industrial equipment is divided into the startup stage, transition stage, stable stage and abnormality detection stage; Start-up phase: 0≤t <T1; Transition stage: T1≤t <T2; Stable stage: t≥T2; Where t is the device operation time; T1 and T2 are the preset time thresholds; Set different collection frequencies based on different operating stages: Startup phase: high frequency acquisition; Transition stage: medium frequency acquisition; Stable stage: low frequency acquisition; Anomaly detection stage: high-frequency acquisition; Based on the running time of different stages, the linear interpolation method is used to gradually reduce the acquisition frequency in the transition stage. The calculation formula is as follows: Among them, f (t) It represents the acquisition frequency in the transition phase; f1 represents the acquisition frequency in the startup phase; and f2 represents the acquisition frequency in the stable phase.

4. The industrial equipment voiceprint monitoring system for equipment defect analysis according to claim 3, characterized in that: The process of setting the time threshold T1 in the startup phase includes the following steps: Extract the nominal values ​​corresponding to the main operating parameters of industrial equipment; Real-time monitoring of the values ​​of the main operating parameters of the industrial equipment after startup; When the main operating parameters include only one operating parameter, the corresponding typical startup time in the equipment manual is extracted as the time threshold T1 corresponding to the startup phase; When the main operating parameter includes multiple operating parameters, the values ​​of all the main operating parameters are monitored in real time when the industrial equipment is started; When the value of the first main operating parameter reaches 30% of the nominal value, obtaining the time period between the moment when the value of the first main operating parameter reaches 30% of the nominal value and the moment when the industrial equipment is started as the first time period data; When the value of the last main operating parameter reaches 30% of the nominal value, obtaining the time period between the moment when the value of the last main operating parameter reaches 30% of the nominal value and the moment when the industrial equipment is turned on as the second time period data; Detecting the percentage of the nominal value reached by other main operating parameters when the value of the last main operating parameter reaches 30% of the nominal value, as parameter percentage data; The time threshold T1 of the startup phase is set using the first time period data, the second time period data and the parameter percentage data.

5. The industrial equipment voiceprint monitoring system for equipment defect analysis according to claim 4, characterized in that: Setting the time threshold T1 of the startup phase using the first time period data, the second time period data, and the parameter percentage data includes the following steps: Retrieve data from the first time period and data from the second time period; Obtaining a time difference using data from the first time period and data from the second time period; Retrieve the corresponding typical startup time from the device manual; Retrieve parameter percentage data corresponding to the percentage of the nominal value reached by other main operating parameters when the value of the last main operating parameter reaches 30% of the nominal value; The typical startup time is adjusted using the time difference between the first time period data and the second time period data in combination with the parameter percentage data, wherein the adjusted startup time is the time threshold T1 of the startup phase.

6. The industrial equipment voiceprint monitoring system for equipment defect analysis according to claim 1, characterized in that: The feature extraction module includes: an audio conversion module configured to convert a sound signal of the industrial equipment into a spectrogram based on a short-time Fourier transform method; a pitch extraction module configured to extract voiceprint features related to pitch for use in analyzing pitch features of sound signals from industrial equipment; The feature fusion module is configured to fuse the Mel spectrum features, linear prediction features and pitch features to comprehensively describe the spectrum, periodicity and voiceprint characteristics of the sound signal of industrial equipment.

7. The industrial equipment voiceprint monitoring system for equipment defect analysis according to claim 1, characterized in that: The voiceprint recognition module includes: a signal collection module configured to collect historical sound signals of industrial equipment in different states, including: a normal operating state and a faulty operating state; A template construction module is configured to construct voiceprint feature templates of different fault types based on historical sound signals and their voiceprint features; The voiceprint matching module is configured to match the voiceprint features of the historical sound signal with the voiceprint feature template as data samples for the convolutional neural network model.

8. The industrial equipment voiceprint monitoring system for equipment defect analysis according to claim 7, characterized in that: The voiceprint recognition module further includes: A sample processing module is configured to convert the voiceprint features in the data sample into a two-dimensional matrix, where each row represents the voiceprint features of a frame of signal and each column represents a feature dimension, as the input set of the convolutional neural network model; and divide the input set into a training set, a validation set, and a test set; The model training module is configured to transmit the voiceprint feature matrix to the input layer and use multiple convolution kernels to extract local features; use the ReLU activation function to introduce nonlinearity and use the maximum pooling layer to reduce the feature dimension; flatten the output of the convolution layer and connect it to one or more fully connected layers; use the Softmax activation function to output the probability distribution of the fault type and select the fault category with the highest probability as the prediction result; The model practice module is configured to deploy the trained convolutional neural network model to the industrial equipment monitoring system to analyze equipment sound signals in real time and detect fault types.

9. A method for industrial equipment voiceprint monitoring for equipment defect analysis, applied to an industrial equipment voiceprint monitoring system for equipment defect analysis according to any one of claims 1 to 8, characterized in that: The following steps are involved: Based on the operating time of industrial equipment, the equipment is divided into the startup phase, transition phase, stable phase, and abnormality detection phase, and the data collection frequency of the microphone is set based on different phases; Use microphone arrays to collect sound signals from industrial equipment during operation, perform denoising, framing, and windowing on the sound signals, and label the device status of each sound signal segment. Extract Mel spectrum features to describe the spectral characteristics of the sound signal; extract linear prediction features to analyze the periodic characteristics of the sound signal; extract pitch features to supplement the description of the periodic changes of the sound signal; Match the voiceprint features of historical sound signals with the voiceprint feature template as data samples for the convolutional neural network model; Convert the voiceprint features into a two-dimensional matrix and build a convolutional neural network model, including convolutional layers, pooling layers, fully connected layers, and output layers. Train the convolutional neural network model using the training set, adjust hyperparameters on the validation set, and evaluate the model's performance on the test set. The trained convolutional neural network model is deployed in the industrial equipment monitoring system to collect industrial equipment sound signals in real time and judge the operating status of industrial equipment.

10. The industrial equipment voiceprint monitoring method for equipment defect analysis according to claim 9, characterized in that: Extracting Mel spectrum features includes the following steps: Based on the processed sound signal, a Mel filter bank is designed, and a set of triangular filters is designed on the Mel frequency scale; Pass the voiceprint spectrum through the Mel filter bank and calculate the output energy of each filter; Take the logarithm of the output energy of each filter to obtain the logarithmic energy spectrum; Perform DCT discrete cosine transform on the logarithmic energy spectrum to obtain the cepstrum coefficients; The first-order and second-order differences of the MFCC Mel-frequency cepstral coefficients are calculated to capture dynamic features and describe the changes of features over time, thereby obtaining the spectral characteristics of the industrial equipment sound signal.

Images