Pump turbine health state detection method and system based on double models of sound, vibration and temperature

Abstract

The invention discloses a pump turbine health state detection method and system based on double models of sound, vibration and temperature. The method comprises the following steps: I, acquiring and transmitting multi-modal data; iI, data preprocessing; iII, feature extraction; iV, feature fusion; v, generalization model identification: inputting the multi-modal characteristic value as a first input value into the generalization model, calculating the first input value and the characteristic vector in the model library, and finally obtaining the final category, namely a first output value, of the section of signal through a scoring module; vI, classification model identification: inputting the multi-modal characteristic value into a classification model to obtain a second output value; vII, classification model self-training; vIII, performing unknown fault optimization; and IX, weight identification. The method has the advantages that the anomaly detection rate is far higher than that of other models, the advantages of an existing neural network are achieved, meanwhile, the defects of the neural network are overcome, and operation of the pump turbine is safer and more reliable.

Description

Technical Field

[0001] This invention relates to a method and system for detecting the health status of water pumps and turbines, and particularly to a method and system for detecting the health status of water pumps and turbines based on a dual-model approach of acoustic vibration and temperature. Background Technology

[0002] Pump-turbines, a new type of pumped-storage unit that emerged in the 1930s, are reversible hydraulic machines that function as pumps when the runner rotates forward and as turbines when it rotates backward. They are the core power equipment of pumped-storage power stations. The operational health of pump-turbines is crucial. Traditional detection methods involve collecting and analyzing various signals such as vibration, temperature, and stress. However, with the continuous development of sound-related technologies in recent years, it has been discovered that the sound spectrum emitted by equipment varies depending on its operating state. Therefore, methods for detecting the operational status of pump-turbines by collecting the sound during operation and analyzing its sound texture have been proposed.

[0003] However, current sound detection methods are relatively independent of traditional vibration and temperature signal detection methods, failing to link sound signals with vibration and temperature signals. Therefore, the detection dimension is relatively singular, limiting accuracy. Furthermore, existing methods for detecting the acoustic signature of water pumps and turbines fall into two categories: one involves pre-collecting sounds from all states of the water pump and turbine to build a pre-built acoustic signature database. This method is challenging, time-consuming, and inaccurate when the sample size is incomplete. The other method uses metric training, employing deep learning with a large number of irrelevant sounds to train a generalization model. This model then extracts feature vectors for similarity comparison. While the generalization model can be quickly deployed based on a small number of positive samples, its accuracy is limited in practice. Moreover, the generalization model relies on comparing each sample in the model database to obtain the final classification, which slows down with a large sample size.

[0004] Existing patent 202310451081.6, "A Method and Device for Detecting Anomalies in Hydro-generator Units under Multiple Operating Conditions Based on Sound Recognition," describes a detection method that requires pre-collection of sound data under all operating conditions of the turbine. However, even in a laboratory environment, it is difficult to collect sound data under all operating conditions, making pre-training challenging. Hydro-generator units generate vibration and temperature signals during operation, but this patent does not describe using these signals for health monitoring. Existing patent 202510193129.7, "A Method and System for Detecting Anomalies in Pump Units Based on Voiceprint Recognition," describes identifying abnormal pump states using a pre-trained convolutional neural network model. The pre-trained model's data comes from historical operating data, but historical operating data cannot cover all current operating states of the equipment. Therefore, when unknown anomalies exist, the accuracy of identification will decrease. Furthermore, in the early stages of system operation when historical data is limited, the accuracy of the convolutional neural network model trained on historical data will be even lower. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for detecting the health status of water pumps and turbines based on a dual model of acoustic-vibration-temperature, which is more sensitive to vibration signal capture, better resistant to environmental interference, has higher recognition accuracy, and higher system error detection rate.

[0006] To address the aforementioned technical problems, the present invention provides a pump-turbine health status detection method based on a dual-model approach of acoustic vibration and temperature, comprising the following steps:

[0007] I. Multimodal data acquisition and transmission: Real-time multimodal data is acquired during the operation of the water pump and turbine, specifically including sound signals, vibration signals, and temperature signals;

[0008] II. Data Preprocessing: The sound signal is pre-emphasized to achieve high-frequency compensation, and the vibration signal is detrended using polynomial fitting; both the sound and vibration signals are framed and windowed.

[0009] III. Feature Extraction: First, FFT transformation is performed on the preprocessed sound signal and vibration signal to obtain frequency domain features; second, the FFT result is quantized by taking the modulus and squares to obtain the frequency domain energy distribution; then, the frequency domain energy distribution is divided into equal intervals on the Mel scale according to the number of filters, and the Mel scale and frequency are nonlinearly transformed and the output result is transformed by DCT; finally, the temperature data is resampled and the gradient of each data point is obtained.

[0010] IV. Feature Fusion: The features of sound signals, vibration signals, and temperature signals are fused to obtain multimodal feature values;

[0011] V. Generalization Model Recognition: The multimodal feature values ​​are input into the generalization model as the first input value, and calculated with the feature vectors in the model library. Finally, the final category of the signal segment, i.e. the first output value, is obtained through the scoring module.

[0012] VI. Classification Model Recognition: Input the multimodal feature values ​​into the classification model to obtain the second output value; when the accuracy of the classification model reaches more than 98% in nearly 100 trials and the recognition rate is higher than that of the generalization model and the average recognition time of the generalization model is greater than the system configuration time, then step V is skipped and the classification model recognition is directly entered.

[0013] VII. Classification Model Self-Training: When the classification model has been trained less than 1000 times, or has fewer than 5 classification categories, or has an accuracy of less than 70% after nearly 100 training iterations, the second output value is not involved in the final decision. In this case, the classification model will perform self-training, that is, using the first input value recognized by the generalization model as the driving force, and using the first output value, the second output value, and the multimodal feature values ​​together as the input to train the model, and performing hot replacement after the model training is completed.

[0014] VIII. Unknown Fault Optimization: If the category of the first output value is an unknown anomaly, then the first output value is stored in the generalization model library and the classification model is driven to self-train.

[0015] IX. Weight Identification: When both the first and second output values ​​participate in the final decision, if only one is identified as an anomaly, that anomaly becomes the final result. When both the first and second output values ​​are identified as anomalies, but the first output value is an unknown anomaly, proceed to step VIII regardless of the second output value. If both are identified as anomalies, and the first output value is not an unknown anomaly, the operator will make the final decision. If the operator determines that the first output value is correct, the classification model will be self-trained. If the operator determines that the second output value is correct, the first input value will be added to the model library of the generalization model.

[0016] The data acquisition method in step I involves installing acoustic sensors, vibration sensors, and temperature sensors on the top cover of the water pump turbine, the bearings, and the motor housing.

[0017] The time-domain expression for the pre-emphasis processing in step II to achieve high-frequency compensation is as follows:

[0018] y(n)=x(n)-αx(n-1)(α=0.95)

[0019] Where x(n) is the sampled value of the current frame, x(n-1) is the sampled value of the previous frame, and y(n) is the pre-emphasized value of the current frame;

[0020] The current frame is divided into frames, and the formula for the dimensions after frame division is:

[0021] Frames[i,j] = y[(i-1)×n+j]

[0022] Where i represents the frame number, j represents the value of each sample point in the frame, n represents the sample point corresponding to the frame shift, and y represents the original audio sample array. After frame division, the original audio sample array is converted into Frames[i,j] for output.

[0023] The framed audio data is windowed using a Hamming window, the function of which is:

[0024] w[n]=0.54-0.46cos(2πn / (N-1))(0≤n≤N-1)

[0025] Where N is the total length of the frame after framing, i.e. the total length of the window function, and n is the sampling point number of the frame. After substituting the parameters, the Hamming window coefficient can be calculated. Then, the Hamming window coefficient and the output result after framing are multiplied point by point to obtain the output after windowing.

[0026] The detrending term is fitted using a polynomial, as shown in the following formula:

[0027] y_detrended=y_original-(a0+a1 t+a2 t 2 +...+a n t n )

[0028] Where y_original is the original time series data, containing the signal with trend and noise, t is the time or location index of the data point in the time series, and a0, a1, ..., a n These are polynomial fitting coefficients, calculated from the data using the least squares method. They are the mathematical form describing the trend, a0 + a1t + ... + a n t n The trend term is the fitted term, representing the slowly changing non-random component in the data, and y_detrended is the output after the trend term is removed;

[0029] The vibration signal is framed and windowed using the same frame shift and frame length as the speech signal to ensure that the number of frames is the same after framing. The vibration signal is windowed using a Hann window to balance frequency resolution and sidelobe attenuation, as shown in the following formula:

[0030]

[0031] Where n is the sequence of sampling points for each frame, ranging from 0 to N-1, and N is the total length of a frame.

[0032] In step III, the output dimensions of the preprocessed sound and vibration signals are (i,j), where i is the sequence of each frame after framing, ranging from 0 to the maximum frame number, and j is the corresponding time-domain sequence of that frame, ranging from 0 to the maximum number of points in a frame. The value of a certain time point in a certain frame can be determined through the two dimensions i and j. The formula for obtaining the frequency domain features by performing FFT transformation on the data of each frame is as follows:

[0033]

[0034] Where N is the maximum number of points in a frame, x(n) is the time-domain signal, and x(k) is the frequency-domain signal output;

[0035] The formula for nonlinearly converting the Mel scale and frequency is as follows:

[0036]

[0037] Where f is the frequency, and the output Mel(f) is the Mel scale. The center frequency of each filter is obtained by the above formula. Then, the FFT result is multiplied by the frequency of each filter and accumulated. The value obtained is the energy value of the frame data in the corresponding frequency band of the filter. The output dimension should be (i,n), where i is the total number of frames and n is the number of Mel filters.

[0038] The energy value is subjected to DCT transformation, and the sound signal and vibration signal retain half the number of filter coefficients respectively, which are the corresponding MFCC coefficients.

[0039] In step III, the number of resampled temperature signal data points is the same as the total number of frames for the sound and vibration signals; the gradient of each data point is calculated using the following gradient calculation formula, and temperature abrupt change detection is introduced:

[0040]

[0041] Where μt is the average temperature of frame t in °C, μt-1 is the average temperature of frame t-1, Δt is the time interval between two frames, and the output is the gradient of that frame.

[0042] In step IV, the feature dimension of the temperature signal is (i,m), the feature dimension of the sound signal is (i,n), and the feature dimension of the vibration signal is (i,s), where i is the total number of frames representing the time dimension, m is 2, representing the frame temperature and gradient of the temperature, n is the MFCC of the sound, and s is the MFCC of the vibration. The feature dimension output after feature fusion is (i,n+s+m).

[0043] The specific process of generalization model identification in step V is as follows: The generalization model includes X-Vector and ECAPA-TDNN, and the loss function of the generalization model adopts AM-Softmax:

[0044]

[0045] Where cosθ is the cosine similarity between the feature vector and the true class weight vector, and N is the number of samples in this batch;

[0046] After the generalization model outputs a feature vector, the scoring module ultimately outputs the final category of the signal segment. The scoring module calculates the similarity between the feature vector and each category using a scoring algorithm that includes the mean, variance, and the highest n terms. The feature vector output by the generalization model is first compared with all samples in the first category using the cosine similarity formula to obtain the similarity between the feature vector and each sample in the first category.

[0047] The cosine similarity formula is as follows:

[0048]

[0049] Where v1 and v2 are two voiceprint embedding vectors output by the generalization model;

[0050] The similarity of each sample in the first category is calculated using the scoring algorithm to obtain the average similarity of the first category. This average similarity is the similarity between the feature vector and the first category. Repeating this process yields the similarity between the feature vector and each category. Finally, the maximum value is taken, which is the most similar category matched by the feature vector in the model library and is used as the first output value. The formula for the average value is as follows:

[0051]

[0052] Where n is the number of samples in each category, x i The output x represents the similarity value between the i-th element of the class and the feature vector. The output x represents the average similarity of the samples in the class, which is the similarity between the class and the feature vector.

[0053] The classification models in step VI include ResNet and VGG; the classification models identify and output the identified categories through pre-trained models as the second output value.

[0054] The system for detecting the health status of water pumps and turbines based on a dual-model approach involving acoustic vibration and temperature includes the following steps:

[0055] A. Acquisition Module: It adopts two modes: interval acquisition and real-time acquisition. The acquisition sensor collects sound, vibration and temperature signals of a specified time series in each round, and gathers them at the acquisition gateway, which then distributes the signals.

[0056] B. Feature Extraction Module: This module performs preprocessing, feature extraction, and feature fusion on sound, vibration, and temperature signals simultaneously to obtain multimodal feature values ​​in the time series.

[0057] C. Generalization Recognition Module: The generalization model is used to identify multimodal feature values, and the feature vector is calculated. The feature vector and the samples in the generalization model library are sent to the scoring module for comparison. The scoring module is configured with a variety of scoring algorithms, including the average value, variance, and taking the highest n terms, so as to obtain the category most similar to the feature vector, which is used as the first output value.

[0058] D. Classification and Recognition Module: This module uses multimodal feature values ​​as input, identifies the data through a pre-trained classification model, and outputs the identified category as the second output value. When the classification model has fewer than 1000 training iterations, fewer than 5 classification categories, or an accuracy rate of less than 70% in the last 100 iterations, only the feature values ​​from this time series and the first output value are used for self-training of the classification model. When the classification model achieves an accuracy rate of over 98% in the last 100 iterations, and this recognition rate is higher than that of the generalization model, and the average recognition time of the generalization model is greater than the system configuration time, then generalization recognition is skipped, and the module directly proceeds to the classification model for recognition.

[0059] E. Weight Recognition Module: When both the first and second output values ​​participate in the final decision, if either is identified as an anomaly, that anomaly becomes the final result. When both the first and second output values ​​are identified as anomalies, but the first output value is an unknown anomaly, proceed to step VIII regardless of the second output value. If both are identified as anomalies, and the first output value is not an unknown anomaly, the operator will make the final decision. If the operator determines that the first output value is correct, the classification model will be self-trained. If the operator determines that the second output value is correct, the first input value will be added to the model library of the generalization model.

[0060] F. Early Warning Module: When the first output value is identified as an abnormal type, an early warning is generated; when one or more of the first and second output values ​​are abnormal, an early warning is generated; the category of the feature value is calibrated according to the actual working condition of the water pump and turbine, and the addition of a generalization model library or self-training of a classification model is triggered.

[0061] Advantages of this invention:

[0062] (1) By inputting multimodal data such as sound, vibration and temperature into the model, the model can learn more operating parameters of water pumps and turbines, thereby improving the model's recognition accuracy;

[0063] (2) The traditional classification model is self-trained by driving the generalization model, and joint detection is finally achieved through the weight recognition module. The generalization model can solve the problem of insufficient pre-training data of traditional neural networks, and can be used with a small amount of data. When the amount of data is small, the traditional classification model is not suitable for use. The input and output of the generalization model can be used as parameters for training the classification model to achieve self-training, which solves the problems of difficult data collection and long training cycle of traditional classification models. When the program runs for a long time and the sample library is large, the generalization model uses the cosine similarity formula and calculates with all samples. The time complexity and the number of samples increase linearly, and the recognition time will gradually increase. At this time, the classification model can solve this problem.

[0064] (3) High error detection rate: For water pump turbines, if an anomaly is not detected in time, it may eventually lead to significant personnel and economic losses. This invention uses acoustic, vibration and temperature multimodal signal features and two different types of models for detection. The anomaly detection rate is much higher than that of other models. While combining the advantages of the neural network currently used for water pump turbine condition health detection, it uses dual models and self-training methods to make up for their respective shortcomings, making the operation of water pump turbines safer and more reliable. Attached Figure Description

[0065] Figure 1 The flowchart shows the pump-turbine health status detection system based on a dual-model of acoustic vibration and temperature according to the present invention.

[0066] Figure 2 The flowchart shows a method for detecting the health status of a water pump turbine based on a dual-model approach of acoustic vibration and temperature.

[0067] Figure 3 The flowchart shows the specific process for generalization model recognition and classification model recognition. Detailed Implementation

[0068] The following detailed description of the pump-turbine health status detection method and system based on a dual-model approach of acoustic vibration and temperature according to the present invention, with reference to the accompanying drawings and specific embodiments, is further provided.

[0069] Example:

[0070] like Figure 2 As shown, the method for detecting the health status of a water pump turbine based on a dual-model approach of acoustic vibration and temperature according to the present invention includes the following steps:

[0071] I. Multimodal data acquisition and transmission: Real-time multimodal data during the operation of the water pump turbine is collected by installing sound sensors, vibration sensors, and temperature sensors in appropriate locations such as the top cover, bearings, and motor housing. The data specifically includes sound signals, vibration signals, and temperature signals.

[0072] II. Data Preprocessing: The audio signal is pre-emphasized to achieve high-frequency compensation. The time-domain expression for high-frequency compensation with pre-emphasis is:

[0073] y(n)=x(n)-αx(n-1)(α=0.95)

[0074] Where x(n) is the sampled value of the current frame, x(n-1) is the sampled value of the previous frame, and y(n) is the pre-emphasized value of the current frame;

[0075] Speech signals exhibit time-varying characteristics, but within a short time range, their characteristics remain essentially unchanged, i.e., relatively stable. Frame length represents the length of each frame after framing, measured in milliseconds (ms). Frame shift represents the overlap between two adjacent frames, also measured in milliseconds. Once the frame length and frame shift are determined, framing can be performed. The formula for the dimensions after framing is:

[0076] Frames[i,j] = y[(i-1)×n+j]

[0077] Where i represents the frame number, j represents the value of each sample point in the frame, n represents the sample point corresponding to the frame shift, and y represents the original audio sample array. After frame division, the original audio sample array is converted into Frames[i,j] for output.

[0078] To address the spectrum leakage issue, the framed data is windowed using a Hamming windowing process. The function is as follows:

[0079] w[n]=0.54-0.46cos(2πn / (N-1))(0≤n≤N-1)

[0080] Where N is the total length of the next frame after framing, i.e., the total length of the window function, and n is the sampling point number of that frame.

[0081] After inputting the parameters, the Hamming window coefficients can be calculated. Then, the Hamming window coefficients and the output results after frame segmentation are multiplied point by point to obtain the output after windowing.

[0082] Vibration signals often suffer from various interferences, which can easily cause zero drift and baseline deviation. A polynomial fitting method is used to remove the trend term, as shown in the following formula:

[0083] y_detrended=y_original-(a0+a1 t+a2 t 2 +...+a n t n )

[0084] Where y_original is the original time series data, containing the signal with trend and noise, t is the time or location index of the data point in the time series representation, and a0, a1, ..., a n These are polynomial fitting coefficients, calculated from the data using the least squares method. They are the mathematical form describing the trend, a0 + a1t + ... + a n t n The trend term is the fitted term, representing the slowly changing non-random component in the data, and y_detrended is the output after the trend term is removed;

[0085] The vibration signal is framed and windowed using the same frame shift and frame length as the speech signal to ensure that the number of frames is the same after framing. The Hann window is used for the vibration signal to better balance frequency resolution and sidelobe attenuation. The formula is as follows:

[0086]

[0087] Where n is the sequence of sampling points for each frame, ranging from 0 to N-1, and N is the total length of a frame.

[0088] III. Feature Extraction: The preprocessed sound and vibration signals are subjected to FFT transformation to obtain frequency domain features; then, the FFT results are quantized by taking the modulus and squares to obtain the frequency domain energy distribution; then, the Mel scale is divided into equal intervals according to the number of filters, and the Mel scale and frequency are nonlinearly transformed and the output results are subjected to DCT transformation; finally, the temperature data is resampled and the gradient of each data point is obtained.

[0089] Specifically, the output dimension of the preprocessed sound and vibration signals is (i,j), where i represents each frame sequence after framing, ranging from 0 to the maximum frame number, and j represents the corresponding time-domain sequence, ranging from 0 to the maximum number of points in a frame. The value at a specific time point within a frame can be determined using the two dimensions i and j. Each frame's data is then subjected to an FFT (Fast Fourier Transform) to obtain the frequency domain features, as shown in the following formula:

[0090]

[0091] Where N is the maximum number of points in a frame, x(n) is the time-domain signal, and x(k) is the frequency-domain signal output;

[0092] Power spectrum: The complex values ​​(including the imaginary part) of the FFT output have no direct physical meaning. They only reflect the amplitude and phase of the frequency components. The frequency domain energy distribution is obtained by taking the modulus and squaring the FFT result.

[0093] Mel spectrum: The second dimension of the power spectrum is usually quite large, and human frequency perception is not linearly correlated. Therefore, the power spectrum contains a lot of redundant information and exposes details of all frequency bands, including interference signals, resulting in insufficient robustness. Obtaining the Mel spectrum through Mel filters can reduce computation and improve accuracy. To obtain the center frequency of each Mel filter, the Mel scale must first be divided at equal intervals according to the number of filters. Then, the Mel scale and frequency are converted. The nonlinear conversion formula from frequency to Mel scale is as follows:

[0094]

[0095] Where f is the frequency, and the output Mel(f) is the Mel scale. Typically, 20-40 Mel filters are selected as a group. The center frequency of each filter is obtained through the above formula. Then, the FFT result is multiplied by the frequency of each filter and accumulated. The resulting value is the energy value of the frame data in the corresponding frequency band of the filter. The output dimension should be (i,n), where i is the total number of frames and n is the number of Mel filters.

[0096] Performing a DCT transformation on the energy value, the sound signal and vibration signal retain half the number of coefficients of the filter, which are the corresponding MFCC coefficients;

[0097] Temperature resampling and gradient: Temperature sensors typically have a sampling frequency of 1-100 Hz. To perform multimodal fusion, temperature data needs to be resampled. The number of resampled data points is the same as the total number of frames for sound and vibration. The gradient of each data point is calculated using the following gradient calculation formula, which is then used to detect sudden temperature changes:

[0098]

[0099] μt is the average temperature of frame t (in °C), and μt-1 is the average temperature of frame (t-1). Δt is the time interval between two frames, and the output is the gradient of that frame.

[0100] IV. Feature Fusion: The features of sound signals, vibration signals, and temperature signals are fused to obtain multimodal feature values. The feature dimension of temperature is (i,m), the feature dimension of sound signal is (i,n), and the feature dimension of vibration signal is (i,s). Here, i is the total number of frames representing the time dimension, m is 2, representing the frame temperature and gradient of temperature, n is the MFCC of sound, and s is the MFCC of vibration. The feature dimension output after feature fusion is (i,n+s+m).

[0101] V. Dual-model diagnostics – Generalization model identification: such as Figure 3As shown, dual-model diagnosis uses a generalization model and a classification model for anomaly identification; multimodal feature values ​​are input into the generalization model as the first input value, and are calculated with the feature vectors in the model library. Finally, the final category of the signal segment, i.e., the first output value, is obtained through the scoring module.

[0102] Specifically, generalized model recognition refers to using multimodal feature values ​​obtained through feature fusion as input values ​​to a generalized model (specific generalized models include X-Vector, ECAPA-TDNN, etc.). The characteristic of generalized models is that they do not require a large number of pump and turbine samples during training; instead, they use a large amount of readily collectable data from nature as training samples. The ultimate goal is to train the model's measurement capabilities. Therefore, generalized models solve the problem of difficult sample collection for pump and turbine operation. The loss function of generalized models typically uses AM-Softmax to give features larger inter-class distances and smaller intra-class distances.

[0103]

[0104] cosθ is the cosine similarity between the feature vector and the true class weight vector, and N is the number of samples in this batch.

[0105] The generalization model uses a neural network to calculate a feature vector from the input values, rather than simply classifying them. It ensures that feature vectors from similar features have smaller inter-class distances, while those from different features have larger inter-class distances. After outputting the feature vector, the model calculates the similarity between the feature vector and the feature vectors in the model library using the cosine similarity formula. The scoring module then outputs the final category of the signal segment. This scoring module is part of the generalization recognition module. Each category in the model library contains a certain number of samples. The cosine similarity formula is used to calculate the similarity between the feature vector and the samples, and the scoring algorithm is used to calculate the similarity between the feature vector and each category. The scoring module incorporates several hot-update scoring algorithms, such as average, variance, and taking the highest n items, to adapt to different scenarios. The default algorithm is the average. The model library contains n possible result categories, each containing multiple samples. By first calculating the cosine similarity between the feature vector output by the generalization model and all samples in the first category, the similarity between the feature vector and each sample in the first category can be obtained.

[0106] The formula for cosine similarity is as follows:

[0107]

[0108] Where v1 and v2 are two voiceprint embedding vectors output by the generalization model.

[0109] The average similarity of each sample in the first category can be calculated using the average formula (which can be replaced with variance, the highest n terms, etc., depending on the situation). This average similarity is the similarity between the feature vector and the first category. Repeating the above process, the similarity between the feature vector and each category can be calculated. Finally, the maximum value is taken, which is the most similar category matched by the feature vector in the model library. This value is used as the first output value. The average formula is as follows:

[0110]

[0111] Where n is the number of samples in each class, xi is the similarity value between the i-th element of the class and the feature vector, and the output x represents the average similarity of the samples in the class, which is the similarity between the class and the feature vector;

[0112] If the classification model achieves an accuracy of over 98% in nearly 100 trials, and this recognition rate is higher than that of the generalization model, and the average recognition time of the generalization model is greater than the system configuration time, then generalization recognition will not be performed, and the classification model will be used for recognition.

[0113] VI. Classification Model Recognition: Input the multimodal feature values ​​into the classification model to obtain the second output value; when the accuracy of the classification model reaches more than 98% in nearly 100 trials and the recognition rate is higher than that of the generalization model and the average recognition time of the generalization model is greater than the system configuration time, then step V is skipped and the classification model recognition is directly entered.

[0114] Specifically, classification models include ResNet and VGG. These models require pre-training with a large amount of operational data from pumps and turbines under various conditions. The pre-trained model identifies the class, which is then used as a second output value. When the traditional classification model has fewer than 1000 training iterations, fewer than 5 classification categories, or an accuracy of less than 70% after nearly 100 iterations, the second output value is not used in the final decision.

[0115] VII. Classification Model Self-Training: When the second output value does not meet the final decision condition, the classification model will perform self-training. That is, it uses the first input value recognized by the generalization model as the driving force, and uses the first output value, the second output value and the multimodal feature value together as the input to train the model, and performs hot replacement after the model training is completed.

[0116] Traditional classification neural network training requires input sample data and manual labeling of the true category of the sample. The model then identifies a classification type, and the identified category and the true category are fed into a loss function. The model parameters are then adjusted through backpropagation based on the loss function to achieve the training effect. This invention eliminates the need for manual labeling. Instead, it uses the first input value identified by the generalization model as the driving force, and uses the first output value, second output value, and input value together as inputs to train the model. After training, the model is hot-replaced. If the second output value meets the final decision, it is processed by the weight recognition module to see if self-training is needed. As the project's operating cycle lengthens, after the classification model is pre-trained with a large amount of data on the operation of water pumps and turbines under various states, the accuracy of the second output value will gradually surpass that of the generalization model, and the recognition speed will also be faster than that of the generalization model.

[0117] VIII. Unknown Fault Optimization: If the category of the first output value is an unknown anomaly, then the first output value is stored in the generalization model library and the classification model is driven to self-train.

[0118] If the generalization model outputs an unknown anomaly after passing through the scoring module, it means that the similarity between this data segment and any class in the feature vector library does not exceed the threshold, which is a new anomaly type. The system will alert the operator, who will then conduct a comprehensive inspection to determine the anomaly type. The system will then store the first output value of this segment in the generalization model library and perform classification model recognition and self-training.

[0119] IX. Weight Identification: When both the first and second output values ​​participate in the final decision, if only one is identified as an anomaly, that anomaly becomes the final result. When both the first and second output values ​​are identified as anomalies, but the first output value is an unknown anomaly, proceed to step VIII regardless of the second output value. If both are identified as anomalies, and the first output value is not an unknown anomaly, the operator will make the final decision. If the operator determines that the first output value is correct, the classification model will be self-trained. If the operator determines that the second output value is correct, the first input value will be added to the model library of the generalization model.

[0120] like Figure 2 As shown, the system for detecting the health status of a pump-turbine based on a dual-model approach involving acoustics, vibration, and temperature includes the following steps:

[0121] A. Acquisition Module: It adopts two modes: interval acquisition and real-time acquisition. The acquisition sensor collects sound, vibration and temperature signals of a specified time series in each round, and gathers them at the acquisition gateway, which then distributes the signals.

[0122] B. Feature Extraction Module: This module performs preprocessing, feature extraction, and feature fusion on sound, vibration, and temperature signals simultaneously to obtain multimodal feature values ​​in the time series.

[0123] C. Generalization Recognition Module: The generalization model is used to identify multimodal feature values, and the feature vector is calculated. The feature vector and the samples in the generalization model library are sent to the scoring module for comparison. The scoring module is configured with a variety of scoring algorithms, including the average value, variance, and taking the highest n terms, so as to obtain the category most similar to the feature vector, which is used as the first output value.

[0124] D. Classification and Recognition Module: This module uses multimodal feature values ​​as input, identifies the data through a pre-trained classification model, and outputs the identified category as the second output value. When the classification model has fewer than 1000 training iterations, fewer than 5 classification categories, or an accuracy rate of less than 70% in the last 100 iterations, only the feature values ​​from this time series and the first output value are used for self-training of the classification model. When the classification model achieves an accuracy rate of over 98% in the last 100 iterations, and this recognition rate is higher than that of the generalization model, and the average recognition time of the generalization model is greater than the system configuration time, then generalization recognition is skipped, and the module directly proceeds to the classification model for recognition.

[0125] E. Weight Recognition Module: When both the first and second output values ​​participate in the final decision, if either is identified as an anomaly, that anomaly becomes the final result. When both the first and second output values ​​are identified as anomalies, but the first output value is an unknown anomaly, proceed to step VIII regardless of the second output value. If both are identified as anomalies, and the first output value is not an unknown anomaly, the operator will make the final decision. If the operator determines that the first output value is correct, the classification model will be self-trained. If the operator determines that the second output value is correct, the first input value will be added to the model library of the generalization model.

[0126] F. Early Warning Module: When the first output value is identified as an abnormal type, an early warning is generated; when one or more of the first and second output values ​​are abnormal, an early warning is generated; the category of the feature value is calibrated according to the actual working condition of the water pump and turbine, and the addition of a generalization model library or self-training of a classification model is triggered.

Claims

1. A method for detecting the health status of a water pump turbine based on a dual-model approach of acoustic vibration and temperature, characterized in that, Includes the following steps: I. Multimodal data acquisition and transmission: Real-time multimodal data is acquired during the operation of the water pump and turbine, specifically including sound signals, vibration signals, and temperature signals; II. Data Preprocessing: The sound signal is pre-emphasized to achieve high-frequency compensation, and the vibration signal is detrended using polynomial fitting; both the sound and vibration signals are framed and windowed. III. Feature Extraction: First, FFT transformation is performed on the preprocessed sound signal and vibration signal to obtain frequency domain features; second, the FFT result is quantized by taking the modulus and squares to obtain the frequency domain energy distribution; then, the frequency domain energy distribution is divided into equal intervals on the Mel scale according to the number of filters, and the Mel scale and frequency are nonlinearly transformed and the output result is transformed by DCT; finally, the temperature data is resampled and the gradient of each data point is obtained. IV. Feature Fusion: The features of sound signals, vibration signals, and temperature signals are fused to obtain multimodal feature values; V. Generalization Model Recognition: The multimodal feature values ​​are input into the generalization model as the first input value, and calculated with the feature vectors in the model library. Finally, the final category of the signal segment, i.e. the first output value, is obtained through the scoring module. VI. Classification Model Recognition: Input the multimodal feature values ​​into the classification model to obtain the second output value; If the classification model achieves an accuracy of over 98% in nearly 100 trials, and this recognition rate is higher than that of the generalization model, and the average recognition time of the generalization model is greater than the system configuration time, then step V is skipped, and the classification model recognition proceeds directly. VII. Classification Model Self-Training: When the classification model has been trained less than 1000 times, or has fewer than 5 classification categories, or has an accuracy of less than 70% after nearly 100 training iterations, the second output value is not involved in the final decision. In this case, the classification model will perform self-training, that is, using the first input value recognized by the generalization model as the driving force, and using the first output value, the second output value, and the multimodal feature values ​​together as the input to train the model, and performing hot replacement after the model training is completed. VIII. Unknown Fault Optimization: If the category of the first output value is an unknown anomaly, then the first output value is stored in the generalization model library and the classification model is driven to self-train. IX. Weight Identification: When both the first and second output values ​​participate in the final decision, if only one is identified as an anomaly, that anomaly becomes the final result. When both the first and second output values ​​are identified as anomalies, but the first output value is an unknown anomaly, proceed to step VIII regardless of the second output value. If both are identified as anomalies, and the first output value is not an unknown anomaly, the operator will make the final decision. If the operator determines that the first output value is correct, the classification model will be self-trained. If the operator determines that the second output value is correct, the first input value will be added to the model library of the generalization model.

2. The method for detecting the health status of a water pump turbine based on a dual-model approach of acoustic vibration and temperature as described in claim 1, characterized in that: The data acquisition method in step I involves installing acoustic sensors, vibration sensors, and temperature sensors on the top cover of the water pump turbine, the bearings, and the motor housing.

3. The method for detecting the health status of a water pump turbine based on a dual-model approach of acoustic vibration and temperature as described in claim 1, characterized in that: The time-domain expression for the pre-emphasis processing in step II to achieve high-frequency compensation is as follows: y(n)=x(n)-αx(n-1)(α=0.95) Where x(n) is the sampled value of the current frame, x(n-1) is the sampled value of the previous frame, and y(n) is the pre-emphasized value of the current frame; The current frame is divided into frames, and the formula for the dimensions after frame division is: Frames[i,j] = y[(i-1)×n+j] Where i represents the frame number, j represents the value of each sample point in the frame, n represents the sample point corresponding to the frame shift, and y represents the original audio sample array. After frame division, the original audio sample array is converted into Frames[i,j] for output. The framed audio data is windowed using a Hamming window, the function of which is: w[n]=0.54-0.46cos(2πn / (N-1))(0≤n≤N-1) Where N is the total length of the frame after framing, i.e. the total length of the window function, and n is the sampling point number of the frame. After substituting the parameters, the Hamming window coefficient can be calculated. Then, the Hamming window coefficient and the output result after framing are multiplied point by point to obtain the output after windowing. The detrending term is fitted using a polynomial, as shown in the following formula: y_detrended=y_original-(a0+a1t+a2t 2 +...+ant n ) Where y_original is the original time series data, containing the trend and noise signals; t is the time or location index of the data point in the time series representation; a0, a1, ..., an are polynomial fitting coefficients, calculated from the data using the least squares method, representing the mathematical form of the trend: a0 + a1t + ... + ant. n The trend term is the fitted term, representing the slowly changing non-random component in the data, and y_detrended is the output after the trend term is removed; The vibration signal is framed and windowed using the same frame shift and frame length as the speech signal to ensure that the number of frames is the same after framing. The vibration signal is windowed using a Hann window to balance frequency resolution and sidelobe attenuation, as shown in the following formula: Where n is the sequence of sampling points for each frame, ranging from 0 to N-1, and N is the total length of a frame.

4. The method for detecting the health status of a water pump turbine based on a dual-model of acoustic vibration and temperature according to claim 1, characterized in that: In step III, the output dimension of the preprocessed sound signal and vibration signal is (i,j), where i is the sequence of each frame after framing, from 0 to the maximum number of frames, and j is the temporal sequence corresponding to the frame, from 0 to the maximum number of points in a frame. The value at a specific time point in a frame can be determined using the two dimensions i and j. The formula for obtaining the frequency domain features by performing an FFT on the data of each frame is as follows: Where N is the maximum number of points in a frame, x(n) is the time-domain signal, and x(k) is the frequency-domain signal output; The formula for nonlinearly converting the Mel scale and frequency is as follows: Where f is the frequency, and the output Mel(f) is the Mel scale. The center frequency of each filter is obtained by the above formula. Then, the FFT result is multiplied by the frequency of each filter and accumulated. The value obtained is the energy value of the frame data in the corresponding frequency band of the filter. The output dimension should be (i,n), where i is the total number of frames and n is the number of Mel filters. The energy value is subjected to DCT transformation, and the sound signal and vibration signal retain half the number of filter coefficients respectively, which are the corresponding MFCC coefficients.

5. The method for detecting the health status of a water pump turbine based on a dual-model approach of acoustic vibration and temperature as described in claim 1, characterized in that: In step III, the number of temperature signal data points after resampling is the same as the total number of frames of the sound signal and vibration signal. The gradient of each data point is calculated using the following gradient calculation formula, which is then used to detect temperature abrupt changes: Where μt is the average temperature of frame t in °C, μt-1 is the average temperature of frame t-1, Δt is the time interval between two frames, and the output is the gradient of that frame.

6. The method for detecting the health status of a water pump turbine based on a dual-model acoustic-vibration-temperature system according to claim 1, characterized in that: In step IV, the feature dimension of the temperature signal is (i,m), the feature dimension of the sound signal is (i,n), and the feature dimension of the vibration signal is (i,s), where i is the total number of frames representing the time dimension, m is 2, representing the frame temperature and gradient of the temperature, n is the MFCC of the sound, and s is the MFCC of the vibration. The feature dimension output after feature fusion is (i,n+s+m).

7. The method for detecting the health status of a water pump turbine based on a dual-model of acoustic vibration and temperature according to claim 1, characterized in that, The specific process of generalization model identification in step V is as follows: The generalization model includes X-Vector and ECAPA-TDNN, and the loss function of the generalization model adopts AM-Softmax: Where cosθ is the cosine similarity between the feature vector and the true class weight vector, and N is the number of samples in this batch; After the generalization model outputs a feature vector, the scoring module ultimately outputs the final category of the signal segment. The scoring module calculates the similarity between the feature vector and each category using a scoring algorithm that includes the mean, variance, and the highest n terms. The feature vector output by the generalization model is first compared with all samples in the first category using the cosine similarity formula to obtain the similarity between the feature vector and each sample in the first category. The cosine similarity formula is as follows: Where v1 and v2 are two voiceprint embedding vectors output by the generalization model; The similarity of each sample in the first category is calculated using the scoring algorithm to obtain the average similarity of the first category. This average similarity is the similarity between the feature vector and the first category. Repeating this process yields the similarity between the feature vector and each category. Finally, the maximum value is taken, which is the most similar category matched by the feature vector in the model library and is used as the first output value. The formula for the average value is as follows: Where n is the number of samples in each category, x i The output x represents the similarity value between the i-th element of the class and the feature vector. The output x represents the average similarity of the samples in the class, which is the similarity between the class and the feature vector.

8. The method for detecting the health status of a water pump turbine based on a dual-model of acoustic vibration and temperature according to claim 1, characterized in that: The classification models in step VI include ResNet and VGG; the classification models identify and output the identified categories through pre-trained models as the second output value.

9. A system for detecting the health status of a water pump turbine based on a dual-model approach of acoustic vibration and temperature as described in any one of claims 1-8, characterized in that, Includes the following steps: A. Acquisition Module: It adopts two modes: interval acquisition and real-time acquisition. The acquisition sensor collects sound, vibration and temperature signals of a specified time series in each round, and gathers them at the acquisition gateway, which then distributes the signals. B. Feature Extraction Module: This module performs preprocessing, feature extraction, and feature fusion on sound, vibration, and temperature signals simultaneously to obtain multimodal feature values ​​in the time series. C. Generalization Recognition Module: The generalization model is used to identify multimodal feature values, and the feature vector is calculated. The feature vector and the samples in the generalization model library are sent to the scoring module for comparison. The scoring module is configured with a variety of scoring algorithms, including the average value, variance, and taking the highest n terms, so as to obtain the category most similar to the feature vector, which is used as the first output value. D. Classification and Recognition Module: Taking multimodal feature values ​​as input values, the module identifies the categories through a pre-trained classification model and outputs the identified categories as the second output value. When the classification model has been trained less than 1000 times, has fewer than 5 categories, or has an accuracy of less than 70% in the last 100 training iterations, only the feature values ​​and the first output value of the time series are used for self-training of the classification model; when the accuracy of the classification model in the last 100 training iterations reaches more than 98%, and the recognition rate is higher than that of the generalization model, and the average recognition time of the generalization model is greater than the system configuration time, then generalization recognition is not performed, and the classification model recognition is directly entered. E. Weight Recognition Module: When both the first and second output values ​​participate in the final decision, if either is identified as an anomaly, that anomaly becomes the final result. When both the first and second output values ​​are identified as anomalies, but the first output value is an unknown anomaly, proceed to step VIII regardless of the second output value. If both are identified as anomalies, and the first output value is not an unknown anomaly, the operator will make the final decision. If the operator determines that the first output value is correct, the classification model will be self-trained. If the operator determines that the second output value is correct, the first input value will be added to the model library of the generalization model. F. Early Warning Module: When the first output value is identified as an abnormal type, an early warning is generated; when one or more of the first and second output values ​​are abnormal, an early warning is generated; the category of the feature value is calibrated according to the actual working condition of the water pump and turbine, and the addition of a generalization model library or self-training of a classification model is triggered.

Images