Power transformation equipment anomaly detection method and device based on voiceprint

Abstract

The invention discloses a voiceprint-based power transformation equipment anomaly detection method and device, relates to the field of power transformation equipment monitoring, and can solve the technical problem of low accuracy of existing voiceprint-based power transformation equipment anomaly detection. The voiceprint-based power transformation equipment anomaly detection method comprises the steps of obtaining historical voiceprint feature data of a plurality of power transformation equipment, and based on a two-dimensional convolutional network, obtaining a plurality of historical voiceprint feature data of the plurality of power transformation equipment; extracting a local time-frequency feature of the historical voiceprint feature data to obtain a first voiceprint feature; performing feature extraction on the first voiceprint feature based on a bidirectional recurrent neural network to obtain a second voiceprint feature; according to the second voiceprint feature, multiple times of iterative training is carried out on a preset initial voiceprint detection model, joint optimization is carried out on model parameters based on comparative learning and perceptron classification during iteration, training is completed until the model parameters meet preset values, and a voiceprint detection model is obtained; and obtaining real-time voiceprint features of the to-be-detected power transformation equipment, and inputting the real-time voiceprint features into the voiceprint detection model to obtain an anomaly detection result of the to-be-detected power transformation equipment.

Description

Technical Field

[0001] The present application relates to the field of substation equipment monitoring, and in particular to a method and device for detecting anomalies of substation equipment based on voiceprint. Background Art

[0002] As the main function equipment of substation, the safe and stable operation of substation equipment plays a vital role in the stability of substation and power system. Therefore, it is necessary to accurately detect the abnormality of substation equipment and make timely response decisions to prevent substation equipment failure.

[0003] In the prior art, there are usually two methods for detecting abnormalities in substation equipment. The first is to obtain the operating status of substation equipment by analyzing the periodic reporting of operating data by substation equipment. Due to the characteristics of periodic reporting, some emergency situations cannot be detected in time, and there is a certain lag. The other is to collect the operating data of substation equipment, mainly voiceprint data, through sensors, especially sound sensors, and analyze the operating status of substation equipment. This method can solve the problem of lag, but the existing analysis of voiceprint data mainly relies on the deep learning prediction method using a single model, which has problems such as difficulty in feature extraction and inadequate model training, resulting in low accuracy of abnormal detection of substation equipment based on voiceprint. Therefore, how to perform more accurate abnormal detection of substation equipment based on voiceprint is still a technical problem that needs to be solved urgently in the prior art. Summary of the invention

[0004] The present application provides a method and device for detecting abnormality of substation equipment based on voiceprint, so as to solve the technical problem that the accuracy of abnormality detection of substation equipment based on voiceprint is low.

[0005] According to a first aspect of the implementation of the present application, a method for detecting abnormality of substation equipment based on voiceprint is provided, comprising:

[0006] Acquire historical voiceprint feature data of multiple substation equipment, and extract local time-frequency features of the historical voiceprint feature data based on a two-dimensional convolutional network to obtain a first voiceprint feature;

[0007] Based on a bidirectional recurrent neural network, feature extraction is performed on the first voiceprint feature to obtain a second voiceprint feature;

[0008] According to the second voiceprint feature, a preset initial voiceprint detection model is iteratively trained for multiple times, and during the iterations, model parameters of the initial voiceprint detection model are jointly optimized based on contrastive learning and perceptron classification, until the training is completed when the model parameters meet preset values, thereby obtaining a voiceprint detection model;

[0009] The real-time voiceprint feature of the substation to be detected is obtained, and the real-time voiceprint feature is input into the voiceprint detection model to obtain an abnormality detection result of the substation to be detected.

[0010] The present application first obtains historical voiceprint feature data of multiple substation equipment, and extracts local time-frequency features based on a two-dimensional convolutional network to obtain a first voiceprint feature, and then extracts features based on a bidirectional recurrent neural network to obtain a second voiceprint feature, which can more accurately extract the time-frequency feature part of the voiceprint feature, and then according to the second voiceprint feature, the preset initial voiceprint detection model is iteratively trained multiple times based on contrastive learning and perceptron classification to obtain a voiceprint detection model. By using the more accurate second voiceprint feature as a training set and combining contrastive learning and perceptron classification to optimize the model parameters, the three work together to improve the accuracy of the final output voiceprint detection model, and then improve the accuracy of anomaly detection when performing anomaly detection on the substation to be detected based on real-time voiceprint features.

[0011] In certain embodiments of the present application, the acquiring of historical voiceprint feature data of multiple substations specifically includes:

[0012] Acquire historical audio data of a plurality of substation devices, and divide the historical audio data into a plurality of sample audio sequences according to a preset sequence length;

[0013] Adaptive time-frequency analysis is performed on the multiple sample audio sequences to obtain historical voiceprint feature data corresponding to the multiple substation equipment.

[0014] The present application first obtains historical audio data of multiple substation equipment, and divides the historical audio data according to a preset sequence length to obtain multiple sample audio sequences. By dividing the audio data to obtain short sequences, it is beneficial to improve the accuracy of subsequent time-frequency analysis, and then when adaptive time-frequency analysis is performed on multiple sample audio accumulations to obtain historical voiceprint feature data, the matching degree between the historical voiceprint feature data and the actual operation of the corresponding substation equipment is improved.

[0015] In certain embodiments of the present application, extracting the local time-frequency features of the historical voiceprint feature data based on a two-dimensional convolutional network to obtain the first voiceprint feature specifically includes:

[0016] Performing data enhancement on the historical voiceprint feature data based on the mask to obtain enhanced voiceprint feature data;

[0017] Based on a two-dimensional convolutional network, the enhanced voiceprint feature data is downsampled to extract local time-frequency features to obtain a first voiceprint feature.

[0018] The present application first performs data enhancement on historical voiceprint feature data based on a mask to obtain enhanced voiceprint feature data, which can increase the amount of sample data and prevent the reduction of feature accuracy due to insufficient samples. Then, when the enhanced voiceprint feature is downsampled based on two-dimensional convolution to extract local time-frequency features to obtain the first voiceprint feature, the accuracy of the first voiceprint feature is maintained.

[0019] In certain embodiments of the present application, the preset initial voiceprint detection model is iteratively trained multiple times according to the second voiceprint feature, and the model parameters of the initial voiceprint detection model are jointly optimized based on contrastive learning and perceptron classification during iteration until the training is completed when the model parameters meet the preset values, thereby obtaining the voiceprint detection model, which specifically includes:

[0020] Performing average pooling on the second voiceprint feature to obtain a third voiceprint feature;

[0021] Based on the third voiceprint feature, the preset initial voiceprint detection model is iteratively trained for multiple times, and during the iteration, the joint total loss is calculated based on contrastive learning and perceptron classification, and the model parameters are optimized according to the joint total loss, until the training is completed when the model parameters meet the preset values, and the voiceprint detection model is obtained; wherein the joint total loss includes the contrast loss based on contrastive learning and the cross entropy loss based on perceptron classification;

[0022] The combined total loss is specifically:

[0023] L=α*L contrast +(1-α)*L ce ;

[0024] Among them, L, L contrast , L ce They are the combined total loss, contrast loss and cross entropy loss respectively, and α is the loss balance parameter;

[0025] The contrast loss is specifically:

[0026]

[0027] Among them, H c is the mapping output of the third voiceprint feature based on the multi-layer perceptron, H + , H - are the positive and negative samples of voiceprint features respectively, sim(a, b) is the similarity between input a and input b, and τ is the temperature parameter.

[0028] This application first performs average pooling on the second voiceprint feature to obtain a third voiceprint feature, and then performs multiple iterative training on the preset initial voiceprint detection model based on the third voiceprint feature and the joint total loss calculated based on the contrast loss of contrastive learning and the cross entropy loss based on perceptron classification to obtain a voiceprint detection model. The combined effect of the three can improve the accuracy of the voiceprint detection model, and thus improve the accuracy of abnormal detection of substation equipment.

[0029] In certain embodiments of the present application, the step of inputting the real-time voiceprint feature into the voiceprint detection model to obtain an abnormality detection result of the substation to be detected specifically includes:

[0030] Inputting the real-time voiceprint feature into the voiceprint detection model to obtain an abnormality detection score of the substation to be detected;

[0031] When the abnormality detection score is greater than a preset abnormality threshold, locating the abnormality generation period of the substation to be detected based on the abnormality detection score and the sliding window method;

[0032] According to the abnormality detection score and the abnormality generation time period, an abnormality detection result of the substation to be detected is obtained.

[0033] The present application first obtains the abnormality detection score of the substation to be detected based on the real-time voiceprint features and the voiceprint detection model, and when the abnormality detection score is greater than the preset abnormality threshold, locates the abnormality generation period based on the sliding window method, and then obtains the abnormality detection result. The preset abnormality threshold can eliminate the misjudgment of abnormality detection, and the sliding window method can more accurately locate the abnormality generation period, so that the obtained abnormality detection result is more in line with the actual abnormality of the substation to be detected.

[0034] According to a second aspect of the implementation of the present application, there is provided a device for detecting abnormalities of substation equipment based on voiceprint, comprising a time-frequency feature acquisition module, a voiceprint feature extraction module, a detection model training module and a device abnormality detection module;

[0035] The time-frequency feature acquisition module is used to acquire historical voiceprint feature data of multiple substations, and extract local time-frequency features of the historical voiceprint feature data based on a two-dimensional convolutional network to obtain a first voiceprint feature;

[0036] The voiceprint feature extraction module is used to extract the first voiceprint feature based on a bidirectional recurrent neural network to obtain a second voiceprint feature;

[0037] The detection model training module is used to perform multiple iterative training on the preset initial voiceprint detection model according to the second voiceprint feature, and jointly optimize the model parameters of the initial voiceprint detection model based on contrastive learning and perceptron classification during iteration, until the training is completed when the model parameters meet the preset values, so as to obtain the voiceprint detection model;

[0038] The device anomaly detection module is used to obtain the real-time voiceprint features of the substation equipment to be detected, and input the real-time voiceprint features into the voiceprint detection model to obtain the anomaly detection result of the substation equipment to be detected.

[0039] In certain embodiments of the present application, the time-frequency feature acquisition module includes a voiceprint feature acquisition submodule; the voiceprint feature acquisition submodule includes an audio acquisition segmentation unit and a sequence time-frequency analysis unit;

[0040] The audio acquisition and segmentation unit is used to acquire historical audio data of multiple substation equipment and segment the historical audio data into multiple sample audio sequences according to a preset sequence length;

[0041] The sequence time-frequency analysis unit is used to perform adaptive time-frequency analysis on the multiple sample audio sequences to obtain historical voiceprint feature data corresponding to the multiple substations.

[0042] In certain embodiments of the present application, the time-frequency feature acquisition module includes a time-frequency feature extraction submodule; the time-frequency feature extraction submodule includes a voiceprint feature enhancement unit and a time-frequency feature extraction unit;

[0043] The voiceprint feature enhancement unit is used to enhance the historical voiceprint feature data based on the mask to obtain enhanced voiceprint feature data;

[0044] The time-frequency feature extraction unit is used to downsample the enhanced voiceprint feature data based on a two-dimensional convolutional network to extract local time-frequency features to obtain a first voiceprint feature.

[0045] In certain embodiments of the present application, the detection model training module includes a voiceprint feature pooling submodule and a detection model training submodule;

[0046] The voiceprint feature pooling submodule is used to average pool the second voiceprint feature to obtain a third voiceprint feature;

[0047] The detection model training submodule is used to perform multiple iterative training on the preset initial voiceprint detection model based on the third voiceprint feature, calculate the joint total loss based on contrastive learning and perceptron classification during iteration, and optimize the model parameters according to the joint total loss until the training is completed when the model parameters meet the preset values, thereby obtaining the voiceprint detection model; wherein the joint total loss includes the contrastive loss based on contrastive learning and the cross entropy loss based on perceptron classification;

[0048] The combined total loss is specifically:

[0049] L=α*L contrast +(1-α)*L ce ;

[0050] Among them, L, L contrast , L ce They are the combined total loss, contrast loss and cross entropy loss respectively, and α is the loss balance parameter;

[0051] The contrast loss is specifically:

[0052]

[0053] Among them, H c is the mapping output of the third voiceprint feature based on the multi-layer perceptron, H + , H - are the positive and negative samples of voiceprint features respectively, sim(a, b) is the similarity between input a and input b, and τ is the temperature parameter.

[0054] In certain embodiments of the present application, the device anomaly detection module includes an anomaly score acquisition submodule, an anomaly time period location submodule, and an anomaly result acquisition submodule;

[0055] The anomaly score acquisition submodule is used to input the real-time voiceprint feature into the voiceprint detection model to obtain an anomaly detection score of the substation to be detected;

[0056] The abnormal period positioning submodule is used to locate the abnormal period of the substation to be detected based on the sliding window method according to the abnormal detection score when the abnormal detection score is greater than the preset abnormal threshold;

[0057] The abnormal result acquisition submodule is used to obtain the abnormal detection result of the substation to be detected according to the abnormal detection score and the abnormality generation time period.

[0058] The present application first obtains historical voiceprint feature data of multiple substation equipment, and extracts local time-frequency features based on a two-dimensional convolutional network to obtain a first voiceprint feature, and then extracts features based on a bidirectional recurrent neural network to obtain a second voiceprint feature, which can more accurately extract the time-frequency feature part of the voiceprint feature, and then according to the second voiceprint feature, the preset initial voiceprint detection model is iteratively trained multiple times based on contrastive learning and perceptron classification to obtain a voiceprint detection model. By using the more accurate second voiceprint feature as a training set and combining contrastive learning and perceptron classification to optimize the model parameters, the three work together to improve the accuracy of the final output voiceprint detection model, and then improve the accuracy of anomaly detection when performing anomaly detection on the substation to be detected based on real-time voiceprint features. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 : A flow chart of a method for detecting abnormality of substation equipment based on voiceprints according to certain embodiments of the present application;

[0060] Figure 2 : A module structure diagram of a voiceprint-based substation abnormality detection device shown in certain embodiments of the present application. DETAILED DESCRIPTION

[0061] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below in conjunction with the accompanying drawings are exemplary and are only used to explain some embodiments of the present application, and should not be construed as limitations on the embodiments of the present application. Based on the embodiments shown in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0062] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, unless otherwise clearly and specifically defined, "multiple" and "several" mean two or more.

[0063] In the prior art, when detecting abnormalities of substation equipment based on voiceprint data, it mainly relies on the deep learning prediction method using a single model, which has problems such as difficulty in feature extraction and inadequate model training, resulting in low accuracy of abnormality detection of substation equipment based on voiceprint. Therefore, how to perform more accurate abnormality detection of substation equipment based on voiceprint is still a technical problem that needs to be solved urgently in the prior art.

[0064] Based on the above technical background, please refer to Figure 1 The embodiment of the present application provides a method for detecting abnormality of substation equipment based on voiceprint, including steps S101 to S104, each of which is as follows:

[0065] Step S101: Acquire historical voiceprint feature data of multiple substations, and extract local time-frequency features of the historical voiceprint feature data based on a two-dimensional convolutional network to obtain a first voiceprint feature.

[0066] In certain embodiments of the present application, the acquiring of historical voiceprint feature data of multiple substations specifically includes:

[0067] Acquire historical audio data of a plurality of substation devices, and divide the historical audio data into a plurality of sample audio sequences according to a preset sequence length;

[0068] Adaptive time-frequency analysis is performed on the multiple sample audio sequences to obtain historical voiceprint feature data corresponding to the multiple substation equipment.

[0069] Specifically, the historical audio data is represented as x(t), and the sampling rate is f s , the preset sequence length is T, the number of sample audio sequences obtained is k<b, b is the amount of data processed at a single time. If the length of the currently processed sample audio sequence does not meet the preset sequence length T, the number 0 is added to the end of the currently processed sample audio sequence to the preset sequence length T.

[0070] In certain embodiments of the present application, the adaptive time-frequency analysis is performed on the multiple sample audio sequences to obtain the historical voiceprint feature data corresponding to the multiple substations, specifically:

[0071] Combining the multiple sample audio sequences to obtain a sample audio tensor X[b, 1, T];

[0072] Adaptive time-frequency analysis is performed on the sample audio tensor X[b, 1, T] to obtain historical voiceprint feature data S[b, F, T′] corresponding to the multiple substations.

[0073] Among them, F is the frequency dimension and T′ is the time dimension (number of time frames).

[0074] Specifically, when performing adaptive time-frequency analysis, the preferred value of the frequency dimension F is 2 8 =256 or 2 9 =512, the time dimension T′ is related to the preset time resolution of the adaptive time-frequency analysis.

[0075] In certain embodiments of the present application, the preferred implementation of the adaptive time-frequency analysis is AWPT.

[0076] The present application first obtains historical audio data of multiple substation equipment, and divides the historical audio data according to a preset sequence length to obtain multiple sample audio sequences. By dividing the audio data to obtain short sequences, it is beneficial to improve the accuracy of subsequent time-frequency analysis, and then when adaptive time-frequency analysis is performed on multiple sample audio accumulations to obtain historical voiceprint feature data, the matching degree between the historical voiceprint feature data and the actual operation of the corresponding substation equipment is improved.

[0077] In certain embodiments of the present application, extracting the local time-frequency features of the historical voiceprint feature data based on a two-dimensional convolutional network to obtain the first voiceprint feature specifically includes:

[0078] Performing data enhancement on the historical voiceprint feature data based on the mask to obtain enhanced voiceprint feature data;

[0079] Based on a two-dimensional convolutional network, the enhanced voiceprint feature data is downsampled to extract local time-frequency features to obtain a first voiceprint feature.

[0080] In certain embodiments of the present application, the historical voiceprint feature data is enhanced based on the mask to obtain enhanced voiceprint feature data, specifically:

[0081] Based on the mask, the historical voiceprint feature data S[b, F, T′] is enhanced in frequency and time to obtain enhanced voiceprint feature data S aug [b, F, T′].

[0082] It should be understood that the mask-based data enhancement method in the present application includes but is not limited to the GridMask-based data enhancement method. The specific implementation method is well known to those skilled in the art and will not be described in detail here.

[0083] In certain embodiments of the present application, the enhanced voiceprint feature data is downsampled based on a two-dimensional convolutional network to extract local time-frequency features to obtain a first voiceprint feature, specifically:

[0084] Based on a two-dimensional convolutional network, the enhanced voiceprint feature data S aug [b, F, T′] is downsampled to extract local time-frequency features and obtain the first voiceprint feature C[b, ch, F′, T″].

[0085] Among them, ch is the number of channels, F′ and T″ are the frequency and time dimensions after downsampling, respectively.

[0086] In certain embodiments of the present application, a possible implementation of the two-dimensional convolutional network is specifically: Conv2d→BatchNorm→ReLU→Pooling, i.e., two-dimensional convolution, data normalization, data activation and data pooling.

[0087] The present application first performs data enhancement on historical voiceprint feature data based on a mask to obtain enhanced voiceprint feature data, which can increase the amount of sample data and prevent the reduction of feature accuracy due to insufficient samples. Then, when the enhanced voiceprint feature is downsampled based on two-dimensional convolution to extract local time-frequency features to obtain the first voiceprint feature, the accuracy of the first voiceprint feature is maintained.

[0088] Step S102: Based on a bidirectional recurrent neural network, feature extraction is performed on the first voiceprint feature to obtain a second voiceprint feature.

[0089] In some embodiments of the present application, the feature extraction of the first voiceprint feature based on the bidirectional recurrent neural network to obtain the second voiceprint feature is specifically:

[0090] The first voiceprint feature C[b, ch, F′, T″] is rearranged to obtain the intermediate voiceprint feature C seq [b, T″, ch*F′];

[0091] Based on a bidirectional recurrent neural network, the intermediate voiceprint feature C seq [b, T″, ch*F′] is used for feature extraction to obtain the second voiceprint feature H[b, T″, 2*D].

[0092] Where D is the hidden state dimension of the bidirectional recurrent neural network.

[0093] In certain embodiments of the present application, the bidirectional recurrent neural network includes but is not limited to a bidirectional RNN and a bidirectional LSTM, and a preferred embodiment is a bidirectional LSTM.

[0094] The use of bidirectional recurrent neural networks for feature extraction is considered because bidirectional recurrent neural networks can better process sequence information and use multi-head self-attention mechanisms to more accurately represent contextual dependencies, thereby obtaining more accurate second voiceprint features.

[0095] Step S103: According to the second voiceprint feature, the preset initial voiceprint detection model is iteratively trained for multiple times. During the iteration, the model parameters of the initial voiceprint detection model are jointly optimized based on contrastive learning and perceptron classification until the training is completed when the model parameters meet the preset values, thereby obtaining the voiceprint detection model.

[0096] In certain embodiments of the present application, the preset initial voiceprint detection model is iteratively trained multiple times according to the second voiceprint feature, and the model parameters of the initial voiceprint detection model are jointly optimized based on contrastive learning and perceptron classification during iteration until the training is completed when the model parameters meet the preset values, thereby obtaining the voiceprint detection model, which specifically includes:

[0097] Performing average pooling on the second voiceprint feature to obtain a third voiceprint feature;

[0098] Based on the third voiceprint feature, the preset initial voiceprint detection model is iteratively trained for multiple times, and during the iteration, the joint total loss is calculated based on contrastive learning and perceptron classification, and the model parameters are optimized according to the joint total loss, until the training is completed when the model parameters meet the preset values, and the voiceprint detection model is obtained; wherein the joint total loss includes the contrast loss based on contrastive learning and the cross entropy loss based on perceptron classification;

[0099] The combined total loss is specifically:

[0100] L=α*L contrast +(1-α)*L ce

[0101] Among them, L, L contrast , L ce They are the combined total loss, contrast loss and cross entropy loss respectively, and α is the loss balance parameter;

[0102] The contrast loss is specifically:

[0103]

[0104] Among them, H c is the mapping output of the third voiceprint feature based on the multi-layer perceptron, H + , H - are the positive and negative samples of voiceprint features respectively, sim(a, b) is the similarity between input a and input b, and τ is the temperature parameter.

[0105] In some embodiments of the present application, the second voiceprint feature is averaged and pooled to obtain the third voiceprint feature, specifically:

[0106] The second voiceprint feature H[b, T″, 2*D] is averaged and pooled to obtain the third voiceprint feature H avg [b, 2*D].

[0107] Specifically, when calculating the contrast loss based on contrastive learning, it is as follows:

[0108] The third voiceprint feature H avg[b, 2*D] input multi-layer perceptron, get mapping output H c [b, D′];

[0109] The positive samples and negative samples corresponding to the mapping output are input into the multi-layer perceptron to obtain the positive mapping sample H + And the negative mapping sample H - ;

[0110] According to the mapping output H c [b, D′], the positive mapping sample H + and the negative map sample H - , calculate the contrast loss L contrast .

[0111] Where D′ is the spatial dimension of contrastive learning.

[0112] In certain embodiments of the present application, the similarity sim(a, b) includes but is not limited to Pearson correlation coefficient, cosine similarity, log-likelihood similarity and Spearman rank correlation coefficient, and the preferred embodiment is cosine similarity.

[0113] In certain embodiments of the present application, the preferred sampling object of the positive sample is the normal operation voiceprint feature of the same substation, and the preferred sampling object of the negative sample is the voiceprint feature of other substations or the synthetic abnormal voiceprint feature of the same substation.

[0114] It should be understood that the temperature parameter τ described in this application is a hyperparameter used to control the distribution of positive and negative samples in contrastive learning. The temperature parameter τ is a commonly used hyperparameter for training in the field of neural network models, so it will not be further expanded here.

[0115] Specifically, when calculating the cross entropy loss based on perceptron classification, it is as follows:

[0116] The third voiceprint feature H avg [b, 2*D] is input into the multilayer perceptron to obtain the mapping anomaly score score[b, 1];

[0117] If the third voiceprint feature H avg The current sample of [b, 2*D] is a labeled sample, and the cross entropy loss L is calculated according to the mapping anomaly score score[b, 1] ce ;

[0118] If the third voiceprint feature H avg The current sample of [b, 2*D] is an unlabeled sample. Based on the self-supervised reconstruction error, the cross entropy loss L is obtained according to the mapping anomaly score score[b, 1] ce .

[0119] In certain embodiments of the present application, the cross entropy loss L ce The preferred implementation of is the binary cross entropy loss.

[0120] This application first performs average pooling on the second voiceprint feature to obtain a third voiceprint feature, and then performs multiple iterative training on the preset initial voiceprint detection model based on the third voiceprint feature and the joint total loss calculated based on the contrast loss of contrastive learning and the cross entropy loss based on perceptron classification to obtain a voiceprint detection model. The combined effect of the three can improve the accuracy of the voiceprint detection model, and thus improve the accuracy of abnormal detection of substation equipment.

[0121] Step S104: obtaining the real-time voiceprint features of the substation to be detected, and inputting the real-time voiceprint features into the voiceprint detection model to obtain an abnormality detection result of the substation to be detected.

[0122] In certain embodiments of the present application, the step of inputting the real-time voiceprint feature into the voiceprint detection model to obtain an abnormality detection result of the substation to be detected specifically includes:

[0123] Inputting the real-time voiceprint feature into the voiceprint detection model to obtain an abnormality detection score of the substation to be detected;

[0124] When the abnormality detection score is greater than a preset abnormality threshold, locating the abnormality generation period of the substation to be detected based on the abnormality detection score and the sliding window method;

[0125] According to the abnormality detection score and the abnormality generation time period, an abnormality detection result of the substation to be detected is obtained.

[0126] The present application first obtains the abnormality detection score of the substation to be detected based on the real-time voiceprint features and the voiceprint detection model, and when the abnormality detection score is greater than the preset abnormality threshold, locates the abnormality generation period based on the sliding window method, and then obtains the abnormality detection result. The preset abnormality threshold can eliminate the misjudgment of abnormality detection, and the sliding window method can more accurately locate the abnormality generation period, so that the obtained abnormality detection result is more in line with the actual abnormality of the substation to be detected.

[0127] Compared with the prior art, the present application first obtains historical voiceprint feature data of multiple substation equipment, and extracts local time-frequency features based on a two-dimensional convolutional network to obtain a first voiceprint feature, and then extracts features based on a bidirectional recurrent neural network to obtain a second voiceprint feature, which can more accurately extract the time-frequency feature part of the voiceprint feature, and then according to the second voiceprint feature, the preset initial voiceprint detection model is iteratively trained multiple times based on contrastive learning and perceptron classification to obtain a voiceprint detection model. By using the more accurate second voiceprint feature as a training set and combining contrastive learning and perceptron classification to optimize the model parameters, the three work together to improve the accuracy of the final output voiceprint detection model, and then improve the accuracy of anomaly detection when performing anomaly detection on the substation to be detected based on real-time voiceprint features.

[0128] Corresponding to the previous method, see Figure 2 , the embodiment of the present application provides a substation abnormality detection device based on voiceprint, including a time-frequency feature acquisition module 210, a voiceprint feature extraction module 220, a detection model training module 230 and a device abnormality detection module 240;

[0129] The time-frequency feature acquisition module 210 is used to acquire historical voiceprint feature data of multiple substations, and extract local time-frequency features of the historical voiceprint feature data based on a two-dimensional convolutional network to obtain a first voiceprint feature;

[0130] The voiceprint feature extraction module 220 is used to extract the first voiceprint feature based on a bidirectional recurrent neural network to obtain a second voiceprint feature;

[0131] The detection model training module 230 is used to perform multiple iterations of training on the preset initial voiceprint detection model according to the second voiceprint feature, and jointly optimize the model parameters of the initial voiceprint detection model based on contrastive learning and perceptron classification during iteration, until the training is completed when the model parameters meet the preset values, so as to obtain the voiceprint detection model;

[0132] The device anomaly detection module 240 is used to obtain the real-time voiceprint features of the substation to be detected, and input the real-time voiceprint features into the voiceprint detection model to obtain the anomaly detection result of the substation to be detected.

[0133] In some embodiments of the present application, the time-frequency feature acquisition module 210 includes a voiceprint feature acquisition submodule; the voiceprint feature acquisition submodule includes an audio acquisition segmentation unit and a sequence time-frequency analysis unit;

[0134] The audio acquisition and segmentation unit is used to acquire historical audio data of multiple substation equipment and segment the historical audio data into multiple sample audio sequences according to a preset sequence length;

[0135] The sequence time-frequency analysis unit is used to perform adaptive time-frequency analysis on the multiple sample audio sequences to obtain historical voiceprint feature data corresponding to the multiple substations.

[0136] In some embodiments of the present application, the time-frequency feature acquisition module 210 includes a time-frequency feature extraction submodule; the time-frequency feature extraction submodule includes a voiceprint feature enhancement unit and a time-frequency feature extraction unit;

[0137] The voiceprint feature enhancement unit is used to enhance the historical voiceprint feature data based on the mask to obtain enhanced voiceprint feature data;

[0138] The time-frequency feature extraction unit is used to downsample the enhanced voiceprint feature data based on a two-dimensional convolutional network to extract local time-frequency features to obtain a first voiceprint feature.

[0139] In certain embodiments of the present application, the detection model training module 230 includes a voiceprint feature pooling submodule and a detection model training submodule;

[0140] The voiceprint feature pooling submodule is used to average pool the second voiceprint feature to obtain a third voiceprint feature;

[0141] The detection model training submodule is used to perform multiple iterative training on the preset initial voiceprint detection model based on the third voiceprint feature, calculate the joint total loss based on contrastive learning and perceptron classification during iteration, and optimize the model parameters according to the joint total loss until the training is completed when the model parameters meet the preset values, thereby obtaining the voiceprint detection model; wherein the joint total loss includes the contrastive loss based on contrastive learning and the cross entropy loss based on perceptron classification;

[0142] The combined total loss is specifically:

[0143] L=α*L contrast +(1-α)*L ce

[0144] Among them, L, L contrast , L ce They are the combined total loss, contrast loss and cross entropy loss respectively, and α is the loss balance parameter;

[0145] The contrast loss is specifically:

[0146]

[0147] Among them, H c is the mapping output of the third voiceprint feature based on the multi-layer perceptron, H + , H -are the positive and negative samples of voiceprint features respectively, sim(a, b) is the similarity between input a and input b, and τ is the temperature parameter.

[0148] In certain embodiments of the present application, the device anomaly detection module 240 includes an anomaly score acquisition submodule, an anomaly time period location submodule, and an anomaly result acquisition submodule;

[0149] The anomaly score acquisition submodule is used to input the real-time voiceprint feature into the voiceprint detection model to obtain an anomaly detection score of the substation to be detected;

[0150] The abnormal period positioning submodule is used to locate the abnormal period of the substation to be detected based on the sliding window method according to the abnormal detection score when the abnormal detection score is greater than the preset abnormal threshold;

[0151] The abnormal result acquisition submodule is used to obtain the abnormal detection result of the substation to be detected according to the abnormal detection score and the abnormality generation time period.

[0152] The present application first obtains historical voiceprint feature data of multiple substation equipment, and extracts local time-frequency features based on a two-dimensional convolutional network to obtain a first voiceprint feature, and then extracts features based on a bidirectional recurrent neural network to obtain a second voiceprint feature, which can more accurately extract the time-frequency feature part of the voiceprint feature, and then according to the second voiceprint feature, the preset initial voiceprint detection model is iteratively trained multiple times based on contrastive learning and perceptron classification to obtain a voiceprint detection model. By using the more accurate second voiceprint feature as a training set and combining contrastive learning and perceptron classification to optimize the model parameters, the three work together to improve the accuracy of the final output voiceprint detection model, and then improve the accuracy of anomaly detection when performing anomaly detection on the substation to be detected based on real-time voiceprint features.

[0153] It should be understood that the device provided in the embodiment of the present application corresponds to the aforementioned method, and a substation equipment abnormality detection device based on voiceprint provided in the embodiment of the present application can implement a substation equipment abnormality detection method based on voiceprint provided in any embodiment of the present application.

[0154] Adaptively, the embodiments of the present application also provide a computer device and a computer-readable storage medium.

[0155] The computer device comprises: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor;

[0156] Wherein, when the processor executes the computer program, a method for detecting abnormalities of substation equipment based on voiceprint is implemented in the present application.

[0157] The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute a method for detecting abnormalities of substation equipment based on voiceprints of the present application.

[0158] The above are some implementation methods of the present application, and the purpose, technical scheme and beneficial effects of the present application are further described in detail. It should be clear that the above-mentioned partial implementation methods of the present application cannot be understood as limiting the present application. It is particularly pointed out that for those skilled in the art, any changes, modifications, equivalent substitutions and variations made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for detecting abnormality of substation equipment based on voiceprint, characterized in that: include: Acquire historical voiceprint feature data of multiple substation equipment, and extract local time-frequency features of the historical voiceprint feature data based on a two-dimensional convolutional network to obtain a first voiceprint feature; Based on a bidirectional recurrent neural network, feature extraction is performed on the first voiceprint feature to obtain a second voiceprint feature; According to the second voiceprint feature, a preset initial voiceprint detection model is iteratively trained for multiple times, and during the iterations, model parameters of the initial voiceprint detection model are jointly optimized based on contrastive learning and perceptron classification, until the training is completed when the model parameters meet preset values, thereby obtaining a voiceprint detection model; The real-time voiceprint feature of the substation to be detected is obtained, and the real-time voiceprint feature is input into the voiceprint detection model to obtain an abnormality detection result of the substation to be detected.

2. According to claim 1, a method for detecting abnormality of substation equipment based on voiceprint, characterized in that: The obtaining of historical voiceprint feature data of multiple substation equipment specifically includes: Acquire historical audio data of a plurality of substation devices, and divide the historical audio data into a plurality of sample audio sequences according to a preset sequence length; Adaptive time-frequency analysis is performed on the multiple sample audio sequences to obtain historical voiceprint feature data corresponding to the multiple substation equipment.

3. The method for detecting abnormality of substation equipment based on voiceprint according to claim 1, characterized in that: The extracting of the local time-frequency features of the historical voiceprint feature data based on the two-dimensional convolutional network to obtain the first voiceprint feature specifically includes: Performing data enhancement on the historical voiceprint feature data based on the mask to obtain enhanced voiceprint feature data; Based on a two-dimensional convolutional network, the enhanced voiceprint feature data is downsampled to extract local time-frequency features to obtain a first voiceprint feature.

4. The method for detecting abnormality of substation equipment based on voiceprint according to claim 1, characterized in that: The method of performing multiple iterative training on a preset initial voiceprint detection model according to the second voiceprint feature, jointly optimizing the model parameters of the initial voiceprint detection model based on contrastive learning and perceptron classification during iteration until the training is completed when the model parameters meet preset values, thereby obtaining a voiceprint detection model, specifically includes: Performing average pooling on the second voiceprint feature to obtain a third voiceprint feature; Based on the third voiceprint feature, the preset initial voiceprint detection model is iteratively trained for multiple times, and during the iteration, the joint total loss is calculated based on contrastive learning and perceptron classification, and the model parameters are optimized according to the joint total loss, until the training is completed when the model parameters meet the preset values, and the voiceprint detection model is obtained; wherein the joint total loss includes the contrast loss based on contrastive learning and the cross entropy loss based on perceptron classification; The combined total loss is specifically: L=α*L contrast +(1-a)*L ce ; Among them, L, L contrast , L ce They are the combined total loss, contrast loss and cross entropy loss respectively, and α is the loss balance parameter; The contrast loss is specifically: Among them, H c is the mapping output of the third voiceprint feature based on the multi-layer perceptron, H + , H - are the positive and negative samples of voiceprint features respectively, sim(a, b) is the similarity between input a and input b, and τ is the temperature parameter.

5. The method for detecting abnormality of substation equipment based on voiceprint according to claim 4, characterized in that: The step of inputting the real-time voiceprint feature into the voiceprint detection model to obtain an abnormality detection result of the substation to be detected specifically includes: Inputting the real-time voiceprint feature into the voiceprint detection model to obtain an abnormality detection score of the substation to be detected; When the abnormality detection score is greater than a preset abnormality threshold, locating the abnormality generation period of the substation to be detected based on the abnormality detection score and the sliding window method; According to the abnormality detection score and the abnormality generation time period, an abnormality detection result of the substation to be detected is obtained.

6. A device for detecting abnormality of substation equipment based on voiceprint, characterized in that: It includes time-frequency feature acquisition module, voiceprint feature extraction module, detection model training module and device anomaly detection module; The time-frequency feature acquisition module is used to acquire historical voiceprint feature data of multiple substations, and extract local time-frequency features of the historical voiceprint feature data based on a two-dimensional convolutional network to obtain a first voiceprint feature; The voiceprint feature extraction module is used to extract the first voiceprint feature based on a bidirectional recurrent neural network to obtain a second voiceprint feature; The detection model training module is used to perform multiple iterative training on the preset initial voiceprint detection model according to the second voiceprint feature, and jointly optimize the model parameters of the initial voiceprint detection model based on contrastive learning and perceptron classification during iteration, until the training is completed when the model parameters meet the preset values, so as to obtain the voiceprint detection model; The device anomaly detection module is used to obtain the real-time voiceprint features of the substation equipment to be detected, and input the real-time voiceprint features into the voiceprint detection model to obtain the anomaly detection result of the substation equipment to be detected.

7. The device for detecting abnormality of substation equipment based on voiceprint according to claim 6, characterized in that: The time-frequency feature acquisition module includes a voiceprint feature acquisition submodule; the voiceprint feature acquisition submodule includes an audio acquisition segmentation unit and a sequence time-frequency analysis unit; The audio acquisition and segmentation unit is used to acquire historical audio data of multiple substation equipment and segment the historical audio data into multiple sample audio sequences according to a preset sequence length; The sequence time-frequency analysis unit is used to perform adaptive time-frequency analysis on the multiple sample audio sequences to obtain historical voiceprint feature data corresponding to the multiple substations.

8. The device for detecting abnormality of substation equipment based on voiceprint according to claim 6, characterized in that: The time-frequency feature acquisition module includes a time-frequency feature extraction submodule; the time-frequency feature extraction submodule includes a voiceprint feature enhancement unit and a time-frequency feature extraction unit; The voiceprint feature enhancement unit is used to enhance the historical voiceprint feature data based on the mask to obtain enhanced voiceprint feature data; The time-frequency feature extraction unit is used to downsample the enhanced voiceprint feature data based on a two-dimensional convolutional network to extract local time-frequency features to obtain a first voiceprint feature.

9. The device for detecting abnormality of substation equipment based on voiceprint according to claim 6, characterized in that: The detection model training module includes a voiceprint feature pooling submodule and a detection model training submodule; The voiceprint feature pooling submodule is used to average pool the second voiceprint feature to obtain a third voiceprint feature; The detection model training submodule is used to perform multiple iterative training on the preset initial voiceprint detection model based on the third voiceprint feature, calculate the joint total loss based on contrastive learning and perceptron classification during iteration, and optimize the model parameters according to the joint total loss until the training is completed when the model parameters meet the preset values, thereby obtaining the voiceprint detection model; wherein the joint total loss includes the contrastive loss based on contrastive learning and the cross entropy loss based on perceptron classification; The combined total loss is specifically: L=α*L contrast +(1-a)*L ce ; Among them, L,L contrast , L ce They are the combined total loss, contrast loss and cross entropy loss respectively, and α is the loss balance parameter; The contrast loss is specifically: Among them, H c is the mapping output of the third voiceprint feature based on the multi-layer perceptron, H + , H - are the positive and negative samples of voiceprint features, sim(a, b) is the similarity between input a and input b, τ is the temperature parameter.

10. The device for detecting abnormality of substation equipment based on voiceprint according to claim 6, characterized in that: The device anomaly detection module includes an anomaly score acquisition submodule, an anomaly time period location submodule and an anomaly result acquisition submodule; The anomaly score acquisition submodule is used to input the real-time voiceprint feature into the voiceprint detection model to obtain an anomaly detection score of the substation to be detected; The abnormal period positioning submodule is used to locate the abnormal period of the substation to be detected based on the sliding window method according to the abnormal detection score when the abnormal detection score is greater than the preset abnormal threshold; The abnormal result acquisition submodule is used to obtain the abnormal detection result of the substation to be detected according to the abnormal detection score and the abnormality generation time period.

Images