Power grid main equipment fault diagnosis method and system based on voiceprint cloud edge cooperation

Abstract

The invention belongs to the technical field of power grid main equipment diagnosis, and provides a power grid main equipment fault diagnosis method and system based on voiceprint cloud edge collaboration, which adopts a new cloud edge collaboration method and divides the analysis of voiceprint monitoring data into two stages. According to the method, the exception detection model is deployed on the edge side, exception analysis of mass voiceprint data is achieved, concurrent requests of a large number of concurrent signals at the cloud side are avoided, and the timeliness of voiceprint monitoring and analysis is improved. In the cloud, a multi-modal defect identification method fusing voiceprint, visible light, infrared and other multi-source data is deployed, secondary classification of sound abnormal signals is realized, and the reliability of equipment operation, maintenance and management is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power grid main equipment diagnosis, and in particular, relates to a power grid main equipment fault diagnosis method and system based on voiceprint cloud-edge collaboration. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the development of technologies such as the Internet of Things, big data, and artificial intelligence, intelligent operation and inspection technologies can monitor power grid equipment in real time, evaluate status, and diagnose faults, thus realizing intelligent management and optimization of power grid operation and reducing the operating costs and fault handling time of power companies. The application of intelligent inspection technologies such as high-definition video, robots, and computer vision has realized objective external status and remote intelligent inspection. However, there is a lack of more comprehensive detection methods for the internal operating status of the main power grid equipment.

[0004] When the main equipment of the power grid, such as the transformer, is operating normally, a uniform humming sound will be emitted. Once the transformer produces uneven sound or abnormal sound, it may be due to transformer overload, sudden load change, system short circuit, overvoltage in the power grid, loose transformer core clamps, internal transformer fault discharge sparking, insulation breakdown or inter-turn short circuit, etc. At the same time, there may be a phenomenon of local temperature rise in the transformer body or components, or appearance defects such as abnormal temperature gauge readings, partial discharge of bushings, discoloration of respirator silicone, oil stains on the surface of components, and abnormal oil level of oil seals. When the transformer is in these abnormal operating states, an alarm signal should be given in time to avoid the occurrence of faults.

[0005] The current fault diagnosis method has the following technical problems:

[0006] 1. The fault diagnosis method mainly analyzes and processes data on the big data analysis platform in the cloud. Due to the large number of data types and large amount of data detection on site, the data processing capacity is insufficient under high concurrency conditions, resulting in reduced real-time performance;

[0007] 2. Device voiceprint diagnosis may be interfered by external environmental noise, such as electromagnetic interference, human voice, bird song, etc. These noises will affect the accuracy of voiceprint diagnosis.

[0008] 3. Limitations of sample data: insufficient sample types and quantities, and only samples of a single defect type, without taking into account the superposition of multiple sound sources on site and the occurrence of multiple defects.

[0009] 4. The ability of integrated diagnosis and analysis between data is insufficient. Various monitoring and operation methods generate a variety of perception data such as voiceprints, visible light, infrared, etc., but the collaborative diagnosis and analysis capabilities of multi-source data are insufficient, and more accurate equipment status evaluation cannot be performed. Summary of the invention

[0010] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides a power grid main equipment fault diagnosis method and system based on voiceprint cloud-edge collaboration. After completing the voiceprint signal acquisition, the signal is subjected to spectral subtraction noise reduction, voiceprint signal-to-noise ratio calculation, anomaly detection and other processes on the edge side, and the detected abnormal signal is uploaded to the cloud platform for accurate classification; the detected fault data is expanded for incremental learning optimization of the model, and the optimized model can be updated online or sent to the edge side hardware.

[0011] In order to achieve the above object, the present invention adopts the following technical solution:

[0012] A first aspect of the present invention provides a method for diagnosing faults of a main power grid device based on voiceprint cloud-edge collaboration, comprising the following steps:

[0013] Acquire voiceprint data, visible light image data and infrared image data of main power grid equipment;

[0014] Performing noise suppression on the acquired voiceprint data to obtain the noise-suppressed voiceprint data;

[0015] On the edge side, the voiceprint features after noise suppression and the set conditional features are used as combined features to train the anomaly detection model to obtain the optimal risk threshold. The trained anomaly detection model is used to calculate the risk value of the voiceprint data to be detected. When the risk value is greater than the optimal risk threshold, it is determined to be abnormal voiceprint data.

[0016] In the cloud, the abnormal voiceprint data obtained by detection is combined with the visible light image data and infrared image data to perform secondary classification on the abnormal voiceprint data and obtain the abnormal voiceprint signal diagnosis result.

[0017] Further, noise suppression is performed on the acquired voiceprint data to obtain the voiceprint data after noise suppression, including:

[0018] According to the background voiceprint estimation result obtained by the background voiceprint estimation module, the signal-to-noise ratio of each frame of data is calculated, and the sequence number of the frame to be denoised is determined according to the set noise consistency threshold;

[0019] The amplitude spectrum information of the background voiceprint signal obtained in the background voiceprint estimation module and the phase spectrum information of the noise segment signal obtained in the noise period estimation module are subjected to data fusion processing to obtain the full spectrum information after spectrum replacement;

[0020] Based on the spectral replacement and fusion of the full spectrum information, IFFT calculation is performed to obtain the denoised time domain signal.

[0021] Furthermore, the method of using the noise-suppressed voiceprint features and the set conditional features as combined features to train the anomaly detection model to obtain the optimal risk threshold includes:

[0022] The transformer voiceprint feature s and the set conditional feature c are combined into the input feature x = (s, c), and then the encoder G is generated. E Get the hidden vector z, and generate the decoder G D Get the reconstructed features of x After an additional encoder E, the reconstructed features of the latent vector are obtained Based on the combined feature x and the reconstructed feature Input to the discriminator D, decision and encoder G E and decoder G D The combined generator obtains the optimal risk threshold through continuous adversarial training.

[0023] Furthermore, using the trained generative encoder G E , generate decoder G D , encoder E is used to calculate the reconstruction loss and encoding loss, and the risk R′ value of the data to be detected is calculated. When the risk R′ value of the data to be detected exceeds the risk threshold R obtained by training, it can be determined as abnormal data.

[0024] Furthermore, the calculation formula for the overall loss of the anomaly detection model is:

[0025] L=L adv +αL rec +βL enc ,

[0026]

[0027]

[0028]

[0029] Among them, L adv To combat the loss, L rec is the reconstruction loss, L enc is the encoding loss, α and β are weight coefficients, x i represents the i-th element of the input feature, represents the i-th element of the reconstructed feature, D(x, c) represents the output of the conditional feature vector after the discriminator, Represents the distribution of positive samples from P data(x) Extract samples x from the y-axis and perform the desired operation on them. Represents the distribution of positive samples from P data(x)Reconstruction features and perform the desired operations on it. Represents the generated reconstruction features The output of the decision device D.

[0030] Furthermore, the voiceprint features include MFCC, time-frequency graph and Mel-spectrogram, and the conditional features include transformer voltage level, cooling method, manufacturer and station.

[0031] Furthermore, the conditional features are combined and encoded using one-hot encoding.

[0032] Furthermore, the abnormal voiceprint data obtained by the combined detection is fused with the visible light image data and the infrared image data, and the abnormal voiceprint data is secondary classified to obtain the abnormal voiceprint signal diagnosis result, including:

[0033] Expanding the voiceprint data to obtain expanded voiceprint data;

[0034] By mapping multiple modalities such as visible light images, infrared images and expanded voiceprint data into a common representation space, a grid main equipment defect image dataset with a multi-source structure that includes appearance defect labels, temperature labels and voiceprint spectra was constructed.

[0035] Based on the constructed grid main equipment defect image dataset and the trained multimodal feature extraction network, visible light image features, infrared image features and voiceprint image features are obtained;

[0036] Through the mutual attention mechanism, the visible light image features, infrared image features and voiceprint image features are weightedly fused to obtain the fused feature vector of the main power grid equipment.

[0037] The abnormal voiceprint signal type is obtained based on the fused feature vector of the main power grid equipment.

[0038] Furthermore, the expanding the voiceprint data to obtain the expanded voiceprint data includes:

[0039] Select a benchmark voiceprint sample and calculate the effective value;

[0040] Calculate the amplitude spectrum of two or more samples respectively, and retain the phase spectrum of the reference sample;

[0041] Based on the sample amplitude spectrum, the peak amplitude sequence is extracted for all samples according to the set peak conditions to obtain the peak amplitude sequence;

[0042] Performing random weighting processing on each peak amplitude sequence to obtain an enhanced peak amplitude sequence;

[0043] The enhanced peak amplitude sequence is summed up according to the sample weights to obtain a mixed peak sequence;

[0044] IFFT calculation is performed according to the effective value and phase spectrum of the reference sequence and the amplitude spectrum of the peak sequence after mixing to obtain the multi-sample mixed enhanced samples.

[0045] Furthermore, the PSOSIFT method is used to perform pixel-level alignment of the three modal images, and multiple modalities are mapped into a common representation space. Then, industrial annotation, Mosaic data enhancement and normalization preprocessing are performed to construct a power grid main equipment defect image dataset with a multi-source structure that includes appearance defect labels, temperature labels, and voiceprint spectra.

[0046] Furthermore, the multimodal feature extraction network is a multi-scale network with two parallel branches of CNN and Transformer network. The first feature layer and the last feature layer of the CNN network are selected and fused with the transformer network, which are denoted as F0 and F_L respectively. According to the target characteristics in different scenes, a weighted fusion strategy is adopted. When detecting small targets in large scenes, the F0 weight fusion ratio is increased; when detecting large targets, the F_L weight fusion ratio is increased.

[0047] The second aspect of the present invention provides a power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration, which adopts the power grid main equipment fault diagnosis method based on voiceprint cloud-edge collaboration described in the first aspect, including a data acquisition module, an edge computing device and a cloud platform; the edge computing device includes a noise suppression module and an anomaly detection module;

[0048] The data acquisition module is configured to: collect voiceprint data, visible light image data and infrared image data of the main power grid device;

[0049] The noise suppression module is configured to: perform noise suppression on the acquired voiceprint data to obtain the voiceprint data after noise suppression;

[0050] The anomaly detection module is configured to: train an anomaly detection model using the noise-suppressed voiceprint features and the set conditional features as combined features to obtain an optimal risk threshold; calculate the risk value of the voiceprint data to be detected using the trained anomaly detection model, and determine the voiceprint data as abnormal when the risk value is greater than the optimal risk threshold;

[0051] The cloud platform is configured to: combine the abnormal voiceprint data detected by the abnormality detection module, fuse the visible light image data and the infrared image data, perform secondary classification on the abnormal voiceprint data, and obtain the abnormal voiceprint signal diagnosis result.

[0052] Furthermore, in the data acquisition module, the voiceprint data is collected by a sound signal acquisition device, and the sound signal acquisition device collects data by using a digital microphone and an analog microphone in collaboration. The digital microphone stores the sound signal in the form of digital data, and the analog microphone has a physical disk that can store vibration signals. The vibration signals are stored in the physical disk in the form of vibration line traces in the form of strokes. A merging and intersection processing unit is provided behind the acquisition device to merge the signals of the digital microphone and the analog microphone, and take the intersection signal as the final acquired output signal.

[0053] Furthermore, in the merge intersection processing unit, the signals of the digital microphone and the analog microphone are merged and processed, including:

[0054] Convert the analog signal into discrete samples, obtain the maximum and minimum sampling amplitudes, take the average value as the mapping value of the analog signal to the digital signal, set the value below the average value to 0, and set the value above the average value to 1;

[0055] The mapped digital signal and the original digital signal are used to calculate the analog signal, retain the true value, and obtain the final combined signal.

[0056] Furthermore, the noise suppression module includes a background voiceprint estimation module, a noise period estimation module, a spectrum replacement calculation module and a data restoration module;

[0057] The noise period estimation module is configured to: calculate the signal-to-noise ratio of each frame of data according to the background voiceprint estimation result obtained by the background voiceprint estimation module, and determine the frame number to be denoised according to the set noise consistency threshold;

[0058] The spectrum replacement calculation module is configured to: perform data fusion processing on the amplitude spectrum information of the background voiceprint signal obtained in the background voiceprint estimation module and the phase spectrum information of the noise segment signal obtained in the noise period estimation module to obtain the spectrum replacement and then fuse the full spectrum information;

[0059] The data restoration module is configured to fuse the full spectrum information based on spectrum replacement and perform IFFT calculation to obtain a denoised time domain signal.

[0060] Furthermore, the anomaly detection module is configured to: train the anomaly detection model using the noise-suppressed voiceprint features and the set conditional features as combined features to obtain an optimal risk threshold, including:

[0061] The transformer voiceprint feature s and the set conditional feature c are combined into the input feature x = (s, c), and then the encoder G is generated. E Get the hidden vector z, and generate the decoder G D Get the reconstructed features of x After an additional encoder E, the reconstructed features of the latent vector are obtained Based on the combined feature x and the reconstructed feature Input to the discriminator D, decision and encoder G E and decoder G D The combined generator obtains the optimal risk threshold through continuous adversarial training.

[0062] Furthermore, the voiceprint features include MFCC, time-frequency graph and Mel-spectrogram, and the conditional features include transformer voltage level, cooling method, manufacturer and station.

[0063] Furthermore, the cloud platform includes a data expansion module, a data set construction module, a multimodal feature extraction module, a multimodal feature fusion and defect identification module;

[0064] The data expansion module is configured to expand the voiceprint image to obtain an expanded voiceprint image;

[0065] The dataset construction module is configured to: map multiple modalities including visible light images, infrared images and expanded voiceprint images into a common representation space, and construct a grid main equipment defect image dataset with a multi-source structure that includes appearance defect labels, temperature labels and voiceprint spectra;

[0066] The multimodal feature extraction module is configured to: obtain visible light image features, infrared image features and voiceprint image features based on the constructed grid main equipment defect image dataset and the trained multimodal feature extraction network;

[0067] The multimodal feature fusion module is configured to: weightedly fuse the visible light image features, infrared image features and voiceprint image features through the mutual attention module to obtain a fused feature vector of the main power grid device;

[0068] The defect identification module is configured to: identify the abnormal type according to the fused feature vector of the main power grid equipment.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] 1. The present invention innovatively proposes a method for fault diagnosis of main power grid equipment based on voiceprint cloud-edge collaboration, adopts a new cloud-edge collaboration method, and divides the analysis of voiceprint monitoring data into two stages. An anomaly detection model is deployed on the edge side to realize anomaly analysis of massive voiceprint data, avoid concurrent requests for a large number of concurrent signals on the cloud, and improve the timeliness of voiceprint monitoring and analysis. In the cloud, a multimodal defect recognition method that integrates multi-source data such as voiceprints, visible light, and infrared is deployed to realize secondary classification of abnormal sound signals and improve the reliability of equipment operation and management.

[0071] 2. The present invention innovatively proposes a spectrum noise reduction method for main power grid equipment based on the evaluation of stable voiceprint signals. According to the stable characteristics of the voiceprint of the main power grid equipment, a reverse thinking method is proposed for non-stationary interference signals, that is, by finding the signal of the non-noise interference period as the target signal, and then spectrally replacing the signal of the noise interference period, retaining the original phase information of the signal, and replacing it with the amplitude information of the minimum energy frame (the weakest interference frame) in the period, to achieve noise suppression of any instantaneous interference. The problem of voiceprint signal pollution and feature distortion of the main power grid equipment due to instantaneous noise and continuous noise interference in the acquisition process in complex environments such as substations is solved, and high-quality acquisition of voiceprint signals of the main power grid equipment is achieved, and the accuracy of remote diagnosis of equipment faults is improved.

[0072] 3. The present invention innovatively proposes a multi-sample data expansion method for the voiceprint of the main power grid equipment, which fully takes into account that most of the sounds collected in the substation are the superposition of direct and reflected sounds from different sound sources, solves the problems of unbalanced voiceprint data and small number of samples, and realizes the diversity expansion of the voiceprint of the main power grid equipment under the condition of superposition of reflections from non-point sound sources.

[0073] 4. The present invention innovatively proposes a method for detecting abnormal voiceprint data of main power grid equipment, uses the voiceprint features after noise suppression and the set conditional features as combined features to train the anomaly detection model, and reasonably calculates the optimal risk threshold, which greatly reduces the time and space complexity of the calculation, and improves the speed of fault detection and the efficiency of subsequent defect diagnosis and identification results.

[0074] 5. The present invention innovatively proposes a grid main equipment defect recognition model that integrates voiceprint, visible light and infrared, which solves the problem of insufficient collaborative diagnosis and analysis capabilities of multi-source data and inability to perform more accurate equipment status evaluation, and realizes accurate diagnosis of abnormal conditions of power equipment. It is capable of simultaneously processing a monitoring system of multiple types of data to provide more comprehensive monitoring results, effectively detect abnormal conditions of power equipment, and provide timely warnings.

[0075] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0077] Figure 1 It is a flow chart of a fault diagnosis method based on voiceprint cloud-edge collaboration of power grid master equipment provided by an embodiment of the present invention;

[0078] Figure 2 This is an improved spectrum subtraction calculation process provided by an embodiment of the present invention;

[0079] Figure 3 is a diagram of an anomaly detection network structure provided by an embodiment of the present invention;

[0080] Figure 4 is a structural diagram of an encoder provided by an embodiment of the present invention;

[0081] Figure 5 is a structural diagram of a decoder provided by an embodiment of the present invention;

[0082] Figure 6 It is a flow chart of multi-sample mixed weighted enhancement and expansion based on voiceprint spectrogram provided by an embodiment of the present invention;

[0083] Figure 7 This is an architecture diagram of a fault diagnosis system based on voiceprint cloud-edge collaboration of power grid master equipment provided by an embodiment of the present invention;

[0084] Figure 8 It is a diagram of the sound signal collection principle of the sound signal collection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0085] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0086] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0087] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0088] Embodiment 1

[0089] like Figure 1 As shown, this embodiment provides a method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration, including the following steps:

[0090] Step 1: Collect voiceprint data, visible light image data and infrared image data of the main equipment of the power grid;

[0091] It should be noted that, in this embodiment, the transformer is taken as an example to illustrate the fault diagnosis process of the main equipment of the power grid, and other embodiments can be selected according to actual conditions.

[0092] Step 2: Suppress the noise of the acquired voiceprint data to obtain the voiceprint data after noise suppression;

[0093] The traditional noise suppression using spectral subtraction includes the following steps:

[0094] Background voiceprint assessment is not exactly the same as speech signal spectral subtraction. Speech signal spectral subtraction uses the noise statistical characteristics of the speech intermittent segment to replace the noise statistical characteristics of the superimposed noise period, and then performs spectral subtraction to obtain the noise-suppressed speech signal. Since the transformer voiceprint signal is a stable and continuous signal, it is impossible to obtain the intermittent segment signal. Therefore, to solve this problem, it is necessary to determine the type and power of the noise. There are three conventional ideas:

[0095] (1) Several typical noises are recorded in advance. When the signal-to-noise ratio of the actual collected signal decreases, the actual collected signal and the noise are correlated to determine the noise type. The noise intensity is determined based on the signal power difference between the noise superposition segment and the noise-free segment, thereby determining the noise segment to be spectrally subtracted. However, this method requires certain prior knowledge.

[0096] (2) Based on the typical interference signals collected in advance, the noise type recognition model is trained to determine in real time whether there is noise interference and the noise period. Then, based on the signal power difference between the noise superposition section and the noise-free section, the noise intensity is determined, thereby determining the noise section to be spectrally subtracted. This method still requires certain prior knowledge.

[0097] (3) Taking typical fan noise as an example, since the soundprint signals generated by transformer operation and fan operation are both stable noise, it is possible to collect the soundprint of a single fan during operation, determine whether the fan is running based on the sound pressure level or the energy of the specified frequency band (fan rotation frequency band), and perform real-time spectrum subtraction. This method is only applicable to stable and measurable interference.

[0098] In response to the above problems, in this embodiment, based on the stable characteristics of the transformer voiceprint, a reverse thinking method is proposed for the non-stationary interference signal, that is, by finding the signal of the non-noise interference period as the target signal, and then spectrally replacing the signal of the noise interference period, retaining the original phase information of the signal, and replacing it with the amplitude information of the frame with the minimum energy (the frame with the weakest interference) in the time period, so as to achieve noise suppression of any instantaneous interference.

[0099] like Figure 2 As shown, the specific steps include:

[0100] Step 201: Use a traditional background voiceprint estimation method to obtain a background voiceprint estimation result;

[0101] Step 202: Calculate the signal-to-noise ratio of each frame of data according to the background voiceprint estimation result obtained by the background voiceprint estimation module, and determine the frame number to be denoised according to the set noise consistency threshold;

[0102] Step 203: Perform data fusion processing on the amplitude spectrum information of the background voiceprint signal obtained in the background voiceprint estimation module and the phase spectrum information of the noise segment signal obtained in the noise period estimation module to obtain the spectrum replacement and fusion full spectrum information;

[0103] Step 204: Based on the spectrum replacement and fusion of the full spectrum information, IFFT calculation is performed to obtain the time domain signal after noise reduction.

[0104] Step 3: Use the noise-suppressed voiceprint features and the set conditional features as combined features to train the anomaly detection model to obtain the optimal risk threshold; use the trained anomaly detection model to calculate the risk value of the voiceprint data to be detected. When the risk value is greater than the optimal risk threshold, it is determined to be abnormal voiceprint data;

[0105] Specifically include:

[0106] Step 301: Training process

[0107] The transformer voiceprint feature s and the set conditional feature c are combined into the input feature x = (s, c), and then the encoder G is generated. E Get the hidden vector z, and generate the decoder G D Get the reconstructed features of x After an additional encoder E, the reconstructed features of the latent vector are obtained Based on the combined feature x and the reconstructed feature Input to the discriminator D, the decision and the generator (encoder G E and decoder G D Combined) Through continuous adversarial training, seek the optimal risk threshold;

[0108] like Figure 3 As shown, specifically including:

[0109] The transformer voiceprint feature s is derived from normal transformer voiceprint data, such as MFCC, time-frequency graph, mel-spectrogram, etc. The conditional feature c includes the transformer voltage level (110kV, 220kV, 500kV, 1000kV), cooling method (oil-immersed air cooling, forced oil circulation air cooling, forced guided oil circulation air cooling), manufacturer and station. All conditional features are combined and one-hot encoded. Then the voiceprint feature and conditional feature are combined into the input feature x = (s, c).

[0110] Generate encoder G E:The input feature x is calculated by the generative encoder to obtain the latent vector z=G E (x, c), generating encoder G E It includes convolutional layers, layer normalization, activation functions, and Dropout, etc., to achieve dimensional compression of the input feature x and represent the potential features of the input feature. Layer Normalization is used here to optimize the problem that batch normalization is easily affected by changes in the distribution of data within the batch. Dropout is only used in the training phase. When testing or evaluating the model, the probability needs to be set to 0 to turn off Dropout. The network structure is as follows: Figure 4 As shown;

[0111] Generate decoder G D : Hidden vector z = G E (x, c) is generated by the decoder G D Afterwards, after the scale is enlarged upward, the reconstructed feature vector is obtained. Generate decoder G D Including transposed convolution layer, normalization layer and activation function layer. When reconstructing the feature vector The difference between the input feature x and the input feature x is very small, indicating that the input feature x can be approximately reconstructed, and then the transformer voiceprint anomaly detection is realized by calculating the error between the two to generate the decoder G E The network structure is as follows Figure 5 As shown; the reconstruction loss is defined using the Smooth L1 loss function as follows:

[0112]

[0113] in, x i Represents the i-th element of the input feature.

[0114] Encoder E: In order to better learn the potential feature distribution of normal voiceprint data, the generated features After being encoded again by encoder E, the reconstructed latent vector is obtained When this vector is consistent with the latent vector z, it indicates the reconstructed feature The potential distribution characteristics of are similar to those of the input feature x, and the encoding loss is still defined using the Smooth L1 loss function as follows:

[0115]

[0116] in x i represents the i-th element of the input feature, Represents the i-th element of the reconstructed feature. This method greatly reduces the time complexity and space complexity of the calculation and improves the speed of fault detection. Encoders E and G E The structure is consistent.

[0117] Discriminator network D:

[0118] The role of the discriminator D is consistent with the traditional adversarial generation network. Input features and conditional features (x, c) are input and reconstructed features The input discriminator D is scaled down multiple times through the transposed convolution layer, batch normalization layer, and leaky linear rectifier activation function layer. The compressed vector is flattened and input into the fully connected layer to achieve the purpose of binary classification (determining whether the input is original feature data or generated feature data).

[0119] The adversarial loss is defined as follows:

[0120]

[0121] Among them, D(x, c) represents the output (probability value) of the conditional feature vector after the discriminator D, P data(x) is the distribution of positive samples, E represents the expectation, Represents the distribution of positive samples from P data(x) Extract samples x from the y-axis and perform the desired operation on them. Represents the distribution of positive samples from P data(x) Reconstruction features and perform the desired operations on it. Represents the generated reconstruction features The output of the decision device D;

[0122] In general, the overall loss of the model during training is as follows:

[0123] L=L adv +αL rec +βL enc ,

[0124] During the training phase, the model is trained using voiceprint monitoring data of different voltage levels, from different sites and with different cooling methods. α and β are weight coefficients. In this embodiment, the weights α=1 and β=0.1.

[0125] In the case of minimal loss, the risk threshold is calculated:

[0126] R=L rec +λL enc ,

[0127] Where λ is an adjustable parameter, which is selected according to the batch data.

[0128] Step 302: Reasoning process

[0129] Use the trained generative encoder G E , generate decoder G D , encoder E is used to calculate the reconstruction loss and encoding loss, and the risk R′ value of the data to be detected is calculated. When the risk R′ value of the data to be detected exceeds the risk threshold R obtained by training, it can be determined as abnormal data.

[0130] Step 4: Combine the abnormal voiceprint data detected by the anomaly detection module, fuse the visible light image data and the infrared image data, perform secondary classification on the abnormal voiceprint data, and obtain the abnormal voiceprint signal diagnosis result.

[0131] In order to accurately identify the defect category, the infrared and visible light data are fused based on the voiceprint data obtained from the edge side to implement a multi-modal method for accurately identifying faults in the main power grid equipment.

[0132] The types of power equipment defects in most substations are complex and traditional power equipment monitoring methods often rely on a single monitoring method, such as infrared imaging or sound signal analysis. Although these methods are effective, they may miss important information. For example, it is difficult to accurately identify physical damage to the appearance of equipment by relying solely on infrared imaging, and it is difficult to determine the cause of temperature abnormalities by relying solely on sound signal analysis. Therefore, a monitoring system that can process multiple types of data simultaneously is needed to provide more comprehensive monitoring results, effectively detect abnormal conditions of power equipment, and provide timely warnings.

[0133] Multimodal data fusion refers to the integration of information from multiple different sensors or data sources to form more comprehensive, accurate and reliable information. These sensors or data sources may include images, videos, voice, text, sensor data, etc. By integrating data from different modalities, multimodal data fusion can improve the accuracy and reliability of data analysis, recognition, classification and prediction.

[0134] Infrared imaging can help detect temperature anomalies in equipment, such as overheated connection points or insulation problems. Visible light imaging can capture the appearance of equipment, such as structural damage, rust or cracks. Acoustic modality involves detecting the operating status of equipment through sound signals. The main equipment of the power grid will produce specific sound characteristics when operating normally, and abnormal sounds may be early signs of equipment failure. Sound signal analysis can be used to identify abnormal vibrations, partial discharge noise, etc. inside the equipment.

[0135] Specifically include:

[0136] Step 401, expanding the voiceprint image to obtain an expanded voiceprint image;

[0137] In this embodiment, considering that the voiceprint of the transformer site has the characteristics of non-single sound source and non-point sound source, and the sound signal collected by the sensor is actually the superposition of different sound sources after direct and reflected, that is, under the same working conditions, the voiceprint data collected after moving the sensor position may appear as the weighted superposition of voiceprint data at other positions. In order to realize the generation of transformer site voiceprint data in more reasonable scenarios, reference is made to Figure 6 The specific steps of the flow chart based on multi-sample mixed weighted enhancement and expansion of voiceprint spectrogram are as follows:

[0138] Select a benchmark voiceprint sample and calculate the effective value;

[0139] Calculate the amplitude spectrum of two or more samples respectively, and retain the phase spectrum of the reference sample;

[0140] Based on the sample amplitude spectrum, the peak amplitude sequence is extracted for all samples according to the set peak conditions. The specific peak conditions can be adjusted according to the actual situation. For example, only the maximum value within a certain range is considered a peak, so as to avoid the enhancement of random peaks in the background noise signal, which may cause signal pollution.

[0141] Performing random weighting processing on each peak amplitude sequence to obtain an enhanced peak amplitude sequence;

[0142] The enhanced peak amplitude sequence is summed up according to the sample weights to obtain a mixed peak sequence;

[0143] Perform IFFT calculation according to the effective value and phase spectrum of the reference sequence and the amplitude spectrum of the peak sequence after mixing to obtain the multi-sample mixed enhanced samples;

[0144] Step 402: multiple modalities of visible light images, infrared images and expanded voiceprint images are mapped into a common representation space, and a grid main equipment defect image dataset with a multi-source structure that includes appearance defect labels, temperature labels and voiceprint spectra is constructed;

[0145] In this embodiment, infrared and visible light cameras and voiceprint devices are used to regularly inspect multiple substations and collect visible light images, infrared images, and voiceprint images of the main equipment at the inspection points. First, the PSOSIFT method is used to align the above three modal images at the pixel level, and multiple modalities are mapped to a common representation space. Then, preprocessing such as manual annotation, Mosaic data enhancement and normalization is performed to construct a grid main equipment defect image dataset with a multi-source structure that includes appearance defect labels, temperature labels, and voiceprint spectra;

[0146] Step 403: Obtain visible light image features, infrared image features, and voiceprint image features based on the constructed grid main equipment defect image dataset and the trained multimodal feature extraction network;

[0147] In the fault diagnosis of main power grid equipment, CNN-Transformer combines the local perception ability of CNN and the global dependency capture ability of Transformer. It is suitable for processing multimodal data and can be used to process image-text-sound information pairs to improve the accuracy of fault diagnosis.

[0148] Visible light feature extraction: Use a dual-branch CNN-Transformer network model to extract features such as edges, textures, and shapes;

[0149] Infrared image feature extraction: The encoding part of the CNN-Transformer network model is also used to extract features such as temperature distribution and hot spots, and to extract temperature gradient maps to identify areas with high temperature change rates;

[0150] Sound signal feature extraction: Use a dual-branch CNN-Transformer network model to extract features such as the main frequency, fundamental frequency, and harmonic distortion in the spectrum graph to identify abnormal vibrations or noise.

[0151] Convolutional neural networks (CNNs) have advantages such as strong local perception and strong robustness, but lack the ability to perceive global information. Different feature layers of CNN networks have different scale receptive fields, positions, details and other information. Shallow networks have higher resolution and rich detail information. Transformer networks are encoder-decoder structure networks that use self-attention mechanisms to learn the association between features. They have global perception capabilities, can capture the overall information and semantic relationships of the image, and expand the network's receptive field to the entire image.

[0152] Existing target detection methods adopt a single-branch network structure. Continuous downsampling and upsampling operations often cause some detail information to be lost. At the same time, feature information at different levels cannot or is inconvenient to interact, resulting in some detail information loss and limited field of view.

[0153] This embodiment designs a multi-scale network with two parallel branches of CNN and Transformer network as the encoder part of the CNN-Transformer network model. The local detail features and global semantic features extracted layer by layer are compensated for their respective feature expressions through mutual information fusion, which expands the feature field of the same level and improves the target judgment and positioning capabilities.

[0154] The entire dual-branch network mainly consists of three parts: backbone network, encoder, decoder, and feedforward network:

[0155] Backbone network: CNN extracts multi-scale features of images;

[0156] Encoder: The different scale features output by the backbone network are fused with the Transformer encoder to form a new encoder.

[0157] Since the first layer of the CNN network output has more detailed local basic feature information, and the last layer has a larger receptive field and global feature information, this embodiment selects the first feature layer, the last feature layer and the network after the transformer fusion, respectively, and records them as F0 and F_L. According to the target characteristics in different scenes, a weighted fusion strategy is adopted. When detecting small targets in large scenes, in order to enhance the ability to express the details of small targets, the F0 weight fusion ratio is increased; when detecting large targets, in order to enhance the ability to express the global features of large targets, the F_L weight fusion ratio is increased.

[0158] Compared with CNN networks and Transformer networks, the amount of calculation is slightly increased while the recognition accuracy is improved, so that the sound can be heard accurately and the appearance of the device can be seen accurately.

[0159] Decoder: The decoder is the same as the standard Transformer decoder, consisting of self-attention and cross-attention layers;

[0160] Feedforward network FNN: The feedforward network consists of a 3-layer perceptron with ReLU activation function and a linear mapping layer. The output prediction result is the normalized target center point coordinates, the width and height of the detection box relative to the input image, and the label of the category to which it belongs.

[0161] Step 404: weighted fusion of visible light image features, infrared image features and voiceprint image features through the mutual attention module, splicing multiple modal features together to form a large feature vector for defect identification.

[0162] This method can extract key information from multiple modal data and automatically learn the relationship between different modes through a deep learning model, thereby better identifying the failure mode of the equipment.

[0163] Step 405: Identify the abnormal type according to the multimodal characteristics of the main power grid equipment;

[0164] After obtaining the fusion features of visible light images, infrared images, and voiceprint images, they are input into the decoder and feedforward network part of the CNN-Transformer network model for processing and output the final result.

[0165] Based on the output results, it is determined whether there are abnormal temperature areas, whether the sound is uniform, and whether there are abnormal readings on the appearance of the equipment. This enables the identification of defects in power grid equipment and improves identification accuracy.

[0166] Embodiment 2

[0167] like Figure 7 As shown, this embodiment provides a power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration, adopting the method of embodiment 1, including a data acquisition module, an edge computing device and a cloud platform;

[0168] The data acquisition module is configured to: collect voiceprint data, visible light image data and infrared image data of the main power grid device;

[0169] The data acquisition module is configured to collect voiceprint data when the main power grid equipment is running;

[0170] In this embodiment, the voiceprint data is collected by a sound signal collection device, and the sound signal collection device uses a digital microphone and an analog microphone to collect the data in a coordinated manner;

[0171] like Figure 8 As shown in the figure, the specific collection process is as follows:

[0172] Digital microphones store sound signals in the form of digital data. The specific implementation is as follows: the digital microphone has its own high-frequency sampling module, which can convert the vibration amount into a digital value, and then convert it into a discrete value. The discrete value is then quantified, mapped to a certain numerical range, and the value is converted into a binary digital signal to realize digital data storage.

[0173] The analog microphone acquisition is specifically implemented as follows: the analog microphone is provided with a physical disk that can store the vibration signal, and the vibration signal is stored in the physical disk in the form of vibration line traces in the form of scratches.

[0174] A merging and intersection processing unit is provided at the rear of the acquisition device to merge the signals of the digital microphone and the analog microphone and take the intersection signal as the final acquired output signal.

[0175] Since digital signals are discrete digital signals on a time scale, they are composed of binary digits (0 and 1), and appear as high level (1) and low level (0) on the acquisition device. Their information is not continuous, and the signal is expressed in a fixed range, such as 0-1. Analog signals are continuous signals on a time scale, and appear as continuous changes in voltage on the acquisition device. The range of expression varies with the strength of the signal, and there is no fixed range.

[0176] In this embodiment, the steps of combining the signals of the digital microphone and the analog microphone and taking the intersection signal as the final collected output signal specifically include the following steps:

[0177] Convert the analog signal into discrete samples and perform high-frequency sampling on the analog signal. The sampling frequency is p. The larger the p, the less signal loss. Obtain the maximum and minimum sampling amplitudes, take the average value, and use it as the mapping value of the analog signal to the digital signal. Values ​​below the average value are set to 0, and values ​​above the average value are set to 1.

[0178] The analog signal is converted into a digital signal and the original digital signal is taken to perform "analog signal calculation, retain the true value, and finally obtain a valid signal after removing noise. The edge computing device includes a signal preprocessing module, a data storage module and an anomaly detection module;

[0179] The signal processing module uses an embedded FPGA platform to realize digital sampling of sound signals through the ADC conversion module, and transmits the data to the defect recognition module after noise suppression processing;

[0180] The signal processing module includes a background voiceprint estimation module, a noise period estimation module, a spectrum replacement calculation module and a data restoration module;

[0181] The background voiceprint estimation module is configured to use a traditional background voiceprint estimation method to obtain a background voiceprint estimation result;

[0182] The noise period estimation module is configured to calculate the signal-to-noise ratio of each frame of data according to the background voiceprint estimation result obtained by the background voiceprint estimation module, and determine the frame number to be denoised according to the set noise consistency threshold;

[0183] The spectrum replacement calculation module is configured to perform data fusion processing on the amplitude spectrum information of the background voiceprint signal obtained in the background voiceprint estimation module and the phase spectrum information of the noise segment signal obtained in the noise period estimation module to obtain spectrum replacement and then fuse the full spectrum information;

[0184] The data restoration module is configured to fuse the full spectrum information based on spectrum replacement and perform IFFT calculation to obtain the denoised time domain signal.

[0185] The data storage module stores the collected signal data;

[0186] The anomaly detection module is based on the ARM platform architecture and deploys a variety of defect recognition algorithms to meet computing power requirements; the real-time communication module: has wired and wireless communication methods to transmit stored data and recognition results to the central monitoring system or cloud platform.

[0187] The anomaly detection module is configured to: train an anomaly detection model using the noise-suppressed voiceprint features and the set conditional features as combined features to obtain an optimal risk threshold; calculate the risk value of the voiceprint data to be detected using the trained anomaly detection model, and determine the voiceprint data as abnormal when the risk value is greater than the optimal risk threshold;

[0188] In this embodiment, the anomaly detection module includes a training module and a reasoning module;

[0189] The training module is configured to: combine the transformer voiceprint feature s and the set conditional feature c into an input feature x=(s, c), and generate an encoder G E Get the hidden vector z, and generate the decoder G D Get the reconstructed features of x After an additional encoder E, the reconstructed features of the latent vector are obtained Based on the combined feature x and the reconstructed feature Input to the discriminator D, the decision and the generator (encoder G E and decoder G D Combined) Through continuous adversarial training, seek the optimal risk threshold;

[0190] The inference module is configured to use the trained generative encoder G E , generate decoder G D , encoder E is used to calculate the reconstruction loss and encoding loss, and the risk R′ value of the data to be detected is calculated. When the risk R′ value of the data to be detected exceeds the risk threshold R obtained by training, it can be determined as abnormal data.

[0191] Deploy a cloud platform in the cloud center, wherein the cloud platform is configured to: receive anomaly detection results on the edge side, fuse multi-source data, perform secondary classification on abnormal sound signals, and obtain abnormal signal diagnosis results;

[0192] In order to accurately identify the defect category, the infrared and visible light data are integrated with the voiceprint data obtained from the edge side to implement a multi-modal method for accurately identifying faults in the main power grid equipment.

[0193] The cloud platform includes a data expansion module, a data set construction module, a multimodal feature extraction module, and a multimodal feature fusion and defect identification module;

[0194] The data expansion module is configured to expand the voiceprint image to obtain an expanded voiceprint image;

[0195] The dataset construction module is configured to: map multiple modalities including visible light images, infrared images and expanded voiceprint images into a common representation space, and construct a grid main equipment defect image dataset with a multi-source structure that includes appearance defect labels, temperature labels and voiceprint spectra;

[0196] The multimodal feature extraction module is configured to: obtain visible light image features, infrared image features and voiceprint image features based on the constructed grid main equipment defect image dataset and the trained multimodal feature extraction network;

[0197] The multimodal feature fusion module is configured to: weightedly fuse the visible light image features, infrared image features and voiceprint image features through the mutual attention module to obtain a fused feature vector of the main power grid device;

[0198] The defect identification module is configured to: identify the abnormal type according to the fused feature vector of the main power grid equipment.

[0199] Embodiment 3

[0200] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in the method of a power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration as described above are implemented.

[0201] Embodiment 4

[0202] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method of a power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration as described above are implemented.

[0203] Embodiment 5

[0204] This embodiment provides a program product, which is a computer program product, including a computer program. When the computer program is executed by a processor, the steps in the method of a power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration as described above are implemented.

[0205] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for fault diagnosis of main equipment of power grid based on voiceprint cloud-edge collaboration, characterized in that: The steps include: Acquire voiceprint data, visible light image data and infrared image data of main power grid equipment; Performing noise suppression on the acquired voiceprint data to obtain the noise-suppressed voiceprint data; On the edge side, the voiceprint features after noise suppression and the set conditional features are used as combined features to train the anomaly detection model to obtain the optimal risk threshold; The trained anomaly detection model is used to calculate the risk value of the voiceprint data to be detected. When the risk value is greater than the optimal risk threshold, it is determined to be abnormal voiceprint data; In the cloud, the abnormal voiceprint data obtained by detection is combined with the visible light image data and infrared image data to perform secondary classification on the abnormal voiceprint data and obtain the abnormal voiceprint signal diagnosis result.

2. A method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration as claimed in claim 1, characterized in that: Noise suppression is performed on the acquired voiceprint data to obtain the voiceprint data after noise suppression, including: According to the background voiceprint estimation result obtained by the background voiceprint estimation module, the signal-to-noise ratio of each frame of data is calculated, and the sequence number of the frame to be denoised is determined according to the set noise consistency threshold; The amplitude spectrum information of the background voiceprint signal obtained in the background voiceprint estimation module and the phase spectrum information of the noise segment signal obtained in the noise period estimation module are subjected to data fusion processing to obtain the full spectrum information after spectrum replacement; Based on the spectral replacement and fusion of the full spectrum information, IFFT calculation is performed to obtain the denoised time domain signal.

3. A method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration as claimed in claim 1, characterized in that: The method of using the noise-suppressed voiceprint features and the set conditional features as combined features to train the anomaly detection model to obtain the optimal risk threshold includes: The transformer voiceprint feature s and the set conditional feature c are combined into the input feature x = (s, c), and then the encoder G is generated. E Get the hidden vector z, and generate the decoder G D Get the reconstructed features of x After an additional encoder E, the reconstructed features of the latent vector are obtained Based on the combined feature x and the reconstructed feature Input to the discriminator D, decision and encoder G E and decoder G D The combined generator obtains the optimal risk threshold through continuous adversarial training.

4. A method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration as claimed in claim 1, characterized in that: Use the trained generative encoder G E , generate decoder G D , encoder E is used to calculate the reconstruction loss and encoding loss, and the risk R′ value of the data to be detected is calculated. When the risk R′ value of the data to be detected exceeds the risk threshold R obtained by training, it can be determined as abnormal data.

5. The method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration according to claim 1, characterized in that: The calculation formula for the overall loss of the anomaly detection model is: L=L adv +αL rec +βL enc , Among them, L adv To combat the loss, L rec is the reconstruction loss, L enc is the encoding loss, α and β are weight coefficients, x i represents the i-th element of the input feature, represents the i-th element of the reconstructed feature, D(x, c) represents the output of the conditional feature vector after the discriminator, Represents the distribution of positive samples from P data (x) extracts sample x and performs the desired operation on it. Represents the distribution of positive samples from P data Reconstructed features in (x) and perform the desired operations on it. Represents the generated reconstruction features The output of the decision device D.

6. A method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration as claimed in claim 1, characterized in that: Voiceprint features include MFCC, time-frequency graph and Mel-spectrogram, and conditional features include transformer voltage level, cooling method, manufacturer and station.

7. A method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration as claimed in claim 1, characterized in that: The conditional features are combined and encoded using one-hot encoding.

8. The method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration according to claim 1, characterized in that: The abnormal voiceprint data obtained by the combined detection is integrated with the visible light image data and the infrared image data to perform secondary classification on the abnormal voiceprint data to obtain the abnormal voiceprint signal diagnosis result, including: Expanding the voiceprint data to obtain expanded voiceprint data; By mapping multiple modalities such as visible light images, infrared images and expanded voiceprint data into a common representation space, a grid main equipment defect image dataset with a multi-source structure that includes appearance defect labels, temperature labels and voiceprint spectra was constructed. Based on the constructed grid main equipment defect image dataset and the trained multimodal feature extraction network, visible light image features, infrared image features and voiceprint image features are obtained; Through the mutual attention mechanism, the visible light image features, infrared image features and voiceprint image features are weightedly fused to obtain the fused feature vector of the main power grid equipment. The abnormal voiceprint signal type is obtained based on the fused feature vector of the main power grid equipment.

9. A method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration as claimed in claim 8, characterized in that: The step of expanding the voiceprint data to obtain the expanded voiceprint data includes: Select a benchmark voiceprint sample and calculate the effective value; Calculate the amplitude spectrum of two or more samples respectively, and retain the phase spectrum of the reference sample; Based on the sample amplitude spectrum, the peak amplitude sequence is extracted for all samples according to the set peak conditions to obtain the peak amplitude sequence; Performing random weighting processing on each peak amplitude sequence to obtain an enhanced peak amplitude sequence; The enhanced peak amplitude sequence is summed up according to the sample weights to obtain a mixed peak sequence; IFFT calculation is performed according to the effective value and phase spectrum of the reference sequence and the amplitude spectrum of the peak sequence after mixing to obtain the multi-sample mixed enhanced samples.

10. A method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration as claimed in claim 8, characterized in that: The PSOSIFT method is used to perform pixel-level alignment of the three modal images, and multiple modalities are mapped into a common representation space. Then, industrial annotation, Mosaic data enhancement and normalization preprocessing are performed to construct a power grid main equipment defect image dataset with a multi-source structure that includes appearance defect labels, temperature labels, and voiceprint spectra.

11. A method for diagnosing faults of main power grid equipment based on voiceprint cloud-edge collaboration as claimed in claim 8, characterized in that: The multimodal feature extraction network is a multi-scale network with two parallel branches of CNN and Transformer network. The first feature layer and the last feature layer of the CNN network are selected and fused with the transformer network, which are denoted as F0 and F_L respectively. According to the target characteristics in different scenes, a weighted fusion strategy is adopted. When detecting small targets in large scenes, the weight fusion ratio of F0 is increased; when detecting large targets, the weight fusion ratio of F_L is increased.

12. A power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration, adopting a power grid main equipment fault diagnosis method based on voiceprint cloud-edge collaboration as described in any one of claims 1-11, characterized in that: It includes a data acquisition module, an edge computing device and a cloud platform; the edge computing device includes a noise suppression module and an anomaly detection module; The data acquisition module is configured to: collect voiceprint data, visible light image data and infrared image data of the main power grid device; The noise suppression module is configured to: perform noise suppression on the acquired voiceprint data to obtain the voiceprint data after noise suppression; The anomaly detection module is configured to: train an anomaly detection model using the noise-suppressed voiceprint features and the set conditional features as combined features to obtain an optimal risk threshold; The trained anomaly detection model is used to calculate the risk value of the voiceprint data to be detected. When the risk value is greater than the optimal risk threshold, it is determined to be abnormal voiceprint data; The cloud platform is configured to: combine the abnormal voiceprint data detected by the abnormality detection module, fuse the visible light image data and the infrared image data, perform secondary classification on the abnormal voiceprint data, and obtain the abnormal voiceprint signal diagnosis result.

13. A power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration as claimed in claim 12, characterized in that: In the data acquisition module, the voiceprint data is collected by a sound signal acquisition device, which collects data by using a digital microphone and an analog microphone in collaboration. The digital microphone stores the sound signal in the form of digital data, and the analog microphone has a physical disk that can store vibration signals. The vibration signals are stored in the physical disk in the form of vibration line traces in the form of strokes. A merging and intersection processing unit is provided behind the acquisition device to merge the signals of the digital microphone and the analog microphone, and take the intersection signal as the final collected output signal.

14. A power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration as claimed in claim 13, characterized in that: In the merge intersection processing unit, the signals of the digital microphone and the analog microphone are merged and processed, including: Convert the analog signal into discrete samples, obtain the maximum and minimum sampling amplitudes, take the average value as the mapping value of the analog signal to the digital signal, set the value below the average value to 0, and set the value above the average value to 1; The mapped digital signal and the original digital signal are used to calculate the analog signal, retain the true value, and obtain the final combined signal.

15. The power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration as claimed in claim 12, characterized in that: The noise suppression module includes a background voiceprint estimation module, a noise period estimation module, a spectrum replacement calculation module and a data restoration module; The noise period estimation module is configured to: calculate the signal-to-noise ratio of each frame of data according to the background voiceprint estimation result obtained by the background voiceprint estimation module, and determine the frame number to be denoised according to the set noise consistency threshold; The spectrum replacement calculation module is configured to: perform data fusion processing on the amplitude spectrum information of the background voiceprint signal obtained in the background voiceprint estimation module and the phase spectrum information of the noise segment signal obtained in the noise period estimation module to obtain the spectrum replacement and then fuse the full spectrum information; The data restoration module is configured to fuse the full spectrum information based on spectrum replacement and perform IFFT calculation to obtain a denoised time domain signal.

16. A power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration as claimed in claim 12, characterized in that: The anomaly detection module is configured to: train the anomaly detection model using the noise-suppressed voiceprint features and the set conditional features as combined features to obtain an optimal risk threshold, including: The transformer voiceprint feature s and the set conditional feature c are combined into the input feature x = (s, c), and then the encoder G is generated. E Get the latent vector z, and get the reconstructed features of x through the generative decoder GD After an additional encoder E, the reconstructed features of the latent vector are obtained Based on the combined feature x and the reconstructed feature Input to the discriminator D, decision and encoder G E and decoder G D The combined generator obtains the optimal risk threshold through continuous adversarial training.

17. A power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration as claimed in claim 12, characterized in that: Voiceprint features include MFCC, time-frequency graph and Mel-spectrogram, and conditional features include transformer voltage level, cooling method, manufacturer and station.

18. The power grid main equipment fault diagnosis system based on voiceprint cloud-edge collaboration as claimed in claim 12, characterized in that: The cloud platform includes a data expansion module, a data set construction module, a multimodal feature extraction module, and a multimodal feature fusion and defect identification module; The data expansion module is configured to expand the voiceprint image to obtain an expanded voiceprint image; The dataset construction module is configured to: map multiple modalities including visible light images, infrared images and expanded voiceprint images into a common representation space, and construct a grid main equipment defect image dataset with a multi-source structure that includes appearance defect labels, temperature labels and voiceprint spectra; The multimodal feature extraction module is configured to: obtain visible light image features, infrared image features and voiceprint image features based on the constructed grid main equipment defect image dataset and the trained multimodal feature extraction network; The multimodal feature fusion module is configured to: weightedly fuse the visible light image features, infrared image features and voiceprint image features through the mutual attention module to obtain a fused feature vector of the main power grid device; The defect identification module is configured to: identify the abnormal type according to the fused feature vector of the main power grid equipment.

Images