GIS device fault acoustic data enhancement method and system based on generative large model

Abstract

The application provides a GIS equipment fault acoustic data enhancement method and system based on a generative large model, the method comprising: acquiring multi-modal data of a target device; the multi-modal data at least comprising acoustic signals, partial discharge signals, vibration signals and working condition information; performing feature extraction and fusion based on the multi-modal data to obtain a unified feature representation; performing multi-modal self-supervised training on an initial acoustic base model based on the unified feature representation to obtain an acoustic base model; constructing an acoustic generation model with conditional input based on the acoustic base model, an acoustic generation network, preset random noise and preset control conditions; generating target fault acoustic samples corresponding to the target device based on the acoustic generation model and the preset control conditions; performing quality evaluation based on the target fault acoustic samples and real acoustic data of the multi-modal data to obtain high-quality acoustic enhancement samples as acoustic diagnosis enhancement samples of the target device.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology and relates to a method and system for enhancing acoustic data of GIS equipment faults based on generative large models. Background Technology

[0002] Gas-insulated switchgear (GIS) is widely used in substations of all voltage levels, undertaking critical breaking, isolation, and protection functions in power systems. GIS equipment has a closed structure and strong gas medium sealing; any abnormalities such as gas leakage, partial discharge, internal discharge, or mechanical failure will directly affect system safety and power supply reliability. Therefore, how to continuously monitor the operating status of GIS equipment and promptly identify potential faults has always been a key research direction in the power industry.

[0003] In recent years, with the rapid development of sensor technology, artificial intelligence, and acoustic analysis technology, acoustic recognition-based fault detection methods for power equipment have gradually attracted attention. Acoustic signals have advantages such as non-contact operation, strong real-time performance, and the ability to penetrate enclosed cavities. They can be used to identify typical anomalies in GIS equipment, including: acoustic characteristics of partial discharge (PD), including pulse-type, spike-type, and high-frequency noise discharge; acoustic characteristics of mechanical faults, such as poor engagement of tap changers, loose mechanisms, and mechanical jamming; and acoustic characteristics of gas leaks, including high-frequency broadband noise such as trace SF6 leaks and minor valve seepage.

[0004] However, in practical applications, acoustic-based GIS fault identification still faces the following major challenges:

[0005] On-site acoustic fault data is extremely scarce. GIS equipment is mostly in a high-reliability operating state, with a low probability of typical faults occurring, and most faults exhibiting transient characteristics. This results in: most operating units struggling to collect accurate fault acoustic data; insufficient acoustic databases and imbalanced samples at most sites; and a lack of publicly available datasets for certain important faults (such as partial discharge precursors and minute leaks). Therefore, data scarcity is the core bottleneck restricting the performance of acoustic diagnostic algorithms.

[0006] The diversity and complexity of fault acoustics make it difficult to simulate. Different GIS models, installation methods, cavity structures, and operating pressures all affect acoustic characteristics, resulting in: significant differences in the acoustics of the same fault at different sites; complex noise environments (wind noise, electromagnetic noise, equipment background noise); and strong time-varying characteristics, making it difficult to augment data using traditional methods such as simple superposition and filtering. Traditional acoustic signal enhancement methods, such as random noise superposition, velocity perturbation, and spectral shift, are insufficient to cover the complexity of real-world operating conditions.

[0007] Acoustic features are coupled with multimodal characteristics such as partial discharge and vibration. Existing data is difficult to systematize. GIS faults are often accompanied by multimodal changes in acoustics, partial discharge, vibration, and temperature rise. However, existing public datasets are mostly: single-modal acoustic data, lacking synchronous records of partial discharge; signals lack a unified format and are not labeled in a standardized manner; and they cannot meet the needs of large-scale model and multimodal fusion training. This makes it difficult for intelligent diagnostic models to extract cross-modal correlation features.

[0008] Existing acoustic generation or data augmentation technologies are not suitable for the power industry. Although generative AI has developed rapidly in recent years in areas such as image processing, speech, and speech conversion, with models like GAN, VAE, Diffusion, and Flow, these models are designed for speech or music, not GIS acoustics. They cannot directly learn the unique structures of discharge spikes, high-frequency random noise, and broadband leakage noise, nor can they simulate the physical mechanisms of GIS cavity response and gas medium absorption characteristics. Therefore, existing generative technologies are difficult to directly use for high-fidelity generation of GIS fault acoustics.

[0009] Field data is highly private and difficult to share. The power industry, being a critical infrastructure sector, faces restrictions on data sharing, resulting in very few public datasets and a lack of a unified "GIS acoustic standard dataset" within the industry. This also hinders algorithm research by creating a lack of universally applicable foundational data. This further exacerbates the difficulties in model training.

[0010] Current fault diagnosis models lack generalization ability. Due to insufficient training samples and limited data scenario coverage, existing acoustic diagnosis models generally suffer from: significant performance degradation after migration to different sites; weak recognition ability when faced with new noise and new equipment types; and difficulty in supporting real-time deployment at the edge. This severely restricts the promotion and application of acoustic-based GIS fault diagnosis methods in engineering sites. Summary of the Invention

[0011] This application provides a method and system for enhancing acoustic data of GIS equipment faults based on generative large models, which is used to solve the problems of data scarcity and imbalanced samples in the field of GIS acoustic fault diagnosis.

[0012] In a first aspect, this application provides a method for enhancing acoustic data of GIS equipment faults based on a generative large model. The method includes: acquiring multimodal data of the target equipment; the multimodal data includes at least acoustic signals, partial discharge signals, vibration signals, and operating condition information; performing feature extraction and fusion based on the multimodal data to obtain a unified feature representation; performing multimodal self-supervised training on an initial acoustic basic model based on the unified feature representation to obtain an acoustic basic model; constructing a fault acoustic generative model with conditional input based on the acoustic basic model, an acoustic data generation network, preset random noise, and preset control conditions; the preset control conditions include at least fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions of the target equipment; and performing multimodal self-supervised training on an initial acoustic basic model based on the unified feature representation to obtain an acoustic basic model; constructing a fault acoustic generative model with conditional input based on the fault acoustic generative model and the preset control conditions; the preset control conditions include at least fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions of the target equipment; and constructing a fault acoustic generative model based on the preset control conditions. The control conditions for generating fault acoustic samples corresponding to the target equipment specifically include: performing noise diffusion processing based on the unified feature representation using the fault acoustic generation model to obtain noise-added acoustic samples; encoding and concatenating the fault type condition, fault level condition, operating condition condition, and equipment structure condition respectively to obtain a unified condition control vector; performing reverse denoising processing based on the unified condition control vector, the noise-added acoustic samples, and the unified feature representation using the fault acoustic generation model to generate corresponding fault acoustic sample features, and performing acoustic reconstruction on the fault acoustic sample features to generate fault acoustic samples corresponding to the target equipment; and performing quality evaluation based on the fault acoustic samples and the real acoustic data of the multimodal data to obtain fault acoustic enhancement samples of the target equipment.

[0013] In one implementation of the first aspect, feature extraction and fusion based on the multimodal data to obtain a unified feature representation includes: preprocessing the multimodal data to obtain preprocessed multimodal data; extracting features based on the preprocessed multimodal data to obtain multimodal features; the multimodal features include at least acoustic features, partial discharge features, and vibration features; and fusing the acoustic features, partial discharge features, and vibration features using a timestamp-based multimodal alignment function to obtain a unified feature representation.

[0014] In one implementation of the first aspect, preprocessing the multimodal data to obtain preprocessed multimodal data includes: performing noise reduction, normalization, and time-frequency transformation on the acoustic signal to obtain a preprocessed acoustic signal; performing pulse sequence detection and envelope analysis on the partial discharge signal to obtain a preprocessed partial discharge signal; performing empirical mode decomposition and envelope analysis on the vibration signal to obtain a preprocessed vibration signal.

[0015] In one implementation of the first aspect, the multimodal self-supervised training includes a Mask learning task, a contrastive learning task, and a multimodal alignment learning task. Multimodal self-supervised training is performed on the initial acoustic base model based on the unified feature representation to obtain the acoustic base model. This includes: Mask learning based on the unified feature representation as input to the initial acoustic base model, performing content prediction and reconstruction on the occluded regions of the unified feature representation to learn the internal acoustic structure; Contrastive learning based on the unified feature representation as input to the initial acoustic base model, constructing positive sample pairs between different modalities at the same time and negative sample pairs between different augmented views of the same modality to learn the representation ability of cross-modal correlation and fault semantic consistency; Multimodal alignment learning based on the unified feature representation as input to the initial acoustic base model, constraining the feature representations of the same semantic content in different modalities to align in a shared semantic space to learn shared high-level semantic information across modalities; Joint optimization based on the Mask learning task, the contrastive learning task, and the multimodal alignment learning task, using their respective loss functions to jointly update the initial acoustic base model to obtain the acoustic base model.

[0016] In one implementation of the first aspect, the fault acoustic generation model includes an input encoding module, a conditional embedding module, a noise diffusion module, a reverse denoising generation module, and an output reconstruction module. The input encoding module converts real-world fault acoustic samples into a unified feature representation suitable for model processing. The conditional embedding module encodes and concatenates preset control conditions to form a unified conditional control vector. The noise diffusion module progressively adds preset random noise to the unified feature representation to form a noise-added acoustic sample that transitions from a clear sample to random noise. The reverse denoising generation module performs reverse denoising processing on the noise-added acoustic sample under the control of preset control conditions to generate corresponding fault acoustic sample features. The output reconstruction module performs acoustic reconstruction on the fault acoustic sample features generated by the reverse denoising generation module to generate and output fault acoustic samples.

[0017] In one implementation of the first aspect, generating fault acoustic samples of the corresponding target device based on the fault acoustic generation model and the preset control conditions further includes: encoding the fault type conditions, fault level conditions, operating condition conditions, and device structure conditions respectively to obtain encoded condition vectors; the encoded condition vectors include fault type condition vectors, fault level condition vectors, operating condition vectors, and device structure condition vectors; concatenating the fault type condition vectors, fault level condition vectors, operating condition vectors, and device structure condition vectors to obtain a unified condition control vector; and using the fault acoustic generation model to perform acoustic sample generation processing based on the unified condition control vector, preset random noise, and the unified feature representation to obtain fault acoustic samples of the target device.

[0018] In one implementation of the first aspect, quality evaluation based on the fault acoustic sample and the real acoustic data of the multimodal data to obtain the fault acoustic enhancement sample of the target device includes: extracting features from the fault acoustic sample and the real acoustic data of the multimodal data respectively to obtain corresponding time-frequency features; the time-frequency features include Mel spectrum, STFT spectrum, and spectral entropy features; calculating the corresponding similarity based on each Mel spectrum, STFT spectrum, and spectral entropy feature; the similarity includes Mel spectrum distance, spectral structure similarity, and spectral entropy difference; and performing feature comparison based on the partial discharge pulse sequence of the fault acoustic sample. The system obtains the consistency of partial discharge pulse distribution; inputs the fault acoustic sample into a discrimination network for authenticity judgment and obtains an authenticity score; based on the Mel spectral distance, the spectral structure similarity, the spectral entropy difference, the consistency of partial discharge pulse distribution, and the authenticity score, a weighted fusion is performed according to preset weights to obtain a comprehensive quality score; if the comprehensive quality score is greater than or equal to a preset threshold, the current fault acoustic sample is determined to be a high-quality acoustic enhancement sample and is used as a fault acoustic enhancement sample for the target device; if the comprehensive quality score is less than the preset threshold, the current fault acoustic sample is determined to be a low-quality acoustic enhancement sample and is not used as a fault acoustic enhancement sample for the target device.

[0019] Secondly, this application provides an acoustic data enhancement system based on a generative large model. The system includes: a multimodal data acquisition module configured to acquire multimodal data of a target device; the multimodal data includes at least acoustic signals, partial discharge signals, vibration signals, and operating condition information; a preprocessing and feature fusion module configured to extract and fuse features based on the multimodal data to obtain a unified feature representation; an acoustic basic model training module configured to perform joint self-supervised training on an initial acoustic basic model based on the unified feature representation to obtain an acoustic basic model; a model construction module configured to construct a fault acoustic generative model with conditional input based on the acoustic basic model, an acoustic data generation network, preset random noise, and preset control conditions; the preset control conditions include at least the fault type condition, fault level condition, operating condition condition, and equipment structure condition of the target device; and a sample generation module. The module is configured to generate fault acoustic samples corresponding to the target device based on the fault acoustic generation model and the preset control conditions. Specifically, it includes: performing noise diffusion processing based on the unified feature representation using the fault acoustic generation model to obtain noise-added acoustic samples; encoding and concatenating the fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions to obtain a unified condition control vector; performing reverse denoising processing based on the unified condition control vector, the noise-added acoustic samples, and the unified feature representation using the fault acoustic generation model to generate corresponding fault acoustic sample features, and performing acoustic reconstruction on the fault acoustic sample features to generate fault acoustic samples corresponding to the target device; and a quality evaluation module is configured to perform quality evaluation based on the fault acoustic samples and the real acoustic data of the multimodal data to obtain fault acoustic enhancement samples of the target device.

[0020] Thirdly, this application provides an electronic device, which includes: a memory for storing a computer program; and a processor, which is communicatively connected to the memory and executes the above-described method for enhancing acoustic data of GIS equipment faults based on a generative large model when the computer program is invoked.

[0021] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for enhancing acoustic data of GIS equipment faults based on a generative large model.

[0022] As described above, the method and system for enhancing acoustic data of GIS equipment faults based on generative large models described in this application have the following beneficial effects:

[0023] This application proposes a method for enhancing acoustic data of GIS equipment faults based on generative large models. It can automatically generate high-fidelity and highly diverse typical fault acoustic samples, including simulated acoustics under multiple types, intensities, scenarios, and equipment structures such as partial discharge, mechanical noise, and gas leakage. The data volume can be expanded by hundreds of times, thereby providing sufficient training samples for intelligent diagnostic algorithms, improving the generalization ability and robustness of intelligent diagnostic models, significantly alleviating the problem of scarce GIS fault data, and solving the long-standing "data shortage" problem in existing technologies.

[0024] The model trained using generated data in this application not only solves the problem of poor model training effect caused by the scarcity of field data, but also simulates complex background noise in substations, acoustic drift caused by equipment aging, and changes in acoustic response caused by structural differences in different GIS models. This enables the synthetic data to improve its coverage of complex real-world working conditions and significantly improves the adaptability of the diagnostic model to multi-source scenarios. The model trained using generated data in this application can adapt to various noise environments and equipment differences in various sites, enabling the model to migrate between different regions, different substations, and different GIS models, and improving the model's cross-scenario migration capability.

[0025] The acoustic generation model proposed in this application can be aligned and jointly modeled with multimodal data such as partial discharge, vibration, and temperature rise, providing a data foundation for building a multimodal intelligent diagnostic system of "acoustics + partial discharge + vibration", thereby promoting the application of equipment condition assessment technology based on large models in the power industry. This application realizes the coupled modeling of "physical constraints + generative AI" in GIS acoustics. The proposed fault acoustic generation model can improve the authenticity and interpretability of generated samples, making the synthetic data usable for various engineering scenarios such as technology verification, model training, and test evaluation.

[0026] This application can use multimodal information such as acoustics, partial discharge, and vibration to constrain and evaluate generated acoustic samples (i.e., fault acoustic samples), providing high-quality training samples for building a multimodal diagnostic system; at the same time, it can generate "lightweight feature" data, providing richer samples for edge-side diagnostic models and improving the reliability of edge devices in complex scenarios. Attached Figure Description

[0027] Figure 1 The diagram shown is a schematic representation of the system architecture of the acoustic data augmentation method based on a generative large model as described in an embodiment of this application.

[0028] Figure 2 The diagram shown is a flowchart of the acoustic data enhancement method based on a generative large model as described in an embodiment of this application.

[0029] Figure 3This is a schematic diagram illustrating the complete process of the acoustic data enhancement method based on a generative large model as described in an embodiment of this application.

[0030] Figure 4 The diagram shown is a flowchart of the GIS multimodal raw data acquisition process described in this application embodiment.

[0031] Figure 5 The diagram shows a flowchart of multimodal data feature extraction and fusion as described in an embodiment of this application.

[0032] Figure 6 The diagram shown is a schematic diagram of the GIS acoustic generation model construction described in the embodiments of this application.

[0033] Figure 7 The diagram shown is a schematic representation of the process for generating target fault acoustic samples as described in an embodiment of this application.

[0034] Figure 8 The diagram shown is a schematic representation of the quality evaluation process described in an embodiment of this application.

[0035] Figure 9 The diagram shown is a schematic of the fault diagnosis model training process described in the embodiments of this application.

[0036] Figure 10 The diagram shown is a structural schematic of the acoustic data enhancement system based on a generative large model as described in an embodiment of this application.

[0037] Figure 11 The diagram shown is a structural schematic of the electronic device described in an embodiment of this application. Detailed Implementation

[0038] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0039] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0040] The following embodiments of this application provide a method and system for enhancing acoustic data of GIS equipment faults based on generative large models, which solves the problems of data scarcity and imbalanced samples in the field of GIS acoustic fault diagnosis.

[0041] like Figure 1 As shown in the diagram, this embodiment provides a system architecture diagram for a GIS equipment fault acoustic data enhancement method based on a generative large model, used to implement the GIS equipment fault acoustic data enhancement method based on a generative large model described in this application embodiment. Specifically, it includes: a data acquisition end, a model training end, and an application end.

[0042] The data acquisition terminal includes a multi-microphone array, a partial discharge sensor, a vibration sensor, an embedded data acquisition module, and an edge preprocessing and compression module. It is used to acquire multimodal data of the target device using the multi-microphone array, partial discharge sensor, vibration sensor, and / or embedded data acquisition module, and to preprocess the acquired multimodal data using the edge preprocessing and compression module to obtain a unified feature representation of the multimodal data. The multimodal data includes at least acoustic signals, partial discharge signals, vibration signals, and operating condition information.

[0043] The model training and generation module includes: an acoustic basic model training module, a fault acoustic generative model training module, a fault acoustic sample generation module, and a data quality evaluation module. The acoustic basic model training module is used to train the acoustic basic model using the unified feature representation of the multimodal data and the acoustic basic model training module. The fault acoustic generative model training module is used to train the generative model using the unified feature representation of the multimodal data, and generate fault acoustic samples corresponding to the target device according to the fault acoustic sample generation module. The data quality evaluation module evaluates the quality of the fault acoustic samples and real acoustic data to obtain high-quality acoustic enhancement samples, which are used as fault acoustic enhancement samples for the target device.

[0044] The application includes: an edge acoustic recognition model, a multimodal fault diagnosis model, and local alarm / platform linkage. It utilizes the edge acoustic recognition model for acoustic recognition, the multimodal fault diagnosis model for fault diagnosis, and issues an alarm using local alarms when a faulty acoustic condition is diagnosed. The multimodal fault diagnosis model is trained using enhanced acoustic diagnostic samples.

[0045] This application aims to address the problems of scarce acoustic data on the operating status of existing GIS (Gas Insulated Switchgear) equipment, difficulty in obtaining typical fault samples, uneven data distribution, difficulty in simulating multiple operating conditions, and the high dependence of acoustic characteristics on equipment structure and environmental noise. It proposes a generative modeling method for enhancing acoustic data of typical faults in gas-insulated switchgear (GIS), covering acoustic signal processing, large-scale model training, generative artificial intelligence, acoustic recognition, multimodal fusion, and fault diagnosis. This method automatically synthesizes acoustic data of typical faults such as partial discharge, mechanical noise, and gas leakage under multiple scenarios, levels, and operating conditions by constructing an acoustic generative model tailored to the characteristics of GIS equipment. It aims to solve the problems of scarce, unevenly distributed, and difficult-to-collect fault data in power field, and improve the robustness, generalization, and practical application capability of intelligent diagnostic models.

[0046] It should be noted that this application is an acoustic data enhancement method based on a generative large model designed for GIS (Gas Insulated Switchgear) equipment. Its core is data enhancement of acoustic signals, acoustic features and fault acoustic samples for typical faults of GIS equipment. However, this application is not limited to this, and the acoustic data enhancement method based on a generative large model described in this embodiment is applicable to any power equipment.

[0047] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0048] like Figure 2 As shown, this embodiment provides a method for enhancing acoustic data of GIS equipment faults based on generative large models. The method includes the following steps S201 to S206.

[0049] Step S201: Acquire multimodal data of the target device; the multimodal data includes at least acoustic signals, partial discharge signals, vibration signals, and operating condition information. The target device is a gas-insulated switchgear (GIS), but is not limited to GIS.

[0050] Figure 3 This diagram illustrates the complete workflow of the acoustic data enhancement method based on a generative large model as described in an embodiment of this application. Figure 3 As shown, in some embodiments, this application uses a multi-microphone array, a partial discharge sensor, a vibration sensor, etc. to synchronously collect data on the operating status of GIS equipment, obtain acoustic signals, partial discharge signals, vibration signals and operating condition information, and align the modal data based on a unified timestamp;

[0051] Step S202: Based on the multimodal data, perform feature extraction and fusion to obtain a unified feature representation.

[0052] In some embodiments, this application extracts time-frequency features such as STFT, Mel spectrum, and CQT by performing noise reduction, normalization, and time-frequency transformation on acoustic signals; performs pulse sequence detection and envelope analysis on partial discharge signals; performs empirical mode decomposition and envelope analysis on vibration signals; and constructs a unified multimodal feature set.

[0053] Step S203: Perform multimodal self-supervised training on the initial acoustic basic model based on the unified feature representation to obtain the acoustic basic model.

[0054] In some embodiments, this application utilizes multimodal data such as acoustics, partial discharge, and vibration collected on-site to construct an acoustic basic model, and extracts deep time-frequency structural features of GIS faults through large-scale self-supervised learning to achieve high-dimensional acoustic semantic space representation.

[0055] In some embodiments, this application performs mask prediction, contrastive learning, and cross-modal alignment learning based on multimodal inputs to obtain an acoustic fundamental model representation for GIS acoustic features.

[0056] Step S204: Construct a fault acoustic generation model with conditional input based on the aforementioned acoustic basic model, acoustic data generation network, preset random noise, and preset control conditions. The preset control conditions include at least the fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions of the target equipment.

[0057] In some embodiments, this application constructs an acoustic data generation network based on a diffusion model, a generative adversarial network, or a flow matching model, receives noise state input and control conditions such as fault type, fault level, operating parameters, and equipment model, and establishes a fault acoustic sample generation mechanism.

[0058] Step S205: Generate fault acoustic samples corresponding to the target device based on the fault acoustic generation model and the preset control conditions, specifically including: performing noise diffusion processing based on the unified feature representation using the fault acoustic generation model to obtain noise-added acoustic samples; encoding and concatenating the fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions respectively to obtain a unified condition control vector; performing reverse denoising processing based on the unified condition control vector, the noise-added acoustic samples, and the unified feature representation using the fault acoustic generation model to generate corresponding fault acoustic sample features, and performing acoustic reconstruction on the fault acoustic sample features to generate fault acoustic samples corresponding to the target device.

[0059] In some embodiments, this application utilizes the fault acoustic generation model to generate acoustic data corresponding to the GIS fault type and intensity based on the input control conditions.

[0060] Step S206: Perform quality evaluation based on the real acoustic data of the fault acoustic samples and the multimodal data to obtain fault acoustic enhancement samples of the target device.

[0061] In some embodiments, this application extracts the same features from real acoustic samples and generated acoustic samples respectively, and evaluates the quality of faulty acoustic samples based on spectral structure similarity, Mel spectral distance, spectral entropy difference, consistency of partial discharge pulse distribution, and discrimination network score.

[0062] In this embodiment, the application constructs a training set by mixing the fault acoustic enhancement samples selected through quality evaluation with real collected data, and uses the training set to train a GIS fault diagnosis model. The trained fault diagnosis model is then used to monitor the GIS equipment in real time, continuously monitor the operating status of the GIS equipment, and promptly identify potential faults, thereby improving the robustness and generalization ability of fault identification.

[0063] Figure 4 This is a schematic diagram illustrating the process of acquiring GIS multimodal raw data as described in an embodiment of this application. Figure 4 As shown, this application employs a multimodal sensor array for acoustic sensing of the GIS equipment's operating status, including: a multi-microphone array (4–16 channels): acquiring broadband acoustic data (2kHz–48kHz), covering partial discharge pulses and high-frequency noise from gas leaks; an ultrasonic sensor (≥40kHz): used to capture weak leakage sounds outside the GIS housing; a partial discharge UHF / HFCT sensor: acquiring corresponding discharge pulse sequences; a vibration (IEPE, MEMS) sensor: recording mechanical collision or jamming characteristics; and auxiliary signals (temperature, pressure, load current): used to construct operating condition tags. Sensor synchronization employs: GPS / PTP time synchronization; and multimodal synchronous sampling with a unified clock (≤1ms deviation). The acquired data forms a multimodal data alignment structure. .

[0064] in, This represents the multimodal data alignment structure (dataset), which is a collection of multimodal acquired data organized together after time synchronization.

[0065] It represents a sequence of acoustic (audio) signals, which are broadband acoustic signals collected by a multi-microphone array and include acoustic information such as partial discharge pulse sound, high-frequency noise from gas leaks, and mechanical noises.

[0066] This represents a partial discharge signal sequence, consisting of discharge pulse signals collected by a UHF / HFCT partial discharge sensor, reflecting the internal insulation status of the GIS equipment.

[0067] This represents a vibration signal sequence, which is a mechanical vibration signal collected by an IEPE or MEMS vibration sensor, reflecting the mechanical operating state of the equipment, such as collision, jamming, loosening, etc.

[0068] This represents operating condition information, i.e., auxiliary metadata of the equipment during operation, including but not limited to parameters such as equipment temperature, air chamber pressure, load current, and ambient humidity, which are used to build operating condition tags;

[0069] The subscript i represents the index identifier of the i-th sampling unit or the i-th time window, indicating that the multimodal data alignment structure is organized one by one according to the sampling time or sampling window;

[0070] In one embodiment of this application, feature extraction and fusion based on the multimodal data to obtain a unified feature representation includes the following steps S2021 to S2023.

[0071] Step S2021: Perform preprocessing based on the multimodal data to obtain preprocessed multimodal data.

[0072] Step S2022: Based on the preprocessed multimodal data, feature extraction is performed to obtain multimodal features; the multimodal features include at least acoustic features, partial discharge features, and vibration features.

[0073] Step S2023: Based on the acoustic features, the partial discharge features, and the vibration features, feature fusion is performed using a timestamp-based multimodal alignment function to obtain a unified feature representation.

[0074] In one embodiment of this application, preprocessing based on the multimodal data to obtain preprocessed multimodal data includes: performing noise reduction, normalization, and time-frequency transformation processing on the acoustic signal to obtain a preprocessed acoustic signal; performing pulse sequence detection and envelope analysis on the partial discharge signal to obtain a preprocessed partial discharge signal; performing empirical mode decomposition and envelope analysis on the vibration signal to obtain a preprocessed vibration signal.

[0075] Figure 5 This diagram illustrates the process of multimodal data feature extraction and fusion as described in an embodiment of this application. Figure 5 As shown, this application performs the following processing on the acquired multimodal signals: acoustic preprocessing, partial discharge pulse preprocessing, vibration signal preprocessing, and multimodal alignment.

[0076] The acoustic preprocessing includes windowed STFT; spectral normalization; spectral subtraction, Wiener filtering, neural network noise reduction; and extraction of acoustic features such as MFCC, Log-Mel, CQT, and time-frequency plots.

[0077] Partial discharge pulse preprocessing includes TEV / UHF pulse detection; peak detection and envelope extraction; and statistical structure of the pulse sequence (pulse frequency, amplitude distribution). After completing TEV / UHF pulse detection, peak detection, and envelope extraction, the temporal characteristics, statistical characteristics, and temporal structure characteristics of the partial discharge signal are further extracted.

[0078] The time-domain features include pulse peak value, pulse width, rise time, fall time, pulse energy, and envelope peak value; the statistical features include the number of pulses per unit time, pulse amplitude distribution, pulse interval distribution, mean, variance, skewness, and kurtosis; the temporal structure features include pulse repetition frequency, pulse cluster distribution characteristics, and the sequence arrangement characteristics of discharge pulses within the observation time window.

[0079] Furthermore, this application can construct the extracted partial discharge features into a fixed-dimensional vector or pulse sequence embedding representation for subsequent alignment and joint modeling with acoustic and vibration features.

[0080] Vibration signal preprocessing includes empirical mode decomposition (EMD), vibration envelope demodulation, and harmonic amplitude spectrum analysis. After completing EMD, vibration envelope demodulation, and harmonic amplitude spectrum analysis, the time-domain, frequency-domain, and time-frequency characteristics of the vibration signal are further extracted.

[0081] The time-domain features include root mean square value, peak value, peak-to-peak value, kurtosis factor, impulse factor, and margin factor; the frequency-domain features include dominant frequency amplitude, harmonic component amplitude, frequency band energy distribution, and spectral centroid; the time-frequency features include the energy proportion of each intrinsic mode function, envelope spectrum features, and short-time energy variation characteristics over time.

[0082] Furthermore, this application maps the aforementioned vibration characteristics into a vibration feature vector of a unified dimension to characterize operating states such as mechanical collisions, mechanism jamming, loosening, and periodic anomalies, for subsequent multimodal joint training.

[0083] Multimodal alignment adopts a timestamp-based alignment rule, as shown in formula (1):

[0084] Formula (1)

[0085] in: : Represents a multimodal data alignment function, used to synchronize and fuse signals from different sources based on timestamps; : Represents an acoustic (audio) signal sequence; : Represents a partial discharge signal sequence; : Represents a vibration signal sequence; : Represents the k-th aligned timestamp, which is a discrete time point obtained through a time synchronization mechanism (such as a unified clock, time window division, or time resampling); : Indicates the timestamp The multimodal feature vector obtained by down-fusion is usually composed of the feature combination of each modality at that time. This represents the aligned multimodal data set, which is a sequence composed of multiple timestamps and their corresponding fused features.

[0086] It should be noted that the multimodal data alignment structure D and the multimodal data alignment function Align() are not the same concept. D is the original dataset format organized in time synchronization after data acquisition; it is a static description. Align(), on the other hand, is a calculation function that performs time alignment and feature fusion on multimodal signals; it is a dynamic process. D is the input source of Align(), and the output of Align() is the aligned fused feature set { Therefore, the “multimodal data alignment structure” can be understood as a textual representation of the output result of formula (1).

[0087] Finally, the input feature set is constructed as shown in formula (2):

[0088] Formula (2)

[0089] in, : Represents the fused multimodal feature set, used to characterize the comprehensive information under the same timestamp or the same sampling unit, and as input for subsequent model training or inference; : Represents feature information extracted from acoustic (audio) signals, including but not limited to Mel spectrum features, time-frequency energy distribution features, or acoustic embedding features; : Represents the feature information extracted from the partial discharge signal, wherein the features include pulse amplitude features, pulse count features, or statistical distribution features; : Represents the feature information extracted from the vibration signal, including time-domain statistical features, frequency-domain spectral features, or vibration energy features; : Represents metadata features related to the operating status of the equipment. The metadata includes, but is not limited to, equipment type, operating condition parameters (such as load, voltage, and current), environmental parameters (such as temperature and humidity), and time information.

[0090] In one embodiment of this application, the multimodal self-supervised training includes a mask learning task, a contrastive learning task, and a multimodal alignment learning task.

[0091] In one embodiment of this application, the initial acoustic basic model is subjected to multimodal self-supervised training based on the unified feature representation to obtain the acoustic basic model, which includes the following steps S2031 to S2034.

[0092] Step S2031: Based on the unified feature representation input, perform mask learning on the initial acoustic basic model, and perform content prediction and reconstruction on the masked area of ​​the unified feature representation in order to learn the internal acoustic structure.

[0093] Step S2032: Based on the unified feature representation input to the initial acoustic basic model, perform comparative learning. For the unified feature representation, construct positive sample pairs between different modalities at the same time, and negative sample pairs between different augmented views of the same modality, to learn the representation ability of cross-modal correlation and fault semantic consistency.

[0094] Step S2033: Based on the unified feature representation input to the initial acoustic basic model, perform multimodal alignment learning. By constraining the feature representations of the same semantic content in different modalities to align in the shared semantic space, the shared high-level semantic information across modalities is learned.

[0095] Step S2034: Perform joint optimization based on the Mask learning task, the contrast learning task, and the multimodal alignment learning task, and update the initial acoustic basic model using their respective loss functions to obtain the acoustic basic model.

[0096] In some embodiments, this application constructs an acoustic basic model, enabling the model to automatically learn the time-frequency structure of typical GIS faults. The Mask prediction, contrastive learning, and cross-modal alignment learning in step S203 of the multimodal self-supervised training are derived from step S202 (…). Figure 5 The training mechanism further executed based on the unified multimodal feature representation output does not belong to... Figure 5 The original preprocessing module in the original module belongs to the subsequent acoustic basic model training module.

[0097] Specifically, Mask prediction acts on Figure 5 The output acoustic time-frequency features or unified feature representations are used to learn the acoustic internal structure by reconstructing the occluded region; contrastive learning is applied to feature representations between different modalities at the same time and between different augmented views of the same modality to enhance the model's ability to represent cross-modal correlations and fault semantic consistency. In other words, Figure 5 The training input is provided, and the Mask prediction and contrastive learning in step S203 are responsible for self-supervised training on the input.

[0098] Self-supervised spectral mask prediction is employed: the model is input with a masked time-frequency plot, and the acoustic structure is learned through reconstruction; contrastive learning: multimodal data from the same time point are used as positive samples to improve the model's understanding of cross-modal correlations; multimodal alignment loss is used. Thus, the basic acoustic model is obtained (as in formula (3)):

[0099] in, This represents the multimodal alignment loss function, one of the loss functions used in self-supervised training. The purpose of this loss function is to constrain the feature representations of different modalities to be aligned as much as possible in a shared semantic space, minimizing the representation distance of the same semantic content across different modalities, and thus constraining the consistency of different modalities within a unified feature space.

[0100] This represents the feature embedding representation of the acoustic signal extracted by the acoustic encoder, that is, the high-dimensional feature vector of the acoustic signal in the shared semantic space after being mapped by the encoder;

[0101] This represents the feature embedding representation of the partial discharge signal extracted by the corresponding encoder, i.e., the high-dimensional feature vector of the partial discharge signal in the shared semantic space after being mapped by the encoder;

[0102] This represents the feature embedding representation of the vibration signal extracted by the corresponding encoder, i.e., the high-dimensional feature vector of the vibration signal in the shared semantic space after being mapped by the encoder;

[0103] : Represents the L2 norm, used to measure the Euclidean distance between different modal feature representations;

[0104] The loss function Using acoustic modes as anchors, the Euclidean distances between acoustic embeddings and partial discharge embeddings, and between acoustic embeddings and vibration embeddings, are calculated and summed. Minimizing this loss function promotes the convergence of feature representations of the three modes in the shared semantic space, thereby learning shared high-level semantic information across modes and enhancing the model's ability to understand multimodal relationships.

[0105] Formula (3)

[0106] in, This represents the output representation of the acoustic foundation model, specifically the high-level semantic embedding representation generated by the model from input multimodal features after multimodal self-supervised pre-training. This representation encodes the deep time-frequency structural features of GIS equipment faults and can be used for downstream tasks such as acoustic generation and fault diagnosis.

[0107] Represents the basic acoustic model, with parameters as follows: The deep neural network is a model obtained by jointly optimizing it through three self-supervised tasks: Mask learning, contrastive learning, and multimodal alignment learning. These are all the trainable parameters of the model;

[0108] This represents the unified multimodal feature set, i.e., the set defined by formula (2). It is a fusion representation of acoustic features, partial discharge features, vibration features and operating condition metadata features, and serves as the input to the basic acoustic model;

[0109] Formula (3) expresses the core mapping relationship of the acoustic basic model, which integrates the fused multimodal feature set. As input, after passing through parameters After deep network processing, it outputs high-level semantic embedding representations. This representation preserves the key features of each modality and establishes the correlation between different modalities at the same runtime, providing high-quality feature initialization and semantic constraints for subsequent acoustic generation models.

[0110] The formula (3) corresponds to Figure 5 The unified feature representation construction process involves, specifically, the acoustic signal, partial discharge signal, and vibration signal, after their respective preprocessing and feature extraction, being time-aligned according to a unified timestamp and mapped to the same feature space through their respective encoders. Finally, these are concatenated or fused to form a unified multimodal feature representation. This unified feature representation retains both the key features of each modality and the correlation between different modalities at the same runtime, and is used for subsequent self-supervised pre-training of the acoustic base model and multimodal joint modeling.

[0111] In one embodiment of this application, the fault acoustic generation model includes an input encoding module, a conditional embedding module, a noise diffusion module, a reverse denoising generation module, and an output reconstruction module, wherein,

[0112] The input encoding module is used to convert real-collected fault acoustic samples into a unified feature representation suitable for model processing.

[0113] The condition embedding module is used to encode and concatenate preset control conditions to form a unified condition control vector.

[0114] The noise diffusion module is used to progressively add preset random noise to the unified feature representation to form a noise-added sample that is transformed from a clear sample to random noise.

[0115] The reverse denoising generation module is used to perform reverse denoising processing on the noise-added acoustic sample under the control of preset control conditions to generate corresponding fault acoustic sample features.

[0116] The output reconstruction module is used to perform acoustic reconstruction on the fault acoustic sample features generated by the reverse denoising generation module, generate fault acoustic samples, and output them.

[0117] Figure 6 The diagram shown illustrates the construction of a GIS acoustic generation model as described in an embodiment of this application. Figure 6 As shown, the acoustic generation model of this application is used to learn the distribution characteristics of typical fault acoustics of GIS equipment under small sample conditions, and to generate synthetic acoustic samples with target fault characteristics under given control conditions. The generation model can be implemented using one or more of the following methods: diffusion model, conditional generative adversarial network, flow-based model, or acoustic Transformer-Decoder model. Preferably, a conditional diffusion model is used as the core generation framework to improve the modeling ability of high-frequency pulses, transient impacts, background noise coupling, and non-stationary time-frequency textures in GIS fault acoustics.

[0118] The fault acoustic generation model mainly includes an input encoding module, a conditional embedding module, a noise diffusion module, a reverse denoising generation module, and an output reconstruction module.

[0119] The input encoding module is used to convert the real-collected acoustic fault samples into a representation suitable for model processing. This representation can be the original waveform, a short-time Fourier transform spectrum, a Mel spectrum, a logarithmic power spectrum, or latent embedded features extracted from the acoustic fundamental model.

[0120] In one implementation, this application employs a combination of Mel spectrograms and latent embedding representations to simultaneously preserve the local time-frequency texture features and global semantic representation of fault acoustics.

[0121] The condition embedding module is used to encode control condition information to form conditional control vectors. These control conditions include fault type, fault level, equipment model, operating conditions, and environmental noise level. Discrete conditions are mapped to low-dimensional vectors using one-hot encoding or embedding lookup tables, while continuous conditions are normalized and mapped to continuous condition vectors via a multilayer perceptron. Finally, the condition vectors of different types are concatenated or weighted and fused to form a unified condition embedding representation c.

[0122] The noise diffusion module is used to gradually add random noise to real fault acoustic samples during the training phase in order to construct a forward diffusion process from clear samples to random noise. Its expression can be expressed as the following formula (4):

[0123] Formula (4)

[0124] in, This represents the original, real-world acoustic sample of a fault. This represents the intermediate state after adding noise at step t. Represents the noise scheduling coefficient at step t. This represents Gaussian random noise. By gradually increasing the noise level, the model learns the distribution and evolution of fault acoustics under different noise intensities.

[0125] The inverse denoising generation module is used to gradually recover fault acoustic samples that meet the target conditions from random noise or high noise states under given control conditions c. The inverse denoising process can be expressed as the following formula (5):

[0126] Formula (5)

[0127] in, () represents a denoising network with parameter θ, used to denoise the current noise state. The control condition c predicts the state of the previous sample that is closer to the actual fault distribution.

[0128] The denoising network can be implemented using a one-dimensional convolutional network, a two-dimensional U-Net, a time-frequency generation network with cross-attention structure, or a Transformer decoding network.

[0129] In one implementation, this application employs a U-Net backbone combined with residual blocks and a cross-attention mechanism, enabling the model to simultaneously focus on local impulse details and conditional semantic constraints during the denoising process.

[0130] The output reconstruction module is used to recover the target acoustic output from the generated features obtained in the inverse denoising process. If the model is generated in the spectral domain, the generated spectrum is recovered into a time-domain audio waveform through spectral inversion or a neural vocoder; if the model is generated in the latent space, the acoustic signal is reconstructed through the corresponding decoder. The final generated sample is denoted as... This is used for subsequent quality evaluation and fault diagnosis model training.

[0131] To improve the model's adaptability to the acoustics of typical GIS faults, this application can introduce multiple joint loss functions during model training, including noise prediction loss, spectrum reconstruction loss, impulse structure preservation loss, and conditional consistency loss.

[0132] The noise prediction loss is used to constrain the model to accurately estimate the random noise added during the diffusion process; the spectrum reconstruction loss is used to constrain the generated samples to be consistent with the real samples in terms of time-frequency texture; the pulse structure preservation loss is used to enhance the reconstruction capability of partial discharge pulses, high-frequency spikes and mechanical shock envelope features; the condition consistency loss is used to ensure that the generated samples and the input control conditions match in terms of fault category, fault intensity and operating condition attributes.

[0133] Furthermore, to improve the model's generalization ability under limited sample conditions, this application may employ transfer learning, parameter fine-tuning, low-rank adaptation, or initialization based on the acoustic fundamental model to train the model, enabling the generated model to learn the effective distribution characteristics of GIS fault acoustics even with fewer real fault samples.

[0134] In this application, the fault acoustic sample refers to the feature representation of GIS equipment under different operating or fault states, formed by time-frequency analysis, feature extraction or model encoding of acoustic signals, which can characterize the differences in equipment state.

[0135] In one embodiment of this application, the preset control conditions include at least the fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions of the target equipment.

[0136] In one embodiment of this application, generating fault acoustic samples of the corresponding target device based on the fault acoustic generation model and the preset control conditions includes the following steps S2051 to S2053.

[0137] Step S2051: Encode the fault type condition, fault level condition, operating condition condition, and equipment structure condition respectively to obtain the encoded condition vector; the encoded condition vector includes the fault type condition vector, fault level condition vector, operating condition condition vector, and equipment structure condition vector.

[0138] Step S2052: Based on the fault type condition vector, the fault level condition vector, the operating condition vector, and the equipment structure condition vector, a unified condition control vector is obtained by concatenating them.

[0139] Step S2053: Based on the unified conditional control vector, preset random noise and the unified feature representation, use the fault acoustic generation model to perform acoustic sample generation processing to obtain fault acoustic samples of the target device.

[0140] In one embodiment of this application, the method further includes the following steps S20531 to S20532.

[0141] Step S20531: Based on the unified feature representation, noise diffusion processing is performed using the fault acoustic generation model to obtain noise-added acoustic samples.

[0142] Step S20532: Based on the unified conditional control vector, the noise-added acoustic samples, and the unified feature representation, the fault acoustic generation model is used to perform acoustic sample generation processing to obtain fault acoustic samples of the target device.

[0143] Figure 7 The diagram shows a flowchart illustrating the generation of fault acoustic samples as described in an embodiment of this application. Figure 7 As shown, this application supports multiple controllable generation methods for fault acoustic samples. Controllable generation refers to introducing explicit conditional control vectors during the acoustic sample generation process, so that the fault acoustic samples output by the model not only have time-frequency characteristics similar to real samples, but also meet preset target requirements in terms of fault type, fault intensity, operating condition characteristics, and equipment attributes, thereby achieving data augmentation for specific diagnostic tasks.

[0144] The control condition vector c can be composed of the following information: fault type control, fault level control, operating condition control, and equipment structure control.

[0145] Fault type control: Fault type control is used to specify the fault category to which the generated sample belongs. The fault categories include, but are not limited to, partial discharge faults, mechanical abnormality faults, and gas leakage faults, which can be expressed as the following formula (6):

[0146] Formula (6)

[0147] in, The fault type control condition vector is used to specify the target fault category that the fault acoustic generation model needs to generate.

[0148] This indicates a partial discharge fault, specifically a partial discharge phenomenon occurring inside the GIS equipment. When... Set as When generating acoustic samples, the model should highlight the time-frequency characteristics of pulse-type, high-frequency spike-type, or discrete burst-type.

[0149] This indicates a gas leak fault, specifically a minor leak of SF6 gas from the GIS equipment or a slight seepage from a valve. Set as When generating acoustic samples, the model should highlight continuous broadband airflow noise and stable background venting texture features;

[0150] This indicates a mechanical malfunction, specifically a mechanical fault in the GIS equipment such as poor engagement of the tap changer, loose mechanism, or mechanical jamming. Set as When generating acoustic samples, the model should highlight the characteristics of periodic impact, friction modulation, or structural vibration coupling.

[0151] so, express The value range is one of the three fault categories mentioned above or a combination thereof, but this application is not limited to these three categories and can be extended according to actual needs.

[0152] Formula (6) defines the basic value space of the fault type control condition. Through this control condition, the fault acoustic generation model can be specified to conform to the target fault mechanism in terms of macroscopic acoustic morphology, thereby achieving targeted data augmentation for specific fault types.

[0153] Specifically, when the control condition is a partial discharge fault, the acoustic samples generated by the model should highlight pulse-type, high-frequency spike-type, or discrete burst-type time-frequency characteristics; when the control condition is a mechanical abnormality fault, the acoustic samples generated by the model should highlight periodic impact, friction modulation, or structural vibration coupling characteristics; when the control condition is a leakage fault, the acoustic samples generated by the model should highlight continuous broadband airflow noise and stable background discharge texture characteristics. This control method ensures that the generated samples conform to the target fault mechanism in terms of macroscopic acoustic morphology.

[0154] Fault level control: Fault level control is used to characterize the severity of a fault or the intensity of an anomaly. It can be described using continuous variables, as shown in the following formula (7):

[0155] Formula (7)

[0156] in, The closer to 0, the milder the fault, the weaker the corresponding abnormal component in the generated acoustics, the lower the pulse energy, and the relatively higher the proportion of background noise. The closer to 1, the more severe the fault, the more obvious the corresponding abnormal features in the generated acoustics, the larger the pulse amplitude, and the more prominent the spectral energy concentration.

[0157] During training, the model learns a continuous mapping relationship between "fault intensity - acoustic performance", which enables it to generate transitional acoustic samples at different severity levels, rather than being limited to discrete category switching.

[0158] Operating condition control: Operating condition control is used to reflect the acoustic variation law of equipment under different operating environments and different load conditions, which can be expressed as the following formula (8):

[0159] Formula (8)

[0160] in, Indicates the background ambient noise level. Indicates the ambient or equipment temperature. This indicates the pressure or related pressure status of the GIS air chamber. Indicates ambient humidity. This indicates the operating load or condition level. This type of condition is used to adjust the background noise level, energy distribution, pulse propagation characteristics, and frequency band response differences in the generated acoustics, enabling the model to generate fault samples that match the target operating condition and avoid confusion in sample distribution under different operating conditions.

[0161] Equipment structure control: Equipment structure control is used to characterize the impact of different GIS equipment models, structural layouts and sensor installation differences on acoustic performance, and can be expressed as the following formula (9):

[0162] Formula (9)

[0163] in, This indicates the equipment model, structural category, or installation location. Different models of GIS equipment differ in their shell structure, cavity dimensions, material properties, and sensor coupling paths, which can lead to variations in acoustic propagation, resonant frequency bands, and envelope morphology for similar faults. By introducing equipment structural conditions, the model can generate fault acoustic samples that match specific equipment objects.

[0164] Construction method of control condition vector: In specific implementation, this application can encode the fault type condition, fault level condition, operating condition and equipment structure condition respectively and then concatenate them to form a unified control vector c, as shown in the following formula (10):

[0165] Formula (10)

[0166] Discrete conditions can be mapped to dense vectors through embedding layers, while continuous conditions can be transformed into continuous embedding representations after normalization and linear mapping. Finally, a unified control representation is obtained through vector concatenation, linear transformation, or attention fusion. This control vector is injected into multiple layers of the generative model, including input layer injection, intermediate feature layer conditional modulation, and cross-attention constraints at the decoding end, to ensure that the conditional information continues to play a role throughout the generation process.

[0167] Conditional injection and generation mechanism: During the generation process, the noise vector is... Or an intermediate state in the diffusion process The control vector c is input into the generator network, ensuring that the model is continuously constrained by control conditions during inverse denoising or conditional decoding. The generation process can be expressed as the following formula (11):

[0168] Formula (11)

[0169] in, This represents a conditional generation network. This represents random noise or initial latent variables. Represents the control vector. This represents the final generated target fault acoustic sample. (By...) By injecting a generative network, the model can achieve a balance between randomness and controllability, that is, while maintaining sample diversity, the generated results can satisfy the predetermined fault attributes.

[0170] Training constraints for controllable generation: To ensure consistency between the generated results and the input conditions, this application further introduces a condition consistency discrimination mechanism during the model training phase. Specifically, an auxiliary classifier, attribute predictor, or condition discriminator can be used to reverse-determine the fault category, intensity level, and operating condition attributes of the generated samples, and the determination result is compared with the input control conditions to form a condition consistency loss. This constraint can suppress the problem of "correct input conditions but generated results deviating from the target attributes," thereby improving the accuracy of controllable generation.

[0171] The role of controllable generation: Through the aforementioned control input and constraint mechanisms, this application can achieve controllable acoustic generation under different constraints. Specifically, this includes generating samples for specified fault categories, generating gradual samples with different fault intensities, generating migration samples adapted to different operating conditions, and generating customized samples adapted to specific equipment structures. This not only expands the training sample size but also covers boundary scenarios, weak fault scenarios, and rare model scenarios that are difficult to fully acquire in real-world data collection, thereby improving the robustness and generalization ability of subsequent fault diagnosis models in complex field environments.

[0172] This application can generate diverse acoustic samples under different fault types, fault intensities, and operating conditions through the aforementioned condition control mechanism, thereby expanding the distribution range of training data and improving the generalization ability of the fault diagnosis model in complex environments.

[0173] In one embodiment of this application, the quality evaluation based on the fault acoustic sample and the real acoustic data of the multimodal data to obtain the fault acoustic enhancement sample of the target device includes the following steps S2061 to S2067.

[0174] Step S2061: Based on the fault acoustic sample and the real acoustic data of the multimodal data, feature extraction is performed to obtain the corresponding time-frequency features; the time-frequency features include Mel spectrum, STFT spectrum and spectral entropy features.

[0175] Step S2062: Calculate the corresponding similarity based on the Mel spectrum, STFT spectrum and spectral entropy features of each of them; the similarity includes Mel spectrum distance, spectral structure similarity and spectral entropy difference.

[0176] Step S2063: Perform feature comparison on the partial discharge pulse sequence of the fault acoustic sample to obtain the consistency of the partial discharge pulse distribution.

[0177] Step S2064: Input the fault acoustic sample into the discrimination network for authenticity judgment and obtain an authenticity score.

[0178] Step S2065: Based on the Mel spectral distance, the spectral structure similarity, the spectral entropy difference, the partial discharge pulse distribution consistency, and the authenticity score, a weighted fusion is performed according to a preset weight to obtain a comprehensive quality score.

[0179] Step S2066: If the comprehensive quality score is greater than or equal to a preset threshold, the current fault acoustic sample is determined to be a high-quality acoustic enhancement sample and used as the fault acoustic enhancement sample of the target device.

[0180] Step S2067: If the overall quality score is less than a preset threshold, the current fault acoustic sample is determined to be a low-quality acoustic enhancement sample and is not used as a fault acoustic enhancement sample for the target device.

[0181] Figure 8 This is a schematic diagram of the quality evaluation process described in an embodiment of this application. Figure 8 As shown, the quality evaluation process includes the following steps:

[0182] First, identical time-frequency features, including Mel spectrum, STFT spectrum, and spectral entropy features, are extracted from the faulty acoustic enhancement samples and their corresponding real acoustic samples. Then, the Mel spectral distance, spectral structure similarity (STFT-SSIM), and spectral entropy difference are calculated. A smaller Mel spectral distance, a larger STFT-SSIM, and a smaller spectral entropy difference indicate that the faulty acoustic enhancement sample is closer to the real sample in terms of time-frequency structure.

[0183] Secondly, statistical feature comparison is performed on the partial discharge pulse sequence corresponding to the fault acoustic enhancement sample to evaluate the consistency between the pulse number distribution, amplitude distribution, pulse interval distribution, and repetition frequency and the real sample, so as to determine whether the fault acoustic enhancement sample maintains the fault mechanism characteristics that match the partial discharge process. The partial discharge pulse alignment evaluation formula is as follows: Formula (12):

[0184] Formula (12)

[0185] in, This represents a measure of the consistency of partial discharge pulse distribution, used to evaluate the degree of statistical distribution difference between the partial discharge pulse sequence in the faulty acoustic enhancement sample and the real sample. The smaller the value, the more consistent the fault acoustic enhancement sample is with the real sample in terms of partial discharge pulse distribution characteristics, and the higher the generation quality.

[0186] It represents the distribution of statistical characteristics of partial discharge pulses corresponding to real acoustic samples, that is, the statistical characteristics of pulse sequences extracted from real partial discharge signals, including feature vectors or distribution histograms composed of statistical quantities such as pulse number distribution, pulse amplitude distribution, pulse interval distribution and pulse repetition frequency.

[0187] This represents the distribution of partial discharge pulse statistical characteristics corresponding to the fault acoustic enhancement samples, i.e., the statistical characteristics of the partial discharge pulse sequence extracted from the synthetic acoustic data output by the fault acoustic generation model. The extraction method is the same as... Consistent;

[0188] This formula (12) evaluates whether the fault acoustic enhancement sample maintains the fault mechanism characteristics that match the real partial discharge process by comparing the statistical feature distribution differences between the fault acoustic enhancement sample and the real sample at the partial discharge pulse sequence level. This index is one of the components of the comprehensive quality score Q.

[0189] Next, the faulty acoustic enhancement samples are input into the discriminant network to obtain a realism score. The score is used to characterize how closely the faulty acoustic enhancement sample approximates the real sample in terms of overall distribution. The evaluation output score of the discriminant network is as follows (13):

[0190] Formula (13)

[0191] in, The score represents the authenticity score, which is the output value of the discriminant network's assessment of the authenticity of the input sample. This score is usually a continuous value in the range [0,1]. The closer the score is to 1, the closer the discriminant network considers the sample to be close to the real acoustic data distribution; the closer the score is to 0, the greater the difference between the sample and the real sample distribution.

[0192] This refers to a discriminant network, which is a binary classification or regression neural network trained on real acoustic samples. This network is used to evaluate the authenticity of faulty acoustic enhancement samples from an overall distribution perspective, determining whether the faulty acoustic enhancement samples closely resemble the statistical characteristics of real samples in terms of spectral structure, time-frequency texture, energy distribution, etc.

[0193] This represents the enhanced fault acoustic sample, which is the synthetic fault acoustic data output by the fault acoustic generation model under given control conditions. This sample serves as the input to the discriminant network and undergoes a realism evaluation.

[0194] Formula (13) expresses the quantitative realism score obtained by inputting faulty acoustic enhancement samples into the discriminant network. The score is used to measure how closely the faulty acoustic enhancement samples resemble real samples in the overall data distribution, and is one of the components of the comprehensive quality score Q.

[0195] Furthermore, for the three modes of acoustics, partial discharge, and vibration, their distribution differences in a unified feature space are calculated to evaluate multimodal consistency and ensure that the generated acoustics not only closely approximates the real sample in terms of individual acoustic features, but also maintains reasonable consistency in cross-modal coupling relationships.

[0196] Finally, the above evaluation results are weighted and integrated according to the preset weights to obtain the comprehensive quality score Q. The calculation method of the comprehensive quality score Q is as follows (14):

[0197] Formula (14)

[0198] in:

[0199] The overall quality score is used to characterize the overall quality level of faulty acoustic enhancement samples and serves as the basis for sample selection and data augmentation decisions.

[0200] This represents an acoustic similarity evaluation metric used to measure the consistency of generated acoustics (faulty acoustic enhancement samples) with real acoustics in terms of time-frequency structure.

[0201] This represents the consistency evaluation index of partial discharge characteristics, used to measure the degree of consistency between the fault acoustic enhancement sample and the real sample in terms of partial discharge pulse distribution, amplitude characteristics and temporal structure.

[0202] This represents the output score of the discrimination network, used to measure how closely the faulty acoustic enhancement sample resembles the real sample in the overall data distribution;

[0203] , , These are respectively represented as the weight coefficients corresponding to each evaluation indicator, used to characterize the importance of different evaluation indicators in the overall quality score, satisfying: .

[0204] Furthermore, the weighting coefficients , , The weighting coefficients can be set or adjusted according to actual application needs. In one implementation, the weighting coefficients can be adaptively adjusted according to different fault types, data distribution characteristics, or evaluation objectives, or optimized and determined through the model training process, so as to achieve the allocation of importance to different evaluation dimensions.

[0205] In this embodiment, when the comprehensive quality score When the score exceeds a preset threshold, the faulty acoustic enhancement sample is determined to be a high-quality, usable sample and included in the training dataset; when the overall quality score... If the sample is below the threshold, the acoustic enhancement sample for the fault will be removed and will not be used for subsequent fault diagnosis model training.

[0206] Figure 9 This is a schematic diagram illustrating the fault diagnosis model training process described in an embodiment of this application. Figure 9 As shown, this application combines generated data with real data to train: an acoustic classification CNN, a Transformer acoustic diagnostic model, a multimodal fault recognition model, and a lightweight edge-side model. Hard example mining, phased training, and a hybrid self-supervised and supervised training method are used to improve performance. The final fault diagnosis model is generated as shown in formula (15):

[0207] Formula (15)

[0208] in, This refers to the fault diagnosis model, specifically the final trained fault diagnosis model for GIS equipment. This model can be an acoustic classification CNN, a Transformer acoustic diagnosis model, a multimodal fault recognition model, or a lightweight edge-side model, etc. After training, this model can perform real-time acoustic monitoring and identification of faults in GIS equipment.

[0209] This represents the model training function / process, indicating the process of supervising or a hybrid of self-supervised and supervised training on a fault diagnosis model using a given training dataset. Training strategies may include hard sample mining, phased training, and a hybrid of self-supervised and supervised training, etc.

[0210] This represents a real acoustic dataset, which is multimodal acoustic data actually collected from GIS equipment on-site, including real fault acoustic samples and corresponding labels;

[0211] The fault acoustic enhancement sample set refers to the set of fault acoustic enhancement samples obtained by controllable generation through the fault acoustic generation model described in this application and after being screened by quality evaluation.

[0212] This refers to a mixed training set, which is a training dataset constructed by merging real-world collected data with generated fault acoustic enhancement samples.

[0213] Formula (15) expresses the training paradigm of the fault diagnosis model. It constructs a training set by mixing fault acoustic enhancement sample data selected through quality evaluation with real collected data, and uses this mixed training set to train the GIS fault diagnosis model. This approach can effectively expand the scale of training samples and enrich sample diversity, thereby improving the robustness and generalization ability of the fault diagnosis model.

[0214] This application proposes a method for enhancing acoustic data of GIS equipment faults based on generative large models. It can automatically generate high-fidelity and highly diverse typical fault acoustic samples, including simulated acoustics under multiple types, intensities, scenarios, and equipment structures such as partial discharge, mechanical noise, and gas leakage. The data volume can be expanded by hundreds of times, thereby providing sufficient training samples for intelligent diagnostic algorithms, improving the generalization ability and robustness of intelligent diagnostic models, significantly alleviating the problem of scarce GIS fault data, and solving the long-standing "data shortage" problem in existing technologies.

[0215] The model trained using generated data in this application not only solves the problem of poor model training effect caused by the scarcity of field data, but also simulates complex background noise in substations, acoustic drift caused by equipment aging, and changes in acoustic response caused by structural differences in different GIS models. This enables the synthetic data to improve its coverage of complex real-world working conditions and significantly improves the adaptability of the diagnostic model to multi-source scenarios. The model trained using generated data in this application can adapt to various noise environments and equipment differences in various sites, enabling the model to migrate between different regions, different substations, and different GIS models, and improving the model's cross-scenario migration capability.

[0216] The acoustic generation model proposed in this application can be aligned and jointly modeled with multimodal data such as partial discharge, vibration, and temperature rise, providing a data foundation for building a multimodal intelligent diagnostic system of "acoustics + partial discharge + vibration", thereby promoting the application of equipment condition assessment technology based on large models in the power industry. This application realizes the coupled modeling of "physical constraints + generative AI" in GIS acoustics. The proposed acoustic generation model can improve the authenticity and interpretability of generated samples, making the synthetic data usable for various engineering scenarios such as technology verification, model training, and test evaluation.

[0217] This application can use multimodal information such as acoustics, partial discharge, and vibration to constrain and evaluate generated acoustic samples (i.e., fault acoustic samples), providing high-quality training samples for building a multimodal diagnostic system; at the same time, it can generate "lightweight feature" data, providing richer samples for edge-side diagnostic models and improving the reliability of edge devices in complex scenarios.

[0218] The scope of protection for the method for enhancing acoustic data of GIS equipment faults based on generative large models described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0219] This application also provides a GIS equipment fault acoustic data enhancement system based on a generative large model. The GIS equipment fault acoustic data enhancement system based on a generative large model can implement the GIS equipment fault acoustic data enhancement method based on a generative large model described in this application. However, the implementation device of the GIS equipment fault acoustic data enhancement method based on a generative large model described in this application includes, but is not limited to, the structure of the GIS equipment fault acoustic data enhancement system based on a generative large model listed in this embodiment. All structural modifications and substitutions of the prior art made according to the principles of this application are included within the protection scope of this application.

[0220] like Figure 10 As shown, this embodiment provides a GIS equipment fault acoustic data enhancement system based on a generative large model. The GIS equipment fault acoustic data enhancement system 100 based on a generative large model includes: a multimodal data acquisition module 101, a preprocessing and feature fusion module 102, an acoustic basic model training module 103, a model construction module 104, a sample generation module 105, and a quality evaluation module 106.

[0221] The multimodal data acquisition module 101 is configured to acquire multimodal data of the target device; the multimodal data includes at least acoustic signals, partial discharge signals, vibration signals, and operating condition information.

[0222] The preprocessing and feature fusion module 102 is configured to extract and fuse features based on the multimodal data to obtain a unified feature representation.

[0223] The acoustic fundamental model training module 103 is configured to perform joint self-supervised training on the initial acoustic fundamental model based on the unified feature representation to obtain the acoustic fundamental model.

[0224] The model building module 104 is configured to build a fault acoustic generation model with conditional input based on the acoustic basic model, acoustic data generation network, preset random noise, and preset control conditions. The preset control conditions include at least the fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions of the target equipment.

[0225] The sample generation module 105 is configured to generate fault acoustic samples corresponding to the target device based on the fault acoustic generation model and the preset control conditions. Specifically, it includes: performing noise diffusion processing based on the unified feature representation using the fault acoustic generation model to obtain noise-added acoustic samples; encoding and concatenating the fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions to obtain a unified condition control vector; performing reverse denoising processing based on the unified condition control vector, the noise-added acoustic samples, and the unified feature representation using the fault acoustic generation model to generate corresponding fault acoustic sample features, and performing acoustic reconstruction on the fault acoustic sample features to generate fault acoustic samples corresponding to the target device.

[0226] The quality evaluation module 106 is configured to perform quality evaluation based on the real acoustic data of the fault acoustic samples and the multimodal data, and obtain the fault acoustic enhancement samples of the target device.

[0227] It should be noted that the functions or operations of the multimodal data acquisition module 101, preprocessing and feature fusion module 102, acoustic basic model training module 103, model construction module 104, sample generation module 105 and quality evaluation module 106 described in this embodiment correspond one-to-one with the steps in the above-described method for enhancing acoustic data of GIS equipment faults based on generative large models, and therefore will not be repeated here.

[0228] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0229] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0230] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0231] This embodiment provides an electronic device, which includes a memory and a processor.

[0232] The memory is used to store computer programs.

[0233] The processor is communicatively connected to the memory and executes the above-described method for enhancing acoustic data of GIS equipment faults based on a generative large model when calling the computer program.

[0234] like Figure 11 As shown, the terminal of this application is presented in the form of a general-purpose computing device. The components of the terminal may include, but are not limited to: one or more processors or processing units 31, memory 32, and bus 33 connecting different system components (including memory 32 and processing unit 31).

[0235] Bus 33 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0236] Terminals typically include various computer system-readable media. These media can be any available media that can be accessed by the terminal, including volatile and non-volatile media, and removable and non-removable media.

[0237] Memory 32 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 321 and / or cache memory 322. The terminal may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 323 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 11Not shown; usually referred to as a "hard drive"). Although Figure 11 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 33 via one or more data media interfaces. Memory 32 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0238] A program / utility 324 having a set (at least one) of program modules 3241 may be stored, for example, in memory 32. Such program modules 3241 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 3241 typically perform the functions and / or methods described in the embodiments of this application.

[0239] The terminal can also communicate with one or more external devices (e.g., keyboard, pointing device, display, etc.), one or more devices that enable user interaction with the terminal, and / or any device that enables the terminal to communicate with one or more other computing devices (e.g., network interface card, modem, etc.). This communication can be performed through input / output (I / O) interface 34. Furthermore, the terminal can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 35. Figure 11 As shown, network adapter 35 communicates with other modules of the terminal via bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the terminal, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0240] This application also provides a computer-readable storage medium. Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)).

[0241] This application embodiment may also provide a computer program product comprising one or more computer instructions. When the computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in this application embodiment are generated. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0242] When the computer program product is executed by a computer, the computer performs the method described in the foregoing method embodiments. The computer program product can be a software installation package; when the foregoing method is required, the computer program product can be downloaded and executed on the computer.

[0243] In summary, the method and system for enhancing acoustic data of GIS equipment faults based on generative large models described in this application have the following beneficial effects:

[0244] This application proposes a method for enhancing acoustic data of GIS equipment faults based on generative large models. It can automatically generate high-fidelity and highly diverse typical fault acoustic samples, including simulated acoustics under multiple types, intensities, scenarios, and equipment structures such as partial discharge, mechanical noise, and gas leakage. The data volume can be expanded by hundreds of times, thereby providing sufficient training samples for intelligent diagnostic algorithms, improving the generalization ability and robustness of intelligent diagnostic models, significantly alleviating the problem of scarce GIS fault data, and solving the long-standing "data shortage" problem in existing technologies.

[0245] The model trained using generated data in this application not only solves the problem of poor model training effect caused by the scarcity of field data, but also simulates complex background noise in substations, acoustic drift caused by equipment aging, and changes in acoustic response caused by structural differences in different GIS models. This enables the synthetic data to improve its coverage of complex real-world working conditions and significantly improves the adaptability of the diagnostic model to multi-source scenarios. The model trained using generated data in this application can adapt to various noise environments and equipment differences in various sites, enabling the model to migrate between different regions, different substations, and different GIS models, and improving the model's cross-scenario migration capability.

[0246] The acoustic generation model proposed in this application can be aligned and jointly modeled with multimodal data such as partial discharge, vibration, and temperature rise, providing a data foundation for building a multimodal intelligent diagnostic system of "acoustics + partial discharge + vibration", thereby promoting the application of equipment condition assessment technology based on large models in the power industry. This application realizes the coupled modeling of "physical constraints + generative AI" in GIS acoustics. The proposed acoustic generation model can improve the authenticity and interpretability of generated samples, making the synthetic data usable for various engineering scenarios such as technology verification, model training, and test evaluation.

[0247] This application can use multimodal information such as acoustics, partial discharge, and vibration to constrain and evaluate generated acoustic samples (i.e., fault acoustic samples), providing high-quality training samples for building a multimodal diagnostic system; at the same time, it can generate "lightweight feature" data, providing richer samples for edge-side diagnostic models and improving the reliability of edge devices in complex scenarios.

[0248] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0249] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for enhancing acoustic data of GIS equipment faults based on generative large models, characterized in that, include: Acquire multimodal data of the target device; the multimodal data includes at least acoustic signals, partial discharge signals, vibration signals, and operating condition information; Feature extraction and fusion are performed based on the multimodal data to obtain a unified feature representation; Based on the unified feature representation, the initial acoustic basic model is subjected to multimodal self-supervised training to obtain the acoustic basic model; Based on the aforementioned acoustic fundamental model, acoustic data generation network, preset random noise, and preset control conditions, a fault acoustic generation model with conditional input is constructed; the preset control conditions include at least the fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions of the target equipment. Generating fault acoustic samples for the target device based on the fault acoustic generation model and the preset control conditions specifically includes: performing noise diffusion processing using the fault acoustic generation model based on the unified feature representation to obtain noise-added acoustic samples; encoding and concatenating the fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions to obtain a unified condition control vector; performing reverse denoising processing using the fault acoustic generation model based on the unified condition control vector, the noise-added acoustic samples, and the unified feature representation to generate corresponding fault acoustic sample features, and performing acoustic reconstruction on the fault acoustic sample features to generate fault acoustic samples for the target device. Quality evaluation is performed based on the fault acoustic samples and the real acoustic data of the multimodal data to obtain fault acoustic enhancement samples of the target device.

2. The method according to claim 1, characterized in that, Based on the multimodal data, feature extraction and fusion are performed to obtain a unified feature representation, including: Preprocessing is performed on the multimodal data to obtain preprocessed multimodal data; Feature extraction is performed based on the preprocessed multimodal data to obtain multimodal features; the multimodal features include at least acoustic features, partial discharge features, and vibration features; Based on the acoustic features, the partial discharge features, and the vibration features, a time-stamp-based multimodal alignment function is used to perform feature fusion to obtain a unified feature representation.

3. The method according to claim 2, characterized in that, Preprocessing is performed on the multimodal data to obtain preprocessed multimodal data, including: The acoustic signal is subjected to noise reduction, normalization and time-frequency transformation to obtain a preprocessed acoustic signal. Based on the partial discharge signal, pulse sequence detection and envelope analysis are performed to obtain the preprocessed partial discharge signal; Empirical mode decomposition and envelope analysis are performed based on the vibration signal to obtain the preprocessed vibration signal.

4. The method according to claim 1, characterized in that, The multimodal self-supervised training includes a mask learning task, a contrastive learning task, and a multimodal alignment learning task. Based on the unified feature representation, the initial acoustic base model is subjected to multimodal self-supervised training to obtain the acoustic base model, which includes: Mask learning is performed on the initial acoustic model based on the unified feature representation, and the content prediction and reconstruction are performed on the masked region of the unified feature representation in order to learn the internal acoustic structure. Based on the unified feature representation input to the initial acoustic base model, comparative learning is performed. For the unified feature representation, positive sample pairs between different modalities at the same time and negative sample pairs between different augmented views of the same modality are constructed to learn the representation ability of cross-modal correlation and fault semantic consistency. Multimodal alignment learning is performed based on the unified feature representation input to the initial acoustic base model. By constraining the feature representations of the same semantic content in different modalities to align in a shared semantic space, shared high-level semantic information across modalities is learned. The initial acoustic base model is jointly optimized based on the Mask learning task, the contrastive learning task, and the multimodal alignment learning task, and the initial acoustic base model is updated using their respective loss functions to obtain the acoustic base model.

5. The method according to claim 1, characterized in that, The fault acoustic generation model includes an input encoding module, a conditional embedding module, a noise diffusion module, a reverse denoising generation module, and an output reconstruction module. The input encoding module is used to convert real-collected fault acoustic samples into a unified feature representation suitable for model processing; The condition embedding module is used to encode and concatenate preset control conditions to form a unified condition control vector; The noise diffusion module is used to progressively add preset random noise to the unified feature representation to form a noise-added sample from clear sample to random noise. The reverse denoising generation module is used to perform reverse denoising processing on the noise-added acoustic sample under the control of preset control conditions to generate corresponding fault acoustic sample features. The output reconstruction module is used to perform acoustic reconstruction on the fault acoustic sample features generated by the inverse denoising generation module, generate fault acoustic samples, and output them.

6. The method according to claim 1, characterized in that, Generating fault acoustic samples for the corresponding target device based on the fault acoustic generation model and the preset control conditions also includes: Encode the fault type condition, fault level condition, operating condition condition, and equipment structure condition respectively to obtain the encoded condition vector; the encoded condition vector includes the fault type condition vector, fault level condition vector, operating condition condition vector, and equipment structure condition vector. A unified condition control vector is obtained by concatenating the fault type condition vector, the fault level condition vector, the operating condition vector, and the equipment structure condition vector. Based on the unified conditional control vector, preset random noise, and the unified feature representation, the fault acoustic generation model is used to generate acoustic samples to obtain fault acoustic samples of the target device.

7. The method according to claim 1, characterized in that, Quality evaluation is performed based on the fault acoustic samples and the real acoustic data of the multimodal data to obtain fault acoustic enhancement samples of the target device, including: Feature extraction is performed on the fault acoustic samples and the real acoustic data of the multimodal data to obtain the corresponding time-frequency features; the time-frequency features include Mel spectrum, STFT spectrum and spectral entropy features; The similarity is calculated based on the Mel spectrum, STFT spectrum, and spectral entropy features of each spectrum; the similarity includes Mel spectrum distance, spectral structure similarity, and spectral entropy difference. Feature comparison is performed on the partial discharge pulse sequence of the fault acoustic sample to obtain the consistency of the partial discharge pulse distribution; The faulty acoustic samples are input into a discrimination network for authenticity assessment, and an authenticity score is obtained. A comprehensive quality score is obtained by weighting and fusing the Mel spectral distance, the spectral structure similarity, the spectral entropy difference, the partial discharge pulse distribution consistency, and the authenticity score according to preset weights. If the overall quality score is greater than or equal to the preset threshold, the current fault acoustic sample is determined to be a high-quality acoustic enhancement sample and is used as the fault acoustic enhancement sample of the target device. If the overall quality score is less than a preset threshold, the current fault acoustic sample is determined to be a low-quality acoustic enhancement sample and is not used as a fault acoustic enhancement sample for the target device.

8. An acoustic data augmentation system based on a generative large model, characterized in that, include: A multimodal data acquisition module is configured to acquire multimodal data of a target device; the multimodal data includes at least acoustic signals, partial discharge signals, vibration signals, and operating condition information. The preprocessing and feature fusion module is configured to extract and fuse features based on the multimodal data to obtain a unified feature representation; The acoustic fundamental model training module is configured to perform joint self-supervised training on the initial acoustic fundamental model based on the unified feature representation to obtain the acoustic fundamental model. The model building module is configured to build a fault acoustic generation model with conditional input based on the acoustic basic model, acoustic data generation network, preset random noise, and preset control conditions. The preset control conditions include at least the fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions of the target equipment; The sample generation module is configured to generate fault acoustic samples corresponding to the target device based on the fault acoustic generation model and the preset control conditions. Specifically, it includes: performing noise diffusion processing based on the unified feature representation using the fault acoustic generation model to obtain noise-added acoustic samples; encoding and concatenating the fault type conditions, fault level conditions, operating condition conditions, and equipment structure conditions to obtain a unified condition control vector; performing reverse denoising processing based on the unified condition control vector, the noise-added acoustic samples, and the unified feature representation using the fault acoustic generation model to generate corresponding fault acoustic sample features, and performing acoustic reconstruction on the fault acoustic sample features to generate fault acoustic samples corresponding to the target device. The quality evaluation module is configured to perform quality evaluation based on the real acoustic data of the fault acoustic samples and the multimodal data, and to obtain the fault acoustic enhancement samples of the target device.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; The processor, which is communicatively connected to the memory, executes the method for enhancing acoustic data of GIS equipment faults based on a generative large model as described in any one of claims 1 to 7 when calling the computer program.

Images