Insulator modeling evaluation method and system based on voiceprint fusion and knowledge distillation

Abstract

The invention discloses a voiceprint fusion and knowledge distillation insulator modeling evaluation method and system. The method comprises the steps that firstly, insulator voiceprint signals and synchronous working condition data are collected, and preprocessing is carried out to generate a multi-dimensional voiceprint feature map and a working condition scalar feature vector; secondly, constructing a teacher model, extracting voiceprint time-frequency features by using a double-flow spatial-temporal feature aggregation architecture, deeply embedding working condition scalar quantities into voiceprint feature tensors through a broadcast mechanism, and generating multi-modal fusion features; thirdly, constructing a lightweight student model, guiding the lightweight student model to perform knowledge distillation training by using multi-modal fusion features of the teacher model, and constructing a multi-task joint loss function optimization model parameter in combination with cross-view feature alignment loss, focus loss and center loss; and finally, deploying the student model subjected to training convergence to a monitoring terminal for real-time reasoning. According to the method, the problems of difficulty in insulator fault feature extraction and difficulty in model edge deployment under complex working conditions are effectively solved, and accurate evaluation of the health state is realized.

Description

Technical Field

[0001] This invention relates to the field of power system equipment monitoring technology, specifically to an insulator modeling and evaluation method and system based on voiceprint fusion and knowledge distillation. Background Technology

[0002] As a core insulating component of high-voltage transmission lines, accurate assessment of the health status of insulators is crucial for the safe and stable operation of the power grid, reliable power supply, and control of operation and maintenance costs. Conducting intelligent research on insulator condition monitoring to quickly identify deterioration states and assess fault levels is a core element in improving the digital operation and maintenance level of the power system.

[0003] Currently, traditional methods for assessing the condition of insulators include infrared thermal imaging, ultrasonic testing, and manual inspection image recognition. However, these methods have certain limitations: for example, infrared testing is not sensitive to early, minor degradation; ultrasonic testing has poor robustness under complex weather conditions; and image recognition methods are greatly affected by lighting and angle, and are difficult to detect internal insulation degradation.

[0004] In recent years, detection methods based on acoustic signature signals have attracted attention due to their advantages such as being non-contact and providing rich information. However, existing acoustic signature detection technologies, such as LSTM-based time-series analysis, while having made some progress, still face the following technical bottlenecks in practical industrial applications: First, existing methods mostly rely on single types of acoustic signature features (such as MFCC or short-time spectrum), neglecting the influence of physical operating parameters such as insulator operating voltage and ambient temperature and humidity on the characteristics of discharge acoustic signatures. Under varying operating conditions such as rain, fog, high humidity, or load fluctuations, single acoustic signature features are prone to feature drift, leading to serious false alarms or missed alarms. Second, early fault samples of insulators are scarce, and discharge signals exhibit transient bursts and irregular frequency domain distribution. Traditional convolutional networks struggle to simultaneously capture long-term temporal dependencies and irregular frequency domain morphological features, resulting in weak model generalization ability in small-sample scenarios and insufficient accuracy in identifying subtle fault features. Finally, complex deep learning models designed to improve accuracy often have a huge number of parameters and high computational costs, while the computing power and power consumption of intelligent monitoring terminals at power transmission line sites are strictly limited. Existing general model compression methods (such as ordinary knowledge distillation) only focus on fitting the output probabilities of teachers and students, ignoring the robustness of features to noise. Under strong background noise interference, they are prone to losing key semantic features, making it difficult to achieve extremely lightweight deployment while ensuring high accuracy and meeting the needs of real-time monitoring.

[0005] Therefore, how to achieve lightweight models while ensuring high evaluation accuracy, so that they can run efficiently in edge computing environments, is a technical challenge that urgently needs to be solved in the field of intelligent monitoring of insulators for high-voltage transmission lines. Summary of the Invention

[0006] To address the shortcomings of existing insulator monitoring technologies, such as difficulties in feature extraction under complex operating conditions and limited computing power at the edge, this invention proposes a method and system for insulator modeling and evaluation based on acoustic signature fusion and knowledge distillation. The method includes: collecting insulator acoustic signature signals and synchronous operating condition data; preprocessing to generate multi-dimensional acoustic signature feature maps and operating condition scalar feature vectors; constructing a teacher model for insulator health status evaluation, extracting acoustic signature features using a dual-stream spatiotemporal feature aggregation architecture, and embedding the operating condition scalar feature vectors into the deep acoustic signature feature tensor through a broadcast mechanism to generate multimodal fusion features; constructing a lightweight student model, using the multimodal fusion features to guide its knowledge distillation training, and optimizing model parameters by constructing a multi-task joint loss function combining cross-view feature alignment loss, focus loss, and center loss; finally, deploying the converged student model on the monitoring terminal for real-time inference. This invention effectively solves the problems of difficult insulator fault feature extraction and model deployment at the edge under complex operating conditions, achieving accurate health status evaluation.

[0007] The present invention adopts the following technical solution.

[0008] In a first aspect, this invention discloses an insulator modeling and evaluation method based on voiceprint fusion and knowledge distillation, comprising: S1: Collect insulator acoustic signature signals and synchronous operating condition data, and generate multi-dimensional acoustic signature feature maps and operating condition scalar feature vectors after preprocessing. S2: Construct a teacher model for assessing the health status of insulators; utilize the dual-stream spatiotemporal feature aggregation architecture of the teacher model for assessing the health status of insulators to extract the temporal sequence dependency features and frequency domain morphological features of the multi-dimensional acoustic signature feature map, and perform multimodal fusion after mapping and expanding the working condition scalar feature vector to generate multimodal fusion features. S3: Construct a lightweight student model; using multi-dimensional voiceprint feature maps and operating condition scalar feature vectors as input, under the condition of fixed parameters of the insulator health status assessment teacher model, use multi-modal fusion features to guide the lightweight student model to perform knowledge distillation training; calculate the multi-task joint loss function and update the parameters of the lightweight student model; the multi-task joint loss function includes the cross-view feature alignment loss calculated based on the basic voiceprint features extracted from the same sample by the lightweight student model and the enhanced voiceprint features after data augmentation; S4: Deploy the trained lightweight student model on the monitoring terminal, and perform inference on the real-time acoustic fingerprint data of the insulator under test to output the insulator health status assessment results.

[0009] Preferably, the insulator acoustic signal includes background sound of normal insulator operation, partial discharge sound, and mechanical vibration sound; the synchronous operating condition data includes insulator operating voltage level, load current, ambient temperature, and ambient humidity.

[0010] Preferably, the insulator acoustic signature signal is preprocessed to generate a multi-dimensional acoustic signature feature map, specifically including: For the same segment of insulator acoustic signature signal collected, feature extraction operations are performed respectively, including: performing short-time Fourier transform on the insulator acoustic signature signal, calculating the power spectrum and mapping it to the Mel scale to generate a Mel spectrum; performing time-frequency analysis on the insulator acoustic signature signal using continuous wavelet transform to extract non-stationary transient features and generate a wavelet time-frequency map; and performing pre-emphasis, framing, and discrete cosine transform on the insulator acoustic signature signal to extract cepstral coefficients and generate an MFCC feature map. The multidimensional voiceprint feature map is composed of at least one or more combinations of the Mel spectrogram, wavelet time-frequency graph, and MFCC feature map.

[0011] Preferably, the dual-stream spatiotemporal feature aggregation architecture of the insulator health status assessment teacher model is used to extract the temporal sequence dependency features and frequency domain morphological features of the multi-dimensional voiceprint feature map, specifically including: Construct parallel time-domain feature extraction branches and frequency-domain feature extraction branches; The temporal feature extraction branch is used to slice the multi-dimensional voiceprint feature map in the time dimension, flatten the frequency dimension and channel dimension contained in the sliced ​​multi-dimensional voiceprint feature map and map the feature sequence, input the bidirectional long short-term memory network, and capture the temporal sequence-dependent features of the insulator voiceprint. The frequency domain feature extraction branch is used to process the multi-dimensional voiceprint feature map using a deformable convolutional network. By learning the sampling offset of the convolution kernel, it adaptively adjusts the convolution sampling position to cover irregular discharge frequency bands and extracts frequency domain morphological features.

[0012] Preferably, after mapping and expanding the operating condition scalar feature vector, multimodal fusion is performed with the time-domain sequence dependency feature and the frequency-domain morphological feature, specifically including: The time-domain sequence dependency features are concatenated with the frequency-domain morphological features to construct a deep voiceprint feature tensor. A fully connected layer is used to map the scalar feature vector of the operating condition to a channel dimension consistent with the deep feature tensor of the voiceprint. A broadcast mechanism is used to copy and expand the mapped working condition scalar feature vector in the spatial dimension, thereby constructing an extended working condition feature with a spatial size consistent with the deep feature tensor of the voiceprint. The extended operating condition features are concatenated with the deep voiceprint feature tensor through channels or added point by point to generate the multimodal fusion features.

[0013] Preferably, the use of multimodal fusion features to guide the knowledge distillation training of a lightweight student model includes: Construct the lightweight student model so that its number of convolutional layers or feature channel dimension is lower than that of the insulator health status assessment teacher model; The multi-dimensional voiceprint feature map and the working condition scalar feature vector are input into the lightweight student model to extract deep student features and student classification probability. Perform feature-level distillation, including calculating feature matching loss, and constraining the student deep features by minimizing the Euclidean or cosine distance between the student deep features and the multimodal fusion features; Performing logic-level distillation includes calculating the probability distribution divergence loss and constraining the student classification probability by minimizing the relative entropy between the student classification probability and the classification probability distribution output by the insulator health status assessment teacher model.

[0014] Preferably, the multi-task joint loss function is constructed by weighted summation of classification loss, intra-class distance loss, knowledge distillation loss, and cross-view feature alignment loss, and the specific calculation includes: The classification loss is calculated using the focus loss function, including dynamically adjusting the loss weights based on the student classification probability to increase the training weight of hard-to-classify samples; The intra-class distance loss is calculated using the center loss function, including calculating the Euclidean distance between the student deep features and the corresponding category feature centers; The feature matching loss and probability distribution divergence loss are obtained, and then weighted and summed to obtain the knowledge distillation loss. The cross-view feature alignment loss is calculated using a cosine embedding loss function, including calculating the cosine similarity in the latent space between the basic voiceprint features and the enhanced voiceprint features extracted by the lightweight student model for the same sample.

[0015] Secondly, an insulator modeling and evaluation system based on voiceprint fusion and knowledge distillation, comprising running the aforementioned insulator modeling and evaluation method based on voiceprint fusion and knowledge distillation, including: The data acquisition and preprocessing module is used to acquire insulator acoustic signature signals and synchronous operating condition data, and after preprocessing, generates multi-dimensional acoustic signature feature maps and operating condition scalar feature vectors respectively. The teacher model construction and fusion module is used to construct a teacher model for insulator health status assessment. Utilizing the dual-stream spatiotemporal feature aggregation architecture of the teacher model for insulator health status assessment, it extracts the temporal sequence dependency features and frequency domain morphological features of the multi-dimensional acoustic signature feature map, and then performs multimodal fusion after mapping and expanding the working condition scalar feature vector to generate multimodal fusion features. The student model distillation training module is used to construct a lightweight student model. It utilizes multi-dimensional acoustic signature feature maps and operational condition scalar feature vectors as inputs. Under the condition of fixed parameters of the insulator health status assessment teacher model, it uses multi-modal fusion features to guide the lightweight student model in knowledge distillation training. It calculates the multi-task joint loss function and updates the parameters of the lightweight student model. The multi-task joint loss function includes the cross-view feature alignment loss calculated based on the basic acoustic signature features extracted from the same sample by the lightweight student model and the enhanced acoustic signature features after data augmentation. The deployment and inference evaluation module is used to deploy the trained lightweight student model on the monitoring terminal, and to infer the real-time acoustic fingerprint data of the insulator under test, and output the insulator health status evaluation results.

[0016] Thirdly, a terminal includes a processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method.

[0017] Fourthly, a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method.

[0018] The beneficial effects of this invention are as follows: 1. Based on the acquisition of insulator acoustic signature signals and synchronization condition data, this invention generates multi-dimensional acoustic signature feature maps and operating condition scalar feature vectors through preprocessing. By constructing multi-dimensional features including Mel spectrum, wavelet time-frequency plots, and MFCC, and combining data augmentation strategies, the diversity of degradation features is increased, thereby improving the model's generalization ability to multi-source data under complex operating conditions (such as different meteorological conditions and electromagnetic interference).

[0019] 2. This invention utilizes a dual-stream spatiotemporal feature aggregation architecture to construct a teacher model for insulator health status assessment. Specifically, the temporal feature extraction branch captures the temporal sequence-dependent features of insulator acoustic signatures (such as the periodic features of intermittent discharges) through a bidirectional long short-term memory network; the frequency domain feature extraction branch uses a deformable convolutional network to adaptively adjust the sampling position to cover irregular discharge frequency bands and extract frequency domain morphological features. This time-frequency dual-stream architecture can simultaneously handle feature extraction from early weak discharge signals (time-sensitive) and severe fault signals (significant frequency domain morphology), achieving accurate assessment of the insulator's health status throughout its entire lifecycle.

[0020] 3. This invention utilizes a voiceprint-operating condition feature fusion module to achieve deep interaction of heterogeneous data. Unlike traditional methods, this module employs a broadcast mechanism to deeply embed the mapped and expanded operating condition scalar feature vector into the deep voiceprint feature tensor, generating multimodal fusion features. This mechanism introduces physical environmental constraints at the feature extraction level, enabling the model to dynamically adjust its discrimination logic based on operating voltage and temperature / humidity, effectively solving the false alarm problem caused by discharge characteristic drift under varying operating conditions.

[0021] 4. This invention introduces cross-view feature alignment loss during training. By calculating the cosine similarity in the latent space between the basic voiceprint features and enhanced voiceprint features extracted by the lightweight student model for the same sample, the consistency of their semantic directions is constrained. This self-supervised feature alignment method forces the model to ignore noise interference and focus on the essence of the fault, improving the robustness of the model in scenarios with strong noise and small sample sizes.

[0022] 5. This invention constructs a multi-task joint loss function that includes classification loss, intra-class distance loss, and knowledge distillation loss. It utilizes a focus loss function to adjust weights and identify hard-to-classify samples, and a center loss function to minimize intra-class distance and improve feature compactness. Simultaneously, through feature matching loss and probability distribution divergence loss, it efficiently transfers the multimodal fusion features and classification probabilities generated by the teacher model to the lightweight student model. This not only solves the problem of complex models being difficult to deploy on edge monitoring terminals, but also effectively reduces early, slight degradation in recognition error and improves overall evaluation performance. Attached Figure Description

[0023] Figure 1 This is a flowchart of an insulator modeling and evaluation method based on voiceprint fusion and knowledge distillation provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0025] Example 1: like Figure 1 As shown, this invention provides an insulator modeling and evaluation method based on voiceprint fusion and knowledge distillation, comprising: S1: Collect insulator acoustic signature signals and synchronization condition data. This step aims to construct a high-quality, multimodal small-sample dataset of insulator health status.

[0026] S11, Multi-source data acquisition and cleaning: This embodiment preferably utilizes an array of microphones deployed along the transmission line insulators to collect insulator acoustic signature signals under different health conditions. These signals include background noise during normal insulator operation, buzzing or popping sounds generated by partial discharge, and mechanical vibrations caused by loose fittings. Simultaneously, sensors record synchronous operating condition data for the insulators, including insulator operating voltage level, load current, ambient temperature, and ambient humidity.

[0027] Based on energy threshold detection (VAD) or sliding window algorithm, the continuously acquired dynamic voiceprint stream is processed to remove silent segments and automatically extract effective voiceprint segments containing significant feature information (e.g., the extraction duration is uniformly 3 seconds).

[0028] To address the issue of insufficient initial measured fault samples, this embodiment also selects insulator degradation samples from publicly available power equipment acoustic fingerprint databases (such as publicly available transmission line fault acoustic fingerprint datasets) and uses them together with measured data to construct an initial acoustic fingerprint dataset, and establishes an online real-time update and expansion mechanism.

[0029] S12, preprocess the insulator acoustic signature signal to generate a multi-dimensional acoustic signature feature map, generating a heterogeneous feature set containing 3 maps and 1 vector: For the same segment of insulator acoustic signature signal collected, feature extraction operations were performed separately: After pre-emphasis, framing, and windowing of the insulator acoustic signature signal, a short-time Fourier transform (STFT) is performed on the insulator acoustic signature signal, the power spectrum is calculated and mapped to the Mel scale through a Mel filter bank, and the logarithm is taken to generate a Mel spectrum diagram that reflects the auditory characteristics of the human ear. To address the non-stationary and highly abrupt characteristics of insulator discharge signals, a time-frequency analysis of the insulator acoustic signature signal is performed using continuous wavelet transform (CWT) (e.g., using the Morlet wavelet basis) to extract non-stationary transient features and generate a wavelet time-frequency diagram. The voiceprint signal is pre-emphasized, framed, and subjected to discrete cosine transform (DCT) to extract cepstral coefficients and generate an MFCC feature map that reflects the characteristics of the vocal tract. Numerical data such as operating voltage levels and ambient temperature and humidity collected synchronously are normalized and then concatenated in a fixed order to construct a one-dimensional scalar feature vector of operating conditions. The Mel spectrum, wavelet time-frequency graph, and MFCC feature map mentioned above together constitute a multi-dimensional acoustic signature feature map, which serves as the tensor input for subsequent models; the scalar feature vector of operating conditions serves as the auxiliary scalar input for subsequent models.

[0030] S13 uses professional annotation tools (such as LabelStudio) to annotate health status labels (normal, mild degradation, such as early weak discharge, severe fault, such as obvious breakdown or continuous arcing) in the form of feature maps, and generates a JSON format annotation file that records the path of the voiceprint file, the corresponding feature map path, the value of the operating condition vector, and the mapping relationship of the health status labels.

[0031] S14, the labeled multi-dimensional voiceprint feature map dataset is divided into training set, validation set and test set according to a certain ratio. In this embodiment, 60% of the data is used for model training (including working conditions with different degrees of degradation and environmental interference), 20% is used to verify the model's generalization ability, and 20% is used to test the accuracy of health status classification. The division ratio can be dynamically adjusted according to the scale of small sample data in the field.

[0032] In particular, to address the long-tail distribution problem and to prepare for cross-view issues in subsequent S3, Figure 1 Consistent learning provides the data foundation, and this embodiment constructs a data augmentation strategy. One or more of the following augmentation operations are performed on the samples in the training set to generate an enhanced voiceprint feature map: Temporal augmentation: random time shift, random pruning; Frequency domain enhancement: Frequency domain masking and temporal masking are performed using the SpecAugment strategy; Environmental simulation: Gaussian white noise or natural environmental background sounds (such as wind and rain) with different signal-to-noise ratios (SNR) are superimposed to simulate real harsh working conditions. Through the above operations, for the same original sample, the system simultaneously holds its basic voiceprint feature map and its enhanced voiceprint feature map to support the robust training of subsequent models.

[0033] S2: Construct a teacher model for assessing the health status of insulators; utilize the dual-stream spatiotemporal feature aggregation architecture of the teacher model for assessing the health status of insulators to extract the temporal sequence dependence features and frequency domain morphological features of the multi-dimensional acoustic signature feature map, and perform multimodal fusion after mapping and expanding the working condition scalar feature vector to generate multimodal fusion features.

[0034] This step aims to construct a high-precision teacher model by extracting spatiotemporal features through a two-stream architecture and deeply fusing physical condition information. Specifically, S2 above includes: S21. Construct a dual-stream spatiotemporal feature aggregation network (STFA). To address the issues of temporal feature loss and irregular frequency domain morphology in traditional convolutional networks when processing insulator discharge signals, this embodiment designs a parallel dual-stream feature extraction architecture, including parallel temporal feature extraction branches and frequency domain feature extraction branches. The input data is the multi-dimensional acoustic signature feature map generated in S1 (tensor dimensions are [B, C, F, T], where B represents the batch size, C=3 corresponds to the three channels of Mel / wavelet / MFCC, F represents the frequency, and T represents the number of time frames).

[0035] S211, the time-domain feature extraction branch aims to capture the temporal evolution of insulator discharge (such as intermittent pulses). First, the multi-dimensional acoustic signature feature map is sliced ​​along the time dimension T, and the frequency dimension F and channel dimension C contained in the sliced ​​multi-dimensional acoustic signature feature map are flattened and mapped to a feature sequence. Finally, this sequence is input into a bidirectional long short-term memory network (Bi-LSTM) to capture the long-distance dependence of the acoustic signature signal on the time axis and output the time-domain sequence dependence features of the insulator acoustic signature. S212, the frequency domain feature extraction branch is designed to adapt to the irregular shapes such as distortion and discreteness of the discharge signal in the frequency domain spectrum. A Deformable CNN is used to process the multi-dimensional acoustic signature feature map. By learning the sampling offset of the convolution kernel, the convolution sampling position is adaptively adjusted so that its receptive field is no longer limited to a rectangle, but can undergo geometric deformation, thereby covering irregular discharge frequency bands (e.g., automatically focusing on the high-frequency discharge region of 3kHz-10kHz while ignoring low-frequency background noise regions), and extracting frequency domain morphological features.

[0036] S22: Design the Voiceprint-Working Feature Fusion Module (SVFM). SVFM can be set in a deep layer of the teacher model (such as at the end of the encoder) or deployed at multiple levels (e.g., fused simultaneously in the middle layer of the encoder and the decoder layer) to achieve step-by-step guidance of working information on voiceprint features at different scales.

[0037] S221, firstly, the dual-stream features output from S21 are dimensionally aligned. Since the temporal sequence-dependent features (output from Bi-LSTM) lack a frequency dimension, this step replicates and expands them along the frequency dimension to construct extended temporal features (both dimensions B×D×F×T) with the same spatial size as the frequency domain morphological features. Then, the two are concatenated along the channel dimension, and a 1×1 convolution is used for channel adjustment to obtain a unified deep voiceprint feature tensor (its dimension is F×T×D, where D is the number of feature channels). Next, the operating condition scalar feature vector (containing values ​​such as voltage, temperature, and humidity) input from S1 is processed. A fully connected layer maps the operating condition scalar feature vector to the same channel dimension D as the deep voiceprint feature tensor. A broadcast mechanism is used to replicate and expand the mapped operating condition scalar feature vector along the spatial dimensions (F and T directions) to construct extended operating condition features with the same spatial size as the deep voiceprint feature tensor.

[0038] S222, the extended operating condition features and the deep voiceprint feature tensor are concatenated by channel or added point-by-point to generate preliminary fusion features. Based on this, a channel attention mechanism is introduced. The channel attention mechanism is used to calculate a cross-dimensional weight matrix, adaptively enhancing the response of degraded related features (such as feature channels corresponding to the 3-10kHz discharge frequency band); multimodal fusion features are output through residual connections. This design allows the model to dynamically adjust the discrimination logic for voiceprint features according to environmental parameters (e.g., automatically reducing sensitivity to corona noise under high humidity).

[0039] S23, Pre-training of the teacher model based on multi-dimensional contrastive learning. To further address the problem of scarce insulator fault samples, as a preferred approach in this embodiment, the STFA backbone network of the teacher model is pre-trained using unlabeled data before formal supervised training.

[0040] S231. In view of the irregular shape of the discharge acoustic pattern of the insulator in the frequency domain, an acoustic pattern comparison learner is constructed based on the MoCo v3 framework. The original acoustic pattern spectrum of the same insulator and the enhanced acoustic pattern spectrum after data enhancement as described in S1 are used as positive sample pairs, and randomly matched samples of different states are used as negative sample pairs. The similarity of positive sample pairs is maximized in the feature space.

[0041] S232 employs a "masking-reconstruction" task: randomly occluding 30% of the voiceprint time frame or frequency channel, forcing the STFA network to recover complete features through multi-dimensional information complementarity. This solves the semantic gap problem of single-dimensional feature pre-training in multi-dimensional scenarios, improves the model's health status assessment accuracy and robustness in low signal-to-noise ratio (SNR≤5dB) environments, and lays the foundation for knowledge transfer in S3.

[0042] As a supplementary solution to this invention, in extremely unreliable environments, a voiceprint feature calibration module (SFCM) can be introduced to perform consistency calibration on multi-dimensional features at each level. SFCM downsamples the basic and enhanced voiceprint features to a uniform size (e.g., 32×32 pixels) through average pooling, calculates the consistency between features using cosine similarity, and performs channel-weighted adjustment on features with deviations exceeding a threshold (0.3). While ensuring information fusion efficiency, it reduces multi-dimensional feature deviations, providing a more accurate feature representation for subsequent health status classification.

[0043] S3: Construct a lightweight student model; using multi-dimensional voiceprint feature maps and operating condition scalar feature vectors as input, under the condition of fixed parameters of the insulator health status assessment teacher model, use multi-modal fusion features to guide the lightweight student model to perform knowledge distillation training; calculate the multi-task joint loss function and update the parameters of the lightweight student model; the multi-task joint loss function includes the cross-view feature alignment loss calculated based on the basic voiceprint features extracted from the same sample by the lightweight student model and the enhanced voiceprint features after data augmentation.

[0044] Specifically, S3 includes: S31, Constructing a lightweight student model and distillation framework. Utilizing multimodal fusion features to guide the lightweight student model in knowledge distillation training, specifically including: First, a lightweight convolutional neural network designed for edge deployment is constructed as a lightweight student model. This student model has significantly fewer network layers or channels than the teacher model; for example, fewer convolutional layers and channels are used, compressing the number of parameters to approximately 30% of the teacher model. For multimodal data processing, the student model employs a lightweight fusion strategy, such as mapping the scalar feature vector of the operating conditions through a fully connected layer and then directly concatenating it with the voiceprint features along the channel dimension.

[0045] The multi-dimensional voiceprint feature map (including the basic voiceprint map and the data-enhanced voiceprint map) preprocessed in S1, along with the scalar feature vector of the working condition, is input into the lightweight student model to extract deep features of the student and the student classification probability. The parameters of the teacher model constructed in S2 are fixed (not participating in gradient updates), and the multimodal fusion features and classification probability distribution generated by it are used as supervision signals to guide the training of the student model.

[0046] S32, Constructing the multi-task joint loss function. To address the issues of scarce insulator fault samples, large intra-class variability, and environmental noise interference, this embodiment designs a multi-task joint loss function consisting of four parts. It is constructed by weighted summation of classification loss, intra-class distance loss, knowledge distillation loss, and cross-view feature alignment loss. The specific calculation includes: The classification loss is calculated using the focus loss function, which includes dynamically adjusting the loss weights based on the student classification probability, reducing the weight of easily classified samples (such as obvious background noise) to increase the training weight of difficult-to-classify samples (such as early weak discharge) and solving the sample imbalance problem.

[0047] The intra-class distance loss is calculated using a center loss function, which includes calculating the Euclidean distance between the student's deep features and the corresponding class feature centers, and minimizing this distance. This aims to compress the distribution of similar features (e.g., all being "slightly degraded") in the feature space, thereby improving the discriminative power of the classification boundary.

[0048] The process involves performing feature-level distillation, including calculating the feature matching loss. This is done by minimizing the Euclidean or cosine distance between the student's deep features and the multimodal fusion features (preferably Euclidean distance in this embodiment), constraining the student's deep features and forcing the student model to learn the problem-solving approach of the teacher model based on the fusion of physical conditions. The process also includes performing logic-level distillation, including calculating the probability distribution divergence loss. This is done by minimizing the relative entropy (KL divergence) between the student's classification probability and the classification probability distribution output by the teacher model for assessing the insulator's health status, constraining the student's classification probability and ensuring that the student model inherits the implicit information from the teacher model. Finally, the feature matching loss and the probability distribution divergence loss are obtained and weighted to obtain the knowledge distillation loss.

[0049] The cross-view feature alignment loss is calculated using a cosine embedding loss function. This includes calculating the cosine similarity in the latent space between the basic voiceprint features (noise-free or weakly noisy) and enhanced voiceprint features (strongly noisy or masked) extracted by the lightweight student model for the same sample, thus constraining the consistency of semantic direction. This loss forces the model to ignore environmental noise interference and focus on the essential features of the fault.

[0050] S33, Iterative Optimization and Convergence Control. Continuously iterate training until the joint loss function for multiple tasks no longer decreases or the accuracy on the validation set no longer improves (e.g., accuracy fluctuation less than 0.1% over 10 consecutive epochs), saving the optimal student model parameters. During training, combine the focus loss and dynamic sample weighting strategy described in S32 to balance the proportion of samples in different health states (normal / mildly degraded / severely faulty); simultaneously, enhance the learning efficiency of degraded features in small samples through feature distillation techniques in knowledge distillation.

[0051] S4: Deploy the trained lightweight student model on the monitoring terminal, and perform inference on the real-time acoustic fingerprint data of the insulator under test to output the insulator health status assessment results.

[0052] Specifically, S4 above includes: S41 integrates the trained lightweight student model into the intelligent monitoring terminal (including a preferred array microphone and edge processor) along the transmission line insulator through the edge computing unit, realizing lightweight deployment and real-time inference of the model.

[0053] S42: Real-time acquisition of multi-dimensional acoustic signature signals from insulator operation, and synchronous acquisition of current operating condition data via sensors. Following the preprocessing procedure described in step S1, real-time acoustic signature feature maps and operating condition scalar feature vectors are generated, and both are input into the lightweight student model. The health status assessment results (normal / slightly deteriorated / severe fault) and confidence scores for each category are obtained. Simultaneously, Grad-CAM (gradient-weighted class activation mapping) technology is used to generate a class activation heatmap. This heatmap visually locates high-response regions (i.e., key fault feature regions) that lead to fault decisions.

[0054] S43, Set a dynamic confidence threshold. Based on real-time calculation of the signal-to-noise ratio (SNR) of the current acoustic signal and synchronous operating condition data (such as ambient humidity), establish a "noise-threshold" linkage mechanism: in low-noise environments (such as high SNR or low-humidity sunny weather), the threshold is set to 0.75; in strong background noise environments (such as high SNR or low-humidity sunny weather), the threshold is automatically increased to 0.85 to reduce the false alarm rate. When the classification result is deteriorated and the confidence exceeds the current threshold, an alarm is triggered, indicating that the insulator has a health hazard or fault.

[0055] Once an alarm is triggered, S44 automatically uploads an acoustic signature containing fault characteristics, a visual image of the fault area overlaid with a heat map, confidence level values, collection timestamps, tower number and location, and coordinates of key fault characteristic areas to the power grid cloud monitoring database. At the same time, it triggers local audible and visual alarms and remote operation and maintenance platform warning pushes.

[0056] Example 2: An insulator modeling and evaluation system based on voiceprint fusion and knowledge distillation, comprising running the aforementioned insulator modeling and evaluation method based on voiceprint fusion and knowledge distillation, including: The data acquisition and preprocessing module is used to acquire insulator acoustic signature signals and synchronous operating condition data, and after preprocessing, generates multi-dimensional acoustic signature feature maps and operating condition scalar feature vectors respectively. The teacher model construction and fusion module is used to construct a teacher model for insulator health status assessment. Utilizing the dual-stream spatiotemporal feature aggregation architecture of the teacher model for insulator health status assessment, it extracts the temporal sequence dependency features and frequency domain morphological features of the multi-dimensional acoustic signature feature map, and then performs multimodal fusion after mapping and expanding the working condition scalar feature vector to generate multimodal fusion features. The student model distillation training module is used to construct a lightweight student model. It utilizes multi-dimensional acoustic signature feature maps and operational condition scalar feature vectors as inputs. Under the condition of fixed parameters of the insulator health status assessment teacher model, it uses multi-modal fusion features to guide the lightweight student model in knowledge distillation training. It calculates the multi-task joint loss function and updates the parameters of the lightweight student model. The multi-task joint loss function includes the cross-view feature alignment loss calculated based on the basic acoustic signature features extracted from the same sample by the lightweight student model and the enhanced acoustic signature features after data augmentation. The deployment and inference evaluation module is used to deploy the trained lightweight student model on the monitoring terminal, and to infer the real-time acoustic fingerprint data of the insulator under test, and output the insulator health status evaluation results.

[0057] Example 3: A terminal, comprising a processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method.

[0058] Example 4: A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method.

[0059] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0060] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0061] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0062] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for acoustic fingerprint fusion and knowledge distillation for insulator modeling evaluation, characterized in that, The method comprises the following steps: S1: Collecting insulator acoustic print signals and synchronous working condition data, and generating multi-dimensional acoustic print feature maps and working condition scalar feature vectors through preprocessing respectively; S2: Constructing an insulator health state evaluation teacher model; using the double-flow spatiotemporal feature aggregation architecture of the insulator health state evaluation teacher model, time-domain sequence-dependent features and frequency-domain morphological features of the multi-dimensional acoustic print feature maps are extracted respectively, and the working condition scalar feature vectors are mapped and expanded for multi-modal fusion to generate multi-modal fusion features; S3: Constructing a lightweight student model; using the multi-dimensional acoustic print feature maps and the working condition scalar feature vectors as inputs, under the condition of fixing the parameters of the insulator health state evaluation teacher model, the multi-modal fusion features are used to guide the knowledge distillation training of the lightweight student model; a multi-task joint loss function is calculated and the parameters of the lightweight student model are updated; the multi-task joint loss function includes a cross-view feature alignment loss calculated based on the basic acoustic print features and the enhanced acoustic print features extracted by the lightweight student model from the same sample after data enhancement; S4: Deploying the trained lightweight student model on a monitoring terminal and performing inference on real-time acoustic print data of the insulator to be tested to output an insulator health state evaluation result.

2. The acoustic fingerprint fusion and knowledge distillation insulator modeling and evaluation method according to claim 1, characterized in that, The insulator acoustic print signals include insulator normal operation background sound, partial discharge sound and mechanical vibration sound; the synchronous working condition data includes insulator operating voltage level, load current, environmental temperature and environmental humidity.

3. The acoustic fingerprint fusion and knowledge distillation insulator modeling evaluation method according to claim 2, characterized in that, The insulator acoustic print signals are preprocessed to generate multi-dimensional acoustic print feature maps, specifically including: For the same segment of insulator acoustic print signals collected, feature extraction operations are performed respectively, including: performing short-time Fourier transform on the insulator acoustic print signals, calculating power spectrum and mapping to mel scale to generate a mel spectrum map; using continuous wavelet transform to perform time-frequency analysis on the insulator acoustic print signals to extract non-stationary transient features and generate a wavelet time-frequency map; pre-emphasizing, framing and discrete cosine transforming the insulator acoustic print signals to extract cepstrum coefficients and generate an MFCC feature map; A combination of at least one or more of the mel spectrum map, the wavelet time-frequency map and the MFCC feature map constitutes the multi-dimensional acoustic print feature map.

4. The acoustic fingerprint fusion and knowledge distillation insulator modeling and evaluation method according to claim 1, characterized in that, Using the double-flow spatiotemporal feature aggregation architecture of the insulator health state evaluation teacher model, time-domain sequence-dependent features and frequency-domain morphological features of the multi-dimensional acoustic print feature maps are extracted respectively, specifically including: Constructing parallel time-domain feature extraction branches and frequency-domain feature extraction branches; The time-domain feature extraction branch is used to slice the multi-dimensional acoustic print feature maps in the time dimension, flatten the frequency dimension and channel dimension contained in the sliced multi-dimensional acoustic print feature maps and map feature sequences, input a bidirectional long short-term memory network, and capture time-domain sequence-dependent features of the insulator acoustic print; The frequency-domain feature extraction branch is used to process the multi-dimensional acoustic print feature maps using a deformable convolutional network, adaptively adjust the convolution sampling position by learning the sampling offset of the convolution kernel to cover irregular discharge frequency bands, and extract frequency-domain morphological features.

5. The acoustic fingerprint fusion and knowledge distillation insulator modeling evaluation method according to claim 4, characterized in that, The extended working condition scalar feature vector is fused with the time domain sequence dependent feature and the frequency domain morphological feature in a multi-modal manner, specifically including: The time domain sequence dependent feature and the frequency domain morphological feature are spliced to construct a voiceprint deep feature tensor; The working condition scalar feature vector is mapped to a channel dimension consistent with the voiceprint deep feature tensor by using a full connection layer; The mapped working condition scalar feature vector is copied and expanded in the spatial dimension by using a broadcast mechanism to construct an expanded working condition feature with a spatial size consistent with the voiceprint deep feature tensor; The expanded working condition feature and the voiceprint deep feature tensor are spliced or added point by point to generate the multi-modal fusion feature.

6. The acoustic fingerprint fusion and knowledge distillation insulator modeling evaluation method according to claim 1, characterized in that, The multi-modal fusion feature is used to guide the knowledge distillation training of a lightweight student model, specifically including: The lightweight student model is constructed to have a lower number of convolutional layers or feature channel dimension than the insulator health state evaluation teacher model; The multi-dimensional voiceprint feature map and the working condition scalar feature vector are input into the lightweight student model to extract student deep features and student classification probabilities; Feature-level distillation is performed, including calculating a feature matching loss, and constraining the student deep features by minimizing the Euclidean distance or cosine distance between the student deep features and the multi-modal fusion feature; Logical-level distillation is performed, including calculating a probability distribution divergence loss, and constraining the student classification probabilities by minimizing the relative entropy between the student classification probabilities and the classification probability distribution output by the insulator health state evaluation teacher model.

7. The acoustic fingerprint fusion and knowledge distillation insulator modeling evaluation method according to claim 6, characterized in that, The multi-task joint loss function is constructed by weighted summation of the classification loss, intra-class distance loss, knowledge distillation loss, and cross-view feature alignment loss, and the specific calculation includes: The classification loss is calculated using a focal loss function, including dynamically adjusting the loss weight according to the student classification probability to increase the training proportion of difficult classification samples; The intra-class distance loss is calculated using a center loss function, including calculating the Euclidean distance between the student deep features and the corresponding class feature center; The feature matching loss and the probability distribution divergence loss are obtained, and the knowledge distillation loss is obtained by weighted summation; The cross-view feature alignment loss is calculated using a cosine embedding loss function, including calculating the cosine similarity in the hidden space between the basic voiceprint features and the enhanced voiceprint features extracted by the lightweight student model for the same sample.

8. A voiceprint fusion and knowledge distillation insulator modeling evaluation system, running a voiceprint fusion and knowledge distillation insulator modeling evaluation method according to any one of claims 1-7, characterized in that, including: A data acquisition and preprocessing module for acquiring insulator voiceprint signals and synchronous working condition data, and generating multi-dimensional voiceprint feature maps and working condition scalar feature vectors after preprocessing; A teacher model construction and fusion module for constructing an insulator health state evaluation teacher model; using a double-flow spatio-temporal feature aggregation architecture of the insulator health state evaluation teacher model to extract time domain sequence dependent features and frequency domain morphological features of multi-dimensional voiceprint feature maps, and performing multi-modal fusion after mapping and expanding the working condition scalar feature vector to generate a multi-modal fusion feature; The student model distillation training module is configured to build a lightweight student model; under the condition that the parameters of the teacher model for insulator health state evaluation are fixed, multi-modal fusion features are used to guide the lightweight student model to perform knowledge distillation training; a multi-task joint loss function is calculated and the parameters of the lightweight student model are updated; the multi-task joint loss function includes a cross-view feature alignment loss calculated based on the basic voiceprint features and the enhanced voiceprint features extracted by the lightweight student model from the same sample respectively after data enhancement; The deployment and inference evaluation module is configured to deploy the trained lightweight student model to a monitoring terminal, and perform inference on real-time voiceprint data of the insulator to be tested, and output an insulator health state evaluation result.

9. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is configured to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-7.

10. A computer readable storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the steps of the method according to any one of claims 1-7.

Images