A Small Sample Fault Diagnosis Method for Voltage Transformers Driven by Multimodal Prototype

Abstract

This invention discloses a multimodal prototype-driven method for small-sample fault diagnosis of voltage transformers. First, the operating time-series signal of the voltage transformer is acquired, and a time-series representation is obtained through normalization, segmentation, and feature encoding. Second, textual semantic features are obtained. Then, with the time-series features as the main driver, cross-modal attention is used to achieve collaborative interaction between visual and textual features, generating a multimodal enhanced representation. A time-series category prototype library and a multi-view image category prototype library are constructed respectively, and prototype classification scores are obtained based on similarity matching. Finally, an adaptive gating network is used to dynamically fuse the time-series feature classification, multimodal enhanced classification, image prototype classification, and time-series prototype classification results to output the final fault diagnosis result. This invention can fully utilize the complementary information of time-series, image, and text modalities, improving the fault identification accuracy, generalization ability, and diagnostic robustness of voltage transformers under small-sample conditions.

Description

Technical Field

[0001] This invention belongs to the fields of power equipment condition monitoring, intelligent fault diagnosis, artificial intelligence, multimodal learning and few-shot learning, and specifically relates to a multimodal prototype-driven method for few-shot fault diagnosis of voltage transformers. Background Technology

[0002] Voltage transformers are crucial devices in power systems, responsible for voltage measurement, metering, and relay protection signal acquisition. Their operational status directly impacts the accuracy of energy metering, the reliability of protection actions, and the safe and stable operation of the power system. With the development of smart grids and digital substations, voltage transformer condition monitoring and fault diagnosis based on operational signals are gradually becoming important technical means to improve equipment operation and maintenance.

[0003] In actual operation, the output signal of a voltage transformer is usually dominated by the power frequency component, and includes harmonics, frequency divisions, waveform distortions, and local disturbances caused by factors such as changes in structural parameters, insulation degradation, core nonlinearity, and external electromagnetic interference. This type of signal has non-stationary, nonlinear, and multi-scale characteristics, making it difficult for diagnostic methods that rely solely on a single time-series mode to fully characterize the waveform changes, frequency domain structure, and dynamic evolution under different fault conditions.

[0004] Existing deep learning diagnostic methods typically rely on large-scale labeled samples to obtain fault discrimination models through end-to-end training. However, obtaining fault samples for voltage transformers is difficult, actual fault occurrence frequency is low, experimental data collection is costly, and signal distribution is prone to change under different operating conditions. Therefore, in engineering applications, these methods often face problems such as small sample training, blurred class boundaries, and insufficient model generalization ability. Under conditions of limited sample size, the model is prone to overfitting the collected samples, making it difficult to form stable discriminations for new operating conditions or similar fault types.

[0005] Multimodal visual language models, through large-scale pre-training with text and images, develop rich cross-modal representation capabilities, providing additional visual and semantic priors for time-series fault diagnosis. After converting voltage transformer time-series signals into image forms such as curves, spectrograms, and recursion graphs, transferable visual features can be extracted using visual language models. Simultaneously, textual prompts constructed using fault diagnosis task background and sample statistical information can provide high-level semantic constraints for the model.

[0006] However, directly transferring visual language models to the task of small-sample fault diagnosis of voltage transformers may still result in problems such as insufficient intermodal information interaction, incomplete consistency between pre-trained semantics and equipment signal semantics, insufficient utilization of prior class structure, and difficulty in effectively fusing multi-branch discrimination results. Therefore, there is an urgent need to propose a voltage transformer fault diagnosis method that can simultaneously utilize time series, image, and text modal information and is applicable to small-sample conditions, in order to improve the accuracy, robustness, and engineering adaptability of fault diagnosis.

[0007] Comparison with patent CN116842459A, "A method and terminal for diagnosing power metering faults based on few-sample learning":

[0008] Patent CN116842459A primarily addresses the issues of data missingness, anomaly repair, and small-sample fault diagnosis in electricity metering scenarios. It utilizes correlation analysis, Pearson correlation coefficient matrix, entropy weighting method, and distance metrics based on Siamese networks to achieve abnormal data repair and fault classification in electricity metering. The patent's key technology lies in leveraging the correlations between multi-dimensional electrical parameters in electricity metering to fill in abnormal or missing data, and constructing a small-sample classification model based on LSTM, CNN, and Siamese networks. Meanwhile, this invention addresses the small-sample fault diagnosis task of voltage transformer operating signals, focusing on solving the problems of insufficient representation of fault features by single time-series modes, semantic shifts easily generated by direct transfer of pre-trained visual language models, and blurred category boundaries under small-sample conditions. This invention does not rely solely on electrical parameter correlation and inter-sample distance measurement for classification. Instead, it integrates time-series signals, multi-view images derived from signal conversion, and textual semantic prompts into a multimodal collaborative representation framework. Furthermore, it constructs a time-series category prototype library and an image category prototype library, and achieves fault category discrimination through multi-branch adaptive gating fusion, thereby enhancing the feature representation capability and generalization performance of small-sample fault diagnosis of voltage transformers.

[0009] Technical comparison with patent CN108614548A, "An intelligent fault diagnosis method based on multimodal fusion deep learning":

[0010] Patent CN108614548A discloses a multimodal fusion deep learning fault diagnosis method for industrial equipment. It extracts fault features from both structured and unstructured data, fuses these different fault features, and finally uses a Softmax classifier to predict the health status and diagnose faults in industrial equipment. The key technical focus of this patent is feature extraction and fusion for multi-source heterogeneous industrial data, exhibiting strong general-purpose fault diagnosis attributes for industrial equipment. However, this invention addresses the specific signal morphology and sample limitations in small-sample fault diagnosis of voltage transformers, rather than simply performing parallel feature extraction on structured and unstructured data and then directly fusing them. This invention first constructs time-series, image, and text branches based on the one-dimensional operating signal of a voltage transformer. The image branch further includes multi-view representations such as curves, spectrograms, and recursive graphs, while the text branch introduces task background and sample statistical semantic information. Second, this invention introduces cross-modal priors through a pre-trained visual language model and constrains the central representation of various faults in different modal feature spaces through a category prototype retrieval mechanism. Finally, this invention employs a multi-branch adaptive gating strategy to dynamically weight and fuse the results of time-series feature classification, time-series prototype classification, image prototype classification, and multi-modal enhanced classification. Therefore, compared to the general multi-modal feature fusion framework of CN108614548A, this invention places greater emphasis on small-sample prototype modeling, multi-view visual semantic enhancement, and sample-level adaptive decision fusion for voltage transformer fault signals.

[0011] Technical comparison with patent CN117491935A "Evaluation method for operating status of voltage transformer based on multi-source data":

[0012] Patent CN117491935A primarily addresses the online operational status assessment of voltage transformers. It acquires multi-source features such as inter-phase residual components, inter-group residual components, individual residual components, inter-phase amplitude ratio, three-phase imbalance of a group, and individual voltage amplitude, and combines these with GASF transform, Transformer encoding, and a Naive Bayes classifier to complete the voltage transformer status assessment. The key technical focus of this patent is utilizing multi-source residual information and amplitude statistics to perform online assessment and early warning of the voltage transformer's operational status. In contrast, this invention addresses the task of identifying voltage transformer fault types with a small sample size, focusing on the ability to distinguish fault categories under conditions of limited sample quantity. Unlike CN117491935A, which primarily relies on residual component transformation, Transformer feature extraction, and state level assessment, this invention constructs the same voltage transformer fault signal into time series representation, curve graph, spectrogram, recursion graph, and textual semantic cues. It utilizes cross-modal priors from a visual language pre-trained model to enhance fault feature representation and establishes category prototype libraries in both the time series feature space and image feature space, using prototype similarity matching to enhance the stability of small-sample category discrimination. Furthermore, this invention dynamically fuses multiple discrimination paths through an adaptive gating mechanism, adjusting decision weights based on the response differences of different samples across modalities and classification branches, thereby improving the robustness of small-sample fault diagnosis.

[0013] Technical comparison with patent CN116702612A "A CVT Health Status Prediction Method and System Based on Multidimensional Index Fusion":

[0014] Patent CN116702612A discloses a multi-dimensional index fusion method and system for predicting the health status of CVTs. It constructs accuracy, reliability, and stability indices using information such as historical error sequences of current transformers, time-series sequences of trend indicators, time-series sequences of rise and fall trends, secondary voltage data, and stable state data, and obtains the CVT health status through a fusion model. The key technical point of this patent is to comprehensively evaluate and predict the CVT health status from multiple index dimensions, including accuracy, reliability, and stability. However, this invention does not focus on weighted evaluation of CVT health status, but rather on small-sample fault diagnosis of voltage transformer fault signals. It addresses the problems of scarce fault samples, insufficient single-modal feature representation, and insufficient transfer stability of pre-trained visual language models in professional fault diagnosis scenarios. This invention extracts local waveform changes and global dynamic dependencies through time-series encoding, introduces time-domain contours, frequency-domain energy distribution, and dynamic reproduction patterns through multi-view image representation, and introduces task semantics and sample statistical information through text prompts. Based on this, it constructs a multi-modal enhanced representation and a multi-modal category prototype retrieval mechanism. Compared to CN116702612A, which focuses on health status indicator system and trend prediction, this invention focuses more on fine-grained identification at the fault category level. It can improve the accuracy of voltage transformer fault diagnosis by using multimodal prototype constraints and gating fusion strategies under the condition of a small number of labeled samples. Summary of the Invention

[0015] To address the technical challenges of existing voltage transformer fault diagnosis methods, such as insufficient feature representation, ambiguous category boundaries, inadequate utilization of cross-modal priors, and insufficient robustness of diagnostic results under conditions of limited sample size, high annotation costs, and complex and variable operating conditions, this invention proposes a multi-modal prototype-driven small-sample fault diagnosis method for voltage transformers. This method, centered on category prototype modeling, integrates time-series features, image visual features, and textual semantic features into a unified diagnostic framework. Furthermore, it achieves collaborative decision-making across multiple discriminative paths through cross-modal interaction and adaptive gating fusion.

[0016] To achieve the above objectives, the technical solution adopted by this invention is: a multi-modal prototype-driven method for small-sample fault diagnosis of voltage transformers, the specific steps of which are as follows:

[0017] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0018] A perceptual domain open-vocabulary semantic segmentation method based on region consistency and fusion attention, the specific steps of which are as follows:

[0019] (1) Signal acquisition and preprocessing. Data of the voltage transformer under five operating states are acquired. Fault states include normal state, voltage divider capacitor gradual change fault, compensation reactor parameter drift fault, insulation aging fault and intermediate transformer core saturation fault. Necessary noise reduction, normalization and sliding window sample division are performed on the time series signal to obtain the input sample to be diagnosed.

[0020] (2) Multimodal collaborative representation. The input samples to be diagnosed are input into the time series modality branch, the image modality branch, and the text modality branch, respectively. The time series modality branch obtains sample-level time series features through fragment encoding and self-attention modeling; the image modality branch converts the same time series signal into curves, spectrograms, and recursive graphs, and extracts multi-view visual features through a pre-trained visual language model; the text modality branch constructs text prompts based on task background and sample statistical information, and extracts text semantic features through a pre-trained visual language model. Subsequently, with time series features as the main component, cross-modal attention interaction is performed with image features and text features to obtain multimodal enhanced representation.

[0021] (3) Multimodal category prototype retrieval. Based on a small sample support set, a time series category prototype library is constructed in the time series feature space, and a multi-view image category prototype library is constructed in the image feature space. For the input sample to be diagnosed, the similarity between its time series features and time series category prototypes, and the similarity between its image query features and image category prototypes are calculated respectively to obtain the time series prototype classification score and the image prototype classification score.

[0022] (4) Multi-branch adaptive gating fusion. The classification scores of time series features, multimodal enhancement, image prototype, and time series prototype are concatenated and input into a lightweight gating network to dynamically generate the fusion weights corresponding to the four branches, and the classification scores of each branch are weighted and fused.

[0023] (5) Fault Category Output. Based on the weighted fusion category scores, the fault category corresponding to the input sample to be diagnosed is determined, and the fault diagnosis results of the small sample of voltage transformer are output.

[0024] The benefits of this application are:

[0025] 1. This invention, through the coordinated representation of three modalities—time series, image, and text—can characterize the fault features of voltage transformers from multiple perspectives, such as local waveform changes, frequency domain energy distribution, dynamic state reproduction patterns, and high-level semantic priors.

[0026] 2. Introducing category-level structural priors through time series and image prototype libraries can enhance intra-class cohesion and reduce inter-class confusion under small sample conditions;

[0027] 3. By using an adaptive gating fusion mechanism to dynamically adjust the contribution of each branch based on the multi-branch response of the current sample, the imbalance of discrimination information caused by fixed-weight fusion can be avoided, thereby improving the accuracy, generalization ability and robustness of small-sample fault diagnosis of voltage transformers. Attached Figure Description

[0028] Figure 1 This is a flowchart of the method disclosed in this invention;

[0029] Figure 2 This is a schematic diagram of the image category retrieval module. Detailed Implementation

[0030] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0031] This invention discloses a multi-modal prototype-driven method for small-sample fault diagnosis of voltage transformers, the overall process of which is as follows: Figure 1 As shown, it mainly includes five steps: signal acquisition and preprocessing, multimodal collaborative representation, multimodal category prototype retrieval, multi-branch adaptive gating fusion, and fault category output.

[0032] Step (1) Signal Acquisition and Preprocessing is as follows: Acquire the operating time series signals of the voltage transformer under normal conditions and different fault conditions. The fault conditions may include voltage divider capacitor gradual change fault, compensation reactor parameter drift fault, insulation aging fault, and intermediate transformer core saturation fault, etc. The original time series sample length corresponding to each fault type is 3.6×10 7 The dataset was divided into three sampling points. A sliding window strategy was used to partition the original long sequence. Specifically, the sliding window length was set to 1024, and the sliding step size was 512, resulting in 70,300 samples for each fault type. Subsequently, to ensure the rationality and independence of the model training, parameter tuning, and performance evaluation processes, the dataset was divided into training, validation, and test sets in an 8:1:1 ratio. The training set was used for model parameter learning, the validation set was used for model hyperparameter tuning and training process monitoring, and the test set was used for final performance evaluation. Data preprocessing and fault category settings are shown in Table 1.

[0033] Table 1 Dataset Preprocessing Settings

[0034]

[0035] Step (2) Multimodal collaborative characterization is as follows: Figure 1As shown, a multimodal collaborative representation network is constructed, consisting of a time-series modal branch, an image modal branch, and a text modal branch. This network uses the time-series signal from the same voltage transformer as a unified information source, extracting complementary features from three levels: temporal structure, visual pattern, and semantic prior. Through cross-modal interaction, it forms a multimodal enhanced representation for subsequent classification. Let the input sample be denoted as:

[0036] ;

[0037] in, Let represent the observation value at time i.

[0038] 1) Time series modal branching

[0039] First, the input sequence is normalized to obtain a standardized input representation. :

[0040] ;

[0041] Time series branching aims to extract discriminative sample-level time series features from the input signal to characterize the local waveform variation patterns and global dynamic dependencies in the voltage transformer operating signal. The normalized input sequence is divided into several local segments of length p. Let the sliding step size be s, then the total number of segments is... It can be represented as:

[0042] ;

[0043] Let the m-th local segment be ? Then, after patch embedding and mapping, a patch-level representation is obtained:

[0044] ;

[0045] in, and Let I represent the weight matrix and bias vector of the patch embedding layer, respectively. By concatenating the I patch-level representations, we can obtain the patch-level representation matrix. :

[0046] ;

[0047] in, The embedding dimension represents the time series features. After obtaining the patch-level representation matrix P, a multi-head self-attention mechanism is further employed to model the contextual dependencies between local segments. For the h-th attention head, its query, key, and value vectors are defined as follows:

[0048] ;

[0049] in, , and Let and represent the linear mapping matrices corresponding to the h-th attention head. The corresponding single-head attention output can be written as:

[0050] ;

[0051] After concatenating and linearly mapping the outputs of all attention heads, the multi-head self-attention result is obtained:

[0052] ;

[0053] Where H represents the number of attention heads, This represents the output projection matrix. Further combining this with residual connections yields the context-enhanced patch representation:

[0054] ;

[0055] Based on this, the enhanced patch features are flattened and linearly mapped to obtain a sample-level time series representation. :

[0056]

[0057] in, This indicates a flattening operation, from which the following is obtained It is used not only for subsequent time series prototype modeling and pure time series classification, but also as the dominant query vector in cross-modal interactions. Furthermore, it is used as the primary feature input to the multilayer perceptron classification head to calculate the time series feature classification score. :

[0058] .

[0059] 2) Image modal branching

[0060] The role of image modal branching is to map one-dimensional time-series signals into two-dimensional visual representations, so as to make full use of the pre-trained knowledge of the visual language model in the visual representation space. Considering that a single image view is difficult to fully represent the multi-level structure of voltage transformer fault signals, this paper constructs three complementary image representations for the same input sample, namely curves. Spectrum diagram and recursion graph , Used to characterize the waveform profile features of a signal in the time domain. Used to characterize its frequency domain energy distribution structure. This is used to describe the state reproduction pattern during the dynamic evolution of the system.

[0061] Next, the three types of images are input into the visual encoder to extract the corresponding visual embeddings. , and :

[0062]

[0063]

[0064]

[0065] To obtain a unified visual representation, this paper performs linear mapping aggregation on three types of visual embeddings, thus achieving a unified visual representation. It can be represented as:

[0066] ;

[0067] From this obtained It can serve as a unified visual representation of image branches and also provide basic visual information for subsequent image prototype retrieval and multimodal enhancement.

[0068] 3) Text Branches

[0069] The text branch aims to enhance the model's ability to understand the high-level semantics of the voltage transformer's operating state by incorporating task background knowledge, prior information about the dataset, and statistical properties of the input samples through natural language descriptions.

[0070] First, this paper presents the unnormalized original input sequence. The statistical measures reflecting the overall amplitude distribution characteristics of the sample are extracted. Considering that the amplitude range and center position of the voltage transformer's operating signal can characterize its operating state differences to a certain extent, this paper selects the maximum, minimum, and median values ​​to construct the statistical description vector. Its definition is:

[0071] ;

[0072] The statistics will then be used to construct text prompts, represented as follows:

[0073] ;

[0074] Finally, the text feature representation is obtained. :

[0075]

[0076] 4) Cross-modal attention and interaction layer

[0077] In obtaining time series representation Visual representation and text representation Subsequently, this paper further constructs a cross-modal attention and interaction module to achieve deep interaction and enhanced discrimination between multimodal information. The architecture of the cross-modal attention and interaction module is as follows: Figure 2 As shown.

[0078] First, the visual representation and the text representation are concatenated along the feature dimension and projected onto a unified fusion space through a linear mapping to obtain the joint image-text representation:

[0079]

[0080] Based on this, it is represented by time series. As a query vector, it is represented by a graph-text union. We construct cross-modal attention interactions using keys and values. Let the mapping forms for queries, keys, and values ​​be as follows:

[0081] ;

[0082] in , , Let be a linear transformation matrix. Then the single-head cross-modal attention output can be expressed as:

[0083] ;

[0084] If a multi-head attention mechanism is adopted, the enhanced multimodal collaborative representation can be expressed as:

[0085] ;

[0086] in ,at last, The multimodal augmented classification score is obtained through the multimodal classification head. , can be represented as:

[0087] .

[0088] Step (3) Multimodal category prototype retrieval is as follows: Based on multimodal collaborative representation, a multimodal category prototype retrieval module is introduced. This module maintains category prototype libraries in both the time series feature space and the image feature space, and uses the modal features of the sample to be diagnosed as the query vector to perform similarity matching with each category prototype, thereby obtaining the category retrieval score. Through category prototype retrieval, the model can explicitly utilize category-level structural priors under small sample conditions, enhance the aggregation degree of similar samples, and suppress confusion between similar fault categories.

[0089] In the image prototype branch, such as Figure 2 As shown, the original time series is first converted into multiple image views, such as a curve graph, a spectrogram, and a recursive graph, and then fed into a unified visual encoder for feature extraction, thereby obtaining the corresponding sample-level visual representations. Let the visual features corresponding to the three views be denoted as ... , and Then it can be expressed as:

[0090] ;

[0091] in, , and These represent the curve graph, spectrogram, and recursive graph in the visual feature space, respectively. In the category prototype retrieval stage, this paper establishes independent category prototype memories for each of the three image views. Let the set of samples of the nth class in the small sample training set be denoted as... The sample size is The category prototypes for the three image views are defined as follows: If there are N categories, the category prototype libraries corresponding to the curve graph, spectrum graph, and recursive graph can be represented as follows:

[0092]

[0093] ;

[0094]

[0095] in, , and These represent the class prototypes of the nth class in the feature spaces of the curve graph view, the spectrum graph view, and the recursive graph view, respectively. , and Let represent the feature representations of the i-th sample in the curve graph view, spectrogram view, and recursive graph view, respectively. Then the image category prototype can be represented as:

[0096] ;

[0097] in Then the similarity score between the input features and all image prototypes can be expressed as:

[0098] ;

[0099] in, This represents the image query features of the input sample. This represents the prototype-level similarity score of a sample to all image prototypes. This is the temperature coefficient for the image retrieval branch.

[0100] To map prototype-level scores to the category space, a label indicator matrix corresponding to the image prototype is further introduced. This yields the category score output by the image prototype branch:

[0101] ;

[0102] in Similar to the image prototype branch, the time series prototype branch maintains a time series prototype for each category in the time series feature space. The time series features obtained from the input samples after passing through the time series encoder are represented as follows: The corresponding time series category prototype library is Then the similarity score between the input feature and all time series prototypes can be expressed as:

[0103] ;

[0104] in, This represents the time series features of the input sample. Represents the time series category prototype library matrix. This represents the class similarity score between a sample and all time series prototypes. This is the temperature coefficient for the time series retrieval branch. It consists of the similarity score and the tag corresponding to the category prototype. The final classification score of the time series prototype branch can be obtained. , can be represented as:

[0105] .

[0106] Step (4) Multi-branch adaptive gating fusion is as follows: After obtaining the time series feature classification score, multimodal enhanced classification score, image prototype classification score, and time series prototype classification score, the category outputs of the four branches are concatenated along the feature dimension to construct a joint representation that reflects the current multi-path discrimination state of the sample. The joint representation is input into a lightweight gating network, and after nonlinear mapping and normalization, the fusion weights corresponding to the four branches are generated.

[0107] The proposed multimodal fault diagnosis network outputs classification scores for four branches: time series feature classification scores. Multimodal augmented classification scores Multi-image category prototype retrieval classification score Time series category retrieval classification score To characterize the overall discriminative state of the four branches on the current sample, the outputs of each branch are first concatenated along the feature dimension to construct a joint representation:

[0108] ;

[0109] in, This represents the joint representation after concatenating the scores of the four branch categories, where N is the number of categories. This representation comprehensively includes the response information of different branches to each category and can be used as the basis for generating gating weights. Based on this, this paper uses a lightweight gating network to map the joint representation and generate adaptive fusion weights corresponding to the four branches. The gating weight vector is defined as:

[0110] ;

[0111] in, This represents the gating weight vector for the four branches. and These are the weight matrix and bias vector of the first-level linear mapping, respectively. The dimension of the gated hidden layer; and Here, represents the weight matrix and bias vector of the second-layer linear mapping, respectively; σ represents the nonlinear activation function; Softmax is used to normalize the weights of the four branches into a probability distribution. Therefore, the gated weight vector can be further expressed as:

[0112] ;

[0113] in, This represents the fusion weight of the i-th branch on the current sample. Since the gating weight is determined by the outputs of the four branches of the current sample, this module can adaptively adjust the contribution ratio of each branch to the final decision based on the response strength of the sample on different discrimination paths.

[0114] Step (5) Fault category output is as follows: Based on the fusion weights generated by the gating network, the time series feature classification score, multimodal enhancement classification score, image prototype classification score, and time series prototype classification score are weighted and summed to obtain the final category score. The category corresponding to the largest final category score is taken as the fault category of the input sample to be diagnosed, and the corresponding diagnosis result is output.

[0115] After obtaining the gating weights, the outputs of the four branches are weighted and summed to obtain the final fused classification result. :

[0116] .

[0117] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any modifications, equivalent substitutions, or improvements made to the above embodiments based on the technical essence of the present invention should be included within the scope of protection claimed by the present invention.

Claims

1. A method for diagnosing small-sample faults in voltage transformers driven by multi-modal prototypes, characterized in that, The method includes the following steps: S1, signal acquisition and preprocessing; acquiring time series signals of voltage transformers under different operating states, dividing the time series signals into sliding windows to obtain input samples to be diagnosed; S2, multimodal collaborative characterization; Using the input sample to be diagnosed as a unified information source, time-series modal branches, image modal branches, and text modal branches are constructed respectively to obtain time-series features, visual features, and text semantic features. Cross-modal interaction is performed with time-series features as the main driver to obtain multimodal enhanced representations; S3, Multimodal category prototype retrieval; Based on the support samples of each fault category in the small sample training set, category prototype libraries are constructed in the time-series feature space and the image feature space respectively. The modal features corresponding to the input sample to be diagnosed are matched with the category prototype libraries to obtain time-series prototype classification scores and image prototype classification scores; S4, Multi-branch adaptive gating fusion; The time-series feature classification scores, multimodal enhanced classification scores, image prototype classification scores, and time-series prototype classification scores are concatenated and input into the gating network to generate adaptive fusion weights corresponding to each branch, and the classification scores of the four branches are weighted and fused. S5, Fault Category Output: Determine the voltage transformer fault category of the input sample to be diagnosed based on the weighted fusion classification results.

2. The method for small-sample fault diagnosis of voltage transformers driven by multi-mode prototypes according to claim 1, characterized in that: In step S1, the time series signal is the output signal of the secondary side of the voltage transformer. The fault categories include one of the following: normal state, voltage divider capacitor gradual change fault, compensation reactor parameter drift fault, insulation aging fault, and intermediate transformer core saturation fault. The sliding window division adopts the sliding window method to extract local samples from the long sequence in order to preserve the local waveform changes and dynamic evolution characteristics of the signal.

3. The method for small-sample fault diagnosis of voltage transformers driven by multi-modal prototypes according to claim 1, characterized in that: In step S2, the time series modal branch performs segmentation and embedding mapping on the normalized input samples to obtain segment-level representations, and uses a multi-head self-attention mechanism to model the contextual dependencies between different local segments. Furthermore, it obtains sample-level time series features through flattening and linear mapping.

4. The method for small-sample fault diagnosis of voltage transformers driven by multi-mode prototypes according to claim 1, characterized in that: In step S2, the image modality branch converts the same time series signal into three complementary image representations: curve graph, spectrogram, and recursive graph. The three complementary image representations are then input into the visual encoder of the pre-trained visual language model to extract multi-view visual embeddings, which are then aggregated through linear mapping to obtain unified visual features.

5. The method for small-sample fault diagnosis of voltage transformers driven by multi-mode prototypes according to claim 1, characterized in that: In step S2, the text modality branch constructs text prompts based on fault diagnosis task information, dataset background information, and input sample statistical information. The input sample statistical information includes one or more of the maximum, minimum, and median values ​​of the time series signal, and extracts text semantic features using a pre-trained visual language model text encoder.

6. The method for small-sample fault diagnosis of voltage transformers driven by multi-mode prototypes according to claim 1, characterized in that: In step S2, the cross-modal interaction includes concatenating visual features and textual semantic features and projecting them into a unified fusion space to obtain a joint image-text representation; using time series features as query vectors and the joint image-text representation as key vectors and value vectors, a multimodal enhanced representation is obtained through cross-modal attention calculation.

7. The method for small-sample fault diagnosis of voltage transformers driven by multi-mode prototypes according to claim 1, characterized in that: In step S3, the category prototype library in the image feature space includes a curve graph category prototype library, a spectrum graph category prototype library, and a recursive graph category prototype library. Each category prototype is obtained by aggregating the visual features of the corresponding fault category supporting samples. The image prototype classification score is determined by the cosine similarity between the image query features of the sample to be diagnosed and all image category prototypes, as well as the category label indication relationship. The category prototype library in the time series feature space includes time series category prototypes corresponding to each fault category. The time series category prototypes are obtained by aggregating the time series features of the supporting samples of the corresponding fault category. The sample to be diagnosed obtains the time series prototype classification score by matching the cosine similarity with each time series category prototype.

8. The method for small-sample fault diagnosis of voltage transformers driven by multi-mode prototypes according to claim 3, characterized in that: In step S4, the time series feature classification score is obtained by inputting the sample-level time series representation as the main feature into the multilayer perceptron classification head.

9. The method for small-sample fault diagnosis of voltage transformers driven by multi-mode prototypes according to claim 1, characterized in that: In step S4, the multimodal augmentation classification score is obtained from the multimodal augmentation representation through the multimodal classification head.

10. The method for small-sample fault diagnosis of voltage transformers driven by multi-mode prototypes according to claim 1, characterized in that: In step S4, the gated network is a lightweight multilayer perceptron network. Its input is a joint representation obtained by concatenating the category scores of the four branches. Its output is normalized to obtain the fusion weights of the four branches. The fusion weights change dynamically with the response state of the sample to be diagnosed on different discrimination paths.

Images