A transformer defect detection method based on multi-modal joint learning

Abstract

The application discloses a transformer defect detection method based on multi-modal joint learning, relates to the low defect detection classification accuracy of an oil-immersed transformer paper insulation part, and discloses the following technical scheme: a defect detection method based on multi-modal joint learning, wherein the method comprises the following steps: acquiring a time sequence of a transformer paper insulation part; acquiring a noise sequence of the transformer paper insulation part; performing feature extraction on the time sequence and the noise sequence; performing feature fusion on the time sequence and the noise sequence; and performing defect classification on the feature fusion result, wherein the feature extraction is performed by using a time sequence feature extraction method, and the feature fusion is performed by using a fine-grained bilinear fusion method.

Description

Technical Field

[0001] This invention relates to the detection of internal defects in paper insulation components of oil-immersed transformers. Background Technology

[0002] There are many types of transformers. Oil-immersed transformers, due to their excellent electrical and mechanical performance, are widely used as key equipment in high-voltage power grids. The internal insulation of oil-immersed transformers is primarily a combination of oil and paper insulation. During manufacturing, assembly, or operation, defects such as air gaps and metal particles may appear in the paper insulation components. These defects distort the electric field distribution within the insulation, triggering partial discharges, which in turn damage the transformer insulation and affect its service life. Research data indicates that a significant portion of transformer safety accidents are caused by insulation defects. Therefore, effective partial discharge detection of transformers is crucial. This can be achieved through the identification of partial discharge patterns using characteristic quantities of various partial discharge timing signals and deep neural networks, enabling timely fault detection and elimination before accidents occur.

[0003] In recent years, due to the significant advantages of deep learning technology in solving problems involving large datasets and complex modeling, an increasing number of researchers have favored using deep learning to address temporal defect detection, achieving results far exceeding those of traditional defect detection methods. While these methods have achieved good detection performance, they are all single-modal temporal defect detection methods, meaning they only consider the temporal features of the time series while ignoring other features, thus limiting further improvement in defect detection performance. To address the limitation of existing single-modal temporal defect detection methods that rely solely on temporal features while neglecting other modal features, thus hindering more effective feature extraction from multiple modal spaces, a multimodal temporal defect detection method is proposed. A noisy modality beneficial for temporal defect detection is designed and generated. By combining the temporal and noisy modalities and introducing a fine-grained bilinear fusion mechanism during the fusion process, more valuable features from both modalities are extracted more fully, thereby overcoming the shortcomings of single-modal temporal defect detection, improving defect detection performance, and increasing accuracy. Summary of the Invention

[0004] The purpose of this invention is to solve the problem of partial discharge and insulation degradation caused by internal defects in the paper insulation components of oil-immersed transformers, and to propose a multimodal joint learning method for transformer defect detection.

[0005] The aforementioned objectives are primarily achieved through the following technical solutions:

[0006] S1. Defect model fabrication and partial discharge signal acquisition and processing;

[0007] Based on the defect types that occur under actual engineering operating conditions, defect-free paperboard, paperboard with air gap defects, and paperboard with metal impurities were prepared for partial discharge testing. These tests correspond to simulated oil discharge caused by defect-free paper insulation components but unreasonable design or manufacturing, internal discharge caused by air gaps in the insulation components, and internal discharge caused by metal impurities in the insulation components, respectively. Following the current engineering method for preparing electrical insulation paperboard, unbleached softwood sulfate pulp of electrical grade was used as raw material. After pulping, loosening, weighing, and draining, the paperboard was divided into two parts while still wet. A polyester film was placed between the two wet paperboards, and the paperboard was pressed and dried. The polyester film was then removed to prepare paperboard samples with air gap defects. When making paperboard with metal impurities, the polyester film in the above process was replaced with metal sheets, and the metal sheets were left in the paperboard after pressing and drying.

[0008] Partial discharge tests were conducted on three sample models of the same size, and data were randomly collected from the discharge initiation time to the near breakdown time. The test was carried out in a partial discharge shielded room, and a partial discharge test system was built. The partial discharge test system mainly includes a voltage regulator, a transformer without partial discharge test, a protective resistor, a voltage divider, a coupling capacitor, and a detection impedance and acquisition system. Two-dimensional partial discharge spectra of defect-free cardboard, cardboard with air gaps, and cardboard with metal impurities were collected at different stages according to the above test method.

[0009] With the application of statistics in the fields of high voltage and insulation, partial discharge type identification based on discharge statistical spectra has become a current research hotspot. By extracting statistical features from various partial discharge spectra, feature parameters that can quantitatively describe the spectra morphology and its changes can be obtained, which can greatly improve the effect of partial discharge pattern recognition. Commonly used statistical parameters include skewness Sk, kurtosis Ku, asymmetry Asy, cross-correlation coefficient CC, and Weibull parameters α and β. Skewness and kurtosis represent the degree of skewness and distribution characteristics of the spectra, respectively, while asymmetry represents the asymmetry of the discharge quantity relative to the phase. The cross-correlation coefficient is used to characterize the similarity of the shape distribution of different types of spectra in the positive and negative half-cycles. The Weibull parameter α is the image scale parameter, and β is the image shape parameter. The calculation formulas for the above parameters are as follows.

[0010]

[0011] In the formula: x i Represents the phase value of the i-th discharge window; μ is the mean value in the phase spectrum; ρ i For x i The probability distribution; σ represents the standard deviation of the phase spectrum of the discharge quantity or number of discharges;

[0012]

[0013] In the formula: x i μ, ρ i The expression for σ is the same as above;

[0014]

[0015] In the formula: N1 and N2 represent the number of phase windows for the positive and negative half-cycles, respectively; and These represent the average discharge amplitude within the i-th phase window of the positive and negative half-cycles, respectively.

[0016]

[0017] In the formula: and These represent the average discharge amplitude within the i-th phase window during the positive and negative half-cycles, respectively; N represents the number of phase windows.

[0018]

[0019] In the formula: x is the number of discharges; x0 is the discharge quantity parameter.

[0020] S2. Prepare time series data and add noise to generate noise data;

[0021] The collected time-series data were processed by adding Gaussian noise distribution to obtain noise data; the Gaussian distribution formula is as follows:

[0022]

[0023] In the formula: μ is the mean, and σ is the standard deviation;

[0024] Finally, each sample is divided into a training set, a test set, and a validation set.

[0025] S3. Input the timing data into the timing signal encoder to extract the low-dimensional embedding representation of the timing mode;

[0026] The input is the original timing signal X = {X (1) ,X (2) ,...,X (n) ,X (t) ∈R 1×M The embedding representation H of the next layer is obtained through nonlinear transformation. C The operating procedure formula is as follows:

[0027]

[0028] In the formula: σ is the activation function LeakyReLU(·), and These represent the weights and biases in this layer, respectively. The weights are shared by all nodes, and the bias can improve the fitting ability of neurons; Z represents the output dimension of this convolutional layer; H C H is the output of this convolutional layer. C ∈R N×Z Then input it into the second convolutional layer. The operation process is as follows:

[0029]

[0030] In the formula: and These represent the weights and biases in this layer, respectively. D indicates that the output dimension BN(·) of this convolutional layer is batch normalized; This is the output of the convolutional layer. The input is then fed into the third convolutional layer, and the operation process is as follows:

[0031]

[0032] In the formula: and These represent the weights and biases in this layer, respectively. V is the output dimension of this convolutional layer; This is the output of the convolutional layer. It is then fed into the fourth convolutional layer, processed using the following formula:

[0033]

[0034] In the formula: and These represent the weights and biases in this layer, respectively. U represents the output dimension of the convolutional layer; This is the output of the convolutional layer. GAP is a global average pooling algorithm that outputs a specific dimension.

[0035] S4. Input the noise data into the noise signal encoder to extract the low-dimensional embedding representation of the noise mode;

[0036] The network structure of the noise signal encoder is the same as that of the time sequence signal encoder. Both are multi-layer one-dimensional convolutional neural networks. The only difference is that the input of the network is a noise signal, and the encoder extracts features from the noise signal. The noise signal is another mode of the time series obtained by transforming the original time sequence signal. The noise signal has features that are not in the original time sequence signal. Therefore, it is necessary to encode the noise signal and the time sequence signal separately to extract effective features.

[0037] S5. The effective features of different modalities are fully extracted through the fine-grained bilinear fusion module, and then the two modalities are fused after encoding.

[0038] The principle of fine-grained bilinear fusion is based on bilinear pooling, which can effectively capture the correlation and complementarity between two features. The core idea of ​​this method is to perform a tensor outer product of the two feature representations and then transform them into a fixed-dimensional feature vector through pooling. The resulting temporal feature vector A∈R is obtained first. M and noise feature vector B∈R N Where m and n represent their dimensions, respectively; performing a tensor outer product operation on the eigenvectors A and B yields a two-dimensional feature matrix C∈R. M×N The elements C of the characteristic matrix ij The feature matrix C is the product of the i-th element of feature vector A and the j-th element of feature vector B, representing the correlation between them; pooling is performed on the feature matrix C to transform it into a fixed-dimensional feature vector D∈R. D Common pooling operations include average pooling and max pooling. Average pooling takes the average value of the elements in the feature matrix, while max pooling takes the maximum value. The purpose of pooling operations is to compress and integrate the information in the feature matrix to obtain a feature vector of fixed length. Nonlinear transformations can be performed on the feature vector D to enhance the expressive power of the features.

[0039] S6. Construct a multimodal joint learning model framework and train a classification model using the data generated in S5;

[0040] The feature Z′ obtained by fusing S5 is input into the decoder, and the feature representation H of the next layer is obtained through one-dimensional deconvolution. T , where Z'={z' (1) ,z' (2) ,...,z' (N) ,z' (n) ∈R 1×M The operating procedure formula is as follows:

[0041]

[0042] In the formula: σ is the activation function LeakyReLU(·), and These are the weights and biases in the first deconvolutional layer, shared by all nodes. BN stands for batch normalization; Z is a hyperparameter representing the output dimension of the deconvolutional layer; H... T H is the output of this deconvolution layer. T ∈R N×Z Then, input it into the next deconvolution layer. The operation process is as follows:

[0043]

[0044] In the formula: and These represent the weights and biases in this layer, respectively. D is a hyperparameter representing the output dimension of the deconvolutional layer; This is the output of the convolutional layer. The input is then fed into the next deconvolutional layer, and the operation process is as follows:

[0045]

[0046] In the formula: and These are the weights and biases of the deconvolution layer. V is a hyperparameter representing the output dimension of the deconvolutional layer; This is the output of the deconvolution layer. The input is then fed into the next deconvolutional layer, and the operation process is as follows:

[0047]

[0048] In the formula: and These are the weights and biases of the current deconvolution layer. U is a hyperparameter representing the output dimension of the deconvolutional layer; This is the output of the current deconvolution layer. The final reconstructed temporal features are obtained by inputting them into the last linear layer, and the operation formula is as follows:

[0049]

[0050] In the formula: and For the weights and biases of the linear layer, X T′ For the reconstructed temporal features, L(·) represents the linear layer.

[0051] Invention Effects

[0052] The purpose of this invention is to address the problem of partial discharge and insulation degradation caused by internal defects in the paper insulation components of oil-immersed transformers, and to propose a transformer defect detection method based on multimodal joint learning. First, this invention proposes a multimodal joint learning method. Noise signals are generated by adding noise to the original data according to a Gaussian distribution. Then, based on the multimodal concept, it can simultaneously capture the cross-modal interaction of time-series features and noise features. Two encoder structures are used to jointly learn the time-series information and the noise information generated from the time-series information. The low-dimensional embeddings of the two modes are fused and used as the input of the decoder, realizing cross-modal interactive learning of the time-series and noise signals. Next, this invention proposes a feature fusion method, enabling the model to effectively capture the effective information of the two modes, achieving cross-modal interactive learning of the time-series and noise signals. Specifically, different modes have different features, which are helpful for time-series defect detection. To fully extract the effective features of different modes, a fine-grained bilinear fusion module is needed to fuse the two encoded modes and effectively extract the features of different modes, improving the defect detection accuracy. Experiments show that the present invention can accurately classify and detect internal defects in paper insulation components of oil-immersed transformers, and can be effectively applied to detection systems. Attached Figure Description

[0053] Figure 1 Here is the main flowchart of the algorithm;

[0054] Figure 2 This is a model structure diagram of a transformer defect detection method based on multimodal joint learning.

[0055] Specific implementation methods Detailed implementation method:

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] like Figure 1 As shown, the purpose of this paper is to address the problem of partial discharge and insulation degradation caused by internal defects in the paper insulation components of oil-immersed transformers. A transformer defect detection method based on multimodal joint learning is proposed, which mainly includes the following steps:

[0058] S1. Prepare time series data and add noise to generate noise data;

[0059] S2. Input the timing data into the timing signal encoder to extract the low-dimensional embedding representation of the timing mode;

[0060] S3. Input the noise data into the noise signal encoder to extract the low-dimensional embedding representation of the noise mode;

[0061] S4. The effective features of different modalities are fully extracted by the fine-grained bilinear fusion module, and then the two modalities are fused after encoding.

[0062] S5. Construct a multimodal joint learning model framework and train a classification model using the data generated in S4.

[0063] In this embodiment of the invention, time-series data containing defect type labels are first collected, and then noise is added to the collected time-series data to generate a noisy mode. Then, the invention proposes a fine-grained bilinear fusion module to fully extract the effective features of different modes and perform fusion of the two modes after encoding. Next, the invention constructs a multimodal joint learning model framework and trains a classification model using the data generated after fusion, thereby improving the accuracy of defect detection.

[0064] The embodiments of the present invention will be described in detail below:

[0065] like Figure 1 The implementation of the algorithm shown includes the following steps:

[0066] S1. Prepare time series data and add noise to generate noise data;

[0067] First, time series data containing defect type labels are collected. Then, noise is added to the collected time series data to generate noisy modes. Finally, each sample data is divided into training set, test set and validation set in a 6:2:2 ratio.

[0068] S2. Input the timing data into the timing signal encoder to extract the low-dimensional embedding representation of the timing mode;

[0069] The input is the original timing signal X = {X (1) ,X (2) ,...,X (n) ,X (t) ∈R 1×M The embedding representation H of the next layer is obtained through nonlinear transformation. C The operating procedure formula is as follows:

[0070]

[0071] In the formula: σ is the activation function LeakyReLU(·), and These represent the weights and biases in this layer, respectively. The weights are shared by all nodes, and the bias can improve the fitting ability of neurons; Z represents the output dimension of this convolutional layer; H C H is the output of this convolutional layer. C ∈R N×Z Then input it into the second convolutional layer. The operation process is as follows:

[0072]

[0073] In the formula: and These represent the weights and biases in this layer, respectively. D represents the output dimension of the convolutional layer, and BN(·) is batch normalization; This is the output of the convolutional layer. The input is then fed into the third convolutional layer, and the operation process is as follows:

[0074]

[0075] In the formula: and These represent the weights and biases in this layer, respectively. V is the output dimension of this convolutional layer; This is the output of the convolutional layer. It is then fed into the fourth convolutional layer, processed using the following formula:

[0076]

[0077] In the formula: and These represent the weights and biases in this layer, respectively. U represents the output dimension of the convolutional layer; This is the output of the convolutional layer. GAP is a global average pooling algorithm that outputs a specific dimension.

[0078] S3. Input the noise data into the noise signal encoder to extract the low-dimensional embedding representation of the noise mode;

[0079] The network structure of the noise signal encoder is the same as that of the time sequence signal encoder. Both are multi-layer one-dimensional convolutional neural networks. The only difference is that the input of the network is a noise signal, and the encoder extracts features from the noise signal. The noise signal is another mode of the time series obtained by transforming the original time sequence signal. The noise signal has features that are not in the original time sequence signal. Therefore, it is necessary to encode the noise signal and the time sequence signal separately to extract effective features.

[0080] S4. The effective features of different modalities are fully extracted by the fine-grained bilinear fusion module, and then the two modalities are fused after encoding.

[0081] The principle of fine-grained bilinear fusion is based on bilinear pooling, which can effectively capture the correlation and complementarity between two features. The core idea of ​​this method is to perform a tensor outer product of the two feature representations and then transform them into a fixed-dimensional feature vector through pooling. The resulting temporal feature vector A∈R is obtained first. M and noise feature vector B∈R N Where m and n represent their dimensions, respectively; performing a tensor outer product operation on the eigenvectors A and B yields a two-dimensional feature matrix C∈R. M×N The elements C of the characteristic matrix ij The feature matrix C is the product of the i-th element of feature vector A and the j-th element of feature vector B, representing the correlation between them; pooling is performed on the feature matrix C to transform it into a fixed-dimensional feature vector D∈R. D Common pooling operations include average pooling and max pooling. Average pooling takes the average value of the elements in the feature matrix, while max pooling takes the maximum value. The purpose of pooling operations is to compress and integrate the information in the feature matrix to obtain a feature vector of fixed length. Nonlinear transformations can be performed on the feature vector D to enhance the expressive power of the features.

[0082] S5. Construct a multimodal joint learning model framework and train a classification model using the data generated in S4;

[0083] In this embodiment, the model training parameters are selected as follows: batch size: 128; learning rate: 0.01;

[0084] The feature Z′ obtained by fusing S4 is input into the decoder, and the feature representation H of the next layer is obtained through one-dimensional deconvolution. T , where Z'={z' (1) ,z' (2) ,...,z' (N), z' (n) ∈R 1×M The operating procedure formula is as follows:

[0085]

[0086] In the formula: σ is the activation function LeakyReLU(·), and These are the weights and biases in the first deconvolutional layer, shared by all nodes. BN stands for batch normalization; Z is a hyperparameter representing the output dimension of the deconvolutional layer; H... T H is the output of this deconvolution layer. T ∈R N×Z Then, input it into the next deconvolution layer. The operation process is as follows:

[0087]

[0088] In the formula: and These represent the weights and biases in this layer, respectively. D is a hyperparameter representing the output dimension of the deconvolutional layer; This is the output of the convolutional layer. The input is then fed into the next deconvolutional layer, and the operation process is as follows:

[0089]

[0090] In the formula: and These are the weights and biases of the deconvolution layer. V is a hyperparameter representing the output dimension of the deconvolutional layer; This is the output of the deconvolution layer. The input is then fed into the next deconvolutional layer, and the operation process is as follows:

[0091]

[0092] In the formula: and These are the weights and biases of the current deconvolution layer. U is a hyperparameter representing the output dimension of the deconvolutional layer; This is the output of the current deconvolution layer. The final reconstructed temporal features are obtained by inputting them into the last linear layer, and the operation formula is as follows:

[0093]

[0094] In the formula: and For the weights and biases of the linear layer, X T′ For the reconstructed temporal features, L(·) represents the linear layer.

[0095] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the scope of this invention.

Claims

1. A transformer defect detection method based on multi-modal joint learning, characterized in that, Includes the following steps: S1. Defect model fabrication and partial discharge signal acquisition and processing; S2. Prepare time series data and add noise to generate noise data; S3. Input the timing data into the timing signal encoder to extract the low-dimensional embedding representation of the timing mode; S4. Input the noise data into the noise signal encoder to extract the low-dimensional embedding representation of the noise mode; S5. The effective features of different modalities are fully extracted through the fine-grained bilinear fusion module, and then the two modalities are fused after encoding. S6. Construct a multimodal joint learning model framework and train a classification model using the data generated in S5; The method for fully extracting effective features from different modalities using a fine-grained bilinear fusion module in step S5, followed by encoding and then fusing the two modalities, is as follows: The principle of fine-grained bilinear fusion is based on a bilinear pooling operation, which can effectively capture the correlation and complementarity between two features; the core idea of the method is to perform tensor outer product on two feature representations, and convert them into fixed-dimensional feature vectors through a pooling operation; first, the time sequence feature vector A∈R M and the noise feature vector B∈R N , where m and n represent their dimensions, respectively; The tensor outer product operation is performed on the feature vectors A and B to obtain a two-dimensional feature matrix C ∈ R M×N ; the elements C ij of the feature matrix are the products of the i-th element of the feature vector A and the j-th element of the feature vector B, representing the correlation between them; the feature matrix C is subjected to a pooling operation to convert it into a fixed-dimensional feature vector D ∈ R D ; the purpose of the pooling operation is to compress and integrate the information in the feature matrix to obtain a fixed-length feature vector; A nonlinear transformation is applied to the feature vector D to enhance the expressive power of the features.

2. The transformer defect detection method based on multi-modal joint learning of claim 1, wherein, The specific steps for creating the defect model and acquiring and processing partial discharge signals as described in step S1 are as follows: Based on the defect types that occur under actual engineering operating conditions, defect-free paperboard, paperboard with air gap defects, and paperboard with metal impurities were prepared for partial discharge testing. These tests correspond to simulated oil discharge caused by defect-free paper insulation components but unreasonable design or manufacturing, internal discharge caused by air gaps in the insulation components, and internal discharge caused by metal impurities in the insulation components, respectively. Following the current engineering method for preparing electrical insulation paperboard, unbleached softwood sulfate pulp of electrical grade was used as raw material. After pulping, loosening, weighing, and draining, the paperboard was divided into two parts while still wet. A polyester film was placed between the two wet paperboards, and the paperboard was pressed and dried. The polyester film was then removed to prepare paperboard samples with air gap defects. When making paperboard with metal impurities, the polyester film in the above process was replaced with metal sheets, and the metal sheets were left in the paperboard after pressing and drying. Partial discharge tests were conducted on three sample models of the same size, and data were randomly collected from the moment of discharge onset to the moment of near breakdown. The test was carried out in a partial discharge shielded room, and a partial discharge test system was built. The partial discharge test system mainly includes a voltage regulator, a transformer without partial discharge test, a protective resistor, a voltage divider, a coupling capacitor, and a detection impedance and acquisition system. According to the above test method, partial discharge two-dimensional spectra of defect-free cardboard, cardboard with air gaps, and cardboard with defects containing metal impurities were collected at different stages. By extracting statistical features from various partial discharge spectra, feature parameters that can quantitatively describe the morphology and changes of the spectra are obtained. Commonly used statistical parameters include skewness Sk, kurtosis Ku, asymmetry Asy, cross-correlation coefficient CC, and Weibull parameters α and β. Skewness and kurtosis represent the degree of skewness and distribution characteristics of the spectra, respectively, while asymmetry represents the asymmetry of the discharge quantity relative to the phase. The cross-correlation coefficient is used to characterize the similarity of the shape distribution of different types of spectra in the positive and negative half-cycles. The Weibull parameter α is the scale parameter of the image, and β is the shape parameter of the image. The calculation formulas for the above parameters are as follows: wherein: x i represents the phase value of the ith discharge window; μ is the mean value in the phase profile; ρ i is the distribution probability of x i ; σ represents the standard deviation of the discharge amount or discharge frequency phase profile; wherein: x i , μ, p i , σ are as above. In the formula, N1 and N2 respectively represent the phase window number of the positive and negative half cycles; and respectively represent the average discharge amplitude in the i th phase window of the positive and negative half cycles. wherein: and respectively represent the average discharge amplitude in the i-th phase window within the positive and negative half cycle. N represents the number of phase windows; In the formula: x represents the number of discharges; x0 represents the discharge quantity parameter.

3. The transformer defect detection method based on multi-modal joint learning of claim 1, wherein, The method for preparing time-series data and generating noise data by adding noise as described in step S2 is as follows: The collected time-series data were processed by adding Gaussian noise distribution to obtain noise data. The Gaussian distribution formula is as follows: In the formula: μ is the mean, and σ is the standard deviation; Finally, each sample is divided into a training set, a test set, and a validation set.

4. The transformer defect detection method based on multi-modal joint learning of claim 1, wherein, The method for extracting low-dimensional embedding representations of timing modes by inputting timing data into a timing signal encoder in step S3 is as follows: The input is the original time series signal X = {X (1) , X (2) ,...,X (n) , X (t) ∈ R 1×M}; The embedding representation H C of the next layer is obtained by a nonlinear transformation, and the operation process is as follows: where σ is the activation function LeakyReLU(·), and are the weights and biases in this layer, respectively, where the weights are shared by all nodes, and the biases can improve the fitting ability of neurons; Z represents the output dimension of this convolutional layer; H C is the output of this convolutional layer, H C ∈R N×Z , which is input to the second convolutional layer, and the operation process is as follows: wherein: and are the weights and biases in the layer, respectively, D denotes the output dimension of the convolutional layer BN(·) is batch normalization; is the output of the convolutional layer, which is input to a third convolutional layer with the following operations: wherein: and are the weights and bias in the layer, respectively, V is the output dimension of the convolutional layer; is the output of the convolutional layer, which is input to a fourth convolutional layer, with the following processing formula: wherein: and are the weights and biases in the layer, respectively, U denotes the output dimension of the convolutional layer; is the output of the convolutional layer, GAP is a global average pooling that outputs a specific dimension.

5. The transformer defect detection method based on multi-modal joint learning of claim 1, wherein, The method for extracting the low-dimensional embedding representation of noise modes by inputting noise data into the noise signal encoder in step S4 is as follows: The network structure of the noise signal encoder is the same as that of the time sequence signal encoder. Both are multi-layer one-dimensional convolutional neural networks. The only difference is that the input of the network is a noise signal, and the encoder extracts features from the noise signal. The noise signal is another mode of the time series obtained by transforming the original time sequence signal. The noise signal has features that are not in the original time sequence signal. Therefore, it is necessary to encode the noise signal and the time sequence signal separately to extract effective features.

6. The transformer defect detection method based on multi-modal joint learning of claim 1, wherein, The method for constructing the multimodal joint learning model framework in step S6 and training the classification model using the data generated in S5 is as follows: The fused feature Z' of S5 is input into a decoder to obtain a feature representation H of the next layer by one-dimensional deconvolution T where Z' = {z' (1) ,z' (2) ,...,z' (N) ,z' (n) ∈ R 1×M The operation process is as follows: where σ is the activation function LeakyReLU(·), and are the weights and bias of the first deconvolutional layer, shared by all the nodes, BN is batch normalization; Z is a hyper-parameter, representing the output dimension of this deconvolutional layer; H T is the output of this deconvolutional layer, H T ∈R N×Z , which is input to the next deconvolutional layer, with the following operation formula: wherein: and are the weights and biases in this layer, respectively, D is a hyperparameter, representing the output dimension of this deconvolutional layer; is the output of this convolutional layer, which is input to the next deconvolutional layer, with the following operations: wherein: and are weights and biases of the deconvolutional layer, V is a hyperparameter, representing the output dimension of the deconvolutional layer; is the output of the deconvolutional layer, which is input to the next deconvolutional layer, the operation process being formulated as follows: wherein: and are the weights and bias of the current deconvolutional layer, U is a hyperparameter, representing the output dimension of this deconvolutional layer; is the output of the current deconvolutional layer, which is input into the last linear layer to get the final reconstructed temporal feature, the operation formula is as follows: where: and are weights and biases of linear layers, X T' are reconstructed temporal features, and L(·) denotes a linear layer.

Images