Power equipment fault diagnosis method based on time sequence-spectrogram-discrete data multi-modal fusion

Abstract

The invention relates to a power equipment fault diagnosis method based on time sequence-spectrogram-discrete data multi-modal fusion, and belongs to the field of power equipment fault diagnosis. A multi-modal feature fusion technology based on an attention mechanism is adopted, and a power equipment fault diagnosis model based on multi-modal information fusion is constructed. Compared with a traditional deep learning fault diagnosis method, the problem that a fault single-modal sample is insufficient in diagnosis model accuracy can be solved through multi-modal data fusion, and the universality and diagnosis precision of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the field of power equipment fault diagnosis, and in particular relates to a power equipment fault diagnosis method based on time series-spectrogram-discrete data multimodal fusion. Background Art

[0002] In recent years, although the online monitoring technology for transformers has developed rapidly, most of the current research has only carried out feature identification and fault analysis on a certain condition monitoring indicator of the transformer. Each online monitoring indicator can be called a modality, and there is a certain correlation between the modal data. Due to the complex structure of the transformer, the analysis results based on a single modal data are often too one-sided, and it is difficult to achieve accurate diagnosis and analysis. It is necessary to combine multi-source feature information and long-term operation and maintenance experience to accurately evaluate and judge the transformer status and fault type. However, when performing correlation analysis, simply treating different modal data equally or splicing and integrating all modal features cannot guarantee the validity of the analysis results. Therefore, it is necessary to study the multi-modal state information fusion and fault analysis reasoning methods suitable for transformers, learn more accurate complex data features through complementary lines between modalities, and improve the diagnosis accuracy and generalization ability of transformer faults. Summary of the invention

[0003] The purpose of the present invention is to solve the problem that single modal data cannot achieve accurate power equipment fault diagnosis, and to provide a power equipment fault diagnosis method based on time series-spectrogram-discrete data multimodal fusion.

[0004] To achieve the above object, the technical solution of the present invention is: a method for fault diagnosis of electric power equipment based on time series-spectrogram-discrete data multimodal fusion, comprising:

[0005] Construct a data fusion model based on the cross-modal attention mechanism to achieve multi-modal feature fusion;

[0006] Build a fault diagnosis model based on multimodal fusion to realize power equipment fault diagnosis.

[0007] In one embodiment of the present invention, before constructing a data fusion model based on a cross-modal attention mechanism, it is necessary to perform feature extraction on multimodal data using a corresponding feature extraction network.

[0008] In one embodiment of the present invention, before feature extraction is performed on the multimodal data, a preprocessing operation needs to be performed on the collected multimodal data to ensure that each modal data is at a uniform scale.

[0009] In one embodiment of the present invention, the multimodal data preprocessing operation is as follows:

[0010] Acoustic vibration data preprocessing: Discrete sampling is performed on the collected equipment acoustic vibration training signals and the acoustic vibration signals to be tested, and the sampled data is normalized and expressed in the form of Mel-level atlas, and the image size is fixedly scaled to 224×224;

[0011] Partial discharge spectrum data preprocessing: Preprocessing is performed on the standard fault data of partial discharge phase distribution PRPD, the actually collected annotated data and the unannotated data, including image resampling and normalization. The resampling adopts the bilinear interpolation method:

[0012]

[0013] Where f(X) is the pixel value at point X, Q 11 =(x1,y1),Q 21 =(x2,y1),Q 12 =(x1,y2),Q 22 = (x2, y2) are the coordinates of the four original image points around the target pixel (x, y). All images are adjusted to the same size by resampling. The resampled data is normalized by dividing the pixel value by 255 to scale the pixel value of the image to the range of [0, 1].

[0014] Numerical data preprocessing: For numerical data, it is scaled by a fixed random matrix to obtain a vector with a dimension of 224.

[0015] In one embodiment of the present invention, the multimodal data feature extraction method is as follows:

[0016] Send the Mel-spectrogram data into the VGG-Net network to obtain the extracted features of the acoustic vibration The partial discharge spectrum data is fed into the Resnet-50 network to obtain the partial discharge extraction features Send the vector of numerical data into a 3-layer fully connected network to obtain the extracted features of numerical data Where N v 、N st and N num They represent the number of samples of three types of multimodal data, and d represents the dimension of extracted features.

[0017] In one embodiment of the present invention, the specific process of constructing a data fusion model based on a cross-modal attention mechanism is as follows:

[0018] Given the audio (A), visual (V), and text (T) feature representations Xa, Xv, and Xt, the feature vectors of the audio, visual, and text modalities are concatenated with the joint feature representation J of all modalities and then passed through a fully connected layer:

[0019]

[0020] Where d = da + dv + dt represents the dimension of J, da, dv and dt are the dimensions of audio, visual and text features respectively, and FC represents the fully connected layer;

[0021] J is input into the joint cross-attention framework of each modality to focus on the feature representation of a single modality, and the cross-correlation relationship between J and Xa is obtained as the joint cross-correlation matrix Ca, which is given by

[0022]

[0023] Where Wja∈R da×d a learnable weight matrix representing the audio modality;

[0024] Similarly, the cross-correlation matrices of the other two modes Cv and Ct are obtained as follows:

[0025]

[0026] Where Wjv∈R dv×d , Wjt∈R dt×d Represent the learnable weight matrices of visual modality and textual modality respectively;

[0027] The joint cross-correlation matrix is ​​used to calculate the attention map of a single modality. For the audio modality, the joint correlation matrix Ca and the corresponding audio features Xa are combined using the learnable weight matrix Wca, and then the ReLU activation function is used to calculate the attention map Ha, which is given by

[0028] H a =ReLU(X a W ca C a ) (5)

[0029] Among them, W ca ∈R K×K represents the learnable weight matrix of the audio modality; similarly, the attention maps of the visual modality and textual modality are

[0030] H v =ReLU(X v W cv C v ) (6)

[0031] H t =ReLU(X t W ct C t ) (7)

[0032] Among them, W cv ∈R K×K , W ct ∈RK×K are the learnable weight matrices for the visual modality and textual modality respectively. Now, the attention map is used to calculate the attending features of the individual modalities as follows:

[0033] X att，a =H a W ha +X a (8)

[0034] X att，v =H v W hv +X v (9)

[0035] X att，t =H t W ht +X t (10)

[0036] Among them, W ha ∈R K×K , W hv ∈R K×K and W ht ∈R K×K Learnable weight matrices representing audio, visual, and textual modalities respectively;

[0037] To obtain a more refined feature representation, the participating features of each modality are again taken as input to their respective joint cross-modal attention modules, which are composed of

[0038]

[0039] Among them, W (l) ha ∈R K×K , W (l) ht ∈R K×K and W (l) ht ∈R K×K denote the learnable weight matrices for audio, visual, and text modalities, respectively, and l denotes the recursive step;

[0040] Connect the participating features of a single modality after l iterations to obtain the multimodal fusion feature representation X (l) att , as shown below:

[0041]

[0042] In one embodiment of the present invention, the specific process of constructing a fault diagnosis model based on multimodal fusion is as follows:

[0043] The multimodal fusion features are sent to the diagnosis module to diagnose equipment failures. The diagnosis loss L labeled The specific definition is:

[0044]

[0045] Among them, f c (·) represents the classifier network, L labeled is the cross entropy loss, and the stochastic gradient descent method is used to optimize the model training process. represents the input features of the neural network, Indicates the corresponding features The weight of .

[0046] In one embodiment of the present invention, the diagnosis module is a three-layer fully connected network, the number of units in each hidden layer is set to 1000-500-200 respectively, and the number of units in the final output layer is the category of the data.

[0047] The present invention also provides an electric power equipment fault diagnosis system based on time series-spectrum-discrete data multimodal fusion, characterized in that it includes a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0048] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0049] Compared with the prior art, the present invention has the following beneficial effects: based on the cross-modal attention technology and the multimodal feature fusion technology, the present invention proposes a multimodal data fusion module based on cross-modal attention, which solves the problem of inaccurate fault diagnosis caused by the traditional deep learning model only training single-modal fault samples, and constructs a multimodal power equipment fault diagnosis model using multimodal fusion data. After the model training is completed, the model can be used to perform fault diagnosis on data of multiple modes in actual tasks, which solves the problem that the traditional deep learning diagnosis model is difficult to perform fault diagnosis when a certain modality of data is missing, thereby enhancing the applicability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The present invention is a flow chart of the method for diagnosing faults of electric power equipment based on multimodal fusion of time series, spectrum and discrete data. DETAILED DESCRIPTION

[0051] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0052] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0053] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0054] The present invention provides a method for diagnosing faults of electric power equipment based on multimodal fusion of time series, spectrogram and discrete data, comprising:

[0055] Construct a data fusion model based on the cross-modal attention mechanism to achieve multi-modal feature fusion;

[0056] Build a fault diagnosis model based on multimodal fusion to realize power equipment fault diagnosis.

[0057] The following is the specific implementation process of the present invention.

[0058] like Figure 1 As shown, this embodiment provides a method for fault diagnosis of electric power equipment based on multimodal fusion of time series-spectrogram-discrete data, and the technical route is as follows: First, the multimodal signals actually collected are preprocessed, including signal denoising, normalization, signal clipping and alignment, etc., to ensure that each modal data is at a unified scale. Secondly, the processed multimodal data is subjected to feature extraction using the corresponding feature extraction network. Thirdly, a multimodal data fusion module based on a cross-modal attention mechanism is constructed to obtain multimodal fusion features. Finally, a fault diagnosis model for electric power equipment is constructed based on multimodal fusion features to support fault diagnosis of electric power equipment, alleviate the problem of insufficient accuracy of the diagnosis model for single modal samples of faults, and improve the versatility and diagnostic accuracy of the model.

[0059] The specific steps are as follows:

[0060] Step 1: Multimodal data preprocessing.

[0061] Step 1.1: Preprocessing of acoustic vibration data. Discrete sampling (sampling frequency 16000 Hz) is performed on the collected equipment acoustic vibration training signals and the acoustic vibration signals to be tested. The sampled data is normalized and further represented in the form of Mel-scale images, and the image size is fixedly scaled to 224×224.

[0062] Step 1.2: Preprocessing of partial discharge spectrum data. Preprocessing is performed on the standard fault data of partial discharge phase distribution (PRPD), the actually collected labeled data and the unlabeled data, including image resampling and normalization. The resampling adopts the bilinear interpolation method:

[0063]

[0064] Where f(X) is the pixel value at point X, Q 11 =(x1,y1),Q 21 =(x2,y1),Q 12 =(x1,y2),Q 22 =(x2,y2) are the coordinates of the four original image points around the target pixel (x,y). All images can be resized to the same size through resampling to meet the subsequent model input requirements. The resampled data is normalized, and the pixel value is divided by 255 to scale the pixel value of the image to the range of [0,1], which improves the stability, generalization ability and robustness of model training, thereby improving the performance and accuracy of the model.

[0065] Step 1.3: Preprocessing of numerical data. For other numerical data, we scale them by a fixed random matrix to obtain a vector with a dimension of 224.

[0066] Step 2: Multimodal data feature extraction. The mel-spectrogram data is fed into the VGG-Net network to obtain the extracted features of the acoustic vibration. The partial discharge data is fed into the Resnet-50 network to obtain the extracted features of the partial discharge. Send the vector of numerical data into a 3-layer fully connected network to obtain the extracted features of numerical data Where N v 、N st and N num They represent the number of samples of three types of multimodal data, and d represents the dimension of extracted features.

[0067] Step 3: Build a multimodal fusion module.

[0068] Given the audio (A), visual (V), and text (T) feature representations Xa, Xv, and Xt, the feature vectors of the audio, visual, and text modalities are concatenated with the joint feature representation of all modalities (J) and then passed through a fully connected layer:

[0069]

[0070] Where d = da + dv + dt represents the dimension of J, da, dv and dt are the dimensions of audio, visual and text features respectively, and FC represents the fully connected layer.

[0071] Now, J is input into Figure 1 In the joint cross-attention framework of the respective modes shown in Fig. 1, the feature representation of a single mode is focused. This helps to simultaneously encode the intra-modal and inter-modal relationships within the same mode and across modes to obtain the attention weights. The cross-correlation relationship between J and Xa is obtained as the joint cross-correlation matrix Ca, which is given by

[0072]

[0073] Where W ja ∈R da×d Indicates cross and The learnable weight matrix .

[0074] Similarly, the cross-correlation matrices of the other two modes Cv and Ct are obtained as:

[0075]

[0076] Among them, W jv ∈R dv×d , W jt ∈R dt×d Represent the learnable weight matrices for visual and textual modes respectively.

[0077] The obtained joint cross-correlation matrix of individual modalities is used to compute the attention weights, thereby capturing the semantic relevance between and within the same modality. The higher the correlation coefficient of the joint cross-correlation matrix, the higher the semantic relevance of the intra-modality and inter-modality relations of the corresponding feature vectors. Now, we use the joint cross-correlation matrix to compute the attention map for a single modality. For the audio modality, the joint correlation matrix Ca and the corresponding audio features Xa are combined using a learnable weight matrix Wca, and then the attention map Ha is computed using the ReLU activation function, where Ha is given by

[0078] H a =ReLU(X a W ca C a ) (5)

[0079] Where W ca ∈R K×K represents the learnable weight matrix for the audio modality. Similarly, the attention maps for the visual and textual modalities are

[0080] H v =ReLU(X v W cv C v ) (6)

[0081] H t =ReLU(Xt W ct C t ) (7)

[0082] Among them, W cv ∈R K×K , W ct ∈R K×K are the learnable weight matrices for the visual mode and text mode respectively. Now, the attention map is used to calculate the attended features of the individual modes as follows:

[0083] X att，a =H a W ha +X a (8)

[0084] X att，v =H v W hv +X v (9)

[0085] X att，t =H t W ht +X t (10)

[0086] Among them, W ha ∈R K×K , W hv ∈R K×K and W ht ∈R K×K Learnable weight matrices representing audio, visual, and textual modalities respectively.

[0087] To obtain a more refined feature representation, the attended features of each modality are again fed as input to their respective joint cross-modal attention modules, which consist of

[0088]

[0089] Among them, W (l) ha ∈R K×K , W (l) ht ∈R K×K and W (l) ht ∈R K×K denote the learnable weight matrices for audio, visual, and text modalities, respectively, and l denotes the recursive step.

[0090] Connect the participating features of a single modality after l iterations to obtain the multimodal feature representation X (l) att , as shown in the figure:

[0091]

[0092] Step 4: Construct a power equipment fault diagnosis model based on multimodal fusion features

[0093] The multimodal fusion features are sent to the diagnosis module to diagnose equipment failures. The module is a three-layer fully connected network. The number of units in each hidden layer is set to 1000-500-200 respectively. The number of units in the final output layer is the category of the data. The diagnosis loss L labeled The specific definition is:

[0094]

[0095] f c (·) represents the classifier network, L labeled It is the cross entropy loss. The optimization methods such as stochastic gradient descent can be used to optimize the model training process.

[0096] Step 5: Overall model training, the pre-processed data information of different modalities is sent to the model together, and the features are extracted through the feature extraction module corresponding to the features, and then the features are sent to the multimodal data fusion module and classifier module based on cross-modal attention, and the loss function corresponding to each loss is calculated, and the gradient between the output and input of the overall network model is calculated. According to the gradient descent method, the feedforward neural network is back-propagated according to the loss function set by the signature, and the weights and biases in the network are iteratively updated. Optimization methods such as adaptive moment estimation can also be used to further optimize the model training process.

[0097] Step 6: Implement fault diagnosis on acoustic and vibroacoustic data from different domains based on the network model.

[0098] After the fault diagnosis model is trained, the single-modal or multi-modal data collected by the device to be diagnosed can be input according to the model to perform fault diagnosis on the device corresponding to the data.

[0099] The present invention also provides an electric power equipment fault diagnosis system based on time series-spectrum-discrete data multimodal fusion, characterized in that it includes a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0100] The present invention also provides a computer-readable storage medium, on which computer program instructions that can be executed by a processor are stored. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0101] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0102] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0103] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0105] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for fault diagnosis of electric power equipment based on multimodal fusion of time series, spectrogram and discrete data, characterized in that: include: Construct a data fusion model based on the cross-modal attention mechanism to achieve multi-modal feature fusion; Build a fault diagnosis model based on multimodal fusion to realize power equipment fault diagnosis.

2. The method for fault diagnosis of electric power equipment based on time series-spectrum-discrete data multimodal fusion according to claim 1 is characterized in that: Before building a data fusion model based on the cross-modal attention mechanism, it is necessary to use the corresponding feature extraction network to extract features from the multimodal data.

3. The method for fault diagnosis of electric power equipment based on time series-spectrum-discrete data multimodal fusion according to claim 2 is characterized in that: Before feature extraction of multimodal data, the collected multimodal data needs to be preprocessed to ensure that each modal data is at a uniform scale.

4. The method for fault diagnosis of electric power equipment based on time series-spectrum-discrete data multimodal fusion according to claim 3 is characterized in that: The multimodal data preprocessing operation is as follows: Acoustic vibration data preprocessing: Discrete sampling is performed on the collected equipment acoustic vibration training signals and the acoustic vibration signals to be tested, and the sampled data is normalized and expressed in the form of Mel-level atlas, and the image size is fixedly scaled to 224×224; Partial discharge spectrum data preprocessing: Preprocessing is performed on the standard fault data of partial discharge phase distribution PRPD, the actually collected annotated data and the unannotated data, including image resampling and normalization. The resampling adopts the bilinear interpolation method: Where f(X) is the pixel value at point X, Q 11 =(x1,y1),Q 21 =(x2,y1),Q 12 =(x1,y2),Q 22 = (x2, y2) are the coordinates of the four original image points around the target pixel (x, y). All images are adjusted to the same size by resampling. The resampled data is normalized by dividing the pixel value by 255 to scale the pixel value of the image to the range of [0, 1]. Numerical data preprocessing: For numerical data, it is scaled by a fixed random matrix to obtain a vector with a dimension of 224.

5. The method for fault diagnosis of electric power equipment based on time series-spectrum-discrete data multimodal fusion according to claim 4 is characterized in that: The multimodal data feature extraction method is as follows: Send the Mel-spectrogram data into the VGG-Net network to obtain the extracted features of the acoustic vibration The partial discharge spectrum data is fed into the Resnet-50 network to obtain the partial discharge extraction features Send the vector of numerical data into a 3-layer fully connected network to obtain the extracted features of numerical data Where N v 、N st and N num They represent the number of samples of three types of multimodal data, and d represents the dimension of extracted features.

6. The method for fault diagnosis of electric power equipment based on time series-spectrum-discrete data multimodal fusion according to claim 5 is characterized in that: The specific process of building a data fusion model based on the cross-modal attention mechanism is as follows: Given the audio (A), visual (V), and text (T) feature representations Xa, Xv, and Xt, the feature vectors of the audio, visual, and text modalities are concatenated with the joint feature representation J of all modalities and then passed through a fully connected layer: Where d = da + dv + dt represents the dimension of J, da, dv and dt are the dimensions of audio, visual and text features respectively, and FC represents the fully connected layer; J is input into the joint cross-attention framework of each modality to focus on the feature representation of a single modality, and the cross-correlation relationship between J and Xa is obtained as the joint cross-correlation matrix Ca, which is given by Among them, W ja ∈R da×d a learnable weight matrix representing the audio modality; Similarly, the cross-correlation matrices of the other two modes Cv and Ct are obtained as follows: Among them, W jv ∈R dv×d , W jt ∈R dt×d Represent the learnable weight matrices of visual modality and textual modality respectively; The joint cross-correlation matrix is ​​used to calculate the attention map of a single modality. For the audio modality, the joint correlation matrix Ca and the corresponding audio features Xa are combined using the learnable weight matrix Wca, and then the ReLU activation function is used to calculate the attention map Ha, which is given by H a =ReLU(X a IN ca C a ) (5) Among them, W ca ∈R K×K represents the learnable weight matrix of the audio modality; similarly, the attention maps of the visual modality and textual modality are H v =ReLU(X v IN cv C v ) (6) H t =ReLU(X t IN ct C t ) (7) Among them, W cv ∈R K×K , W ct ∈R K×K are the learnable weight matrices for the visual modality and textual modality respectively. Now, the attention map is used to calculate the attending features of the individual modalities as follows: X att，a =H a W ha +X a (8) X att，v =H v W hv +X v (9) X att，t =H t W ht +X t (10) Among them, W ha ∈R K×K , W hv ∈R K×K and W ht ∈R K×K Learnable weight matrices representing audio, visual, and textual modalities respectively; To obtain a more refined feature representation, the participating features of each modality are again taken as input to their respective joint cross-modal attention modules, which are composed of Among them, W (1) ha ∈R K×K , W (1) ht ∈R K×K and W (1) ht ∈R K×K denote the learnable weight matrices for audio, visual, and text modalities, respectively, and l denotes the recursive step; Connect the participating features of a single modality after 1 iteration to obtain the multimodal fusion feature representation X (1) att , as shown below:

7. The method for fault diagnosis of electric power equipment based on time series-spectrum-discrete data multimodal fusion according to claim 6 is characterized in that: The specific process of building a fault diagnosis model based on multimodal fusion is as follows: The multimodal fusion features are sent to the diagnosis module to diagnose equipment failures. The diagnosis loss L labeled The specific definition is: Among them, f c (·) represents the classifier network, L labeled is the cross entropy loss, and the stochastic gradient descent method is used to optimize the model training process. represents the input features of the neural network, Indicates the corresponding features The weight of .

8. The method for fault diagnosis of electric power equipment based on time series-spectrum-discrete data multimodal fusion according to claim 7 is characterized in that: The diagnosis module is a three-layer fully connected network. The number of units in each hidden layer is set to 1000-500-200 respectively, and the number of units in the final output layer is the category of the data.

9. A power equipment fault diagnosis system based on time series-spectrum-discrete data multimodal fusion, characterized in that: The method comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps as claimed in any one of claims 1 to 8 can be implemented.

10. A computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor, and when the processor executes the computer program instructions, the method steps according to any one of claims 1 to 8 can be implemented.

Images