Pump storage unit defect identification and tracing method and system based on multi-modal fusion

Abstract

The application discloses a kind of based on multimodal fusion pumped storage unit defect identification trace method and system, it is related to pumped storage unit defect identification and defect trace technical field, comprising: the multimodal data generated in the operation of pumped storage unit is respectively carried out graph structure representation learning, constructs timing modality graph representation, image modality graph representation and text modality graph representation;Timing modality graph representation, image modality graph representation and text modality graph representation are fused based on multilevel long short-term memory network, obtain multimodal fusion graph structure representation;Multimodal fusion graph structure representation is compressed based on graph distillation method, and compressed graph structure representation is obtained;Compressed graph structure representation is carried out graph clustering analysis and graph path search based on causal inference method, and the defect identification result and defect trace result of pumped storage unit are obtained.The application alleviates the technical problems that state evaluation is not comprehensive and operation early warning is difficult in the prior art.

Description

Technical Field

[0001] This invention relates to the field of defect identification and cause tracing technology for pumped storage units, specifically a method and system for defect identification and cause tracing of pumped storage units based on multimodal fusion. Background Technology

[0002] Against the backdrop of my country's ever-expanding power grid, pumped storage units can effectively improve power quality, alleviate peak-valley imbalances, and enhance the safe and stable operation of the power system. Currently, pumped storage power stations primarily rely on manual methods for equipment condition assessment, which fails to obtain sufficient and comprehensive data. This results in incomplete condition assessments and difficulties in implementing early warning systems, hindering the provision of low-cost, highly reliable auxiliary decision-making support for condition-based maintenance of pumped storage units. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for identifying and tracing the causes of defects in pumped storage units based on multimodal fusion in order to solve at least one of the above-mentioned technical problems.

[0004] In a first aspect, embodiments of the present invention provide a method for defect identification and cause tracing of pumped storage units based on multimodal fusion, comprising: performing graph structure representation learning on multimodal data generated during the operation of the pumped storage unit to construct time-series modal graph representation, image modal graph representation, and text modal graph representation; the multimodal data includes: monitoring data, equipment images, and defect analysis reports; fusing the time-series modal graph representation, the image modal graph representation, and the text modal graph representation based on a multi-level long short-term memory network to obtain a multimodal fused graph structure representation; compressing the information of the multimodal fused graph structure representation based on a graph distillation method to obtain a compressed graph structure representation; performing graph clustering analysis on the compressed graph structure representation to obtain the defect identification result of the pumped storage unit; and performing graph path search on the compressed graph structure representation based on a causal reasoning method to obtain the defect cause tracing result of the pumped storage unit.

[0005] Furthermore, constructing a time-series modal graph representation includes: representing the mapping weight of the target time-series segment to the preset time-series state feature by quantifying the distance between the target time-series segment and the preset time-series state feature of the monitoring data of the pumped storage unit; using each preset time-series state feature as a node in the graph representation, using the transition relationship between the preset time-series state features as the edge of the graph representation, determining the corresponding transition weight based on the mapping weight, and constructing the time-series modal graph representation.

[0006] Further, constructing an image modal graph representation includes: dividing the device image into N parts, and treating the feature vector of each part as an unordered node in the graph structure representation, resulting in N unordered nodes; determining the K nearest neighbor set for each unordered node based on the K nearest neighbor algorithm, and adding an edge connected to the corresponding unordered node to all nodes in the K nearest neighbor set, thus obtaining the image modal graph representation.

[0007] Furthermore, constructing a text modal graph representation includes: extracting semantic hierarchical information from the defect analysis report; the semantic information includes entities and relationships between entities; constructing the text modal graph representation by using each entity as a node in the graph structure and the relationships between entities as edges connecting the nodes.

[0008] Furthermore, the multi-level long short-term memory network includes a first-layer long short-term memory network, a second-layer long short-term memory network, and a third-layer long short-term memory network; the fusion of the temporal modality representation, the image modality representation, and the text modality representation based on the multi-level long short-term memory network includes: inputting the text modality representation into the first-layer long short-term memory network to obtain the first hidden layer state of each neuron; concatenating the temporal modality representation with the first hidden layer state and inputting it into the second-layer long short-term memory network to obtain the second hidden layer state of each neuron; concatenating the image modality representation with the second hidden layer state and inputting it into the third-layer long short-term memory network to obtain the third hidden layer state of each neuron.

[0009] Furthermore, information compression of the multimodal fusion graph structure representation based on graph distillation includes: a method for multimodal graph distillation based on a teacher-student model to compress information in the multimodal fusion graph structure representation.

[0010] Furthermore, the method also includes: outputting the defect identification results and the defect cause tracing results in the form of text and corresponding confidence probability sets, and arranging them according to the confidence probability.

[0011] Secondly, embodiments of the present invention also provide a multimodal fusion-based defect identification and cause tracing system for pumped storage units, comprising: a construction module, a fusion module, a compression module, a defect identification module, and a defect cause tracing module; wherein, the construction module is used to perform graph structure representation learning on the multimodal data generated during the operation of the pumped storage unit, respectively, to construct time-series modal graph representations, image modal graph representations, and text modal graph representations; the multimodal data includes: monitoring data, equipment images, and defect analysis reports; the fusion ... The temporal modal graph representation, the image modal graph representation, and the text modal graph representation are fused to obtain a multimodal fused graph structure representation; the compression module is used to compress the information of the multimodal fused graph structure representation based on the graph distillation method to obtain a compressed graph structure representation; the defect identification module is used to perform graph clustering analysis on the compressed graph structure representation to obtain the defect identification result of the pumped storage unit; the defect tracing module is used to perform graph path search on the compressed graph structure representation based on the causal reasoning method to obtain the defect tracing result of the pumped storage unit.

[0012] Thirdly, embodiments of the present invention also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in the first aspect.

[0013] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method described in the first aspect.

[0014] This invention provides a method and system for defect identification and tracing of pumped storage units based on multimodal fusion. It addresses the challenges of using different models for analyzing and reasoning different modal data generated during pumped storage unit operation, such as monitoring data, equipment images, and defect analysis reports. This is achieved through a graph-based multimodal knowledge fusion method, which unifies the knowledge representation of pumped storage unit operation. Using multimodal knowledge as the data foundation, it employs graph clustering analysis and causal reasoning to identify and trace the causes of pumped storage unit defects. This supports highly intelligent and reliable maintenance decision support for pumped storage units, improves the work efficiency of on-site personnel, and alleviates the technical problems of incomplete status evaluation and difficulty in implementing operational early warning in existing technologies. Attached Figure Description

[0015] The accompanying drawings, which form part of this specification, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0016] Figure 1 A flowchart of a method for identifying and tracing the causes of defects in pumped storage units based on multimodal fusion, provided for an embodiment of the present invention;

[0017] Figure 2 A flowchart of another method for identifying and tracing the causes of defects in pumped storage units based on multimodal fusion, provided for an embodiment of the present invention;

[0018] Figure 3 This is a schematic diagram of a multimodal graph fusion process provided in an embodiment of the present invention;

[0019] Figure 4 A schematic diagram of a pumped storage unit defect identification and tracing system based on multimodal fusion provided in an embodiment of the present invention;

[0020] Figure 5 This is a schematic diagram of a construction module provided in an embodiment of the present invention. Detailed Implementation

[0021] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0022] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this invention is for describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0023] Example 1

[0024] Figure 1 This is a flowchart of a method for identifying and tracing the causes of defects in pumped storage units based on multimodal fusion, according to an embodiment of the present invention. Figure 2 This is a flowchart of another method for defect identification and cause tracing of pumped storage units based on multimodal fusion, provided by an embodiment of the present invention. Figure 1 and Figure 2 As shown, the method specifically includes the following steps:

[0025] Step S102: Graph structure representation learning is performed on the multimodal data generated during the operation of the pumped storage unit to construct time-series modal graph representation, image modal graph representation, and text modal graph representation; the multimodal data includes: monitoring data, equipment images, and defect analysis reports.

[0026] Step S104: Based on a multi-level long short-term memory network, the temporal modality graph representation, image modality graph representation, and text modality graph representation are fused to obtain a multi-modal fused graph structure representation.

[0027] Step S106: Based on the graph distillation method, information compression is performed on the multimodal fusion graph structure representation to obtain the compressed graph structure representation.

[0028] Step S108: Perform graph clustering analysis on the compressed graph structure representation to obtain the defect identification results of the pumped storage unit.

[0029] Step S110: Based on the causal reasoning method, perform graph path search on the compressed graph structure representation to obtain the defect tracing results of the pumped storage unit.

[0030] Specifically, this invention performs graph structure representation learning on multimodal data such as monitoring data, equipment images, and defect analysis reports generated during the operation of pumped storage units to form a time-series modal graph representation G. ts =(V ts E ts Image modality representation G f =(V f E f Text modal diagram representation of G d =(V d E d ).

[0031] Specifically, the construction of a time-series modal graph representation includes: representing the mapping weight of the target time-series segment to the preset time-series state feature by quantifying the distance between the target time-series segment and the preset time-series state feature of the monitoring data of the pumped storage unit; taking each preset time-series state feature as a node in the graph representation, taking the transition relationship between the preset time-series state features as the edge of the graph representation, determining the corresponding transition weight based on the mapping weight, and constructing a time-series modal graph representation.

[0032] In an optional embodiment of the present invention, firstly, time-series state identification is performed. N types of time-series state features Θ = [v1, v2, ..., v] can be defined based on the operating status of the pumped storage unit or equipment. N The symbols ] represent normal operation, minor defects, moderate defects, and severe defects, respectively. For a given time series segment X... t The weight of mapping a time series segment to a time series state feature is represented by quantifying the distance between the time series segment and the time series state feature:

[0033]

[0034] Wherein, D(X) t ,v i(Refers to time series segment X) t With temporal state features v i The distance between them can be calculated using Euclidean distance or dynamic time normalization.

[0035] Then, for time segment X t-1 To X t The N temporal state features are considered as N nodes V = [v1, v2, ..., v] in a graph structure. N ], each edge e ij Indicates from v i to v j The transition relationship has a corresponding transition weight of m. ij The calculation is performed using the following formula:

[0036]

[0037] The transfer weight is also X. t-1 Recognized as state v i and X t Recognized as state v j The joint weights.

[0038] Specifically, constructing an image modal graph representation includes: dividing the device image into N parts, and treating the feature vector of each part as an unordered node in the graph structure representation, resulting in N unordered nodes; determining the K nearest neighbor set for each unordered node based on the K nearest neighbor algorithm, and adding an edge connected to the corresponding unordered node to all nodes in the K nearest neighbor set, thus obtaining the image modal graph representation.

[0039] In an optional embodiment of the present invention, the image is first divided into N square parts, and the feature vectors of the N parts are labeled as a sequence X = [x1, x2, ..., x...]. N Simultaneously, the N eigenvectors are considered as N unordered nodes V = [v1, v2, ..., v] in a graph structure. N Then, for each node v i Find node v using the KNN algorithm i K nearest neighbors N(v) i ), and N(v i All nodes v in ) j Add a line with node v i Connected edge e ij Thus, the image modality representation G is completed. f =(V f E f The construction of ).

[0040] Specifically, constructing a text modal graph representation includes: extracting semantic hierarchical information from the defect analysis report; the semantic information includes entities and relationships between entities; and constructing a text modal graph representation with each entity as a node in the graph structure and the relationships between entities as edges connecting the nodes.

[0041] In an optional embodiment of the present invention, semantic hierarchical information is first extracted based on the text data in the defect analysis report. Entities are extracted according to categories such as defect name, defect phenomenon, defect cause, and handling suggestions; relationships are extracted according to categories such as relevance and cause between entities. Each entity is treated as a node in a graph structure, and the relationships between entities are treated as edges connecting nodes, thereby completing the text modal graph representation G. d =(V d E d The construction of ).

[0042] The method provided by this invention is based on a multi-layered long short-term memory (LSTM) network, which fuses three modal features at the model level. Preferably, the multi-layered LTM network includes a first-layer LTM network, a second-layer LTM network, and a third-layer LTM network.

[0043] Figure 3 This is a schematic diagram of a multimodal graph fusion process provided by an embodiment of the present invention. Figure 3 As shown, step S104 specifically includes the following steps:

[0044] Step S1041: Input the text modality representation into the first layer of the long short-term memory network to obtain the first hidden layer state Φ1 of each neuron;

[0045] Step S1042: The temporal modality diagram representation is concatenated with the first hidden layer state Φ1 and input into the second long short-term memory network to obtain the second hidden layer state Φ2 of each neuron;

[0046] Step S1043: The image modality graph representation is concatenated with the second hidden layer state Φ2 and input into the third long short-term memory network to obtain the third hidden layer state Φ3 for each neuron. Graph fusion of the three modalities is completed.

[0047] In an optional embodiment of the present invention, in order to improve the applicability of graph representation, make full use of the topological information of graph representation, and improve model performance through information compression, a method of multimodal graph distillation based on teacher-student model is adopted to compress information on the multimodal fusion graph structure representation.

[0048] Specifically, the three hidden layer states Φ1, Φ2, and Φ3 obtained in step S104 are used as inputs to train three teacher network models. Subsequently, under the same data, the predictions of the teacher networks for the sample are used as the prediction targets of the student models to guide their learning. At the same time, weights are applied to the prediction results of the three teacher networks, such as [0.2, 0.3, 0.5], to ensure that the graph representation containing more modalities provides more knowledge.

[0049] Optionally, the method provided in this embodiment of the invention further includes: outputting the defect identification results and defect cause tracing results in the form of text and corresponding confidence probability sets, and arranging them according to confidence probability.

[0050] In an optional embodiment of this invention, the status description of pumped-storage units or equipment discovered by operational personnel during inspections, along with alarm information from the operation monitoring system, are used as input. The trained models in steps S102 and S104 are then used to achieve graph representation learning and multimodal graph fusion distillation. Graph clustering and causal reasoning algorithms are employed to identify and trace the causes of defects in the pumped-storage units, outputting the results as text and corresponding confidence probability sets, arranged from high to low confidence probability.

[0051] For example, inputting information such as "Relay R29 is normally energized", "Relay K58 is not normally energized", and "The voltage between points XDO6:57 and XDO6:58 is 24V, and the circuit is not open" can yield a set of inference results based on a trained graph neural network model and graph clustering mining method: {1: {"Autosynchronous synchronization failed", "87%"}; 2: {"Objects were missed", "13%"}}.

[0052] For example, given the input unit status description "Operating condition transition failed, pumping phase adjustment operator manually switched to shutdown" and the alarm message "Unit self-synchronization failed", based on a trained graph neural network model, the following inference result set can be obtained through causal link reasoning prediction: {1: {"Drag command not given to unit synchronization device", "72%"}; 2: {"Phase PT secondary circuit breaker tripped", "21%"}; 3: {"Related relay fault", "7%"}}.

[0053] As described above, the embodiments of the present invention provide a method for identifying and tracing the causes of defects in pumped storage units based on multimodal fusion, which has the following advantages compared with the prior art:

[0054] (1) Multimodal Fusion and Unified Knowledge Representation. For various data generated during the operation of pumped storage units, such as monitoring data, equipment images, and defect analysis reports, a graph-based multimodal knowledge fusion method is used to achieve a unified knowledge representation of pumped storage unit operation, solving the problem that different models are needed for analysis and reasoning of different modal data.

[0055] (2) Defect identification and cause tracing method for pumped storage units. Based on multimodal knowledge, defect identification and cause tracing of pumped storage units are achieved using methods such as graph clustering analysis and causal reasoning. This supports intelligent and reliable maintenance auxiliary decision-making for pumped storage units and improves the work efficiency of on-site personnel.

[0056] Example 2

[0057] Figure 4 This is a schematic diagram of a multimodal fusion-based defect identification and tracing system for pumped storage units according to an embodiment of the present invention. Figure 4 As shown, the system includes: a construction module 10, a fusion module 20, a compression module 30, a defect identification module 40, and a defect tracing module 50.

[0058] Specifically, module 10 is used to perform graph structure representation learning on the multimodal data generated during the operation of the pumped storage unit, and to construct time-series modal graph representation, image modal graph representation and text modal graph representation; the multimodal data includes: monitoring data, equipment images and defect analysis reports.

[0059] The fusion module 20 is used to fuse temporal modality graph representation, image modality graph representation and text modality graph representation based on a multi-level long short-term memory network to obtain a multi-modal fused graph structure representation.

[0060] Compression module 30 is used to compress information of multimodal fusion graph structure representation based on graph distillation method to obtain compressed graph structure representation.

[0061] The defect identification module 40 is used to perform graph clustering analysis on the compressed graph structure representation to obtain the defect identification results of the pumped storage unit.

[0062] The defect tracing module 50 is used to perform graph path search on the compressed graph structure representation based on the causal reasoning method to obtain the defect tracing results of the pumped storage unit.

[0063] Figure 5 This is a schematic diagram of a building module provided according to an embodiment of the present invention. Figure 5 As shown, the building module 10 also includes a first building unit 11, a second building unit 12 and a third building unit 13.

[0064] Specifically, the first building unit 11 is used to build a temporal modality graph representation; the second building unit 12 is used to build an image modality graph representation; and the third building unit 13 is used to build a text modality graph representation.

[0065] Specifically, the first building unit 11 is also used for:

[0066] The distance between the target time series segment and the preset time series state characteristics of the monitoring data of the pumped storage unit is quantified to represent the mapping weight of the target time series segment to the preset time series state characteristics.

[0067] Each preset temporal state feature is used as a node in the graph representation, and the transition relationship between the preset temporal state features is used as the edge of the graph representation. The corresponding transition weights are determined based on the mapping weights to construct a temporal modality graph representation.

[0068] The second building unit 12 is also used for:

[0069] The device image is divided into N parts, and the feature vector of each part is regarded as an unordered node in the graph structure representation, resulting in N unordered nodes;

[0070] The K-nearest neighbor algorithm is used to determine the K-nearest neighbor set of each unordered node, and an edge is added to each node in the K-nearest neighbor set to connect to the corresponding unordered node, thus obtaining the image modal graph representation.

[0071] The third building unit 13 is also used for:

[0072] Semantic hierarchical information extraction is performed on the defect analysis report to extract semantic information, including entities and relationships between entities.

[0073] A text modal graph representation is constructed by using each entity as a node in the graph structure and the relationships between entities as edges connecting the nodes.

[0074] Preferably, the multi-level long short-term memory network includes a first-layer long short-term memory network, a second-layer long short-term memory network, and a third-layer long short-term memory network. Specifically, the fusion module 20 is also used for:

[0075] The text modality representation is input into the first layer of the long short-term memory network to obtain the first hidden layer state of each neuron;

[0076] The temporal modality diagram representation is concatenated with the state of the first hidden layer and input into the second long short-term memory network to obtain the state of the second hidden layer of each neuron.

[0077] The image modality representation is concatenated with the state of the second hidden layer and input into the third long short-term memory network to obtain the state of the third hidden layer for each neuron.

[0078] Specifically, the compression module 30 is also used for: a method for multimodal graph distillation based on the teacher-student model to compress information in the multimodal fusion graph structure representation.

[0079] Specifically, such as Figure 4 As shown, the system also includes an output module 60, which is used to output the defect identification results and defect cause tracing results in the form of text and corresponding confidence probability sets, and arrange them according to confidence probability.

[0080] This invention also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method provided in this invention.

[0081] This invention also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method provided in this invention.

[0082] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative in all respects and are not the only ones. All modifications within the scope of this invention or equivalent to the scope of this invention are included in this invention.

[0083] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0085] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for defect identification and cause tracing of pumped storage units based on multimodal fusion, characterized in that, include: Graph structure representation learning is performed on the multimodal data generated during the operation of pumped storage units to construct time-series modal graph representations, image modal graph representations, and text modal graph representations. The multimodal data includes: monitoring data, equipment images, and defect analysis reports; The temporal modality graph representation, the image modality graph representation, and the text modality graph representation are fused based on a multi-level long short-term memory network to obtain a multi-modal fused graph structure representation; The multimodal fusion graph structure representation is compressed using a graph distillation method to obtain a compressed graph structure representation. Graph clustering analysis is performed on the compressed graph structure representation to obtain the defect identification results of the pumped storage unit; Based on the causal reasoning method, a graph path search is performed on the compressed graph structure representation to obtain the defect tracing results of the pumped storage unit. The multi-level long short-term memory network includes a first-layer long short-term memory network, a second-layer long short-term memory network, and a third-layer long short-term memory network; the fusion of the temporal modality representation, the image modality representation, and the text modality representation based on the multi-level long short-term memory network includes: The text modality representation is input into the first layer of the long short-term memory network to obtain the first hidden layer state of each neuron; The temporal modality diagram representation is concatenated with the first hidden layer state and input into the second long short-term memory network to obtain the second hidden layer state of each neuron; The image modality representation is concatenated with the second hidden layer state and input into the third long short-term memory network to obtain the third hidden layer state of each neuron.

2. The method according to claim 1, characterized in that: Constructing a temporal modality diagram representation includes: By quantifying the distance between the target time series segment and the preset time series state feature of the monitoring data of the pumped storage unit, the mapping weight of the target time series segment to the preset time series state feature is represented. Each preset temporal state feature is used as a node in the graph representation, and the transition relationship between the preset temporal state features is used as the edge of the graph representation. The corresponding transition weight is determined based on the mapping weight, and the temporal modality graph representation is constructed.

3. The method according to claim 1, characterized in that: Constructing image modality representations includes: The device image is divided into N parts, and the feature vector of each part is regarded as an unordered node in the graph structure representation, resulting in N unordered nodes; The K-nearest neighbor algorithm is used to determine the K-nearest neighbor set of each unordered node, and an edge is added to each node in the K-nearest neighbor set to connect to the corresponding unordered node, thus obtaining the image modal graph representation.

4. The method according to claim 1, characterized in that: Constructing a text modality graph representation includes: Semantic hierarchical information extraction is performed on the defect analysis report to extract semantic information; the semantic information includes entities and relationships between entities; The text modal graph representation is constructed by using each entity as a node in the graph structure and the relationships between entities as edges connecting the nodes.

5. The method according to claim 1, characterized in that: Information compression of the multimodal fusion graph structure representation based on graph distillation includes: a method for multimodal graph distillation based on a teacher-student model to compress information in the multimodal fusion graph structure representation.

6. The method according to claim 1, characterized in that: The method further includes: outputting the defect identification results and the defect cause tracing results in the form of text and corresponding confidence probability sets, and arranging them according to the confidence probability.

7. A defect identification and tracing system for pumped storage units based on multimodal fusion, characterized in that, include: The module comprises a construction module, a fusion module, a compression module, a defect identification module, and a defect root cause tracing module; among which, The construction module is used to perform graph structure representation learning on the multimodal data generated during the operation of the pumped storage unit, and to construct time-series modal graph representation, image modal graph representation, and text modal graph representation; the multimodal data includes: monitoring data, equipment images, and defect analysis reports; The fusion module is used to fuse the temporal modality graph representation, the image modality graph representation, and the text modality graph representation based on a multi-level long short-term memory network to obtain a multi-modal fused graph structure representation; The compression module is used to compress the information of the multimodal fused graph structure representation based on the graph distillation method to obtain a compressed graph structure representation. The defect identification module is used to perform graph clustering analysis on the compressed graph structure representation to obtain the defect identification results of the pumped storage unit. The defect tracing module is used to perform graph path search on the compressed graph structure representation based on the causal reasoning method to obtain the defect tracing result of the pumped storage unit. The multi-level long short-term memory network includes a first-layer long short-term memory network, a second-layer long short-term memory network, and a third-layer long short-term memory network; the fusion of the temporal modality representation, the image modality representation, and the text modality representation based on the multi-level long short-term memory network includes: The text modality representation is input into the first layer of the long short-term memory network to obtain the first hidden layer state of each neuron; The temporal modality diagram representation is concatenated with the first hidden layer state and input into the second long short-term memory network to obtain the second hidden layer state of each neuron; The image modality representation is concatenated with the second hidden layer state and input into the third long short-term memory network to obtain the third hidden layer state of each neuron.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as claimed in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-6.

Images