Fault diagnosis method and device based on multi-modal data of flexible direct current converter station, terminal equipment and storage medium

Abstract

The invention discloses a fault diagnosis method and device based on multi-modal data of a flexible direct current converter station, terminal equipment and a storage medium, and relates to the field of fault diagnosis, and the method comprises the steps: obtaining the multi-modal data of the flexible direct current converter station; each piece of modal data comprises time sequence data and corresponding spatial data; inputting the multi-modal data into a feature extraction model, and for each modal data, performing time sequence feature extraction on the time sequence data to obtain a time sequence feature vector; local space structure feature extraction is carried out on the space data to obtain a space feature vector; performing linear transformation and feature integration on the time sequence feature vector and the spatial feature vector to obtain a high-dimensional semantic vector; performing weighted fusion on the high-dimensional semantic vectors of all modal data to obtain a fused feature vector; and performing fault diagnosis on the converter station according to the fusion feature vector. According to the invention, the problem that the time-space relevance in the operation state data is ignored in the prior art is solved, and the accuracy of fault diagnosis is improved.

Description

Technical Field

[0001] The present invention relates to the field of fault diagnosis, and in particular to a fault diagnosis method, apparatus, terminal equipment and storage medium based on multimodal data of a flexible direct current converter station. Background Art

[0002] As the hub of the HVDC transmission system, flexible DC converter stations undertake key functions such as AC / DC power conversion, power control, and system stability regulation. The reliability of converter station operations directly determines the safe and stable operation of the power system, the efficiency of energy transmission, and the quality of regional power supply. With the large-scale integration of renewable energy and the growing demand for cross-regional power transmission, the stable operation of flexible DC converter stations is becoming increasingly critical to ensuring energy supply and grid security.

[0003] In existing technologies, fault diagnosis in converter stations primarily relies on traditional rule-based data analysis methods, which process and diagnose data using manually preset fixed thresholds or logical judgment conditions. However, converter station operating status data is multimodal and complex. Because rule-based data analysis methods typically only perform simple threshold comparisons and logical judgments on single data points, they ignore the temporal and spatial correlations in operating status data, making it difficult to accurately locate faults in complex fault scenarios. Summary of the Invention

[0004] Embodiments of the present invention provide a fault diagnosis method, apparatus, terminal device, and storage medium based on multimodal data of a flexible DC converter station, which solve the problem of the prior art ignoring the temporal correlation and spatial correlation in operating status data and improve the accuracy of converter station fault diagnosis.

[0005] An embodiment of the present invention provides a fault diagnosis method based on multimodal data of a flexible DC converter station, comprising:

[0006] Acquire multimodal data of the flexible DC converter station; each modal data includes: time series data and corresponding spatial data; the time series data includes: electrical parameters, status data or environmental data; the spatial data includes: a spectrum of the time series data, an infrared image of the converter station surface, and a thermal image of the converter station surface;

[0007] Input the multimodal data into a preset feature extraction model, so that the feature extraction model extracts time series features from the time series data for each modal data to obtain a time series feature vector; extracts local spatial structure features from the spatial data corresponding to the time series data to obtain a spatial feature vector; linearly transforms and integrates the time series feature vector and the spatial feature vector to obtain a high-dimensional semantic vector;

[0008] Perform weighted fusion on the high-dimensional semantic vectors of all modal data to obtain a fused feature vector;

[0009] According to the fused eigenvectors, the fault diagnosis of the flexible DC converter station is performed to obtain the fault diagnosis results.

[0010] Furthermore, based on the fused feature vector, fault diagnosis is performed on the flexible DC converter station to obtain the fault diagnosis results, including:

[0011] The fused feature vector is input into the preset fault diagnosis model so that the fault diagnosis model performs attention feature enhancement through the built-in Transformer encoder to obtain an enhanced feature vector;

[0012] Mapping the enhanced feature vector to a built-in fault diagnosis graph network to determine a first mapping node in the fault diagnosis graph network;

[0013] generating a plurality of candidate fault diagnosis paths according to the adjacency relationship of the first mapping node in the fault diagnosis graph network;

[0014] For each candidate fault diagnosis path, the confidence of the candidate fault diagnosis path is calculated according to the preset weight in the candidate fault diagnosis path;

[0015] From all the fault diagnosis candidate paths, select the fault diagnosis candidate path with the highest confidence as the fault diagnosis target path;

[0016] Decode the characteristics of the fault diagnosis target path and generate the fault diagnosis results;

[0017] When the fault diagnosis result indicates that a fault exists, the fault cause and fault location are generated.

[0018] Furthermore, when the fault diagnosis result indicates that a fault exists, after generating the fault cause and the fault location, the method further includes:

[0019] Inputting the fault cause and the fault location into a preset fault processing model so that the fault processing model maps the fault cause and the fault location to a built-in fault solution graph network, and determining a second mapping node in the fault solution graph network;

[0020] generating a plurality of candidate fault handling paths according to the adjacency relationship of the second mapping nodes in the fault solution graph network;

[0021] Calculate the semantic similarity of each candidate fault handling path to obtain the path matching score of each candidate fault handling path;

[0022] From all candidate fault handling paths, select the candidate fault handling path with the highest path matching score as the target fault handling path;

[0023] Determine the fault handling plan based on the fault handling target path.

[0024] Furthermore, after the fault diagnosis result of the flexible DC converter station is obtained based on the fused feature vector, the following steps are further included:

[0025] Obtain weak labels corresponding to the multimodal data of the flexible DC converter station; the weak labels are manual fault determination results of the multimodal data of the flexible DC converter station;

[0026] According to the multimodal data of the flexible DC converter station and its corresponding weak labels, the preset weights on the fault diagnosis target path in the fault diagnosis graph network built into the fault diagnosis model are iteratively optimized through gradient descent to obtain an updated fault diagnosis model.

[0027] Furthermore, after obtaining the multimodal data of the flexible DC converter station, the following is also included:

[0028] The time series data in each modal data is processed for outliers, missing values ​​are filled, and normalization is performed to obtain the preprocessed time series data.

[0029] Furthermore, the high-dimensional semantic vectors of all modal data are weightedly fused to obtain a fused feature vector, including:

[0030] Generate query vector, key vector and value vector of each modal data according to the high-dimensional semantic vector of each modal data;

[0031] For each modality combination, the attention score between the two modal data in the modality combination is calculated according to the query vector and key vector of the two modal data in the modality combination;

[0032] According to the attention score between the two modal data in each modality combination, the value vector of each modal data is summed up with attention weights to obtain the cross-modal enhancement vector of each modal data;

[0033] The cross-modal enhancement vectors of each modality data are feature concatenated to obtain a fused feature vector.

[0034] Based on the above method embodiment, the present invention provides a corresponding device embodiment, including: a multimodal data acquisition module, a feature extraction module, a feature fusion module and a fault diagnosis module;

[0035] A multimodal data acquisition module is used to acquire multimodal data of the flexible DC converter station. Each modal data includes: a time series data and corresponding spatial data. The time series data includes: electrical parameters, status data or environmental data. The spatial data includes: a spectrum diagram of the time series data, an infrared image of the converter station surface, and a thermal image of the converter station surface.

[0036] The feature extraction module is used to input multimodal data into a preset feature extraction model so that the feature extraction model performs time series feature extraction on the time series data for each modal data to obtain a time series feature vector; performs local spatial structure feature extraction on the spatial data corresponding to the time series data to obtain a spatial feature vector; and performs linear transformation and feature integration on the time series feature vector and the spatial feature vector to obtain a high-dimensional semantic vector;

[0037] The feature fusion module is used to perform weighted fusion on the high-dimensional semantic vectors of all modal data to obtain a fused feature vector;

[0038] The fault diagnosis module is used to perform fault diagnosis on the flexible DC converter station based on the fused feature vector and obtain the fault diagnosis result.

[0039] Furthermore, the fault diagnosis device based on multimodal data of the flexible DC converter station further includes: a data preprocessing module;

[0040] The data preprocessing module is used to perform outlier processing, missing value filling and normalization on the time series data in each modal data to obtain the preprocessed time series data.

[0041] Based on the above-mentioned method embodiment, the present invention provides a corresponding terminal device embodiment, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the fault diagnosis method based on multimodal data of the flexible DC converter station as described in the present invention.

[0042] Based on the above-mentioned method embodiment, the present invention provides a corresponding computer-readable storage medium embodiment, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to execute the steps of the fault diagnosis method based on multimodal data of the flexible DC converter station as described in the present invention.

[0043] Compared with the prior art, the beneficial effects of the embodiment of this solution are:

[0044] The present invention first obtains multimodal data of a flexible DC converter station, where each modal data includes a time series data and corresponding spatial data, wherein the time series data includes electrical parameters, state data or environmental data, and the spatial data includes a spectrum diagram of the time series data, an infrared image of the converter station surface and a thermal image of the converter station surface. Next, the multimodal data is input into a preset feature extraction model so that the feature extraction model performs time series feature extraction on the time series data for each modal data, captures the time series dependency, and obtains a time series feature vector containing time correlation. The feature extraction model performs local spatial structure feature extraction on the spatial data corresponding to the time series data, mines the spatial dimension correlation, and obtains a spatial feature vector. The deep features of different modal data are extracted modally. The time series feature vector and the spatial feature vector are then linearly transformed and mapped to a unified semantic space, and feature integration is performed to associate different modal spatial data with the time series data to obtain a high-dimensional semantic vector. The high-dimensional semantic vectors of all modal data are weighted fused to obtain a fused feature vector. By fusion of multi-source data, collaborative diagnosis of multi-source data is achieved, and the recognition accuracy of complex faults is improved. Finally, the fault diagnosis of the flexible DC converter station is performed based on the fused feature vector and the fault diagnosis results are obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 1 is a flow chart of a fault diagnosis method based on multimodal data of a flexible DC converter station provided by an embodiment of the present invention;

[0046] Figure 2 It is a structural diagram of a fault diagnosis device based on multimodal data of a flexible DC converter station provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0049] like Figure 1 As shown, in order to solve the problem that the existing technology ignores the temporal correlation and spatial correlation in the operating status data, an embodiment of the present invention provides a fault diagnosis method based on multimodal data of a flexible DC converter station, which includes at least the following steps:

[0050] Step S1: Acquire multimodal data of the flexible DC converter station; each modal data includes: time series data and corresponding spatial data; the time series data includes: electrical parameters, status data or environmental data; the spatial data includes: a spectrum of the time series data, an infrared image of the converter station surface, and a thermal image of the converter station surface;

[0051] In step S1, multimodal data of the flexible DC converter station is first acquired. Each type of modal data consists of time series data and spatial data. Time series data includes time-varying sequences such as electrical parameters, state data, and environmental data, and is used to characterize the data's evolution over time. Specifically, electrical parameters include bus voltage, bus current, active power, reactive power, current harmonic content, frequency, and three-phase voltage imbalance, reflecting the fundamental characteristics of the converter station's electrical operation. Status data includes converter valve module temperature, cooling system water pressure, water temperature, flow, circuit breaker status, trigger pulse signal status, and operating status indicators (such as ON / OFF), used to monitor the operating conditions of the converter station's core equipment. Environmental data includes indoor and outdoor temperature, humidity, wind speed, atmospheric pressure, dust concentration, or noise level, reflecting the impact of the converter station's environment on equipment operation. Spatial data, which is a spectrum converted or collected from time series data, reflects the frequency domain characteristics of the time series data, as well as infrared and thermal images of the converter station's surface, which are used to present the spatial temperature distribution and structural status of the equipment and assist in identifying potential faults in the spatial dimension, such as abnormal heating. By leveraging the temporal evolution characteristics of time series data and the structural characteristics of spatial data, it is possible to perceive and discern the complex conditions of the converter station.

[0052] In a preferred embodiment, after obtaining the multimodal data of the HVDC flexible converter station, the method further includes:

[0053] The time series data in each modal data is processed for outliers, missing values ​​are filled, and normalization is performed to obtain the preprocessed time series data.

[0054] In one embodiment of the present invention, in order to ensure that the data quality meets the requirements of subsequent feature extraction and model training, the time series data in each modal data is preprocessed. Specifically, first, outlier processing is performed to identify and correct abnormal points in the time series data caused by equipment failure or interference to prevent the abnormal values ​​from affecting subsequent model analysis; then, missing value filling is performed, and interpolation methods (such as linear interpolation) are used to supplement missing data points due to acquisition equipment failure or communication interruption to ensure the continuity of the time series data; finally, normalization processing is performed to map time series data of different dimensions and value ranges (such as voltage values ​​and temperature values) to a unified interval [0,1] to eliminate the impact of dimensional differences and improve the efficiency and stability of subsequent model training. The time series data after the above preprocessing can be used for subsequent steps.

[0055] Step S2: Input the multimodal data into a preset feature extraction model, so that the feature extraction model performs time series feature extraction on the time series data for each modal data to obtain a time series feature vector; performs local spatial structure feature extraction on the spatial data corresponding to the time series data to obtain a spatial feature vector; performs linear transformation and feature integration on the time series feature vector and the spatial feature vector to obtain a high-dimensional semantic vector;

[0056] For step S2, the multimodal data is input into the preset feature extraction model. The feature extraction model first captures the long-term dependency in the time series data for each modal data through the built-in long short-term memory network (LSTM), extracts the time dimension information, and obtains the time series feature vector F time Then, the built-in convolutional neural network (CNN) performs layer-by-layer convolution on the spatial data corresponding to the time series data (i.e., the spectrum of the time series data, infrared images, and thermal images), extracts the edge and texture features in the image, mines the structural information of the spatial dimension, and obtains the spatial feature vector F space .

[0057] Finally, the time series feature vector F extracted by LSTM is time And the spatial feature vector F extracted by CNN space , mapped to the semantic feature space of the same dimension through the built-in linear mapping layer, and combined the temporal evolution information of the time series and the structural information of the space through feature concatenation or tensor fusion to generate a high-dimensional semantic vector F for each modal data joint =[F′ space , F′ time This high-dimensional semantic vector condenses the key semantics of the modal data and can provide high-quality features for subsequent converter station operation fault diagnosis, allowing the model to more accurately understand the complex operating status of the converter station.

[0058] Step S3: Perform weighted fusion on the high-dimensional semantic vectors of all modal data to obtain a fused feature vector;

[0059] In a preferred embodiment, high-dimensional semantic vectors of all modal data are weightedly fused to obtain a fused feature vector, including:

[0060] Generate query vector, key vector and value vector of each modal data according to the high-dimensional semantic vector of each modal data;

[0061] For each modality combination, the attention score between the two modal data in the modality combination is calculated according to the query vector and key vector of the two modal data in the modality combination;

[0062] According to the attention score between the two modal data in each modality combination, the value vector of each modal data is summed up with attention weights to obtain the cross-modal enhancement vector of each modal data;

[0063] The cross-modal enhancement vectors of each modality data are feature concatenated to obtain a fused feature vector.

[0064] For step S3, since different modal data have different contributions to the converter station status, an attention mechanism is introduced to automatically learn the correlation strength between modalities and dynamically assign fusion weights to different modalities. Specifically, the high-dimensional semantic vector of each modality is linearly transformed to generate a query vector (Q), a key vector (K), and a value vector (V). All modal combinations are traversed. For each modal combination, the two modal data are recorded as modality i and modality j. The query vector and key vector of the two modal data in the combination are used to calculate the attention score a by the following formula ij , to quantify the strength of feature correlation between modalities:

[0065]

[0066] Among them, a ij represents the attention score between the i-th modal data and the j-th modal data, represents the transposed vector of the query vector of the i-th modality data, K j represents the key vector of the jth modal data, d k Indicates the dimension of the key vector.

[0067] Then, based on the attention score of each modal combination, the value vectors of each modal data are weighted and summed, so that the value vector of each modality integrates its own core features while incorporating the features of other modalities associated with it, thereby generating a cross-modal enhancement vector:

[0068]

[0069] Among them, F fused,i represents the cross-modal enhancement vector of the i-th modal data, and N represents the number of modal data types.

[0070] Finally, the cross-modal enhancement vectors of all modal data are concatenated by dimension to obtain the fused feature vector F that integrates multi-modal correlation information, providing more comprehensive feature support for subsequent converter station operation status judgment:

[0071] F=[F fused,1 ; F fused,2 ;…;F fused,N ].

[0072] Step S4: performing fault diagnosis on the flexible DC converter station according to the fused feature vector to obtain a fault diagnosis result.

[0073] In a preferred embodiment, fault diagnosis is performed on the flexible DC converter station based on the fused eigenvector to obtain a fault diagnosis result, including:

[0074] The fused feature vector is input into the preset fault diagnosis model so that the fault diagnosis model performs attention feature enhancement through the built-in Transformer encoder to obtain an enhanced feature vector;

[0075] Mapping the enhanced feature vector to a built-in fault diagnosis graph network to determine a first mapping node in the fault diagnosis graph network;

[0076] generating a plurality of candidate fault diagnosis paths according to the adjacency relationship of the first mapping node in the fault diagnosis graph network;

[0077] For each candidate fault diagnosis path, the confidence of the candidate fault diagnosis path is calculated according to the preset weight in the candidate fault diagnosis path;

[0078] From all the fault diagnosis candidate paths, select the fault diagnosis candidate path with the highest confidence as the fault diagnosis target path;

[0079] Decode the characteristics of the fault diagnosis target path and generate the fault diagnosis results;

[0080] When the fault diagnosis result indicates that a fault exists, the fault cause and fault location are generated.

[0081] In step S4, the fused feature vector F obtained in step S3 is input into the preset fault diagnosis model, and secondary feature enhancement is performed through the built-in Transformer encoder. Specifically, the context dependency of the fused feature vector F is learned through the multi-head self-attention mechanism in the Transformer encoder, and the sequence embedding representation is output to obtain the enhanced feature vector F. enhanced .

[0082] In order to achieve guided learning of domain knowledge and interpretable modeling of fault paths, the present invention introduces a knowledge graph module (i.e., a built-in fault diagnosis graph network) into the Transformer structure of the fault diagnosis model. The knowledge graph is constructed based on existing power equipment operation and maintenance specifications, expert rule bases, and SCADA (Supervisory Control And Data Acquisition) historical data. It includes equipment entities (such as converter valves, cooling systems, busbars, etc.), state attributes (such as overtemperature, power failure, cooling failure, etc.) and the logical associations between them (such as "overtemperature → cooling system failure"). The representations of each node and edge in the graph are respectively embedded in a graph neural network (GNN) or a TransE algorithm to generate a structured knowledge vector representation, so that the model can use prior knowledge in the power field to assist in fault diagnosis.

[0083] When the enhanced feature vector F enhanced When inputting the built-in fault diagnosis graph network, the feature vector F enhanced Calculate cosine similarity with the embedding vectors of each node in the fault diagnosis graph network, and activate the node with the highest similarity as the first mapping node. Starting from the first mapping node, expand along the edges in the fault diagnosis graph network to generate possible fault propagation paths as candidate paths. In this embodiment, starting with "converter valve" and "overtemperature," activate the relevant subgraphs in the fault diagnosis graph network, and generate the following candidate paths: Path 1: converter valve → overtemperature → cooling system failure → cooling pump failure; Path 2: converter valve → overtemperature → insulation aging → internal short circuit; Path 3: converter valve → overtemperature → temperature sensor failure.

[0084] To screen the most likely fault path, a confidence score is calculated based on the preset weight of each path, and the candidate fault diagnosis path with the highest confidence score is selected as the target fault diagnosis path. In this embodiment, path 1 with the highest confidence score is selected as the target fault diagnosis path: converter valve → overtemperature → cooling system failure → cooling pump failure.

[0085] It should be noted that the preset weights are determined based on knowledge of the power field and historical fault data, and reflect the fault association probability of nodes and edges in the path.

[0086] The fault diagnosis target path is feature decoded. Specifically, it is determined whether the confidence of the fault diagnosis target path is lower than the preset normal threshold. If so, the converter station operation data does not show characteristics that conform to the fault propagation law. It can be considered that the converter station is not currently abnormal due to the fault mode corresponding to this type of fault path, and the fault diagnosis result is output as normal. If not, the converter station operation data shows characteristics that conform to the fault path. The converter station is very likely to have a fault, and the fault diagnosis result is output as the presence of a fault. At this time, the node and edge associations in the fault diagnosis graph network are converted into fault causes and fault locations.

[0087] It should be noted that in actual applications, multiple paths with higher confidence levels can be retained simultaneously to generate multiple possible diagnostic results, for example, possible cause 1: cooling pump failure (confidence level 56%); possible cause 2: temperature sensor failure (confidence level 10%).

[0088] In a preferred embodiment, the training process of the fault diagnosis model is as follows:

[0089] Obtain the historical multimodal data of the flexible DC converter station, the diagnostic results annotated by experts, and the knowledge graph data constructed based on expert experience, perform feature extraction and fusion on the historical multimodal data, obtain the corresponding fused feature sample vector, and input the fused feature sample vector into the fault diagnosis model to be trained. The fault diagnosis model includes a Transformer encoder, a fault diagnosis graph network, a first path generation module, and a first decoding module. The Transformer encoder captures the long-distance dependencies between features through a multi-head self-attention mechanism and outputs an enhanced feature sample vector; the fault diagnosis graph network is constructed based on the knowledge graph, with nodes representing device entities and state attributes, and edges representing causal relationships. Node vectors are generated through GNN or TransE embedding; the first path generation module is used to generate candidate fault paths based on the enhanced feature activation graph network nodes; the first decoding module is used to calculate the confidence level through preset weights and decode it into the fault location and cause. Multi-task joint optimization is adopted during the training process, and the loss function expression is:

[0090]

[0091] in, represents the total loss function of the fault diagnosis model, Represents the feature enhancement loss, which enhances the feature sample vector F through contrastive learning enhanced，c Maximize the distance between fault features and normal features, represents the knowledge graph regularization loss, represents the path classification loss, and adopts the cross entropy function to make the model more confident in the real fault path than the false path. α and β represent the task weights.

[0092] According to the loss function Optimize the linear transformation matrix of Transformer, such as the mapping matrix of query vector, key vector and value vector, as well as the edge weight of the graph network, and use batch gradient descent for iterative optimization in each round.

[0093] The fault diagnosis model of the present invention is based on the Transformer structure combined with the knowledge graph to construct a diagnostic framework that deeply integrates multi-source data and is guided by domain knowledge: the Transformer multi-head self-attention mechanism is used to efficiently capture the long-distance dependencies of the multimodal time series data of the flexible DC converter station, and the efficiency of multi-source data feature extraction is improved by 40% compared with traditional methods; at the same time, a knowledge graph is constructed based on the power equipment operation and maintenance specifications, expert rule base and historical data, and the equipment entities, state attributes and causal relationships are embedded in the model, introducing the diagnostic logic constrained by prior knowledge in the power field. The fault diagnosis accuracy of the fault diagnosis model of the present invention is increased to 98%, which not only retains the advantage of Transformer in capturing time series features, but also ensures that the diagnostic results are consistent with the physical laws of the power system and have traceability from data association to causal explanation chain.

[0094] In a preferred embodiment, when the fault diagnosis result indicates that a fault exists, after generating the fault cause and the fault location, the method further includes:

[0095] Inputting the fault cause and the fault location into a preset fault processing model so that the fault processing model maps the fault cause and the fault location to a built-in fault solution graph network, and determining a second mapping node in the fault solution graph network;

[0096] generating a plurality of candidate fault handling paths according to the adjacency relationship of the second mapping nodes in the fault solution graph network;

[0097] Calculate the semantic similarity of each candidate fault handling path to obtain the path matching score of each candidate fault handling path;

[0098] From all candidate fault handling paths, select the candidate fault handling path with the highest path matching score as the target fault handling path;

[0099] Determine the fault handling plan based on the fault handling target path.

[0100] In one embodiment of the present invention, the fault cause and fault location obtained by diagnosis are input into a preset fault processing model, and the semantic similarity between the fault cause and fault location and the nodes in the fault plan graph network is calculated through a fully connected layer, and the node with the highest similarity is activated as the second mapping node.

[0101] It should be noted that the nodes in the fault plan graph network include fault types, handling measures, tools and materials, and safety specifications. The edges in the fault plan graph network represent the logical relationship of the handling process (such as power off → dismantling the pump body → installing a new pump → power on and testing) or constraints (such as requiring two-person operation or requiring insulating tools).

[0102] Next, starting from the second mapping node, the directly connected handling nodes are traversed to generate candidate fault handling paths. In this example, starting with "cooling pump failure," the relevant subgraphs in the fault plan graph network are activated, generating the following candidate paths: Path 1: Cooling pump failure → Power off → Pump disassembly → Install new pump → Power on and test → Record and archive; Path 2: Cooling pump failure → Circuit check → Repair control circuit → Test and run → Record and archive.

[0103] For each candidate path, each node and edge in the path is converted into a semantic vector. The cosine similarity between the path vector and the fault cause and location vectors is calculated using semantic similarity. This yields a path matching score for the candidate fault handling path. The path with the highest matching score is selected as the target path for fault handling. Finally, structured instructions are generated according to the path sequence of the target path for fault handling.

[0104] In a preferred embodiment, the training process of the fault handling model is as follows:

[0105] Obtain historical fault handling records for flexible DC converter stations, standard handling solutions annotated by experts, and a knowledge graph of fault solutions constructed based on power equipment operation and maintenance specifications and fault handling experience. Analyze historical fault handling records to extract information such as the cause and location of the fault, the corresponding sequence of effective handling steps, resource usage (such as tools and spare parts), and handling time.

[0106] The fault cause and fault location are input into the fault handling model to be trained. The fault handling model includes a fault plan graph network, a second path generation module, and a second decoding module. The fault plan graph network is constructed based on the fault plan knowledge graph. The nodes are fault handling related entities (such as fault type, handling operation, tool spare parts, safety specification requirements, etc.), and the edges are logical associations of the handling process (such as the sequence relationship between "power off operation" and "replacement of pump body", the constraint relationship between "using insulating tools" and "live work", etc.). Node vectors are generated through graph neural networks (GNN) or TransE embedding algorithms to give entities computable semantic representations; the second path generation module is used to generate several candidate fault handling paths based on node adjacency relationships; the second decoding module is used to calculate the path confidence for each candidate fault handling path, combined with the preset process rationality weight, resource matching weight, etc., and decode the high-confidence path into a specific executable fault handling plan. Multi-task joint optimization is used in the training process, and the loss function expression is:

[0107]

[0108] in, represents the total loss function of the fault handling model, Represents the mapping loss between fault information and solution diagram network nodes. Through comparative learning, the distance between the fusion feature vector of the fault cause and location and the correct node in the fault solution diagram network is minimized, and the distance between the fusion feature vector and the wrong node is maximized to ensure accurate mapping. represents the regularization loss of the fault scenario knowledge graph, represents the classification loss of the processing path. The cross entropy function is used to make the model more confident in the real and effective fault processing path than the false and inefficient path. γ and δ represent the task weights.

[0109] According to the loss function Optimize the node embedding vectors and edge weights of the fault solution graph network, as well as the logical parameters for path expansion in the second path generation module and the confidence calculation and decoding parameters of the second decoding module. Use a batch gradient descent algorithm, inputting batches of fused feature sample vectors, and perform multiple rounds of iterative optimization until the model achieves a perfect match for the fault solution on the validation set.

[0110] In a preferred embodiment, after performing fault diagnosis on the flexible DC converter station based on the fused eigenvector and obtaining the fault diagnosis result, the method further includes:

[0111] Obtain weak labels corresponding to the multimodal data of the flexible DC converter station; the weak labels are manual fault determination results of the multimodal data of the flexible DC converter station;

[0112] According to the multimodal data of the flexible DC converter station and its corresponding weak labels, the preset weights on the fault diagnosis target path in the fault diagnosis graph network built into the fault diagnosis model are iteratively optimized through gradient descent to obtain an updated fault diagnosis model.

[0113] In one embodiment of the present invention, after completing the fault diagnosis of the flexible DC converter station and obtaining the results, the present invention further designs a model iteration optimization mechanism, which retains the trainable interface after the fault diagnosis model is deployed, and performs local parameter updates regularly or in a triggered manner to continuously adapt to the actual operation scenario. Specifically, the weak labels obtained by manually performing fault judgment on the multimodal data of the converter station are first obtained. Such weak labels do not need to accurately locate the cause of the fault, but only make preliminary marks on whether it is abnormal. Based on the multimodal data of the converter station and the corresponding weak labels, the gradient descent algorithm is used to iteratively optimize the preset weights on the fault diagnosis target path in the fault diagnosis graph network in the fault diagnosis model, thereby updating the fault diagnosis model.

[0114] At the same time, a parameter rollback mechanism is introduced as a robustness guarantee. If performance anomalies occur after the model is updated, such as a decrease in the accuracy of the verification set or a serious deviation of the fault path, a rollback will be automatically triggered to restore the parameters to the stable state before the update.

[0115] By synergizing the self-learning mechanism with abnormal stabilization, the present invention enables the fault diagnosis model to break through the limitations of rigid deployment, continuously absorb operation and maintenance experience, and adapt to complex scenarios, ensuring that the diagnostic accuracy of the flexible DC converter station steadily improves with the evolution of operating conditions. The model response time is shortened to milliseconds, effectively responding to the uncertainty challenges in the environment.

[0116] like Figure 2 As shown, based on the above method embodiment, a corresponding device embodiment is provided;

[0117] An embodiment of the present invention provides a fault diagnosis device based on multimodal data of a flexible DC converter station, comprising: a multimodal data acquisition module, a feature extraction module, a feature fusion module, and a fault diagnosis module;

[0118] A multimodal data acquisition module is used to acquire multimodal data of the flexible DC converter station. Each modal data includes: a time series data and corresponding spatial data. The time series data includes: electrical parameters, status data or environmental data. The spatial data includes: a spectrum diagram of the time series data, an infrared image of the converter station surface, and a thermal image of the converter station surface.

[0119] The feature extraction module is used to input multimodal data into a preset feature extraction model so that the feature extraction model performs time series feature extraction on the time series data for each modal data to obtain a time series feature vector; performs local spatial structure feature extraction on the spatial data corresponding to the time series data to obtain a spatial feature vector; and performs linear transformation and feature integration on the time series feature vector and the spatial feature vector to obtain a high-dimensional semantic vector;

[0120] The feature fusion module is used to perform weighted fusion on the high-dimensional semantic vectors of all modal data to obtain a fused feature vector;

[0121] The fault diagnosis module is used to perform fault diagnosis on the flexible DC converter station based on the fused feature vector and obtain the fault diagnosis result.

[0122] In a preferred embodiment, the fault diagnosis device based on multi-modal data of a flexible HVDC converter station further includes: a data pre-processing module;

[0123] The data preprocessing module is used to perform outlier processing, missing value filling and normalization on the time series data in each modal data to obtain the preprocessed time series data.

[0124] It can be understood that the above-mentioned device embodiment corresponds to the method embodiment of the present invention, which can implement the fault diagnosis method based on multimodal data of the flexible DC converter station provided by any of the above-mentioned method embodiments of the present invention.

[0125] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Furthermore, in the drawings of the device embodiments provided by the present invention, the connection relationship between modules indicates that they have a communication connection, which may be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement the present invention without inventive effort.

[0126] Based on the above-mentioned embodiment of the fault diagnosis method based on multimodal data of a flexible DC converter station, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the fault diagnosis method based on multimodal data of a flexible DC converter station according to any embodiment of the present invention is implemented.

[0127] For example, in this embodiment, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to implement the present invention. The one or more module elements may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0128] The terminal device may be a computing device such as a desktop computer, a notebook computer, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0129] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, connecting various parts of the entire terminal device using various interfaces and lines.

[0130] Based on the above method embodiment, another embodiment is provided: another embodiment of the present invention provides a computer-readable storage medium, including a stored computer program, wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the fault diagnosis method based on multimodal data of the flexible DC converter station described in any one of the above method embodiments of the present invention.

[0131] Wherein, the module / unit integrated with the fault diagnosis device / terminal equipment based on the multimodal data of the flexible DC converter station, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0132] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A fault diagnosis method based on multimodal data of a flexible DC converter station, characterized in that: include: Obtain multimodal data of flexible DC converter stations; Each modal data includes: a time series data and corresponding spatial data; the time series data includes: electrical parameters, state data or environmental data; the spatial data includes: a spectrum of the time series data, an infrared image of the converter station surface and a thermal image of the converter station surface; Input the multimodal data into a preset feature extraction model, so that the feature extraction model performs time series feature extraction on the time series data for each modal data to obtain a time series feature vector; performs local spatial structure feature extraction on the spatial data corresponding to the time series data to obtain a spatial feature vector; and performs linear transformation and feature integration on the time series feature vector and the spatial feature vector to obtain a high-dimensional semantic vector; Perform weighted fusion on the high-dimensional semantic vectors of all modal data to obtain a fused feature vector; According to the fused feature vector, fault diagnosis is performed on the flexible DC converter station to obtain a fault diagnosis result.

2. The fault diagnosis method based on multimodal data of a flexible DC converter station according to claim 1 is characterized in that: According to the fused feature vector, fault diagnosis is performed on the flexible DC converter station to obtain a fault diagnosis result, including: Inputting the fused feature vector into a preset fault diagnosis model so that the fault diagnosis model performs attention feature enhancement through a built-in Transformer encoder to obtain an enhanced feature vector; Mapping the enhanced feature vector to a built-in fault diagnosis graph network to determine a first mapping node in the fault diagnosis graph network; generating a plurality of candidate fault diagnosis paths according to the adjacency relationship of the first mapping node in the fault diagnosis graph network; For each candidate fault diagnosis path, the confidence of the candidate fault diagnosis path is calculated according to the preset weight in the candidate fault diagnosis path; From all the fault diagnosis candidate paths, select the fault diagnosis candidate path with the highest confidence as the fault diagnosis target path; Performing feature decoding on the fault diagnosis target path to generate a fault diagnosis result; When the fault diagnosis result indicates that a fault exists, the fault cause and fault location are generated.

3. The fault diagnosis method based on multimodal data of a flexible DC converter station according to claim 2 is characterized in that: If the fault diagnosis result indicates that a fault exists, after generating the fault cause and fault location, the following is also included: Inputting the fault cause and fault location into a preset fault processing model so that the fault processing model maps the fault cause and fault location to a built-in fault solution graph network, and determining a second mapping node in the fault solution graph network; generating a plurality of candidate fault handling paths according to the adjacency relationship of the second mapping nodes in the fault solution graph network; Calculate the semantic similarity of each candidate fault handling path to obtain the path matching score of each candidate fault handling path; From all candidate fault handling paths, select the candidate fault handling path with the highest path matching score as the target fault handling path; Determine the fault handling plan based on the fault handling target path.

4. The fault diagnosis method based on multimodal data of a flexible DC converter station according to claim 2, characterized in that: After performing fault diagnosis on the flexible DC converter station based on the fused feature vector and obtaining a fault diagnosis result, the method further includes: Acquire a weak label corresponding to the multimodal data of the flexible DC converter station; the weak label is a result of manual fault determination of the multimodal data of the flexible DC converter station; According to the multimodal data of the flexible DC converter station and its corresponding weak labels, the preset weights on the fault diagnosis target path in the fault diagnosis graph network set in the fault diagnosis model are iteratively optimized through gradient descent to obtain an updated fault diagnosis model.

5. The fault diagnosis method based on multimodal data of a flexible DC converter station according to claim 1, characterized in that: After obtaining the multimodal data of the flexible DC converter station, the following is also included: The time series data in each modal data is processed for outliers, missing values ​​are filled, and normalization is performed to obtain the preprocessed time series data.

6. The fault diagnosis method based on multimodal data of a HVDC flexible converter station according to claim 1, characterized in that: The high-dimensional semantic vectors of all modal data are weightedly fused to obtain the fused feature vector, including: Generate query vector, key vector and value vector of each modal data according to the high-dimensional semantic vector of each modal data; For each modality combination, the attention score between the two modal data in the modality combination is calculated according to the query vector and key vector of the two modal data in the modality combination; According to the attention score between the two modal data in each modality combination, the value vector of each modal data is summed up with attention weights to obtain the cross-modal enhancement vector of each modal data; The cross-modal enhancement vectors of each modality data are feature concatenated to obtain a fused feature vector.

7. A fault diagnosis device based on multimodal data of a flexible DC converter station, characterized in that: include: Multimodal data acquisition module, feature extraction module, feature fusion module and fault diagnosis module; The multimodal data acquisition module is used to acquire multimodal data of the flexible DC converter station; Each modal data includes: a time series data and corresponding spatial data; the time series data includes: electrical parameters, state data or environmental data; the spatial data includes: a spectrum of the time series data, an infrared image of the converter station surface and a thermal image of the converter station surface; The feature extraction module is used to input multimodal data into a preset feature extraction model, so that the feature extraction model performs time series feature extraction on the time series data for each modal data to obtain a time series feature vector; performs local spatial structure feature extraction on the spatial data corresponding to the time series data to obtain a spatial feature vector; and performs linear transformation and feature integration on the time series feature vector and the spatial feature vector to obtain a high-dimensional semantic vector; The feature fusion module is used to perform weighted fusion on the high-dimensional semantic vectors of all modal data to obtain a fused feature vector; The fault diagnosis module is used to perform fault diagnosis on the flexible DC converter station according to the fused feature vector to obtain a fault diagnosis result.

8. The fault diagnosis device based on multimodal data of a flexible DC converter station according to claim 7, characterized in that: Also includes: Data preprocessing module; The data preprocessing module is used to perform outlier processing, missing value filling and normalization on the time series data in each modal data to obtain preprocessed time series data.

9. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for fault diagnosis based on multimodal data of a flexible DC converter station according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the fault diagnosis method based on multi-modal data of a flexible DC converter station according to any one of claims 1 to 6.

Images