Transform-based power grid data multi-modal soft fusion method and system

Abstract

The invention provides a power grid data multi-mode soft fusion method and system based on Transform, and belongs to the technical field of power grid data processing. According to the scheme, the method comprises the steps of obtaining text resources, multimedia resources, maintenance data, a power grid model, operation data and power transmission and transformation equipment monitoring information, and constructing a power grid data source; carrying out the preprocessing of a power grid data source through employing an improved K-means clustering method, a reinforcement learning method and a synthetic sampling SMOTE method; performing data modeling, data mining, knowledge processing and knowledge management on the preprocessed data to form a power grid data knowledge base; and fusing the power grid data in the power grid data knowledge base by adopting a trained Transform multi-mode soft fusion model. By constructing a knowledge graph and based on technologies of Transform fusion, GAN and the like, the problems of large quality difference of state data of power transmission and transformation equipment and heterogeneous data fusion are solved, and deep fusion of multi-source data is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of power grid data processing, and in particular relates to a Transformer-based power grid data multi-modal soft fusion method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of computer technology and artificial intelligence technology, information fusion technology has been further promoted. Modern information fusion technology makes full use of multi-sensor data resources at different times and spaces, such as sensors, databases, knowledge bases, and information obtained by humans themselves, and uses computer technology to analyze, synthesize, control and use multi-sensor observation data obtained in time series to obtain a consistent interpretation and description of the measured object, and then realize the corresponding decision-making and estimation. The core of information fusion technology lies in the coordinated optimization and comprehensive processing of multi-source information. It involves multiple links such as detection, interconnection, correlation, estimation and information combination, aiming to achieve a comprehensive evaluation of the state, identity and entire tactical situation of the measured object or target through multi-level and multi-faceted processing. This technology not only improves the reliability and fault tolerance of the system, but also significantly enhances the system's adaptability to complex environments.

[0004] However, most of the multi-source information fusion capabilities for power-related industries are poor, mainly manifested in the differences between equipment, systems and software platforms, the processing and storage of massive power data, real-time and accuracy issues, and the lack of unified standards and specifications. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a Transformer-based multimodal soft fusion method and system for power grid data, which solves the problem of difficulty in deep fusion of image, text and other data by constructing a knowledge graph of multidimensional information fusion for power equipment, and breaks through the current situation of data surge; designs a synthetic sampling sample optimization method based on reinforcement learning to solve problems such as low data value density and imbalance, improves analysis accuracy, designs and implements a soft fusion model of data, realizes seamless fusion and intelligent analysis of multi-source heterogeneous data, and then builds a complete, accurate and timely knowledge base, and realizes the provision of basic services for transmission equipment through the data in the soft fusion model, including expert system, knowledge subject editing, knowledge search, knowledge publishing and program interface services, which helps the intelligent transformation and efficiency improvement of transmission and transformation equipment monitoring.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0007] The first aspect of the present invention provides a Transformer-based multi-modal soft fusion method for power grid data;

[0008] A Transformer-based multimodal soft fusion method for power grid data, comprising:

[0009] Acquiring data related to power grid operation and preprocessing the data;

[0010] Constructing a state assessment model and a fault diagnosis model for power transmission equipment based on the preprocessed data; using association rules to perform data mining on the relationship between the data generated in the constructed state assessment model and the fault diagnosis model for power transmission equipment, and converting the results of the data mining into a centralized monitoring knowledge base for power transmission equipment, wherein the centralized monitoring knowledge base for power transmission equipment includes data related to power grid operation and potential associations between the constructed state assessment model and the fault diagnosis model for power transmission equipment;

[0011] The trained Transformer multimodal soft fusion model is used to fuse the data in the centralized monitoring knowledge base of the power transmission equipment and construct a knowledge graph; based on the fusion results and the knowledge graph, intelligent service support is provided to the power transmission equipment.

[0012] Among them, the Transformer multimodal soft fusion model is used to extract the multimodal features of the data in the centralized monitoring knowledge base of the power transmission equipment, and perform cross-modal alignment; the multimodal features after cross-modal alignment are fused using the attention mechanism to construct a knowledge graph.

[0013] As a further technical solution, the relevant data of the power grid operation includes: text resource data, multimedia resource data, maintenance data, power grid model data and power transmission and transformation equipment monitoring information data.

[0014] As a further technical solution, the improved K-means clustering, reinforcement learning and synthetic sampling SMOTE methods are used to pre-process the relevant data of power grid operation;

[0015] Among them, K-means clustering makes the cluster center tend to the area with dense data distribution, thereby improving the convergence speed and accuracy; its formula is as follows:

[0016]

[0017] Among them, λ is the balance coefficient, which controls the influence of cluster center density and ensures that more samples are selected in areas with sparse data distribution; J is the objective function, which is used to measure the quality of cluster centers; by minimizing J, a better cluster center is found, so that the data point is as close to the center of its cluster as possible, and more samples are selected in areas with sparse data distribution; x is a sample point, which represents a specific data point in the data set and is the object to be clustered; each x belongs to a specific cluster G i ;μ i is the cluster center, indicating the i-th cluster G i By adjusting the location of the cluster center, the distance from the sample point to the center is reduced, and the clustering effect is improved; Density (C i ) is the density function, which represents the cluster G i The density of reflects the distribution density of data points around the cluster center.

[0018] As a further technical solution, the process of using association rules to perform data mining on the relationship between data generated in the constructed power transmission equipment status evaluation and fault diagnosis model is as follows:

[0019] Considering operations and faults as item sets, we calculate the frequency of a certain item set in all transactions. For the joint item set X∪Y of operation X and fault Y, the support calculation formula is:

[0020]

[0021] Calculate the confidence level, which indicates the probability of Y occurring when X is known to occur. The confidence calculation formula is:

[0022]

[0023] According to the set minimum support and minimum confidence thresholds, strong association rules are screened out and the rules that meet the conditions are stored in the knowledge base.

[0024] As a further technical solution, the process of using the trained Transformer multimodal soft fusion model to fuse the data in the centralized monitoring knowledge base of the power transmission equipment and construct a knowledge graph also includes:

[0025] Determine the specific requirements for multimodal data fusion, analyze the data characteristics, advantages and limitations in the centralized monitoring knowledge base of power transmission equipment, and determine the fusion strategy;

[0026] Based on the fusion strategy, the attention mechanism is used to fuse the multimodal features after cross-modal alignment to construct and update the knowledge graph.

[0027] As a further technical solution, the process of training the Transformer multimodal soft fusion model is as follows:

[0028] Use the preprocessed data to train the multimodal soft fusion model and adjust the hyperparameters to optimize the performance;

[0029] The model performance is evaluated through cross-validation and test set evaluation, and the model is iteratively optimized based on the evaluation results.

[0030] As a further technical solution, the intelligent service support includes: fault prediction and early warning, power transmission equipment health management, operation optimization, intelligent diagnosis and decision support;

[0031] The fault prediction and early warning are used to predict power transmission equipment failures and issue early warnings;

[0032] The power transmission equipment health management is used to monitor the equipment status in real time and provide maintenance and care suggestions;

[0033] The operation optimization is used to automatically adjust the load of the power transmission equipment and improve the operation efficiency;

[0034] The intelligent diagnosis is used to automatically identify power transmission equipment problems and provide solutions;

[0035] The decision support is used to provide accurate decision suggestions to operation and maintenance personnel.

[0036] A second aspect of the present invention provides a Transformer-based multi-modal soft fusion system for power grid data.

[0037] A Transformer-based multi-modal soft fusion system for power grid data, comprising:

[0038] The data acquisition and preprocessing module is configured to: acquire data related to the operation of the power grid and preprocess the data;

[0039] The knowledge base construction module is configured to: construct a state assessment model and a fault diagnosis model of the power transmission equipment based on the preprocessed data; use association rules to perform data mining on the relationship between the data generated in the constructed state assessment model and the fault diagnosis model of the power transmission equipment, and convert the results of the data mining into a centralized monitoring knowledge base of the power transmission equipment, wherein the centralized monitoring knowledge base of the power transmission equipment includes data related to the operation of the power grid and potential associations between the constructed state assessment model and the fault diagnosis model of the power transmission equipment;

[0040] The multimodal feature soft fusion module is configured to: use the trained Transformer multimodal soft fusion model to fuse the data in the power transmission equipment centralized monitoring knowledge base and construct a knowledge graph; wherein the Transformer multimodal soft fusion model is used to extract the multimodal features of the data in the power transmission equipment centralized monitoring knowledge base and perform cross-modal alignment; the attention mechanism is used to fuse the multimodal features after cross-modal alignment to construct a knowledge graph;

[0041] The service support module is configured to provide intelligent service support for power transmission equipment based on the fusion results and knowledge graph; wherein, based on the fusion results and knowledge graph, the knowledge system in the field of power transmission equipment monitoring is enriched, the fusion results and knowledge graph are integrated into the power transmission equipment monitoring system, and intelligent service support is provided.

[0042] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in a Transformer-based multimodal soft fusion method for power grid data as described in the first aspect of the present invention.

[0043] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the Transformer-based multimodal soft fusion method for power grid data as described in the first aspect of the present invention are implemented.

[0044] One or more of the above technical solutions have the following beneficial effects:

[0045] The present invention solves the problem of the difficulty in deep integration of image, text and other data by constructing a knowledge graph of multi-dimensional information fusion for power equipment, and breaks through the current situation of data explosion; designs a synthetic sampling sample optimization method based on reinforcement learning to solve the problems of low data value density and imbalance, improves analysis accuracy, designs and implements a soft fusion model of data, realizes seamless integration and intelligent analysis of multi-source heterogeneous data, and then builds a complete, accurate and timely knowledge base, and realizes the provision of basic services for power transmission equipment through the data in the soft fusion model, including expert system, knowledge theme editing, knowledge search, knowledge publishing and program interface services, which helps the intelligent transformation and efficiency improvement of power transmission and transformation equipment monitoring.

[0046] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0048] Figure 1 This is a flow chart of the method of the first embodiment.

[0049] Figure 2 It is a system structure diagram of the second embodiment. DETAILED DESCRIPTION

[0050] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0051] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0052] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0053] The multi-source information fusion capability of the power-related industry is relatively poor, mainly manifested in the processing and storage of technical compatibility issues, real-time and accuracy issues, and the lack of unified standards and specifications. In the following embodiments, by constructing a knowledge graph, based on Transformer fusion and GAN technologies, the problem of large differences in the quality of power transmission and transformation equipment status data and heterogeneous data fusion is solved, and the purpose of deep fusion of multi-source data is achieved.

[0054] Embodiment 1

[0055] This embodiment discloses a transformer-based multimodal soft fusion method for power grid data; wherein, soft fusion refers to a data fusion technology that combines data of different modes (such as text, images, videos, structured data, etc.) to achieve seamless fusion and intelligent analysis of multi-source heterogeneous data. Unlike hard fusion, soft fusion does not directly perform mandatory splicing or conversion on the original data, but extracts, weights and fuses the features of each mode through a model (such as a transformer-based deep learning model), so that the data of each mode is represented in a unified feature space.

[0056] like Figure 1 As shown, a Transformer-based multi-modal soft fusion method for power grid data includes:

[0057] Step S1, acquiring data related to the operation of the power grid and preprocessing the data;

[0058] In step S1, data related to the operation of the power grid includes text resource data, multimedia resource data, maintenance data, power grid model data and power transmission and transformation equipment monitoring information.

[0059] Among them, text resources refer to text information related to the power grid, including: data information of power transmission equipment contained in technical documents such as equipment manuals, maintenance guides, technical specifications, etc.; fault cause analysis and maintenance records, equipment maintenance history, maintenance suggestions and other information contained in fault reports and maintenance records; equipment operation logs, daily inspections and equipment status analysis reports and other information contained in operation logs and operation and maintenance reports.

[0060] Multimedia resources refer to non-textual perceptual data, including inspection photos, operation videos, audio data and other information.

[0061] The grid model data includes grid topology, operation status data, historical operation records and real-time monitoring data. The grid topology contains the physical and logical connection relationships of various devices in the grid, such as transformers, transmission lines, generators, etc. The operation status data includes basic electrical parameters such as voltage, current, frequency, as well as the switch status of the equipment, the status of the protection device, etc. The historical operation records store past operation data, including fault records, maintenance records, etc., which are used to analyze and optimize grid operation. Real-time monitoring data includes load, power flow, fault indication, etc., which are usually used for real-time monitoring and control systems to ensure the stable operation of the grid.

[0062] Through crawler technology and API interface calls, text resources such as documents and fault reports of relevant power transmission equipment, as well as image and video resources such as equipment operation status videos and inspection photos are collected. Among them, crawler technology is used to realize automatic data capture, and specific data is extracted from the website by simulating the behavior of users browsing web pages. Crawlers can automatically access web pages, parse HTML structures, and capture text, pictures and other information. API interface calls access data on remote servers through predefined interfaces. API (application programming interface) allows data exchange between different systems. When calling API, the client sends a request to the server, and the server returns the required data, usually in structured JSON or XML format, which is often used to obtain real-time data from online databases and platforms and obtain historical maintenance data from maintenance record systems. At the same time, real-time operation data such as the topology of the power grid is obtained from the power grid management system, and the operation status and alarm information of the equipment are captured from the monitoring system of the power transmission equipment in real time or regularly.

[0063] Furthermore, the relevant data of the power grid operation is preprocessed. In the above preprocessing process, improved K-means clustering, reinforcement learning and synthetic sampling SMOTE are used to balance the data ratio, and a sample optimization method is designed and implemented to ensure the representativeness and integrity of the data sample. The preprocessing process includes:

[0064] S11. Clean the relevant data on power grid operation obtained, use deduplication algorithms or perform uniqueness checks on primary keys, delete duplicate data to ensure data accuracy; use default values ​​(such as 0 or NULL), linear interpolation, or average values ​​based on context data to supplement missing data to ensure the integrity of the data set; use statistical or model-based methods to identify and process outliers, including deleting obviously erroneous data points, or using reasonable alternative values ​​(such as means, predicted values) to correct abnormal data to maintain data consistency.

[0065] S12, convert all data into a unified format standard and merge data from different sources into a unified data set; for example, convert non-ASCII characters in text data into standard encoding.

[0066] S13, merge the power grid operation data from different sources into a unified data set; then associate the different fields in the data set to resolve conflicts and inconsistencies between fields, and ensure the accuracy and consistency of the data after integration.

[0067] S14, use SMOTE to perform synthetic sampling on minority class samples to increase their proportion in the data set and achieve class balance; minority class samples refer to samples with a smaller number of categories in the data set. Compared with majority class samples, the number of such samples is very limited, which usually leads to data imbalance problems. In power grid data, minority class samples may refer to some rare equipment failures, abnormal conditions, or equipment operation data under extreme conditions. Such samples may be ignored during model training due to their small number, making it difficult for the model to accurately identify and predict these special situations.

[0068] The formula for SMOTE sampling is:

[0069]

[0070] Among them, λ is the weight parameter, x min is a minority class sample, x near is the neighboring sample, Density(x min ) represents the local density of minority class samples and is used to control the generation method of synthetic samples, thereby improving the quality of data distribution after sampling.

[0071] Based on the improved K-means clustering method, representative samples are selected to reduce redundant data; the traditional K-means algorithm is usually based on randomly initialized cluster centers, which may lead to different local optimal solutions. In order to avoid the above problems, the improved K-means algorithm is used to make the cluster centers closer to the areas with dense data distribution, thereby improving the convergence speed and accuracy. The specific improvement can be achieved through the following formula:

[0072]

[0073] Among them, λ is the balance coefficient, which controls the influence of cluster center density and ensures that more samples are selected in areas with sparse data distribution; J is the objective function, which is used to measure the quality of cluster centers. By minimizing J, a better cluster center can be found, so that the data point is as close to the center of its cluster as possible, and more samples are selected in areas with sparse data distribution; x is a sample point, which represents a specific data point in the data set and is the object to be clustered. Each x belongs to a specific cluster G. i ;μ i is the cluster center, indicating the i-th cluster G i By adjusting the location of the cluster center, the distance from the sample point to the center can be reduced, thus improving the clustering effect; Density (C i ) is the density function, which represents the cluster G i The density of reflects the density of data points around the cluster center. The introduction of density function can select more sample points in areas with sparse data distribution, which helps to improve the representativeness of clustering.

[0074] Reinforcement learning is used to optimize data sampling. First, the state space (S) is defined as all possible states of power grid operation, such as equipment load and fault status, and the action space (A) is set to operations such as adjusting load and switching backup equipment. The reward function (R) is designed to evaluate the effect of the action, and positive rewards are obtained by reducing the number of faults or reducing operating costs. When initializing the parameters, the learning rate (α) is set to 0.1, the discount factor (γ) is set to 0.9, and the exploration rate (ε) is initially 0.2 to balance exploration and utilization. In the interaction with the environment, the ε-greedy strategy is used to select actions, and the Q value is updated through the Q-learning algorithm. The formula is

[0075] (,)←(,)+[+max(,)-(,)]

[0076] Where Q(s,a) represents the value of selecting action a in state s, and r is the current reward.

[0077] Reinforcement learning can balance the distribution of categories in power grid data, especially when dealing with unbalanced data, which can effectively improve the prediction performance of subsequent models. During data preprocessing and model training, reinforcement learning can automatically adjust algorithm parameters, such as learning rate, number of clusters, etc., to achieve the best data processing effect.

[0078] S15, select the features that have the greatest impact on the target variable through statistical methods or machine learning models; use PCA (principal component analysis), LDA (linear discriminant analysis) and other technologies to reduce data dimensions, improve computing efficiency, and finally complete the preprocessing of multi-source power grid operation data.

[0079] Step S2, constructing a state assessment model and a fault diagnosis model of the power transmission equipment based on the preprocessed data; using association rules to perform data mining on the relationship between the data generated in the constructed state assessment model and the fault diagnosis model of the power transmission equipment, and converting the results of the data mining into a centralized monitoring knowledge base for the power transmission equipment, wherein the centralized monitoring knowledge base for the power transmission equipment includes data related to the operation of the power grid and potential associations between the two constructed models of the power transmission equipment.

[0080] In step S2, the process of constructing a centralized monitoring knowledge base for power transmission equipment includes:

[0081] Step S21, data modeling is performed based on the preprocessed data. By analyzing the preprocessed data, a state assessment model and a fault diagnosis model of the power transmission equipment are constructed to assess the state of the power transmission equipment and predict faults. In step S21, a state assessment model and a fault diagnosis model of the power transmission equipment based on a neural network can be constructed.

[0082] Step S22, using association rules to perform data mining on the relationship between the data generated in the state assessment model and the fault diagnosis model of the power transmission equipment, and obtain the potential relationship between the two models of the power transmission equipment; wherein, the process of using association rules to perform data mining on the relationship between the data generated in the state assessment and fault diagnosis models of the power transmission equipment is as follows:

[0083] When using association rule algorithms to mine the correlation between faults and specific operations, we first consider operations and faults as item sets. In a data set, each record represents an operation and the state of the device (whether it is faulty or not). For example, a record may include "switch 1 operation" and "fault occurred".

[0084] Next, calculate the support, that is, the frequency of a certain item set appearing in all transactions. For the joint item set X∪Y of operation X and fault Y, the support calculation formula is:

[0085]

[0086] Then, we calculate the confidence level, which represents the probability that Y will occur given that X has occurred. The confidence calculation formula is:

[0087]

[0088] Next, strong association rules are screened out according to the set minimum support and minimum confidence thresholds. Finally, the qualified rules are stored in the knowledge base for subsequent fault diagnosis and decision support.

[0089] Furthermore, obtaining the potential correlation between the two models of transmission equipment refers to the deep-level regularities or correlations in the data that cannot be easily discovered through direct observation or simple statistics. These patterns and relationships are usually revealed through data mining techniques, which can help identify potential problems, trends or correlations in equipment operation. In the power grid operation data, these hidden patterns and relationships include but are not limited to:

[0090] Potential causes of equipment failure: By analyzing the equipment's operating status, environmental conditions, and historical maintenance records, we can find out why equipment is more likely to fail under certain conditions. For example, a certain temperature change or a specific voltage fluctuation may be highly correlated with equipment failure.

[0091] Association between equipment operating states: There may be collaborative or linkage effects between equipment. Through data mining, it is possible to identify the impact of the operating states of certain equipment on other equipment. For example, a change in the load of a transformer may cause an overload problem for adjacent equipment.

[0092] The relationship between maintenance cycle and failure probability: By analyzing historical maintenance data, the optimal maintenance cycle for equipment can be discovered, thereby predicting when the equipment is most likely to fail.

[0093] Abnormal operation modes: Certain transition states between normal operation and failure of equipment may hide abnormal modes that are difficult to identify through direct observation but can be discovered through clustering or classification algorithms. For example, equipment may show hidden changes in unstable power output before failure.

[0094] Step S23, performing knowledge processing on the above data mining results, and expressing the data mining results in a knowledge form that is easy to understand and use, such as a rule base, case base, etc.; and by establishing a knowledge base management system, storing, updating, retrieving and sharing the knowledge in the rule base or case base, and finally obtaining a centralized monitoring knowledge base for transmission equipment to provide support for intelligent decision-making.

[0095] Step S3, using the trained Transformer multimodal soft fusion model to fuse the data in the centralized monitoring knowledge base of the power transmission equipment and construct a knowledge graph; based on the fusion results and the knowledge graph, providing intelligent service support for the power transmission equipment.

[0096] Among them, the Transformer multimodal soft fusion model is used to extract the multimodal features of the data in the centralized monitoring knowledge base of the power transmission equipment, and cross-modal alignment is performed; the multimodal features after cross-modal alignment are fused using the attention mechanism to construct a knowledge graph; based on the fusion results and the knowledge graph, the knowledge system in the field of power transmission equipment monitoring is enriched, and the fusion results and the knowledge graph are integrated into the power transmission equipment monitoring system to provide intelligent service support.

[0097] In this embodiment, a multimodal soft fusion model is trained using a labeled data set, and hyperparameters such as learning rate (α), batch size, and number of training rounds (epochs) are adjusted to optimize performance. The formula is as follows:

[0098] Performance=Evaluate(model(α,batch size,epochs),X train ,Y train );

[0099] In the formula, Evaluate is the model evaluation function, X train and Y train are the training data and labels.

[0100] The model performance is evaluated through cross-validation, test set evaluation and other methods. According to the evaluation result feedback, the model is iteratively optimized. The K-fold cross-validation method is used to divide the training data into K subsets, and each subset is used as the validation set in turn, and the other K-1 subsets are used as the training set. The performance calculation formula after each training is:

[0101]

[0102] Among them, Performance represents the performance evaluation result of the model on the validation set. and is the validation set data and label of the i-th fold.

[0103] Furthermore, in step S3, the fusion process includes:

[0104] S31, determine the specific needs of multimodal data fusion, such as improving the accuracy of fault diagnosis, optimizing operation and maintenance decisions, etc. Analyze the data characteristics, advantages and limitations in the centralized monitoring knowledge base of power transmission equipment, and determine the fusion strategy. The purpose of the fusion strategy determined in this step is to improve the accuracy of fault diagnosis and optimize the specific methods and strategies proposed for operation and maintenance decisions. By effectively combining different types of data (such as multimodal data: text, video data, etc.), the system can more comprehensively analyze the status and faults of power transmission equipment, thereby improving the system's intelligent decision-making capabilities;

[0105] S32, extract the multimodal features of the data in the centralized monitoring knowledge base of power transmission equipment, and use natural language processing technology (such as TF-IDF, Word2Vec, BERT, etc.) to extract the semantic features of text resources. For multimedia resources such as images and videos, convolutional neural networks (CNN), deep learning and other methods are used to extract visual features. For structured data such as maintenance data and power grid model data, statistical analysis, time series analysis and other methods are used to extract features; for the extracted multimodal features, alignment functions are used to ensure the mutual mapping of different modal features, achieve cross-modal alignment, and solve the semantic gap between different modal data. The formula is:

[0106]

[0107] In the formula, A ij is the similarity or correlation between device i and device j (calculated by cosine similarity). The higher the similarity or correlation, the better the modal feature alignment effect, which means that the information of different modalities is more consistent and relevant in the same space. i is the multimodal feature vector of device i, which contains all modal data of the device (such as text, image, video data, etc.); F j is the multimodal feature vector of device j, which contains all modal data of the device; ||F i || is the norm (length) of the feature vector F of device i; ||||F j |||| is the feature vector F of device j j The norm (length) of .

[0108] S33, based on the fusion strategy, the multimodal features after cross-modal alignment are input into the trained Transformer multimodal soft fusion model for multimodal feature fusion, to construct a knowledge graph and enrich the knowledge system in the field of power transmission equipment monitoring. Specifically, the multimodal features are processed by the self-attention mechanism to generate query features (Q), key features (K) and value features (V), and the similarity between features is calculated by the dot product of Q and K, and the formula is as follows:

[0109]

[0110] Among them, the query feature (Q) represents the feature information in the image data, and is used as the main modal feature to guide the model and to find relevant information in other modalities; the key feature (K) is the feature in the text or sensor data, which is used to perform similarity matching with the query feature to help the model determine the degree of association between other modal information and image features; the value feature (V) is the content paired with the key feature, which contains the actual text description or sensor data information. After the query feature is matched with the key feature, the weighted sum is calculated according to the weight to generate the fusion feature Z.

[0111] The Softmax function normalizes the similarity into weights, so that the attention mechanism can focus on features with higher correlation. Through this weighting, the query feature can give priority to the value features that are closely related to the key features, and finally obtain the fused feature Z. The fused feature Z is the weighted fusion result of the multimodal information after the Transformer attention mechanism, and is closely related to the acquired multimodal features. The aligned multimodal features such as images, texts, and sensors generate the fused feature Z in the Transformer model, which uniformly represents the key information of different modalities. The multi-head attention mechanism further enriches the feature expression and ensures that the diverse associations between different modal information can be effectively captured.

[0112] The fused multimodal feature Z is integrated into the knowledge graph to update the status information and fault diagnosis data of the power equipment, and to establish and strengthen the relationship between different entities (such as equipment, fault events, operating status, etc.) in the graph. For example, the fused multimodal feature Z can identify the equipment damage characteristics in the image and update the equipment risk status in the knowledge graph. At the same time, the fused multimodal feature Z supports real-time status assessment and fault warning, and generates health monitoring and warning information by integrating the features of each modality, thereby improving the intelligence level and decision support capabilities of the system. Through this dynamic update, the knowledge graph is continuously enriched, providing comprehensive and accurate feature support for the health management, fault prediction and intelligent diagnosis of power equipment. At the same time, the new associations in the knowledge graph can also help the system reveal the potential risks and health status of the equipment, improve the intelligence level and warning capabilities of monitoring, and thus provide strong decision support for power grid operation.

[0113] The fused feature Z is integrated into the knowledge graph to update the status assessment and fault diagnosis information. The fused features are used to establish or update the relationship graph between entities using the knowledge graph, thereby enriching the knowledge system in the field of power transmission equipment monitoring. At the same time, in this embodiment, the fused data is also parsed to extract useful information, such as equipment health status, potential fault risks, etc. Therefore, the Transformer multimodal soft fusion model provides basic data and feature support for the construction of the knowledge graph, and the two work together to achieve intelligent monitoring and decision-making of power grid equipment.

[0114] Step S4, providing intelligent service support for power transmission equipment based on the fusion results and knowledge graph. In step S4, the fusion results and knowledge graph are integrated into the power transmission equipment monitoring system to provide intelligent service support. Among them, intelligent service support includes: fault prediction and early warning, power transmission equipment health management, operation optimization, intelligent diagnosis and decision support; the fault prediction and early warning are used to predict power transmission equipment failures and issue early warnings; the power transmission equipment health management is used to monitor the equipment status in real time and provide maintenance and maintenance suggestions; the operation optimization is used to automatically adjust the load of power transmission equipment and improve operation efficiency; the intelligent diagnosis is used to automatically identify power transmission equipment problems and provide solutions; the decision support is used to provide accurate decision suggestions for operation and maintenance personnel.

[0115] Embodiment 2

[0116] This embodiment discloses a Transformer-based power grid data multi-modal soft fusion system;

[0117] like Figure 2 As shown, a Transformer-based power grid data multimodal soft fusion system includes:

[0118] The data acquisition and preprocessing module is configured to: acquire data related to the operation of the power grid and preprocess the data;

[0119] The knowledge base construction module is configured to: construct a transmission equipment status assessment and fault diagnosis model based on preprocessed data; use association rules to perform data mining on the relationship between the data generated in the constructed transmission equipment status assessment and fault diagnosis model to obtain the hidden patterns and relationships between the data; convert the data mining results into a transmission equipment centralized monitoring knowledge base, and establish a knowledge base management system to store, update, retrieve and share the knowledge in the knowledge base to provide support for intelligent decision-making;

[0120] The multimodal feature soft fusion module is configured to: use the trained Transformer multimodal soft fusion model to fuse the data in the power transmission equipment centralized monitoring knowledge base and construct a knowledge graph; wherein the Transformer multimodal soft fusion model is used to extract the multimodal features of the data in the power transmission equipment centralized monitoring knowledge base and perform cross-modal alignment; the attention mechanism is used to fuse the multimodal features after cross-modal alignment to construct a knowledge graph;

[0121] The service support module is configured to provide intelligent service support for power transmission equipment based on the fusion results and knowledge graph; wherein, based on the fusion results and knowledge graph, the knowledge system in the field of power transmission equipment monitoring is enriched, the fusion results and knowledge graph are integrated into the power transmission equipment monitoring system, and intelligent service support is provided.

[0122] Embodiment 3

[0123] The purpose of this embodiment is to provide a computer-readable storage medium.

[0124] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in a transformer-based multimodal soft fusion method for power grid data as described in Example 1 of the present disclosure.

[0125] Embodiment 4

[0126] The purpose of this embodiment is to provide an electronic device.

[0127] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in a transformer-based multimodal soft fusion method for power grid data as described in Example 1 of the present disclosure are implemented.

[0128] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0129] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0130] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A Transformer-based multimodal soft fusion method for power grid data, characterized in that: include: Acquire data related to power grid operation and pre-process the data; Constructing a state assessment model and a fault diagnosis model for power transmission equipment based on the preprocessed data; using association rules to perform data mining on the relationship between the data generated in the constructed state assessment model and the fault diagnosis model for power transmission equipment, and converting the results of the data mining into a centralized monitoring knowledge base for power transmission equipment, wherein the centralized monitoring knowledge base for power transmission equipment includes data related to power grid operation and potential associations between the constructed state assessment model and the fault diagnosis model for power transmission equipment; The trained Transformer multimodal soft fusion model is used to fuse the data in the centralized monitoring knowledge base of the power transmission equipment and construct a knowledge graph; based on the fusion results and the knowledge graph, intelligent service support is provided to the power transmission equipment. The Transformer multimodal soft fusion model is used to extract multimodal features of data in the centralized monitoring knowledge base of power transmission equipment, and perform cross-modal alignment; The attention mechanism is used to fuse the multimodal features after cross-modal alignment to construct a knowledge graph.

2. The Transformer-based multi-modal soft fusion method for power grid data according to claim 1, characterized in that: The relevant data of the power grid operation include: text resource data, multimedia resource data, maintenance data, power grid model data and power transmission and transformation equipment monitoring information data.

3. The Transformer-based multi-modal soft fusion method for power grid data according to claim 1, characterized in that: Improved K-means clustering, reinforcement learning and synthetic sampling SMOTE methods are used to preprocess the relevant data of power grid operation; Among them, K-means clustering makes the cluster center tend to the area with dense data distribution, thereby improving the convergence speed and accuracy; its formula is as follows: Among them, λ is the balance coefficient, which controls the influence of cluster center density and ensures that more samples are selected in areas with sparse data distribution; J is the objective function, which is used to measure the quality of cluster centers; by minimizing J, a better cluster center is found, so that the data point is as close to the center of its cluster as possible, and more samples are selected in areas with sparse data distribution; x is a sample point, which represents a specific data point in the data set and is the object to be clustered; each x belongs to a specific cluster G i ;μ i is the cluster center, indicating the i-th cluster G i By adjusting the location of the cluster center, the distance from the sample point to the center is reduced, and the clustering effect is improved; Density (C i ) is the density function, which represents the cluster G i The density of reflects the distribution density of data points around the cluster center.

4. The Transformer-based multi-modal soft fusion method for power grid data according to claim 1, characterized in that: The process of using association rules to perform data mining on the relationship between data generated in the constructed power transmission equipment state evaluation and fault diagnosis model is as follows: Considering operations and faults as item sets, we calculate the frequency of a certain item set appearing in all transactions. For the joint item set X∪Y of operation X and fault Y, the support calculation formula is: Calculate the confidence level, which indicates the probability of Y occurring when X is known to occur. The confidence calculation formula is: According to the set minimum support and minimum confidence thresholds, strong association rules are screened out and the rules that meet the conditions are stored in the knowledge base.

5. The Transformer-based multi-modal soft fusion method for power grid data according to claim 1, characterized in that: The process of fusing the data in the centralized monitoring knowledge base of the power transmission equipment using the trained Transformer multimodal soft fusion model and constructing a knowledge graph also includes: Determine the specific requirements for multimodal data fusion, analyze the data characteristics, advantages and limitations in the centralized monitoring knowledge base of power transmission equipment, and determine the fusion strategy; Based on the fusion strategy, the attention mechanism is used to fuse the multimodal features after cross-modal alignment to construct and update the knowledge graph.

6. The Transformer-based multi-modal soft fusion method for power grid data according to claim 1, characterized in that: The process of training the Transformer multimodal soft fusion model is as follows: Use the preprocessed data to train the multimodal soft fusion model and adjust the hyperparameters to optimize the performance; The model performance is evaluated through cross-validation and test set evaluation, and the model is iteratively optimized based on the evaluation results.

7. The Transformer-based multi-modal soft fusion method for power grid data according to claim 1, characterized in that: The intelligent service support includes: fault prediction and early warning, power transmission equipment health management, operation optimization, intelligent diagnosis and decision support; The fault prediction and early warning are used to predict power transmission equipment failures and issue early warnings; The power transmission equipment health management is used to monitor the equipment status in real time and provide maintenance and care suggestions; The operation optimization is used to automatically adjust the load of the power transmission equipment and improve the operation efficiency; The intelligent diagnosis is used to automatically identify power transmission equipment problems and provide solutions; The decision support is used to provide accurate decision suggestions to operation and maintenance personnel.

8. A Transformer-based multi-modal soft fusion system for power grid data, characterized by: include: The data acquisition and preprocessing module is configured to: acquire data related to the operation of the power grid and preprocess the data; The knowledge base construction module is configured to: construct a state assessment model and a fault diagnosis model of the power transmission equipment based on the preprocessed data; use association rules to perform data mining on the relationship between the data generated in the constructed state assessment model and the fault diagnosis model of the power transmission equipment, and convert the results of the data mining into a centralized monitoring knowledge base of the power transmission equipment, wherein the centralized monitoring knowledge base of the power transmission equipment includes data related to the operation of the power grid and potential associations between the constructed state assessment model and the fault diagnosis model of the power transmission equipment; The multimodal feature fusion module is configured to: use the trained Transformer multimodal soft fusion model to fuse the data in the power transmission equipment centralized monitoring knowledge base and construct a knowledge graph; wherein the Transformer multimodal soft fusion model is used to extract the multimodal features of the data in the power transmission equipment centralized monitoring knowledge base and perform cross-modal alignment; the attention mechanism is used to fuse the multimodal features after cross-modal alignment to construct a knowledge graph; The service support module is configured to: provide intelligent service support for power transmission equipment based on fusion results and knowledge graphs; enrich the knowledge system in the field of power transmission equipment monitoring based on the fusion results and knowledge graphs, integrate the fusion results and knowledge graphs into the power transmission equipment monitoring system, and provide intelligent service support.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the Transformer-based multimodal soft fusion method of power grid data as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the Transformer-based multimodal soft fusion method of power grid data as described in any one of claims 1-7 are implemented.

Images