Power grid multi-modal data compliance monitoring method based on knowledge graph

Abstract

The application relates to a power grid multi-modal data compliance monitoring method based on a knowledge graph, which comprises the following steps: S1: acquiring multi-source heterogeneous data of a power grid and performing preprocessing to generate structured and semantic data representation; S2: constructing an initial knowledge graph according to the generated structured and semantic data representation; S3: identifying compliance standards and rules of the power grid operation environment changing over time according to the initial knowledge graph and combining a time perception mechanism (TGN), and combining rule-based logical reasoning and a deep neural model through neural symbol fusion to realize compliance judgment under fuzzy conditions and construct a knowledge intelligent agent; S4: performing multi-modal compliance analysis based on the knowledge intelligent agent and combining deep learning and symbolic reasoning to obtain analysis results; and S5: automatically generating an explanation report for a violation event based on the analysis results and combining a natural language generation technology. The application realizes dynamic intelligent compliance analysis and significantly improves the intelligent and automatic level of power grid management.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data monitoring, and in particular to a power grid multi-modal data compliance monitoring method based on a knowledge graph. BACKGROUND

[0002] With the continuous acceleration of the digitization and intelligentization of modern power systems, power grid enterprises accumulate and transmit massive multi-source heterogeneous data in the aspects of operation scheduling, equipment operation and maintenance, market transactions, and information interaction. These data cover real-time SCADA monitoring information, equipment operation parameters, operation and maintenance records, scheduling instructions, power market transaction data, and text files and image and video materials related to management systems and safety regulations. With the promotion of energy internet and data interconnection and sharing, a large amount of power grid business data needs to be circulated and shared between enterprises, regions, and even cross-industry scenarios. Power grid data is diverse, sensitive, complex in structure, and updated frequently, and the security levels, privacy attributes, and compliance requirements of different data are significantly different. Therefore, a systematic and intelligent compliance monitoring mechanism needs to be established in the process of data circulation to ensure the safety, legality, and controllability of data use.

[0003] However, the existing data compliance monitoring method mainly relies on static rule configuration and manual review, and has limited semantic understanding ability for data content, making it difficult to accurately identify the business semantics, sensitive elements, and context relationships implied in power grid data, and thus unable to effectively determine whether the data circulation behavior touches the safety red line or violates the management regulations. In addition, the traditional method is difficult to cope with real-time compliance judgment and strategy adjustment in dynamic data flow scenarios, and lacks intelligent support for complex scenarios such as data hierarchical classification management, desensitization processing verification, and cross-modal security review. This not only leads to low efficiency, lagging response, and high false positive rate in the monitoring process, but also makes it difficult to discover and trace the illegal data circulation behavior in a timely manner. SUMMARY

[0004] To solve the above problems, the purpose of the present application is to provide a power grid multi-modal data compliance monitoring method based on a knowledge graph, which realizes dynamic and intelligent compliance analysis and significantly improves the intelligentization and automation level of power grid management.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] The power grid multi-modal data compliance monitoring method based on a knowledge graph comprises the following steps:

[0007] S1: Obtain multi-source heterogeneous data of the power grid and perform preprocessing to generate structured and semantic data representation;

[0008] S2: Construct an initial knowledge graph according to the structured and semantic data representation produced;

[0009] S3: According to the initial knowledge graph, the compliance standards and rules of the power grid operation environment changing over time are identified in combination with the time perception mechanism TGN, and through neural symbol fusion, the rule-based logical reasoning and deep neural model are combined to realize the compliance judgment under fuzzy conditions, and the initial knowledge base is optimized into a dynamically evolving knowledge agent;

[0010] S4: Based on the knowledge agent, multi-modal compliance analysis is carried out by combining deep learning and symbolic reasoning to obtain the analysis result;

[0011] S5: Based on the analysis result, combined with natural language generation technology, an explanation report is automatically generated for the violation event.

[0012] Further, the multi-source heterogeneous data of the power grid is obtained, specifically as follows: a unified access and collection mechanism for power grid multi-source heterogeneous data is established, and for real-time monitoring data, the real-time operation parameters, switch state, alarm information time series data of the substation, line, generator equipment are obtained through standardized communication protocol; For device operation logs, log parsing engines are used to extract fault records, maintenance information, and state change textual records from various device management systems; For job image and video data, field operation photos, device inspection videos, personnel operation videos, and multimedia information are obtained through field operation management platforms, inspection robots, and surveillance cameras; The operation and maintenance work ticket data is obtained from the work order management system and the maintenance plan system, which contains operation content, execution personnel, and time node key information structured forms; The procedure documents are collected from the document management system, including various safety procedures, operation manuals, and technical standards unstructured texts; The dispatching command records are obtained from the dispatching automation system, including dispatching instructions, execution status, response time, and other information command logs.

[0013] Further, preprocessing is performed, specifically as follows: after data collection is completed, intelligent cleaning and preprocessing strategies are used to ensure data quality, for structured data, statistical anomaly detection algorithms are used to identify and process missing values, repeated values, and outliers, time series smoothing and interpolation algorithms are used to repair data discontinuity problems, and a unified data dictionary is established for field mapping and unit standardization; For semi-structured log data, regular expressions and NLP techniques are used for format analysis to extract key business fields and remove redundant information and system noise; For unstructured text data, a deep learning-based text cleaning model is used to remove irrelevant content, correct errors, and unify terminology, and named entity recognition technology is used to automatically identify device names, personnel information, and time and place key entities; For multimedia data such as images and videos, computer vision preprocessing techniques are used for denoising, enhancement, and standardization, and target detection and scene recognition algorithms are used to extract key visual elements;

[0014] For text data, a pre-trained language model is used for word embedding and sentence encoding to generate high-dimensional semantic vector representation, while a professional domain dictionary and ontology library are used for semantic annotation and concept mapping. For numerical monitoring data, statistical features, time domain and frequency domain features, trend features, etc. are extracted through feature engineering methods, and normalization, standardization, etc. are used to generate standardized numerical representation. For image and video data, convolutional neural network and visual Transformer model are used to extract visual feature vectors, and multi-modal fusion technology is used to align and map visual features with text descriptions. Finally, cross-modal representation learning method is used to project different types of feature vectors into a unified semantic space to form structured and semantic data representation containing rich semantic information and convenient for calculation and processing

[0015] Further, according to the structured and semantic data representation produced, an initial knowledge graph is constructed, as follows: After the structured and semantic data representation, a pre-trained named entity recognition model is used to automatically identify and classify key entities in the power grid field; By constructing an ontology library in the power grid field and an entity linking algorithm, the identified entities are matched with the standardized entity library to eliminate different forms of expression of the same entity and ensure the consistency and accuracy of the entity identification.

[0016] Based on the identified entities, a relationship extraction model is used to mine the semantic relationships between entities to construct a relationship network of power grid business. The relationship extraction model uses a BERT-based relationship extraction based on attention mechanism to extract multiple types of binary relationships such as device-belonging-substation, personnel-performing-operation, device-failing, and operation-following-procedure from text data. At the same time, for the characteristics of multi-modal data, a cross-modal relationship extraction method is used to establish semantic associations between image-text and video-operation records. After entity recognition and relationship extraction, a knowledge fusion model is used to integrate scattered knowledge fragments into an initial knowledge graph.

[0017] Further, a knowledge fusion model is used to integrate scattered knowledge fragments into an initial knowledge graph, as follows: First, entity alignment and deduplication are performed, and duplicate entities are identified using entity embedding similarity calculation, string matching, and attribute comparison methods. Different entity representations representing the same real object are merged through clustering algorithm. Then, relationship fusion and conflict resolution are performed, and for mutually contradictory or repeated relationship triples, time stamp comparison is used for conflict processing. Based on the extracted entities and relationships, an RDF triple or attribute graph model is used to construct an initial knowledge graph, in which nodes represent power grid entities, edges represent relationships between entities, and node and edge attributes carry detailed semantic information.

[0018] Further, the time-aware mechanism TGN is used to identify the compliance standards and rules of the power grid operation environment changing over time, as follows: based on the constructed initial knowledge graph, a time-aware graph neural network TGN is used to deeply model the time-varying characteristics of the power grid operation environment; the TGN can capture the dynamic change patterns of the nodes and edges in the knowledge graph over time by introducing a time encoding mechanism and a memory update module, identify seasonal operation rules, equipment aging trends, and operation habit evolution time sequence characteristics, embed the timestamp information of the power grid operation data into the graph structure, construct a time-enhanced knowledge representation Gt, and form a triple structure:

[0019] ;

[0020] wherein V is a node set, containing all entities in the power grid; E t is an edge set at time t, representing the relationship between entities at a specific time point; T is a timestamp set, recording the specific time points of all related events, and the Unix timestamp is used;

[0021] When a new interaction event occurs, the memory state of the related node will be updated according to the new information:

[0022] ;

[0023] wherein s i (t - ) is the memory state vector of node i before time t; m i (t) is the message vector received by node i at time t, containing information from neighbor nodes; Δt is the time interval, representing the time difference from the last update to the current time; Memory(.) is a memory update function; s i (t) is the memory state vector of node i at time t;

[0024] At each time step, the node collects information from its neighbors, including the state of the neighbor, the feature of the connecting edge, and the time information:

[0025] ;

[0026] wherein Ni(t) is the neighbor set of node i at time t; MSG(.) is a message function; e ij (t) is the edge feature vector connecting nodes i and j, containing relationship type, weight, and confidence signal; AGG(.) is an aggregation function; s j (t - ) is the memory state vector of node j before time t; s j (t) is the memory state vector of node j at time t; TimeEnc(t) is the time encoding vector at time t;

[0027] The historical interaction sequence is modeled by a recurrent neural unit (RNN), and the time sequence dependency of the entity state is learned. Meanwhile, the importance weight of the compliance standard in different time periods is dynamically adjusted by using an attention mechanism, and the compliance rule strongly related to time is automatically identified. Through a continuous learning mechanism, the time sequence mode of the knowledge graph can be continuously updated according to the newly added operation data, so as to realize the adaptive perception and prediction of the change of the power grid operation environment.

[0028] Further, the rule-based logical reasoning is combined with the deep neural model to realize the compliance judgment under the fuzzy condition, as follows: a neural symbol fusion reasoning framework is constructed to organically combine the traditional rule-based symbol reasoning and the distributed representation learning of the deep neural network. On the symbol level, the explicit knowledge of the power grid safety regulations, operation standards and management systems is converted into logical rules and constraint conditions, and the rule modeling is performed by using the first-order logic formalization language. On the neural level, a graph neural network (GNN) is used to perform representation learning on the initial knowledge graph to generate low-dimensional embedding vectors of entities and relationships, and the non-linear mapping capability of the neural network is used to process the noise, uncertainty and fuzziness in the data. The two levels are coupled through a neural module network and a differentiable neural symbol reasoner to realize the differentiability of the symbol reasoning and the explainability of the neural calculation. When facing the fuzzy condition, the deterministic reasoning of the symbol rule and the pattern recognition capability of the neural network can be combined to perform the probabilistic compliance judgment and confidence evaluation.

[0029] Further, based on the knowledge agent, the deep learning and symbol reasoning are combined to perform multi-modal compliance analysis to obtain the analysis result, as follows:

[0030] Based on the constructed knowledge agent, first, the multi-source heterogeneous data generated in the power grid operation process are deeply fused and semantically aligned. Through the cross-modal alignment technology, the features of different modalities are projected into a unified multi-modal representation space. The similarity between related modalities is maximized and the similarity between irrelevant modalities is minimized by using the contrast learning method, so as to realize the effective fusion of multi-modal information and the guarantee of semantic consistency.

[0031] On the basis of the multi-modal feature fusion, the compliance analysis is performed by combining the symbol reasoning and the neural reasoning. On the symbol reasoning level, the formal rule base constructed according to the power grid safety regulations, operation standards and management systems is used to perform accurate logical reasoning and deduction on the explicit compliance condition. On the neural reasoning level, the graph neural network is used to perform representation learning on the entity relationship in the knowledge agent, and the complex correlation mode is captured through multi-layer message propagation and attention mechanism to process the fuzzy and uncertain compliance scene. After the multi-modal fusion and collaborative reasoning analysis, the comprehensive compliance analysis result is generated.

[0032] Further, a comprehensive compliance analysis result is generated, specifically as follows: first, the results of symbolic reasoning and neural reasoning are fused by weighting, the weights are dynamically adjusted according to the reliability and integrity of different evidence sources, and the final compliance probability distribution is generated; second, the uncertainty in the reasoning process is modeled and quantified using the Bayesian reasoning framework, the parameter uncertainty is estimated by the Monte Carlo sampling method, and the model uncertainty is evaluated using the ensemble model, to provide a confidence interval and a credibility score for each compliance judgment.

[0033] Further, based on the analysis result, a violation event is automatically generated by combining natural language generation technology, specifically as follows:

[0034] Based on the multi-modal compliance analysis result of the S4 stage, first, the detected violation event is deeply structured and key evidence is extracted, the event graph construction technology is used to decompose the violation event into time, place, personnel, equipment, operation and rule elements, and the complete chain and context information of the violation behavior are traced through the entity relationship network in the knowledge graph; after completing the violation event analysis and evidence extraction, an intelligent report generation architecture based on template driving and deep learning is constructed, first, according to the standardization requirements of the power industry and the format specifications of the regulatory department, a multi-level report template system is designed, including the structured framework of the violation summary template, the detailed analysis template and the rectification suggestion template; a sequence-to-sequence generation model based on the Transformer architecture is used, combined with the professional term dictionary and corpus of the power field to adapt to the field, so that the model can accurately understand and generate professional expressions conforming to the specifications of the power industry; conditional text generation technology is introduced, and according to the violation type, severity and equipment condition information, the appropriate language style and expression method are dynamically selected.

[0035] A computer storage medium stores a plurality of instructions, the instructions are suitable for being loaded and executed by a processor to perform the method steps as described above.

[0036] The present application has the following beneficial effects:

[0037] 1、The present application takes knowledge graph as the core, realizes deep fusion of power grid multi-source heterogeneous data, through systematic data preprocessing and unified semantic modeling, not only breaks through the originally fragmented different types of data and data islands, but also constructs a unified and structured knowledge expression space, effectively improves the data utilization rate and information coverage, on this basis, the knowledge graph establishes a clear semantic network for the complex correlation between equipment, personnel, operation and procedures, provides a solid data foundation for subsequent complex compliance detection, risk tracing and behavior analysis, greatly improves the comprehensiveness and accuracy of compliance monitoring;

[0038] 2、The application introduces a time-aware graph neural network and a neural-symbol fusion reasoning framework, which not only dynamically captures the evolution law of the power grid operation state and behavior mode over time, but also effectively balances the strictness of rule reasoning and the fault tolerance ability of deep models under fuzzy and uncertain conditions. The rule system at the symbol layer ensures high safety and explainability, and the representation learning of the neural layer improves the flexibility and intelligence of abnormal detection and compliance judgment. The fusion of the two makes the entire knowledge base have self-learning, continuous evolution and abnormal adaptation ability, which can meet the complex monitoring needs of frequent sudden conditions and information missing in actual power grid scenarios, and ensure the high robustness and dynamic self-evolution ability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The method flowchart of the application. DETAILED DESCRIPTION

[0040] The application will be further described in detail below in combination with the drawings and specific embodiments:

[0041] Reference Figure 1 In this embodiment, a power grid multi-modal data compliance monitoring method based on a knowledge graph is provided, including the following steps:

[0042] S1: Obtain multi-source heterogeneous data of the power grid, including SCADA real-time monitoring data, device operation logs, operation image videos, operation tickets, procedure documents and dispatching command records, and perform preprocessing to generate structured and semantic data representations;

[0043] S2: Construct an initial knowledge graph according to the structured and semantic data representations produced;

[0044] S3: According to the initial knowledge graph, combine the time-aware mechanism TGN to identify the compliance standards and rules of the power grid operation environment changing over time, and through neural-symbol fusion, combine rule-based logical reasoning with deep neural models to realize compliance judgment under fuzzy conditions, and optimize the initial knowledge base into a dynamically evolving knowledge agent;

[0045] S4: Based on the knowledge agent, combine deep learning and symbolic reasoning to perform multi-modal compliance analysis and obtain analysis results;

[0046] S5: Based on the analysis results, combine natural language generation technology to automatically generate an explanation report for the violation event, for example: "The system detects that the disconnecting switch QF2 has not been maintained according to the specified period, and according to Article 5 of the "Operation and Maintenance Regulations", there is a risk of overage of the maintenance period."

[0047] In the embodiment, multi-source heterogeneous data of the power grid is acquired, specifically as follows: a unified access and collection mechanism of power grid multi-source heterogeneous data is established, for real-time monitoring data, real-time operation parameters, switch state, alarm information time sequence data of transformer substations, lines and generators are acquired through standardized communication protocols (such as IEC 61850, DNP3.0); for device operation logs, log analysis engines are used to extract fault records, maintenance information and state change textual records from various device management systems; for job image and video data, field operation photos, device inspection videos, personnel operation video multimedia information are acquired through field operation management platforms, inspection robots and monitoring cameras; operation and maintenance work ticket data are acquired from work order management systems and maintenance plan systems, including operation content, execution personnel and time node key information structured forms; regulations documents are collected through document management systems, including various safety regulations, operation manuals and technical standards unstructured texts; dispatching command records are acquired from dispatching automation systems, including dispatching instructions, execution status, response time and other information command logs.

[0048] In the embodiment, preprocessing is specifically as follows: after data collection is completed, intelligent cleaning and preprocessing strategies are used to ensure data quality, for structured data, statistical anomaly detection algorithms are used to identify and process missing values, repeated values and abnormal values, time series smoothing and interpolation algorithms are used to repair data discontinuity problems, and a unified data dictionary is established for field mapping and unit standardization; for semi-structured log data, regular expressions and NLP technology are used for format analysis, key business fields are extracted, and redundant information and system noise are removed; for unstructured text data, a deep learning-based text cleaning model is used to remove irrelevant content, correct errors and unify terminology expression, and naming entity recognition technology is used to automatically identify device names, personnel information, time and place key entities; for image and video multimedia data, computer vision preprocessing technology is used for denoising, enhancement and standardization, and target detection and scene recognition algorithms are used to extract key visual elements;

[0049] For text data, pre-trained language models such as BERT and RoBERTa are used for word embedding and sentence encoding to generate high-dimensional semantic vector representations. Meanwhile, professional domain dictionaries and ontology libraries are used for semantic annotation and concept mapping. For numerical monitoring data, statistical features, time and frequency domain features, and trend features are extracted through feature engineering methods, and normalization and standardization methods are used to generate standardized numerical representations. For image and video data, convolutional neural networks and visual Transformer models are used to extract visual feature vectors, and multi-modal fusion techniques are used to align and map visual features with text descriptions. Finally, cross-modal representation learning methods are used to project different types of feature vectors into a unified semantic space, forming structured and semantic data representations that contain rich semantic information and are easy to calculate and process

[0050] In this embodiment, based on the structured and semantic data representation produced, an initial knowledge graph is constructed as follows: After the structured and semantic data representation, a pre-trained named entity recognition model is used to automatically identify and classify key entities in the power grid field. For the complexity of power grid professional terms and device names, a BERT-CRF sequence labeling model is used, combined with a power industry-specific dictionary and labeled corpus for domain adaptability fine-tuning, to accurately identify device entities such as substations, transmission lines, generator sets, switch devices, and protection devices. At the same time, multiple types of entities are identified, such as personnel entities (e.g., operators, maintenance personnel, dispatchers), time entities (operation time, maintenance period, alarm time), geographic entities (substation location, line direction), and state entities (operation state, alarm level, operation type). Through the construction of a power grid domain ontology library and entity linking algorithm, the identified entities are matched with the standardized entity library to eliminate different forms of expression of the same entity and ensure the consistency and accuracy of entity identification. In addition, for image and video data, target detection and scene understanding techniques are used to identify visual entities such as device appearance, work tools, and safety equipment, and they are associated and mapped with corresponding text entities.

[0051] Based on the identified entities, the semantic relationship between entities is mined by using a relationship extraction model to construct a relationship network of power grid business. The relationship extraction model adopts a BERT-based relationship extraction based on an attention mechanism to extract multiple types of binary relationships such as device-belonging-substation, personnel-performing-operation, device-failing, and operation-following-procedure from text data. For the characteristics of time series data, special attention is paid to the extraction of time-dependent relationships, such as "device A-at time T-state change to-abnormal" and "operation B-duration-30 minutes". At the same time, for the characteristics of multi-modal data, a cross-modal relationship extraction method is used to establish semantic associations between image-text and video-operation records, such as "on-site image-shows-device state" and "operation video-records-operation process". After completing entity recognition and relationship extraction, a knowledge fusion model is used to integrate scattered knowledge fragments into an initial knowledge graph.

[0052] In this embodiment, the knowledge fusion model is used to integrate scattered knowledge fragments into an initial knowledge graph, as follows. First, entity alignment and deduplication are performed. Repetitive entities are identified by using entity embedding similarity calculation, string matching, and attribute comparison methods. Different entity representations representing the same real object are merged by using a clustering algorithm. Then, relationship fusion and conflict resolution are performed. For mutually contradictory or repetitive relationship triples, time stamp comparison is used for conflict processing. Based on the extracted entities and relationships, an initial knowledge graph is constructed using RDF triples or attribute graph models, in which nodes represent power grid entities, edges represent relationships between entities, and the attributes of nodes and edges carry detailed semantic information.

[0053] In this embodiment, the time-aware mechanism TGN is used to identify the compliance standards and rules of the power grid operation environment over time, as follows. Based on the constructed initial knowledge graph, the time-aware graph neural network TGN is used to deeply model the time-varying characteristics of the power grid operation environment. TGN can capture the dynamic change patterns of nodes and edges in the knowledge graph over time by introducing a time encoding mechanism and a memory update module, identify seasonal operation rules, device aging trends, and operation habit evolution time sequence characteristics, embed the time stamp information of the power grid operation data into the graph structure, construct a time-enhanced knowledge representation Gt, and form a triple structure (subject-relation-object-time):

[0054] ;

[0055] wherein V is a node set containing all entities in the power grid, such as transformers, circuit breakers, relay protection devices, operation and maintenance personnel, dispatchers, etc.; E tE(t) is the edge set at time t, representing the relationship between entities at a specific time point, such as "Device A - connected to - Bus B", "Person C - operates - Switch D"; T is the timestamp set, recording the specific time point of all related events, using Unix timestamp;

[0056] When a new interaction event occurs, the memory state of the relevant node will be updated according to the new information:

[0057] ;

[0058] Where s i (t - ) is the memory state vector of node i before time t; m i (t) is the message vector received by node i at time t, containing information from neighboring nodes; Δt is the time interval, representing the time difference from the last update to the current time; Memory(.) is the memory update function; s i (t) is the memory state vector of node i at time t;

[0059] At each time step, the node collects information from its neighbors, including the state of the neighbors, the characteristics of the connecting edges, and the time information:

[0060] ;

[0061] Where Ni(t) is the neighbor set of node i at time t; MSG(.) is the message function, which is a multi-layer perception (MLP) with the same output dimension as the node embedding dimension; e ij (t) is the edge feature vector connecting nodes i and j, containing relationship type, weight, confidence signal; AGG(.) is the aggregation function; s j (t - ) is the memory state vector of node j before time t; s j (t) is the memory state vector of node j at time t; TimeEnc(t) is the time encoding vector at time t;

[0062] The historical interaction sequence is modeled by a recurrent neural unit RNN, learning the temporal dependence of entity states; At the same time, the importance weight of compliance standards in different time periods is dynamically adjusted by using attention mechanism, automatically identifying "patrol frequency requirements during summer high load period", "on-duty regulations during holidays", "safety measures during equipment maintenance season" and other time-dependent compliance rules; Through the continuous learning mechanism, the temporal pattern of the knowledge graph can be constantly updated according to the newly added operation data, realizing the adaptive perception and prediction of the changes in the power grid operation environment.

[0063] In this embodiment, rule-based logical reasoning is combined with deep neural models to realize compliance judgment under fuzzy conditions, as follows: a neural-symbol fusion reasoning framework is constructed to organically combine traditional rule-based symbolic reasoning and distributed representation learning of deep neural networks; at the symbolic level, explicit knowledge of power grid safety regulations, operation standards, and management systems is converted into logical rules and constraint conditions, rule modeling is performed using a first-order logic formal language, and accurate logical reasoning and deduction are supported; at the neural level, a graph neural network (GNN) is used to perform representation learning on the initial knowledge graph to generate low-dimensional embedding vectors of entities and relationships, and the non-linear mapping capability of the neural network is used to process noise, uncertainty, and fuzziness in the data; the two levels are coupled through a neural module network and a differentiable neural-symbol reasoner to realize the differentiability of symbolic reasoning and the interpretability of neural computation, and when facing fuzzy conditions such as "device state signal missing but surrounding environment abnormal" and "incomplete operation record but on-site image shows violation", the system can combine the deterministic reasoning of symbolic rules and the pattern recognition capability of neural networks to make probabilistic compliance judgment and confidence evaluation.

[0064] In this embodiment, a knowledge agent is used to combine deep learning and symbolic reasoning for multi-modal compliance analysis to obtain analysis results, as follows:

[0065] Based on the constructed knowledge agent, first, multi-source heterogeneous data generated during power grid operation are deeply fused and semantically aligned, pre-trained language models (BERT, RoBERTa) are used to extract semantic features for text modal data (such as operation logs, regulation documents, and dispatch instructions), and entity linking technology in the knowledge graph is used to map key elements such as device names, operation types, and time information in the text to a unified semantic space; for image modal data (on-site operation photos, device inspection images), convolutional neural networks and visual Transformers are used to extract visual features, and target detection algorithms are used to identify key visual elements such as device status, personnel behavior, and safety equipment wearing status; for time series numerical data (SCADA monitoring data, sensor readings), time series convolutional networks and LSTM networks are used to extract time domain and frequency domain features to identify device operation trends and abnormal patterns; cross-modal alignment technology is used to project features of different modalities into a unified multi-modal representation space, and a contrast learning method is used to maximize similarity between related modalities and minimize similarity between unrelated modalities, thereby realizing effective fusion of multi-modal information and guaranteeing semantic consistency;

[0066] On the basis of multi-modal feature fusion, a compliance analysis is performed in a manner of combining symbolic reasoning and neural reasoning. At the symbolic reasoning level, a formal rule base constructed according to power grid safety regulations, operation standards and management systems is used to perform accurate logical reasoning and deduction on explicit compliance conditions, such as the deterministic rules of "high-voltage equipment maintenance must be connected with a ground wire", "personnel entering a high-voltage area must wear insulating equipment", etc. The rules are strictly verified by a symbolic reasoning engine. At the neural reasoning level, a graph neural network is used to perform representation learning on entity relationships in a knowledge intelligent agent, and through multi-layer message propagation and attention mechanisms, complex correlation patterns are captured to handle fuzzy and uncertain compliance scenarios, such as "the device state signal is abnormal but there is no explicit fault indication", "the operation record is incomplete but the on-site image shows a standard operation", etc. After multi-modal fusion and collaborative reasoning analysis, a comprehensive compliance analysis result is generated.

[0067] In this embodiment, a comprehensive compliance analysis result is generated, specifically as follows: first, the results of symbolic reasoning and neural reasoning are fused by weighting, the weights are dynamically adjusted according to the reliability and integrity of different evidence sources, and a final compliance probability distribution is generated; second, a Bayesian reasoning framework is used to model and quantify the uncertainty in the reasoning process, the parameter uncertainty is estimated by a Monte Carlo sampling method, and the model uncertainty is evaluated by an ensemble model, thereby providing a confidence interval and a credibility score for each compliance judgment.

[0068] In this embodiment, based on the analysis result, a violation event is automatically generated to generate an explanation report in combination with natural language generation technology, specifically as follows:

[0069] Based on the multi-modal compliance analysis results of the S4 stage, first, the detected violation events are deeply structured and key evidence is extracted, the event graph construction technology is used to decompose the violation events into time, place, personnel, equipment, operation, and rule elements, and through the entity relationship network in the knowledge graph, the complete chain and context information of the violation behavior are traced back; for text evidence, key information such as "violation time: October 15, 2023, 14:30", "involved equipment: 110kV transformer T1", "operation personnel: Zhang", "violate rules: Article 4.2.3 of the Electric Power Safety Work Regulations" and other structured elements are extracted from operation logs and work order records by using named entity recognition and relationship extraction technology; for image evidence, computer vision technology is used to identify on-site violation behaviors such as "not wearing a safety helmet", "violating safety barriers", "not hanging a ground wire during equipment maintenance", and other key visual features, and the specific location of the violation behavior in the image is located through target detection and scene understanding algorithms; for numerical evidence, abnormal detection algorithms are used to identify monitoring data that deviates from the normal range, such as "device temperature exceeds 50%", "insulation resistance value is lower than the safety threshold", and other quantitative indicators. The system establishes an evidence credibility evaluation mechanism, according to the authority of the data source, the consistency of the time, the mutual confirmation degree of multi-source evidence and other factors, each piece of evidence is configured with a credibility weight, to ensure the accuracy and reliability of the subsequent report generation; after completing the violation event analysis and evidence extraction, an intelligent report generation architecture based on template driving and deep learning is constructed, first, according to the standardization requirements of the electric power industry and the format specifications of the regulatory departments, a multi-level report template system is designed, including the structured framework of violation summary template, detailed analysis template, and rectification suggestion template; a sequence-to-sequence generation model (such as T5, BART) based on the Transformer architecture is used, combined with the professional term dictionary and corpus of the electric power field for domain adaptability fine-tuning, so that the model can accurately understand and generate professional expressions conforming to the specifications of the electric power industry; conditional text generation technology is introduced, according to the violation type, severity, and involved equipment condition information, the appropriate language style and expression method are dynamically selected, such as using a serious warning tone for safety violations and using an objective description tone for operation violations.

[0070] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems, or computer program products. Accordingly, the application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can be embodied in the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) having computer usable program code embodied thereon.

[0071] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0072] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0073] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0074] The above descriptions are only preferred embodiments of the present application, and are not intended to limit the present application to other forms. Any person skilled in the art can make modifications or improvements on the basis of the above disclosed technical content without departing from the technical scope of the present application. Any simple modifications, equivalent changes and improvements made on the basis of the above embodiments without departing from the technical scope of the present application shall fall within the scope of the present application.

Claims

1. A power grid multi-modal data compliance monitoring method based on a knowledge graph, characterized in that, The method comprises the following steps: S1: obtaining multi-source heterogeneous data of the power grid and preprocessing to generate structured and semantic data representation; S2: constructing an initial knowledge graph according to the generated structured and semantic data representation; S3: identifying the compliance standards and rules of the power grid operation environment changing over time according to the initial knowledge graph combined with the time-aware mechanism TGN, and combining rule-based logical reasoning with a deep neural model through neural-symbolic fusion to realize compliance judgment under fuzzy conditions, and optimizing the initial knowledge base into a dynamically evolving knowledge agent; S4: based on the knowledge agent, combining deep learning and symbolic reasoning for multi-modal compliance analysis to obtain analysis results; S5: based on the analysis results, combining natural language generation technology to automatically generate an explanation report for the violation event; The rule-based logical reasoning is combined with the deep neural model to realize compliance judgment under fuzzy conditions, specifically as follows: a neural-symbolic fusion reasoning framework is constructed to organically combine traditional rule-based symbolic reasoning and distributed representation learning of deep neural networks; on the symbolic level, explicit knowledge of power grid safety regulations, operation standards and management systems is converted into logical rules and constraint conditions, and rule modeling is performed using first-order logic formalization language; on the neural level, graph neural network GNN is used to perform representation learning on the initial knowledge graph to generate low-dimensional embedding vectors of entities and relationships, and the non-linear mapping capability of the neural network is used to process noise, uncertainty and fuzziness in the data; The two levels are coupled through a neural module network and a differentiable neural-symbolic reasoner to realize the differentiability of symbolic reasoning and the explainability of neural computing, and when facing fuzzy conditions, the system can combine the deterministic reasoning of symbolic rules and the pattern recognition capability of neural networks to make probabilistic compliance judgment and confidence evaluation; The knowledge agent is combined with deep learning and symbolic reasoning to perform multi-modal compliance analysis and obtain analysis results, specifically as follows: Based on the constructed knowledge agent, first, multi-source heterogeneous data generated during power grid operation are deeply fused and semantically aligned; through cross-modal alignment technology, features of different modalities are projected into a unified multi-modal representation space, and a contrast learning method is used to maximize the similarity between related modalities and minimize the similarity between unrelated modalities, thereby realizing effective fusion of multi-modal information and guaranteeing semantic consistency; On the basis of multi-modal feature fusion, a combination of symbolic reasoning and neural reasoning is used for compliance analysis; on the symbolic reasoning level, the formal rule base constructed according to power grid safety regulations, operation standards and management systems is used for accurate logical reasoning and deduction of clear compliance conditions; on the neural reasoning level, graph neural networks are used to perform representation learning on entities and relationships in the knowledge agent, and through multi-layer message propagation and attention mechanisms, complex correlation patterns are captured to handle fuzzy and uncertain compliance scenarios; after multi-modal fusion and collaborative reasoning analysis, comprehensive compliance analysis results are generated.

2. The knowledge graph based power grid multi-modal data compliance monitoring method of claim 1, wherein, The multi-source heterogeneous data of the power grid is acquired, specifically as follows: a unified access and collection mechanism of power grid multi-source heterogeneous data is established, real-time operation parameters, switch states, alarm information time sequence data of substations, lines and generator equipment are acquired through standardized communication protocols for real-time monitoring data; for device operation logs, log analysis engines are used to extract fault records, maintenance information and state change textual records from various device management systems; for job image and video data, on-site job photos, device inspection videos and personnel operation video multimedia information are acquired through on-site job management platforms, inspection robots and monitoring cameras; operation and maintenance work ticket data are acquired from work order management systems and maintenance plan systems, including operation content, execution personnel and time node key information structured forms; regulations and documents are collected through document management systems, including various safety regulations, operation manuals and technical standards unstructured texts; and dispatching command records are acquired from dispatching automation systems, including dispatching instructions, execution status and response time information command logs.

3. The knowledge graph based power grid multi-modal data compliance monitoring method of claim 2, wherein, The preprocessing is specifically as follows: after data collection is completed, intelligent cleaning and preprocessing strategies are used to ensure data quality, for structured data, statistical anomaly detection algorithms are used to identify and process missing values, repeated values and abnormal values, time series smoothing and interpolation algorithms are used to repair data discontinuity problems, and a unified data dictionary is established for field mapping and unit standardization; for semi-structured log data, regular expressions and NLP techniques are used for format analysis, key business fields are extracted, and redundant information and system noise are removed; for unstructured text data, a deep learning-based text cleaning model is used to remove irrelevant content, correct errors and unify terminology, and naming entity recognition technology is used to automatically identify device names, personnel information and time and place key entities; for image and video multimedia data, computer vision preprocessing techniques are used for denoising, enhancement and standardization, and target detection and scene recognition algorithms are used to extract key visual elements; for text data, a pre-trained language model is used for word embedding and sentence encoding to generate high-dimensional semantic vector representations, and professional field dictionaries and ontology libraries are used for semantic annotation and concept mapping; for numerical monitoring data, statistical features, time and frequency domain features and trend features are extracted through feature engineering methods, and normalized and standardized methods are used to generate standardized numerical representations; for image and video data, convolutional neural networks and visual Transformer models are used to extract visual feature vectors, and multi-modal fusion techniques are used to align and map visual features and text descriptions; finally, cross-modal representation learning methods are used to project different types of feature vectors into a unified semantic space to form structured and semantic data representations containing rich semantic information and facilitating calculation and processing.

4. The knowledge graph based power grid multi-modal data compliance monitoring method of claim 1, wherein, The initial knowledge graph is constructed according to the generated structured and semantic data representation, specifically as follows: after the structured and semantic data representation, the key entities in the power grid field are automatically identified and classified by using a pre-trained named entity recognition model; by constructing an ontology library in the power grid field and an entity linking algorithm, the identified entities are matched with the standardized entity library to eliminate different forms of expression of the same entity and ensure the consistency and accuracy of the entity identification; based on the identified entities, a relationship extraction model is used to mine the semantic relationships between entities and construct a relationship network of power grid business; the relationship extraction model uses a BERT-based relationship extraction based on an attention mechanism to extract multiple types of binary relationships such as equipment-belonging-substation, personnel-performing-operation, equipment-failure, and operation-following-procedure from text data; at the same time, for the characteristics of multi-modal data, a cross-modal relationship extraction method is used to establish semantic associations between image-text and video-operation records; after completing entity identification and relationship extraction, a knowledge fusion model is used to integrate scattered knowledge fragments into an initial knowledge graph.

5. The knowledge graph based power grid multi-modal data compliance monitoring method of claim 4, wherein, The initial knowledge graph is constructed by using a knowledge fusion model to integrate scattered knowledge fragments, specifically as follows: first, entity alignment and deduplication are performed, repeated entities are identified by using entity embedding similarity calculation, string matching, and attribute comparison methods, and different entity expressions representing the same real object are merged by using a clustering algorithm; then, relationship fusion and conflict resolution are performed, for mutually contradictory or repeated relationship triples, time stamp comparison is used for conflict processing; based on the extracted entities and relationships, an RDF triple or attribute graph model is used to construct an initial knowledge graph, wherein nodes represent power grid entities, edges represent relationships between entities, and the attributes of nodes and edges carry detailed semantic information.

6. The knowledge graph based power grid multi-modal data compliance monitoring method of claim 1, wherein, The time-aware mechanism TGN is combined to identify the compliance standards and rules of the power grid operation environment changing over time, specifically as follows: based on the constructed initial knowledge graph, the time-aware mechanism TGN is used to deeply model the time-varying characteristics of the power grid operation environment; TGN can capture the dynamic change patterns of nodes and edges in the knowledge graph over time by introducing a time encoding mechanism and a memory update module, identify seasonal operation rules, equipment aging trends, and operation habit evolution time sequence characteristics, embed the timestamp information of the power grid operation data into the graph structure, construct a time-enhanced knowledge representation Gt, and form a triple structure: Gt=(V,Et,T); Wherein, V is a node set, containing all entities in the power grid; E t is an edge set at time t; T is a timestamp set, recording the specific time points of all related events, using Unix timestamp; When a new interaction event occurs, the memory state of the related node will be updated according to the new information: ； where s i (t - ) is the memory state vector of node i before time t; m i (t) is the message vector received by node i at time t, containing information from neighboring nodes; Δt is the time interval, representing the time difference from the last update to the current time; Memory(.) is the memory update function; s i (t) is the memory state vector of node i at time t; At each time step, the node collects information from its neighbors, including the state of the neighbor, the features of the connecting edge, and the time information: ； where Ni(t) is the neighbor set of node i at time t; is a message function; e ij is an edge feature vector connecting node i and j, containing relationship type, weight, confidence; is an aggregation function; s j is a memory state vector of node j before time t; s - is a memory state vector of node j at time t; s j is a memory state vector of node j at time t; TimeEnc(t) is a time encoding vector at time t. By modeling historical interaction sequences using recurrent neural networks (RNNs), the temporal dependencies of entity states are learned. Simultaneously, an attention mechanism is used to dynamically adjust the importance weights of compliance standards in different time periods, automatically identifying compliance patterns that are strongly correlated with time. Through a continuous learning mechanism, the temporal patterns of the knowledge graph can be continuously updated based on newly added operational data, enabling adaptive perception and prediction of changes in the power grid operating environment.

7. The knowledge graph based power grid multi-modal data compliance monitoring method of claim 1, wherein, The process of generating comprehensive compliance analysis results is as follows: First, the results of symbolic reasoning and neural reasoning are weighted and fused, and the weights are dynamically adjusted according to the reliability and completeness of different evidence sources to generate the final compliance probability distribution. Second, a Bayesian reasoning framework is used to model and quantify the uncertainty in the reasoning process, the parameter uncertainty is estimated by Monte Carlo sampling, the model uncertainty is evaluated by ensemble model, and a confidence interval and credibility score are provided for each compliance judgment.

8. The knowledge graph based power grid multi-modal data compliance monitoring method of claim 1, wherein, Based on the analysis results and combined with natural language generation technology, an explanation report is automatically generated for the violation incident, as detailed below: Based on the multimodal compliance analysis results of the S4 stage, the detected violations are first analyzed in a deep structure and key evidence is extracted. Event graph construction technology is used to decompose the violations into elements of time, location, personnel, equipment, operation, and rules. The complete chain and contextual information of the violation are traced through the entity relationship network in the knowledge graph. After completing the analysis of the violations and the extraction of evidence, an intelligent report generation architecture based on a combination of template-driven and deep learning is constructed. First, according to the standardization requirements of the power industry and the format specifications of regulatory authorities, a multi-level report template system is designed, including a structured framework of violation summary templates, detailed analysis templates, and rectification suggestion templates. A sequence-to-sequence generation model based on the Transformer architecture is adopted, and domain-adaptive fine-tuning is performed by combining a professional terminology dictionary and corpus in the power industry, enabling the model to accurately understand and generate professional expressions that conform to power industry standards. Conditional text generation technology is introduced to dynamically select appropriate language styles and expressions based on the type of violation, severity, and equipment conditions involved.

Images