Intelligent Supervision and Management Methods and Platform Based on Multi-Terminal Interaction and Data Fusion

Abstract

This invention provides an intelligent supervision management method and platform based on multi-terminal interaction and data fusion. The method includes the following steps: acquiring standardized supervision documents and available supervision resources for power engineering projects; extracting key entities and identifying entity relationships using natural language processing technology; constructing a multi-level project knowledge graph based on these entities and their relationships; and forming a heterogeneous graph neural network inference engine by combining a graph neural network model with the graph knowledge graph. Multi-dimensional supervision data is collected through an intelligent supervision terminal, and the data is processed using an adaptive cross-modal attention mechanism to generate a representation of the causes of project events. Based on this representation, multi-level inference is performed in the inference engine using an improved graph traversal algorithm to predict the set of abnormal event results caused by the causes of project events within a future time period. Combining the available supervision resources and the set of abnormal event results, a reinforcement learning optimization algorithm is used to generate the optimal supervision assignment strategy. This invention can achieve efficient supervision and optimized resource allocation.

Description

Technical Field

[0001] This invention belongs to the field of supervision data management technology, specifically involving an intelligent supervision management method and platform based on multi-terminal interaction and data fusion. Background Technology

[0002] Power engineering projects are typically characterized by their large scale, high complexity, and long duration, involving numerous procedures, resources, and dependencies. For example, a large substation construction project may include multiple phases such as civil construction, equipment installation, and commissioning, each phase containing multiple sub-procedures with complex dependencies between them. Furthermore, power engineering projects require the coordination of various resources, including manpower, equipment, and materials; the allocation and use of these resources directly impact the project's progress and quality.

[0003] With the rapid development of intelligent technologies, especially the advancements in deep learning and natural language processing, intelligent supervision and management methods have emerged. Traditional intelligent supervision and management methods primarily rely on human experience and simple rule-based processing. This approach struggles to comprehensively capture key information in complex power engineering projects. For example, complex dependencies between work processes, dynamic resource allocation needs, and changes in environmental factors cannot be effectively handled through simple rules. Secondly, traditional technologies lack the ability to deeply analyze and integrate multi-dimensional data. Power engineering projects involve diverse data types, including work progress, resource usage, and environmental monitoring data. These data exhibit complex interrelationships, making it difficult for traditional technologies to fully extract the potential information within complex projects. Ultimately, this results in automated work order dispatch strategies that are resource-inefficient and wasteful of resources. Summary of the Invention

[0004] This invention provides an intelligent supervision and management method and platform based on multi-terminal interaction and data fusion, in order to solve the problem that the automatic dispatch strategy generated by existing intelligent supervision and management methods is prone to wasting resources and has poor efficiency.

[0005] In a first aspect, the present invention provides an intelligent supervision and management method based on multi-terminal interaction and data fusion, the method comprising the following steps:

[0006] Obtain standardized project supervision documents and available supervision resources for power engineering projects;

[0007] The key entities of the power project are extracted from the standardized project supervision documents using deep learning-enhanced natural language processing technology, and the entity association relationships between the key entities of the power project are identified. The key entities of the power project include power project process entities, power project resource entities, and project process dependency entities. The entity association relationships include entity dependency relationships and entity coreference relationships.

[0008] Based on the key entities of the power project and according to the relationships between the entities, a multi-level project knowledge graph of the power engineering project is constructed.

[0009] Based on the multi-level project knowledge graph, a heterogeneous graph neural network inference engine for the power engineering project is constructed by combining it with a graph neural network model.

[0010] Multi-dimensional project supervision data of the power engineering project during the current time period is collected through various intelligent supervision terminals;

[0011] The multi-dimensional project supervision data is integrated into multimodal supervision data, and the multimodal supervision data is processed using an adaptive cross-modal attention mechanism to generate a representation of the cause of project events in the power engineering project under the current time period.

[0012] Based on the project event cause representation, an improved graph traversal algorithm is used to perform multi-level reasoning in the heterogeneous graph neural network inference engine to predict the abnormal event result set caused by the project event cause in the preset future time period of the power engineering project.

[0013] Combining the available supervision resources and the abnormal event result set, a reinforcement learning optimization algorithm is used to generate the optimal supervision assignment strategy for the power engineering project in the future time period.

[0014] Optionally, the step of extracting key entities of the power project from the standardized project supervision documents using deep learning-enhanced natural language processing technology and identifying the entity relationships between the key entities of the power project includes the following steps:

[0015] The project supervision standardization documents were embedded using a pre-trained BERT model with context-sensitive word embeddings.

[0016] Based on the word embedding results, a named entity recognition model combining bidirectional long short-term memory network and conditional random field is used to extract key entities of the power project from the project supervision standardization documents. The key entities of the power project include power project process entities, power project resource entities, and project process dependency entities.

[0017] Based on the contextual relationships of the key entities in the power project in the project supervision standardization documents, the entity dependency relationships between different key entities in the power project are identified.

[0018] Based on the key entities of the power project and using the coreference resolution model to traverse the text of the standardized project supervision documents in reverse, the reference relationships of all the key entities of the power project in the standardized project supervision documents are identified and parsed to obtain the entity coreference relationships between different key entities of the power project.

[0019] Optionally, the method further includes the following steps:

[0020] The knowledge of the pre-trained large language model is transferred to the BERT model, the named entity recognition model, the coreference resolution model, and the heterogeneous graph neural network inference engine using knowledge distillation technology.

[0021] Optionally, the construction of a multi-level project knowledge graph for the power project based on the key entities of the power project and according to the entity relationships includes the following steps:

[0022] An initial graph ontology model is designed based on multiple pre-defined hierarchical concepts, including macroscopic, mesoscopic, and microscopic levels.

[0023] An entity tree structure is constructed for all key entities of the power project by combining the entity dependency relationship and the entity coreference relationship. The entity nodes in the entity tree structure represent the key entities of the power project. The parent-child relationship between the entity nodes in the entity tree structure represents the entity dependency relationship. The entity nodes with the entity coreference relationship belong to the same level tree structure in the entity tree structure.

[0024] Count the number of tree structure layers in the entity tree structure and the number of structural branches in each tree structure layer of the entity tree structure;

[0025] The entity tree structure is divided into different hierarchical concepts by combining the number of tree structure layers and the number of structural branches, and the entity tree structure is mapped to the initial graph ontology model based on the hierarchical concepts to form a multi-level project knowledge graph of the power engineering project.

[0026] Optionally, the step of splitting the entity tree structure into different hierarchical concepts by combining the number of tree structure layers and the number of structural branches, and mapping the entity tree structure to the initial graph ontology model based on the hierarchical concepts to form a multi-level project knowledge graph for the power engineering project includes the following steps:

[0027] Combining the number of tree structure layers and the number of structural branches, each layer of the entity tree structure is sequentially divided into the macro level, the meso level, and the micro level from the root node to the leaf node, and node level tags corresponding to the level concept are added to all entity nodes;

[0028] Traverse each pair of parent-child structural relationships in the entity tree structure. If the parent node and child node connected by the parent-child structural relationship have the same node hierarchy label, then add a relationship hierarchy label with the same hierarchy concept as the parent node or the child node to the parent-child structural relationship.

[0029] If the parent node and child node connected by the parent-child structure relationship do not have the same node hierarchy label, then add a cross-level relationship label to the parent-child structure relationship;

[0030] For each layer of the graph model in the initial graph ontology model, target entity nodes and target parent-child structural relationships belonging to the same layer concept as the graph model are selected based on the node hierarchy label and the relationship hierarchy label. Graph embedding technology is used to generate a low-dimensional vector representation for each target entity node and target parent-child structural relationship in the graph model, thereby constructing the project knowledge graph corresponding to the graph model.

[0031] Once the project knowledge graph to which all the hierarchical concepts belong is constructed, the vertical connection relationship between the project knowledge graphs at different levels is generated based on the parent-child structure relationship with the cross-level relationship label, thus obtaining the multi-level project knowledge graph of the power engineering project.

[0032] Optionally, the project supervision data includes project visual data, project audio data, and project multi-sensor data. The step of fusing the multi-dimensional project supervision data into multimodal supervision data, processing the multimodal supervision data using an adaptive cross-modal attention mechanism, and generating a project event cause representation for the power engineering project in the current time period includes the following steps:

[0033] The project visual features are extracted from the project visual data using a pre-trained convolutional neural network.

[0034] The acoustic features of the project are extracted from the project's audio data by combining Mel frequency cepstral coefficients and short-time Fourier transform.

[0035] A sensor fusion algorithm is used to fuse multi-sensor data of the project into comprehensive sensor data, and wavelet transform method is used to extract sensor data features from the comprehensive sensor data;

[0036] For any one of the project's visual features, acoustic features, and sensor data features, a feature attention score is calculated between the target feature and the other two features based on an adaptive cross-modal attention mechanism.

[0037] Based on the feature attention score, the project's visual features, acoustic features, and sensor data features are integrated into a comprehensive feature representation of the power engineering project in the current time period;

[0038] By mapping the comprehensive feature representation to a fixed-dimensional vector space using a multilayer perceptron, the cause representation of the project events of the power engineering project in the current time period is obtained.

[0039] Optionally, the heterogeneous graph neural network inference engine includes a heterogeneous graph attention module, a multi-head attention mechanism module, a gated loop module, a skip connection structure, and a batch normalization module. The heterogeneous graph attention module is used to process the graph nodes and graph node edges of different levels of the project knowledge graph in the multi-level project knowledge graph. The multi-head attention mechanism module is used to capture the information transmission between different levels of project knowledge graphs. The loss function of the heterogeneous graph neural network inference engine is constructed based on a contrastive learning strategy.

[0040] The step of predicting the set of abnormal events caused by the project event causes in the power engineering project within a preset future time period, based on the project event cause representation and through multi-level reasoning in the heterogeneous graph neural network inference engine using an improved graph traversal algorithm, includes the following steps:

[0041] Based on the project event cause representation, the reasoning starting node in the multi-level project knowledge graph is determined, and the project event cause representation is replaced with the node feature vector of the reasoning starting node;

[0042] The graph is iteratively traversed from the inference starting node using the heterogeneous graph neural network inference engine and a depth-first search algorithm. The depth-first search algorithm incorporates heuristic rules based on knowledge of the power engineering project domain. The number of iterations of the graph iterative traversal is set according to the time interval between a preset future time period and the current time period.

[0043] During the graph iterative traversal, for each target graph node that is traversed and visited, the heterogeneous graph attention module aggregates the neighbor node information of all neighbor nodes of the target graph node in the same level of the project knowledge graph, and dynamically updates the node features and all node edge features of the target graph node based on the neighbor node information.

[0044] Once any of the target graph nodes has completed information aggregation and feature dynamic update, the Monte Carlo tree search method is used to select the next graph node to be visited in the project knowledge graph at the level where the target graph node is located.

[0045] For any level of project knowledge graph, the node update information of all target graph nodes in the project knowledge graph is aggregated, and the aggregated node update information is transmitted to all other levels of project knowledge graph through the multi-head attention mechanism module.

[0046] After the final round of graph iteration traversal, multiple abnormal event chains are obtained in the multi-level project knowledge graph. The corresponding abnormal event result representations are extracted from each of the abnormal event chains, and all the abnormal event result representations are integrated into the abnormal event result set of the power engineering project in the future time period.

[0047] Optionally, the step of combining the available supervision resources and the abnormal event result set to generate the optimal supervision assignment strategy for the power engineering project in the future time period using a reinforcement learning optimization algorithm includes the following steps:

[0048] A state space is constructed by combining the abnormal event result set and the available supervision resources;

[0049] Based on the available supervision resources, a variety of different supervision assignment strategies are generated, and an action space is constructed using all the supervision assignment strategies.

[0050] A supervisory dispatch decision model is constructed by combining the state space and the action space;

[0051] With maximizing the efficiency of supervision resource utilization as the optimization objective, the optimal supervision dispatch strategy for the power engineering project in the future time period is output based on the supervision dispatch decision model and an offline strategy algorithm for iterative optimization.

[0052] Secondly, the present invention also provides an intelligent supervision and management platform based on multi-terminal interaction and data fusion, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the intelligent supervision and management method based on multi-terminal interaction and data fusion as described in the first aspect.

[0053] The beneficial effects of this invention are:

[0054] This invention acquires standardized supervision documents and available supervision resources for power engineering projects, and utilizes natural language processing technology to extract key entities and their relationships within the projects. This enables a more comprehensive and accurate identification of project processes, resources, and dependencies, solving the problem of incomplete and inaccurate information extraction caused by traditional methods relying on human experience and simple rules. This provides a more reliable data foundation for subsequent supervision management. Secondly, based on the extracted key entities and their relationships, this invention constructs a multi-level project knowledge graph and combines it with a graph neural network model to form a heterogeneous graph neural network inference engine. This allows for a deeper exploration of complex relationships and multi-level structures within the projects, overcoming the limitations of traditional methods that cannot fully reflect project complexity due to limited knowledge representation capabilities. This provides more comprehensive knowledge support for supervision management. Furthermore, this invention collects multi-dimensional project supervision data through various intelligent supervision terminals and integrates it into multi-modal data. Utilizing an adaptive cross-modal attention mechanism, it can more efficiently integrate and analyze multi-source heterogeneous data, solving the problem of traditional methods being unable to fully utilize multi-dimensional information due to insufficient data processing capabilities. This improves the depth and breadth of data analysis. Furthermore, this invention improves the graph traversal algorithm to perform multi-level reasoning in the inference engine, enabling more accurate prediction of the set of abnormal event results that may be caused by project events in the future. This overcomes the shortcomings of traditional methods, which are difficult to predict potential risks due to limited reasoning capabilities, and provides a more scientific basis for supervision management. Finally, this invention combines available supervision resources and the set of abnormal event results, and uses a reinforcement learning optimization algorithm to generate the optimal supervision assignment strategy. This enables more rational allocation of resources, reduces resource waste, and improves supervision efficiency, solving the problems of unreasonable resource allocation and low efficiency caused by insufficient optimization capabilities in traditional methods. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating one embodiment of the intelligent supervision and management method based on multi-terminal interaction and data fusion in this application.

[0056] Figure 2 This is a schematic diagram illustrating the relationship between the entity tree structure and the initial graph ontology model in one embodiment of this application. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0058] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0059] Figure 1 This is a flowchart illustrating an intelligent supervision and management method based on multi-terminal interaction and data fusion in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps. For example Figure 1 As shown, the intelligent supervision and management method based on multi-terminal interaction and data fusion disclosed in this invention specifically includes the following steps:

[0060] S101. Obtain standardized project supervision documents and available supervision resources for power engineering projects.

[0061] In the initial phase of power engineering projects, a distributed document management system is established to store and manage various standardized documents. These documents include, but are not limited to, national and industry standards such as the "Construction Supervision Specification for Power Construction Projects," the "Quality Acceptance and Evaluation Regulations for Power Engineering Construction," and the "Technical Specification for Construction and Acceptance of Overhead Transmission Lines for Power Transmission and Transformation Projects," as well as project-specific technical specifications and management systems. The document management system supports multiple formats, such as PDF, Word, and CAD drawings, and uses OCR technology to process scanned documents and extract text information. For drawings, computer vision technology is used to identify graphic elements and extract key information. Distributed storage technology is used for file storage to ensure high availability and fast access. Each file generates a unique digital fingerprint for version control and integrity verification. The system integrates a semantic search engine, supporting intelligent retrieval based on natural language; for example, entering "substation grounding requirements" can quickly locate relevant clauses. Simultaneously, the system automatically extracts key information from documents, generates summaries and tags, and establishes an index linking documents. For example, construction specifications are linked to corresponding quality acceptance standards to facilitate subsequent cross-referencing and consistency checks.

[0062] For available supervision resources, a real-time updated resource management database is established. This database contains all available human resources (such as supervising engineers and professional technicians) and material resources (such as testing equipment and safety equipment). Each resource item has a detailed attribute description, such as the professional qualifications, work experience, and current working status of the supervising personnel, and the model, performance parameters, and usage records of the equipment. The resource management system adopts a dynamic update mechanism to reflect the availability and status changes of resources in real time. For example, when a supervising engineer is assigned to a project, the system automatically updates their status to ensure the accuracy of resource allocation. In addition, the system also includes resource scheduling history records for analyzing resource utilization efficiency and optimizing future resource allocation. Through this comprehensive and intelligent document and resource management approach, a solid data foundation is laid for subsequent deep learning and natural language processing, ensuring the standardization, comprehensiveness, and efficient utilization of resources in project supervision work. The implementation of this step can significantly improve the efficiency of the initial stage of the project, reduce information retrieval time, avoid resource conflicts, and provide strong support for the smooth progress of the entire project.

[0063] S102. Use deep learning-enhanced natural language processing technology to extract key entities of the power project from the standardized project supervision documents, and identify the entity relationships between key entities of the power project.

[0064] In this step, key entities in the power project include power project process entities, power project resource entities, and project process dependency entities. Entity relationships include entity dependency relationships and entity coreference relationships. Specifically, a deep learning model based on BERT (Bidirectional Encoder Representations from Transformers) is used to process the standardized supervision documents for power engineering projects. First, a BERT model fine-tuned for the power field is used for text encoding to generate context-sensitive word embeddings. For example, for a phrase like "installing high-voltage cables," the model can understand the specific meaning of "high voltage" in the power context. Next, Named Entity Recognition (NER) technology is used to identify key entities in the text, such as process names (e.g., "transformer installation") and resource types (e.g., "500kV transformer"). The NER model uses a BiLSTM-CRF (Bidirectional Long Short-Term Memory Network with Conditional Random Field) structure, which can effectively capture long-distance dependencies. For the identified entities, a relation extraction model is used to extract their dependencies. Here, a Graph Attention Network (GAT) is used for relation extraction, such as identifying the dependency that "transformer installation" must be performed after "basic engineering completion." To handle the specialized terminology and abbreviations specific to power engineering, domain-specific dictionaries and rules are introduced to enhance processing effectiveness. For example, "GIS" is correctly resolved as "gas-insulated switchgear". Furthermore, image processing techniques are used to analyze engineering drawings, extracting spatial information such as equipment layout and line routing, and then integrating this information with textual data. Finally, knowledge graph completion technology is employed to infer potential implicit relationships based on the extracted information. Through this series of deep learning-enhanced natural language processing techniques, it is possible to accurately extract process information (such as "substation main transformer installation" and "overhead line erection"), resource information (such as "500kV transformer" and "insulator"), and dependencies between them from complex supervision documents in power engineering projects (such as "cable laying" must follow "cable trench construction"). This lays the foundation for subsequent knowledge graph construction.

[0065] S103. Construct a multi-level project knowledge graph for power engineering projects based on key entities of power projects and according to the relationships between entities.

[0066] The design employs a hierarchical ontology model, defining entity and relation types at the macro, meso, and micro levels. The macro level includes top-level project categories such as "transmission network construction" and "substation construction"; the meso level includes specific sub-projects such as "500kV substation main transformer installation" and "220kV transmission line erection"; and the micro level is refined to specific process steps such as "transformer foundation pouring" and "insulator installation." Next, a hierarchical clustering algorithm (such as AGNES) is used to process the entities and relations extracted from S2 at different levels. For example, "transformer installation" is categorized into the meso level, while "transformer tank sealing inspection" is categorized into the micro level. During the construction process, special attention is paid to the specific characteristics of power engineering, such as considering voltage levels and types of power equipment for classification. Inter-layer relationships are established through edges such as "composition" and "refinement." For example, the meso level "substation main transformer installation" is connected to multiple specific steps in the micro level through "refinement" edges. To handle the complex dependencies in power engineering, temporal logic reasoning is introduced to ensure that constraints such as "high-voltage equipment gas filling" must be correctly represented after "vacuum degree detection is qualified." Simultaneously, graph embedding techniques (such as TransE) are used to generate low-dimensional dense vector representations for each node and edge, facilitating subsequent similarity calculations and reasoning. Furthermore, a knowledge graph completion algorithm is introduced to automatically infer and add potential relationships based on knowledge in the power engineering domain, such as automatically inferring the required number of main transformers based on substation capacity. Finally, a graph database (such as Neo4j) is used to store the constructed knowledge graph, supporting efficient graph query and update operations. This multi-layered knowledge graph not only comprehensively represents the complex structure of power engineering projects but also supports project analysis and decision-making at different granularities, laying the foundation for subsequent intelligent supervision and management.

[0067] S104. Based on a multi-level project knowledge graph, a heterogeneous graph neural network inference engine for power engineering projects is constructed by combining a graph neural network model.

[0068] First, a heterogeneous graph attention module is designed to handle different types of nodes (such as substation equipment, line components, human resources, etc.) and edges (such as dependencies, temporal relationships, spatial relationships, etc.) in power engineering. For each type of edge r, a transformation matrix W_r and an attention vector a_r are defined. For example, edges of the "dependency" type are assigned a higher attention weight because they directly affect the sequence of processes. The attention calculation formula is: α_ij^r=softmax(LeakyReLU(a_r^T[W_rh_i||W_rh_j])), where h_i and h_j are the feature vectors of nodes i and j, respectively. Next, a multi-head attention mechanism module is constructed to capture the information transfer between different levels of project knowledge graphs. The node feature update formula is: h_i'=||_k=1^Kσ(Σ_j∈N_i^rα_ij^(r,k)W_r^kh_j). To handle long-term dependencies in power engineering (such as the entire process from project initiation to final grid connection), a gated recurrent unit (GRU) module is introduced. The GRU update process considers the historical information of nodes, making it particularly suitable for handling long-cycle processes such as power equipment installation, commissioning, and operation. To alleviate the gradient vanishing problem in deep networks, a skip connection structure is designed to ensure that important features at the lower levels can be directly transmitted to higher levels. Considering the complexity of power engineering data, batch normalization is used to improve the training stability of the model. Finally, a contrastive learning strategy is employed to enhance the model's ability to represent graph structures, especially its understanding of the topological relationships between key components in power systems. This heterogeneous graph neural network inference engine can effectively handle complex relationships and dependencies in power engineering projects, providing strong support for subsequent intelligent decision-making and risk prediction.

[0069] S105. Collect multi-dimensional project supervision data of power engineering projects in the current time period through various intelligent supervision terminals.

[0070] The system includes various intelligent monitoring terminals such as a high-definition camera system, an audio acquisition system, and a multi-dimensional sensor network. The high-definition camera system comprises fixed cameras and mobile drone cameras. Fixed cameras are installed in key locations, such as the main transformer area and GIS equipment area of ​​substations, providing 24 / 7 uninterrupted monitoring. Drone cameras are used to inspect the construction status of transmission lines and towers. These cameras feature 4K resolution, a frame rate of 60fps, and infrared night vision capabilities, ensuring clear images are captured under various lighting conditions. The comprehensive audio acquisition system includes high-sensitivity microphone arrays installed in key areas such as substations, with a sampling rate of 48kHz and 24-bit quantization, capable of capturing sounds ranging from low-frequency transformer hum to high-frequency corona discharge. These microphones employ beamforming technology to accurately locate sound sources, facilitating the timely detection of abnormal equipment.

[0071] The multi-dimensional sensor network includes electrical parameter sensors (such as voltage, current, and power factor monitors), environmental parameter sensors (such as temperature, humidity, and wind speed sensors), and structural health monitoring sensors (such as strain gauges and vibration sensors). Partial discharge sensors and infrared thermal imagers are installed, particularly around high-voltage equipment, for early fault detection. These sensors feature industrial-grade design, strong anti-interference capabilities, and sampling frequencies up to 1kHz. All acquired data is transmitted in real-time to the central processing unit via a 5G network, ensuring data real-time performance and integrity. Furthermore, edge computing technology is introduced for preliminary data processing and analysis locally, reducing transmission bandwidth requirements. For example, lightweight target detection algorithms can run on edge devices, transmitting only detected anomalies. To ensure data security, end-to-end encryption and strict access control are implemented. This comprehensive, multi-modal data acquisition system enables real-time and comprehensive monitoring of all aspects of power engineering projects, providing a rich and reliable data foundation for subsequent intelligent analysis and decision-making.

[0072] S106. Integrate multi-dimensional project supervision data into multimodal supervision data, use an adaptive cross-modal attention mechanism to process the multimodal supervision data, and generate a representation of the causes of project events in the power engineering project under the current time period.

[0073] In power engineering projects, multi-dimensional project supervision data includes visual data (such as high-definition images and videos), audio data (such as equipment operating sounds), and various sensor data (such as temperature, humidity, vibration, etc.). First, these heterogeneous data are preprocessed and feature extracted. For visual data, a pre-trained convolutional neural network (such as ResNet-50) is used to extract visual features; for audio data, Mel-frequency cepstral coefficients (MFCC) and short-time Fourier transform (STFT) are used to extract acoustic features; for sensor data, wavelet transform is used to extract time-frequency features. Next, an adaptive cross-modal attention mechanism is designed to dynamically adjust the weights between different modalities. Specifically, for each modality m, its attention score with other modalities n is calculated. This mechanism allows the model to dynamically adjust the attention given to different modal information based on the current situation. For example, when detecting transformer anomalies, more attention may be paid to audio and thermal imaging data; while when monitoring transmission line construction, more emphasis may be placed on visual data. Finally, a multilayer perceptron is used to map the fused features to a fixed-dimensional vector space; this vector is the "project event cause representation".

[0074] For example, in an ongoing transformer installation process, the event cause representation might be a 300-dimensional vector, where different dimensions encode information such as equipment status, environmental conditions, and operational procedures. Specifically, the first 100 dimensions of the vector might represent visual information (e.g., whether the equipment is in the correct position, whether the worker's operation is standardized), the middle 100 dimensions might represent audio information (e.g., whether the equipment is operating normally, whether alarms are sounding), and the last 100 dimensions might represent sensor data (e.g., temperature, humidity, vibration intensity). Each element of this vector has a value between 0 and 1, representing the probability of a specific state or event. In this way, complex multimodal information is compressed into a unified and easily processed representation. This representation not only captures key information in the current time period but also preserves the interrelationships between different modalities, providing a strong foundation for subsequent event reasoning and prediction. Implementing this method can significantly improve the understanding of complex power engineering environments, enabling early identification and warning of potential problems.

[0075] S107. Based on the project event cause representation, multi-level reasoning is performed in the heterogeneous graph neural network inference engine by improving the graph traversal algorithm to predict the abnormal event result set caused by the project event cause in the preset future time period of the power engineering project.

[0076] In this approach, the event cause representation is used as the feature vector of the starting node. For example, if the event cause representation shows an abnormally high transformer temperature, this would become the starting point for inference. Next, an improved Depth-First Search (DFS) algorithm is used for graph traversal. The traditional DFS algorithm is modified to adapt to the specificities of power engineering, incorporating domain-knowledge-based heuristics. The heuristic function guides the algorithm to prioritize paths more likely to lead to serious consequences. During the traversal, a heterogeneous graph neural network inference engine dynamically updates the features of nodes and edges. For each visited node, its state is updated by aggregating information from its neighbors. This dynamic update mechanism allows the inference process to adapt to complex power system state changes. To handle uncertainties in power engineering, Monte Carlo Tree Search (MCTS) is introduced. Each time a node is selected for exploration, multiple simulations are performed to evaluate possible outcomes, and then the most promising path is chosen. This helps find the most likely chain of anomalous events in a large-scale state space.

[0077] During multi-level reasoning, the algorithm jumps between different levels of the knowledge graph. For example, "transformer temperature anomaly" at the micro level may escalate to "substation operation anomaly" at the meso level, and then to "power grid stability threat" at the macro level. This multi-level reasoning can comprehensively assess the potential impact of events. Finally, an abnormal event result set is generated based on the traversal results. This result set is a probability distribution representing the likelihood of different abnormal events occurring within a preset future time period. For example, {("transformer overheating", 0.7, "within 24 hours"), ("insulation failure", 0.4, "within 48 hours"), ("partial power outage", 0.2, "within 72 hours")}, where each tuple contains the abnormal event type, probability of occurrence, and expected occurrence time. Through this improved graph traversal algorithm and multi-level reasoning, the system can comprehensively analyze various abnormal situations that may be caused by project events and provide time-dimensional predictions, providing important basis for subsequent risk management and decision-making. The implementation of this method can significantly improve the risk prediction capability of power engineering projects, enabling early warning and timely intervention for potential problems.

[0078] S108. Combining available supervision resources and abnormal event result sets, a reinforcement learning optimization algorithm is used to generate the optimal supervision assignment strategy for power engineering projects in the future time period.

[0079] A Markov Decision Process (MDP) model considering multiple objectives can be constructed. The state space S includes the current resource state (e.g., available supervisors and equipment), predicted abnormal events (type, probability, and expected occurrence time), and project schedule. The action space A includes different assignment decisions, such as assigning specific supervisors to specific work areas and scheduling detection equipment. The reward function R comprehensively considers multiple objectives, including risk minimization, resource utilization efficiency, and project schedule. The Soft Actor-Critic (SAC) algorithm is used as the core of reinforcement learning. The SAC algorithm achieves a balance between exploration and exploitation by maximizing the weighted sum of expected reward and policy entropy. The policy network π(a|s) outputs action probabilities using a Gaussian distribution, and the value function Q(s,a) evaluates the value of state-action pairs. Through this reinforcement learning optimization algorithm, the system can generate a highly adaptive and globally optimal supervisor assignment strategy. Specifically, the generated strategy includes the following: 1) Based on the predicted "transformer overheating" event, dispatch 3 additional supervising engineers with transformer expertise to the relevant substation within the next 24 hours and deploy 2 infrared thermal imagers for continuous monitoring. 2) For potential "insulation failure" issues, schedule 5 insulation resistance tests within the next 48 hours, at the 12th, 24th, 36th, 42nd, and 47th hours, with each test lasting 2 hours, and assign supervising personnel with high-voltage equipment testing qualifications to perform the tests. 3) Considering the potential risk of "partial power outages," conduct a comprehensive line inspection every 8 hours within the next 72 hours, focusing on weak points that may lead to power outages, such as aging insulators and loose clamps. 4) Adjust the original routine inspection plan, reducing the inspection frequency of non-critical areas by 50% and reallocating the saved manpower to high-risk areas. 5) During the predicted high-risk period (e.g., hours 48-60), arrange 24-hour uninterrupted monitoring and prepare an emergency response team to ensure immediate action in the event of any abnormal situation.

[0080] This task assignment strategy not only addresses potential problems in a targeted manner but also optimizes resource allocation and ensures quality control at critical time points. Through dynamic adjustment and real-time optimization, the system can continuously adjust its strategy based on environmental changes and new forecasts, maintaining the efficiency and focus of supervision work. The implementation of this method can significantly improve the management efficiency and safety of power engineering projects, reduce human error, optimize resource utilization, and enhance the ability to respond to emergencies. Ultimately, it will help reduce project risks, improve engineering quality, and ensure the smooth completion and long-term stable operation of power engineering projects.

[0081] In one implementation, extracting key entities of a power project from standardized project supervision documents using deep learning-enhanced natural language processing techniques and identifying entity relationships between these key entities includes the following steps:

[0082] Context-sensitive word embeddings were performed on standardized project supervision documents using a pre-trained BERT model.

[0083] Based on word embedding results, a named entity recognition model combining bidirectional long short-term memory network and conditional random field is used to extract key entities of power projects from standardized project supervision documents. Key entities of power projects include power project process entities, power project resource entities, and project process dependency entities.

[0084] Based on the contextual relationships of key entities in power projects within standardized project supervision documents, the entity dependencies between different key entities in power projects are identified.

[0085] Based on the key entities of power projects and using the coreference resolution model to traverse the document text of the standardized project supervision documents in reverse, the reference relationships of all key entities of power projects in the standardized project supervision documents are identified and parsed to obtain the entity coreference relationships between key entities of different power projects.

[0086] In this implementation, a pre-trained BERT model is used to perform context-sensitive word embeddings on standardized project supervision documents. BERT, or Bidirectional Encoder Representations from Transformers, is a deep learning model based on the Transformer architecture. By pre-training on large-scale text data, it can capture deep semantic information of words in text. In this step, the BERT model first receives text input from standardized project supervision documents, which typically contain various specifications, processes, and standards from power engineering projects. The BERT model performs layer-by-layer semantic encoding on the input text through multiple Transformer encoder layers. Each layer of encoding takes into account the contextual information of words; that is, the meaning of a word is influenced by the words before and after it. In this way, the BERT model can generate high-dimensional vector representations for each word. These vectors not only contain the semantics of the word itself but also its meaning in a specific context. This word embedding provides a rich semantic foundation for subsequent entity recognition and relation extraction. For example, in power engineering projects, the term "transformer" may refer to different entities in different contexts, such as equipment entities or process entities. The BERT model can generate different vector representations based on the context, thereby providing accurate semantic information for subsequent steps.

[0087] Next, based on the word embeddings generated by the BERT model, a named entity recognition model combining a Bidirectional Long Short-Term Memory (BiLSTM) network and a Conditional Random Field (CRF) is used to extract key entities of the power project from the standardized project supervision documents. BiLSTM is a special type of recurrent neural network (RNN) that can simultaneously consider both forward and backward information in a text sequence, thus better capturing long-distance dependencies in the text. CRF is a statistical model for sequence labeling that can predict the most likely output label sequence given an input sequence. In this step, BiLSTM first receives the word embedding vectors generated by the BERT model as input, and then processes the text sequence word by word through its internal LSTM units. Each LSTM unit outputs a hidden state containing information about the current word and its context. Then, the CRF model receives the hidden state sequence output by BiLSTM and labels each word according to a predefined label set (such as power project process entities, power project resource entities, and project process dependency entities). In this way, the model can accurately identify key entities in the text. For example, the phrase "transformer installation" can be identified in text and labeled as a power project process entity. This entity recognition method can not only handle complex text structures but also effectively address the issues of nesting and overlapping between entities, thereby improving the accuracy and robustness of entity recognition.

[0088] Next, based on the contextual relationships of key entities in the standardized project supervision documents, the entity dependency relationships between different key entities in the power project are identified. Entity dependency relationships refer to semantic dependencies between entities; for example, a process entity may depend on a resource entity. In this step, a dependency relationship parsing model is first constructed, typically based on a graph or tree structure to represent the relationships between entities. The model receives the key entities identified in the previous step and their contextual information as input, and then constructs a dependency relationship graph by analyzing the syntactic and semantic relationships between entities. For example, if the sentence "Transformer installation requires transformer equipment" is identified in the text, the model will analyze that the process entity "transformer installation" depends on the resource entity "transformer equipment." This identification of dependency relationships relies not only on the surface form of words but also on a deep understanding of the semantic structure of the text. In this way, the model can accurately capture the complex relationships between entities, thus providing a foundation for subsequent knowledge graph construction.

[0089] Next, based on the key entities of the power project and using a coreference resolution model, the document text of the standardized project supervision documents is traversed in reverse. This process identifies and parses the referential relationships of all key entities in the standardized project supervision documents, obtaining the coreference relationships between different key entities. Coreference refers to the phenomenon where different words or phrases in the text refer to the same entity; for example, "transformer" and "this equipment" may refer to the same entity. In this step, the coreference resolution model first receives the key entities identified in the previous step and their contextual information as input. Then, by analyzing pronouns, indicator words, and other referential phenomena in the text, it identifies all words or phrases that refer to the same entity. For example, if the sentence "Transformer installation requires transformer equipment, and this equipment must meet the standard" is identified in the text, the model will analyze that "this equipment" refers to the entity "transformer equipment". Coreference resolution models are typically implemented based on rule-based, statistical, or deep learning methods, with deep learning models better able to handle complex referential phenomena. In this way, the model can accurately identify coreference relationships in the text, thus providing more complete and accurate semantic information for subsequent knowledge graph construction. The identification of such coreference relationships can not only improve the integrity of the knowledge graph, but also enhance its reasoning ability, thereby providing stronger support for the intelligent supervision and management of power engineering projects.

[0090] In one embodiment, the method further includes the following steps:

[0091] Knowledge distillation technology is used to transfer the model knowledge of pre-trained large language models to BERT models, named entity recognition models, coreference resolution models, and heterogeneous graph neural network inference engines.

[0092] In this embodiment, knowledge distillation is a model compression and knowledge transfer technique. Its core idea is to guide the training process of a relatively simple model (usually called a student model) using a pre-trained, complex model (often called a teacher model). The teacher model typically possesses stronger expressive power and higher accuracy, while the student model needs to maintain high performance while minimizing the consumption of computational resources and storage space. In this step, a pre-trained large language model, such as GPT-3 or T5, is first selected as the teacher model. These large language models are typically pre-trained on large-scale text data and can capture rich linguistic knowledge and semantic information.

[0093] Next, knowledge distillation is used to transfer the knowledge from the teacher model to the BERT model, named entity recognition model, coreference resolution model, and heterogeneous graph neural network inference engine. Specifically, the knowledge distillation process typically includes two stages: soft label generation and student model training. In the soft label generation stage, the teacher model processes the input text and generates probability distributions for each word or phrase; these probability distributions are called soft labels. Soft labels not only contain the teacher model's predictions for each word or phrase but also information about the teacher model's confidence in these predictions. In the student model training stage, the student model receives the same input text and is trained by minimizing the difference between its output and the soft labels generated by the teacher model. This difference is typically measured using the cross-entropy loss function, calculated as: L=-∑(y i *log(p i ), where y i These are soft labels generated by the teacher model, p i It is the predicted probability of the student model.

[0094] In this way, student models can learn from the knowledge of teacher models, thereby reducing model complexity and computational resource consumption while maintaining high performance. For example, during the training of the BERT model, knowledge distillation allows the BERT model to learn the deep understanding of word context semantics from a large language model, thus improving its performance in specific tasks. During the training of named entity recognition models, knowledge distillation helps the model better capture entity boundaries and category information in text, thereby improving the accuracy of entity recognition. During the training of coreference resolution models, knowledge distillation helps the model better understand referential relationships in text, thereby improving the accuracy of coreference resolution. During the training of heterogeneous graph neural network inference engines, knowledge distillation helps the model better capture complex relationships in knowledge graphs, thereby improving the accuracy and efficiency of inference. Through knowledge distillation, these models not only perform well in specific tasks but also maintain high performance and efficiency under limited computational resources and storage space. This knowledge transfer method not only improves the model's generalization ability but also enhances its adaptability and robustness in practical applications, thus providing more reliable and efficient technical support for the intelligent supervision and management of power engineering projects.

[0095] In one implementation, constructing a multi-level project knowledge graph for a power project based on key entities and their relationships includes the following steps:

[0096] The initial graph ontology model is designed based on several different pre-defined hierarchical concepts, including macroscopic, mesoscopic, and microscopic levels.

[0097] A tree structure of key entities for all power projects is constructed by combining entity dependency and entity coreference. Entity nodes in the entity tree structure represent key entities of power projects, and the parent-child relationship between entity nodes in the entity tree structure represents entity dependency. Entity nodes with entity coreference belong to the same level of the tree structure in the entity tree structure.

[0098] Count the number of tree structure layers in the entity tree structure and the number of structural branches in each tree structure layer of the entity tree structure;

[0099] By combining the number of tree structure layers and the number of structural branches, the entity tree structure is divided into different hierarchical concepts, and based on the hierarchical concepts, the entity tree structure is mapped to the initial graph ontology model to form a multi-level project knowledge graph for power engineering projects.

[0100] In this embodiment, the graph ontology model is a structured model used to represent knowledge. It describes knowledge in a specific domain by defining entities, attributes, and the relationships between them. In this step, the definition and scope of each level of concept must first be clarified. The macro level typically represents high-level concepts and the overall framework of a power engineering project, such as the overall project objectives, main phases, and key milestones. The meso level focuses on intermediate-level concepts in the project, such as specific procedures, resource allocation, and task dependencies. The micro level delves into the detailed aspects of the project, such as specific operational steps, equipment parameters, and personnel division of labor. When designing the initial graph ontology model, corresponding entity types, attribute types, and relationship types need to be defined for each level of concept. For example, at the macro level, the entity type "Project Phase" can be defined, with attributes including "Phase Name," "Start Time," and "End Time"; at the meso level, the entity type "Procedure" can be defined, with attributes including "Procedure Name," "Required Resources," and "Depending Procedures"; and at the micro level, the entity type "Operational Step" can be defined, with attributes including "Step Name," "Executor," and "Equipment Parameters." In this way, the initial graph ontology model can provide a clear structural framework for the subsequent construction of entity tree structures and the formation of multi-level knowledge graphs.

[0101] Next, we construct an entity tree structure for all key entities in the power project by combining entity dependency relationships and entity coreference relationships. An entity tree structure is a tree-like data structure used to represent hierarchical relationships and dependencies between entities. In this step, we first need to represent the key entities of the power project as nodes in the entity tree structure. Each node represents an entity, such as the "Transformer Installation" process entity or the "Transformer Equipment" resource entity. The parent-child relationship between entity nodes represents the entity dependency relationship. For example, the "Transformer Installation" process entity depends on the "Transformer Equipment" resource entity; therefore, in the entity tree structure, the "Transformer Equipment" node is the parent node of the "Transformer Installation" node. Entity nodes with entity coreference relationships belong to the same level of the tree structure. For example, "Transformer Equipment" and "the equipment" point to the same entity; therefore, in the entity tree structure, these two nodes belong to the same level of the tree structure. In this way, the entity tree structure can clearly represent the hierarchical relationships and dependencies between key entities in the power project, thus providing a foundation for the subsequent construction of a multi-level knowledge graph.

[0102] Next, we need to count the number of tree levels in the entity tree structure and the number of branches at each level. The number of tree levels refers to the maximum depth of the entity tree structure from the root node to a leaf node, and the number of branches refers to the number of branches in each level of the tree structure. In this step, we first need to traverse the entity tree structure, starting from the root node and counting the number of nodes at each level downwards. For example... Figure 2 As shown, in the entity tree structure, the root node is "Power Engineering Project," and its child nodes include "Project Phase 1," "Project Phase 2," and "Project Phase 3." These child nodes each have their own child nodes, such as "Process 1," "Process 2," and "Process 3." In this way, the total number of levels in the entity tree structure and the number of nodes in each level can be counted. Finally, based on the number of levels and the number of branches in each level, a logical rule can be introduced to dynamically divide the entity tree structure into macro-level, meso-level, and micro-level structures.

[0103] In one implementation, the entity tree structure is divided into different hierarchical concepts based on the number of tree structure layers and branches, and the entity tree structure is mapped to the initial graph ontology model based on the hierarchical concepts to form a multi-level project knowledge graph for the power engineering project. This includes the following steps:

[0104] Combining the number of tree structure layers and the number of structural branches, each layer of the entity tree structure is divided into macro level, meso level and micro level in the order from the root node to the leaf node, and node level tags with corresponding level concepts are added to all entity nodes.

[0105] Traverse each parent-child relationship in the entity tree structure. If the parent node and child node connected by the parent-child relationship have the same node level label, then add a relationship level label with the same level concept as the parent node or child node to the parent-child relationship.

[0106] If the parent and child nodes connected by the parent-child structure do not have the same node hierarchy label, then add a cross-level relationship label to the parent-child structure.

[0107] For each layer of the graph model in the initial graph ontology model, target entity nodes and target parent-child structure relationships belonging to the same level of concepts as the graph model are selected based on node hierarchy labels and relationship hierarchy labels. Graph embedding technology is used to generate low-dimensional vector representations for each target entity node and target parent-child structure relationship in the graph model, thereby constructing the project knowledge graph corresponding to the graph model.

[0108] Once the project knowledge graphs for all levels of concepts are constructed, the vertical connection relationships between project knowledge graphs at different levels are generated based on the parent-child structure relationships with cross-level relationship tags, resulting in a multi-level project knowledge graph for the power engineering project.

[0109] In this embodiment, the tree structure is divided into macro, meso, and micro levels sequentially from the root node to the leaf node, based on the number of tree structure layers and the number of branches in each layer. The core idea of ​​the level division rule is to divide the tree structure into macro, meso, and micro levels sequentially from the root node to the leaf node according to the number of tree structure layers and the number of branches in each layer. The specific rules are as follows:

[0110] Macro-level: Contains the first N levels of the entity tree structure, where the value of N is determined by the total number of levels and branches in the tree structure. The macro-level typically represents high-level concepts in power engineering projects, such as overall project objectives, major phases, and key milestones.

[0111] Meso-level: This includes the intermediate M layers of the entity tree structure. The value of M is determined by the total number of layers and branches in the tree structure. Meso-level typically represents intermediate concepts within a project, such as specific processes, resource allocation, and task dependencies.

[0112] Micro-level: This includes the remaining levels of the entity tree structure, typically representing the detailed aspects of the project, such as specific operational steps, equipment parameters, and personnel division of labor.

[0113] Let L be the total number of levels in the tree structure, and B be the average number of branches per level. The specific partitioning formula is as follows:

[0114] The number of macroscopic levels N = ceil(log_B(L)), where ceil represents rounding up and log_B represents the logarithm to the base B.

[0115] The number of meso-level layers M = ceil(log_B(L / 2)).

[0116] The number of micro-level layers is the number of remaining layers, i.e., LNM.

[0117] Next, add a corresponding hierarchy marker to each node. For example... Figure 2 As shown, the "Power Engineering Project" node in layer 1 and the "Project Phase 1" node in layer 2 are labeled as macro-level, the "Process 1" node in layer 3 is labeled as meso-level, and the "Operation Step 1" node in layer 4 and the "Equipment Parameter 1" node in layer 5 are labeled as micro-level. In this way, each node in the entity tree structure is accurately assigned to its corresponding hierarchical concept, thus providing a clear hierarchical structure for subsequent knowledge graph construction. Then, each parent-child relationship in the entity tree structure is traversed. If the parent and child nodes connected by the parent-child relationship have the same node hierarchical label, a relationship hierarchical label with the same hierarchical concept as the parent or child node is added to the parent-child relationship. The core objective of this step is to identify and label the hierarchical relationships within the entity tree structure, that is, the relationships between nodes within the same level. First, all parent-child relationships in the entity tree structure need to be traversed. For example... Figure 2 As shown, the parent node is "Project Phase 1" (macro level), and the child node is "Project Phase 2" (macro level). Therefore, this parent-child relationship belongs to the same level. Next, add a relationship hierarchy marker to this type of relationship that shares the same hierarchical concept as the parent or child node. For example... Figure 2 As shown, a macro-level relationship hierarchy marker is added to the relationship between "Project Phase 1" and "Project Phase 2". In this way, the hierarchical relationships within the entity tree structure are accurately marked, providing clear hierarchical relationship information for subsequent knowledge graph construction. This hierarchical relationship marking not only improves the structuring of the knowledge graph but also enhances its query and reasoning efficiency.

[0118] If the parent and child nodes connected by a parent-child relationship do not have the same node hierarchy label, a cross-level relationship label is added to the parent-child relationship. The core objective of this step is to identify and label cross-level relationships in the entity tree structure, that is, the relationships between nodes at different levels. First, all parent-child relationships in the entity tree structure need to be traversed. For example... Figure 2As shown, the parent node is "Project Phase 1" (macro level), and the child node is "Process 1" (meso level). This parent-child relationship is a cross-level relationship. Next, cross-level relationship markers are added to these relationships. For example, a cross-level relationship marker is added to the relationship between "Project Phase 1" and "Process 1". In this way, cross-level relationships in the entity tree structure are accurately marked, providing clear cross-level relationship information for subsequent knowledge graph construction. This marking of cross-level relationships not only improves the completeness of the knowledge graph but also enhances its multi-level reasoning capabilities.

[0119] Next, the nodes and relationships in the entity tree structure are mapped to the initial graph ontology model, generating low-dimensional vector representations to construct a multi-level project knowledge graph. First, target entity nodes and target parent-child relationships belonging to the same level of concepts in the graph model are selected based on node and relationship level labels. For example, in the macro-level graph model, all nodes and relationships labeled as macro-level are selected. Next, graph embedding techniques are used to generate low-dimensional vector representations for each target entity node and target parent-child relationship. Graph embedding is a method that maps nodes and relationships in a graph structure to a low-dimensional vector space; commonly used graph embedding algorithms include Node2Vec and GraphSAGE. Through graph embedding, each node and relationship is represented as a low-dimensional vector, facilitating subsequent queries and reasoning. In this way, each layer of the graph model is constructed as a corresponding project knowledge graph.

[0120] Once the project knowledge graphs for all levels of concepts are constructed, the project knowledge graphs at different levels are connected through cross-level relationships to form a complete multi-level knowledge graph. First, all parent-child structural relationships marked with cross-level relationships need to be traversed. For example, if there is a cross-level relationship between the "Project Phase 1" node in the macro-level graph model and the "Process 1" node in the meso-level graph model, then the "Project Phase 1" node and the "Process 1" node are connected through this cross-level relationship. In this way, the project knowledge graphs at the macro, meso, and micro levels are connected into a complete multi-level knowledge graph. This multi-level knowledge graph not only clearly represents the hierarchical relationships and dependencies between different levels of concepts in a power engineering project, but also supports efficient querying and reasoning.

[0121] In one implementation, the project supervision data includes project visual data, project audio data, and project multi-sensor data. The multi-dimensional project supervision data is fused into multimodal supervision data. An adaptive cross-modal attention mechanism is used to process the multimodal supervision data to generate a representation of the causes of project events in the power engineering project for the current time period. This process includes the following steps:

[0122] Use a pre-trained convolutional neural network to extract visual features of the project from the project visual data;

[0123] The acoustic features of a project are extracted from its audio data by combining Mel frequency cepstral coefficients and short-time Fourier transform.

[0124] A sensor fusion algorithm is used to fuse multi-sensor data into comprehensive sensor data, and wavelet transform is used to extract sensor data features from the comprehensive sensor data.

[0125] For any one of the target features among the project's visual features, project's acoustic features, and sensor data features, the feature attention score between the target feature and the other two features is calculated based on an adaptive cross-modal attention mechanism;

[0126] Based on feature attention scores, the project's visual features, acoustic features, and sensor data features are integrated into a comprehensive feature representation of the power engineering project in the current time period;

[0127] By mapping the comprehensive feature representation to a fixed-dimensional vector space using a multilayer perceptron, the cause representation of project events in the power engineering project under the current time period is obtained.

[0128] In this embodiment, a convolutional neural network (CNN) is a deep learning model specifically designed for processing image data. Its core idea is to extract multi-level feature representations from images through structures such as convolutional layers, pooling layers, and fully connected layers. Pre-trained CNN models are typically trained on large-scale image datasets (such as ImageNet) and can capture common features in images, such as edges, textures, and shapes. In this step, the project's visual data (e.g., images or video frames from a construction site) is first input into the pre-trained CNN model. The CNN model processes the image layer by layer through its internal convolutional layers, with each layer extracting features at different levels. For example, lower convolutional layers might extract edge and texture features, while higher convolutional layers might extract semantic features, such as equipment type or construction status. In this way, the CNN model can generate high-dimensional feature vectors containing rich information about the image. For example, in a power engineering project, the CNN model can extract visual features such as the location of transformers, the operating status of equipment, and the activities of construction workers from images of the construction site.

[0129] Next, the project's acoustic features are extracted from the audio data by combining Mel-frequency cepstral coefficients (MFCC) and short-time Fourier transform (STFT). MFCC is a commonly used audio feature extraction method that converts audio signals into a series of coefficients reflecting the spectral characteristics of sound by simulating the human ear's perception of sound. STFT is a method that converts time-domain audio signals into frequency-domain representations, capturing the changes in frequency components over time. In this step, the project's audio data (e.g., the sound of equipment operation or construction workers' conversations) is first preprocessed, including framing, windowing, and noise reduction. Next, the MFCC and STFT are calculated for each frame of the audio signal. The calculation of the MFCC involves performing a Fourier transform on the audio signal, converting the frequency scale to a Mel scale, taking the logarithm, and performing a discrete cosine transform. The STFT generates its frequency-domain representation by performing a Fourier transform on each frame of the audio signal. In this way, the audio data is converted into a series of coefficients and a spectrum reflecting the sound characteristics. For example, in power engineering projects, MFCC and STFT can extract the vibration frequency, operating status, and abnormal sound characteristics of the equipment from the sound of the equipment running.

[0130] Next, a sensor fusion algorithm is used to fuse multi-sensor data from the project into comprehensive sensor data, and wavelet transform is used to extract sensor data features from this comprehensive data. Sensor fusion algorithms are methods that integrate and process data from multiple sensors, improving data accuracy and reliability. Wavelet transform is a method that decomposes a signal into different frequency components, capturing local features and transient changes in the signal. In this step, the multi-sensor data (e.g., temperature, humidity, vibration, and pressure) is first preprocessed, including data alignment, denoising, and normalization. Next, a sensor fusion algorithm is used to fuse the multi-sensor data into comprehensive sensor data. Commonly used sensor fusion algorithms include Kalman filtering, weighted averaging, and principal component analysis. For example, Kalman filtering can fuse data from temperature and humidity sensors to generate comprehensive data reflecting environmental conditions. Then, wavelet transform is used to process the comprehensive sensor data and extract sensor data features. Wavelet transform decomposes signals at multiple scales, generating coefficients that reflect local signal features. For example, wavelet transform can extract the vibration frequency and abnormal vibration modes of equipment from vibration sensor data.

[0131] Next, the visual features, acoustic features, and sensor data features of the project are input into the adaptive cross-modal attention mechanism. For any target feature, its feature attention score is calculated between it and the other two features. The calculation of the feature attention score is usually based on the similarity of feature vectors, and the correlation between features can be measured by dot product or cosine similarity. For example, assuming the target feature is the visual feature of the project, the feature attention score between it and the acoustic and sensor data features of the project is calculated. The formula for calculating the feature attention score is: Attention Score = softmax((Q*K) T ) / sqrt(d k ), where Q is the query vector of the target feature, K is the key vector of other features, and d k This refers to the dimension of the vector. In this way, the adaptive cross-modal attention mechanism can dynamically adjust the weights of each modal feature, thereby capturing the correlations between multimodal data. Then, each modal feature is weighted according to its feature attention score. For example, for a project's visual features, its weight is the feature attention score between it and the project's acoustic features and sensor data features. Next, the weighted multimodal features are summed to generate a comprehensive feature representation. In this way, the comprehensive feature representation can reflect multi-dimensional information such as the equipment's operating status, environmental conditions, and construction progress.

[0132] Finally, a Multilayer Perceptron (MLP) is used to map the comprehensive feature representation to a fixed-dimensional vector space, obtaining the event cause representation of the power engineering project in the current time period. The MLP is a commonly used neural network model; its core idea is to map input features to a target space through multiple fully connected layers and nonlinear activation functions. In this step, the comprehensive feature representation is first input into the MLP. The MLP processes the input features layer by layer through its internal fully connected layers, with each layer performing a nonlinear transformation on the features. For example, the first fully connected layer maps the input features to the hidden layer space, and the second fully connected layer maps the hidden layer features to the output space. In this way, the MLP can map the comprehensive feature representation to a fixed-dimensional vector space, thereby effectively transforming the multimodal data of the power engineering project into an event cause representation.

[0133] In one implementation, based on the project event cause representation, multi-level reasoning is performed in a heterogeneous graph neural network inference engine using an improved graph traversal algorithm to predict the set of abnormal event results caused by the project event causes in a preset future time period of the power engineering project. This includes the following steps:

[0134] Based on the project event cause representation, the reasoning starting node in the multi-level project knowledge graph is determined, and the project event cause representation is replaced with the node feature vector of the reasoning starting node.

[0135] The graph iterative traversal is performed from the inference starting node using a heterogeneous graph neural network inference engine and a depth-first search algorithm. The depth-first search algorithm incorporates heuristic rules based on knowledge of the power engineering project domain. The number of iterations of the graph iterative traversal is set according to the time interval between the preset future time period and the current time period.

[0136] During the graph iterative traversal, for each target graph node that is traversed and visited, the heterogeneous graph attention module aggregates the neighbor node information of all neighbor nodes of the target graph node in the same level of project knowledge graph, and dynamically updates the node features and all node edge features of the target graph node based on the neighbor node information.

[0137] Once any target graph node has completed information aggregation and dynamic feature updates, the Monte Carlo tree search method is used to select the next graph node to be visited in the project knowledge graph at the level where the target graph node is located.

[0138] For any level of project knowledge graph, the node update information of all target graph nodes in the project knowledge graph is aggregated, and the aggregated node update information is passed to all other levels of project knowledge graph through the multi-head attention mechanism module.

[0139] After the final round of graph iteration traversal, multiple abnormal event chains are obtained in the multi-level project knowledge graph. The corresponding abnormal event result representations are extracted from each abnormal event chain, and all abnormal event result representations are integrated into an abnormal event result set for the power engineering project in the future time period.

[0140] In this implementation, firstly, the reasoning starting node in the multi-level project knowledge graph is determined based on the project event cause representation, and the project event cause representation is replaced with the node feature vector of the reasoning starting node. The specific implementation principle of this step is to calculate the similarity between the project event cause representation and the feature vectors of each node in the multi-level project knowledge graph, and select the node with the highest similarity as the reasoning starting node. The similarity calculation can use the cosine similarity method, that is, treating the project event cause representation and the node feature vector as vectors in a multi-dimensional space, and calculating the cosine value of the angle between them. The formula for calculating cosine similarity is: cos(θ)=(A·B) / (||A||||B||), where A and B represent the project event cause representation and the node feature vector, respectively, and ||A|| and ||B|| represent the magnitude of the vectors. For example, assuming the event cause is represented as [0.5, 0.3, 0.2], and the feature vector of a certain node is [0.4, 0.4, 0.2], their cosine similarity is (0.5*0.4+0.3*0.4+0.2*0.2) / (sqrt(0.5^2+0.3^2+0.2^2)*sqrt(0.4^2+0.4^2+0.2^2))≈0.989. After calculating the starting node for inference, the event cause representation is directly assigned to the feature vector of that node, achieving feature replacement. The effect of this step is to determine a suitable starting point for the subsequent graph traversal process, ensuring that inference starts from the node most relevant to the event cause, thus improving the accuracy and efficiency of inference.

[0141] Next, a heterogeneous graph neural network inference engine is used, and a depth-first search algorithm is employed to iteratively traverse the graph starting from the inference starting node. The basic principle of the depth-first search (DFS) algorithm is to start from the starting node and search as deeply as possible along the edges of the graph until it is impossible to proceed further. When it does, it backtracks to the previous node and continues to search for other unvisited adjacent nodes. In this scheme, the DFS algorithm is optimized to adapt to the characteristics of power engineering projects. Specific implementation methods include: (1) setting a search depth limit to prevent getting stuck in an excessively deep search; (2) introducing heuristic rules based on domain knowledge, such as prioritizing the access of edges that are strongly related to the current node. The correlation can be represented by a predefined weight matrix; (3) dynamically adjusting the search strategy during the search process, and deciding whether to continue searching deeper based on the importance of the node (which can be measured by indicators such as the degree centrality of the node). The method for setting the number of iterations can be a linear mapping, i.e., number of iterations = α * (preset future time period - current time period) + β, where α and β are parameters that are adjusted according to the actual situation. For example, if the preset future timeframe is 30 days from now, the current timeframe is today, α = 0.5, and β = 10, then the number of iterations is 0.5 * 30 + 10 = 25. This setup ensures that the time span of the reasoning matches the actual project cycle. The effect of this step is that it effectively explores possible propagation paths of abnormal events within a multi-layered project knowledge graph, while simultaneously improving the targeting and efficiency of the search by integrating domain knowledge.

[0142] During the graph iterative traversal, for each target graph node visited, the heterogeneous graph attention module aggregates the neighbor node information of all neighbor nodes in the same-level project knowledge graph. Based on this neighbor node information, the node features and edge features of the target graph node are dynamically updated. The core idea of ​​the heterogeneous graph attention module is to selectively aggregate information from different types of neighbor nodes by calculating attention weights. The specific implementation can be described as follows: (1) For the target node v, first collect the set N(v) of all its neighbor nodes of all types; (2) For each type of neighbor node u∈N(v), calculate the attention coefficient α(v,u)=softmax(LeakyReLU(a^T[Wh_v||Wh_u])), where a is a learnable attention vector, W is a linear transformation matrix, and h_v and h_u are the feature vectors of nodes v and u, respectively; (3) Aggregate the neighbor information according to the attention coefficient: h'_v=σ(∑(α(v,u)*Wh_u)), where σ is a non-linear activation function such as ReLU; (4) Update the target node features: h_v^new=h_v+h'_v; (5) For the edge connecting the target node and its neighbor nodes, update the edge features e(v,u)^new=e(v,u)+MLP([h_v^new||h_u^new]), where MLP is a multilayer perceptron. For example, suppose there is a target node representing a power device, and its neighboring nodes include maintenance records, operating status, and related personnel. By calculating attention weights, it might be found that maintenance records and operating status have a greater impact on the current state of the device. Therefore, these two types of nodes are given higher weights when aggregating information. The effect of this method is that it can capture the complex relationships between different types of nodes in a heterogeneous graph, improving the accuracy of node representation and the precision of anomaly prediction.

[0143] Once any target graph node completes information aggregation and dynamic feature updates, the Monte Carlo Tree Search (MCTS) method is used to select the next graph node to be visited in the project knowledge graph at the level where the target graph node is located. MCTS is a heuristic search algorithm, particularly suitable for problems with large state spaces. In this scheme, the implementation principle of MCTS includes four main steps: selection, expansion, simulation, and backtracking. The specific implementation is as follows: (1) Selection: Starting from the root node (the current target graph node), the most promising child node is recursively selected using the UpperConfidenceBound (UCB) formula until the leaf node is reached. The UCB formula is: UCB=X_j+C*sqrt(ln(n) / n_j), where X_j is the average return of node j, n is the number of visits to the parent node, n_j is the number of visits to node j, and C is the exploration parameter; (2) Expansion: If the selected leaf node is not in a terminated state and has not been fully expanded, an unexplored action is randomly selected to expand the tree; (3) Simulation: Starting from the newly expanded node, simulation is performed using a random strategy until the terminated state is reached. In power engineering projects, these simulations can be performed based on predefined rules or simplified project models; (4) Backtracking: The simulation results are backtracked along the selected path to update the statistics of each node. For example, suppose that at a certain power equipment node, MCTS considers three actions: "maintenance", "replacement of parts" and "continued operation". Through multiple simulations, it was found that the "maintenance" action can reduce the failure risk by an average of 60%, "replacement of parts" can reduce it by 80%, while "continued operation" may lead to a 20% increase in the failure rate. MCTS prioritizes the "replace parts" action because it offers the highest average reward. This approach effectively balances exploration and exploitation within a complex decision space, identifying the most likely paths to anomalies and improving the accuracy and comprehensiveness of anomaly prediction.

[0144] For any level of the project knowledge graph, the node update information of all target graph nodes in the project knowledge graph is aggregated, and the aggregated node update information is passed to all other levels of the project knowledge graph through a multi-head attention mechanism module. The implementation principle of this step is based on the multi-head attention mechanism, which allows the model to pay attention to different representation subspaces of information simultaneously. The specific implementation can be described as follows: (1) For each attention head, first perform a linear transformation on the input to obtain the query Q, key K and value V: Q = W_Q*X, K = W_K*X, V = W_V*X, where X is the input, and W_Q, W_K, and W_V are learnable weight matrices; (2) Calculate the attention score: Attention(Q,K,V) = softmax(QK^T / sqrt(d_k))V, where d_k is the dimension of the key; (3) Concatenate the outputs of multiple attention heads and perform a linear transformation: MultiHead(Q,K,V) = Concat(head_1,...,head_h)W_O, where h is the number of attention heads, and W_O is the output projection matrix; (4) For cross-layer information transmission, the graph representations of different levels can be used as the inputs of Q, K, and V to realize the interaction of information between layers. For example, suppose there are three levels of knowledge graphs: macroscopic, mesoscopic, and microscopic, and the graph representation of each level is a matrix. Through a multi-head attention mechanism, information at the macro level can selectively focus on important features at the meso and micro levels, thus capturing detailed information while maintaining high-level abstraction. Specifically, if there is a node representing "power grid stability" at the macro level, the multi-head attention mechanism might simultaneously focus on "substation operating status" at the meso level and "key equipment parameters" at the micro level, resulting in a more comprehensive understanding. The effectiveness of this method lies in enabling effective information exchange between knowledge graphs at different levels, improving the model's ability to understand complex systems, and enhancing the accuracy and interpretability of anomaly prediction.

[0145] After the final round of graph iteration traversal, multiple abnormal event chains are obtained in the multi-level project knowledge graph. The corresponding abnormal event result representations are extracted from each abnormal event chain, and all abnormal event result representations are integrated into an abnormal event result set for the power engineering project in the future time period. The implementation principle of this step is to identify and extract the key node sequence that may lead to abnormal events by analyzing the paths formed during the graph traversal. The specific implementation method can be described as follows: (1) Score each traversal path. The scoring criteria may include path length, node importance, edge weight, etc. The scoring function can be expressed as: Score=∑(w_i*f_i), where w_i is the weight of each factor, and f_i is the corresponding feature function; (2) Select the top-k paths as abnormal event chains according to the scores; (3) For each abnormal event chain, extract the key nodes (such as nodes with an abnormal probability exceeding the threshold) and their attributes to form an abnormal event result representation. The representation can be a vector containing information such as event type, probability of occurrence, and scope of influence; (4) Aggregate all abnormal event result representations into a set. It may also be necessary to remove duplicate or highly similar events.

[0146] In one implementation, combining available supervision resources and anomaly event result sets, the generation of the optimal supervision assignment strategy for a power engineering project over a future time period using a reinforcement learning optimization algorithm includes the following steps:

[0147] The state space is constructed by combining the result set of abnormal events and the available supervision resources;

[0148] Based on the available supervision resources, generate a variety of different supervision assignment strategies, and construct an action space based on all supervision assignment strategies;

[0149] A supervisory dispatch decision model is constructed by combining state space and action space;

[0150] With the goal of maximizing the utilization efficiency of supervision resources, the optimal supervision assignment strategy for power engineering projects in the future time period is output based on the supervision assignment decision model and the offline strategy algorithm.

[0151] In this implementation, the abnormal event result set first needs to be quantified and encoded. Each abnormal event can be represented by a vector, containing attributes such as event type, probability of occurrence, scope of impact, and expected occurrence time. Simultaneously, the available supervision resources also need to be quantified, which can include attributes such as the number of supervisors, their professional type, and experience level. For example, a supervision resource might be represented as [3,2,4], representing 3 electrical, 2 civil, and 4 equipment supervisors. The state space is constructed by combining these quantified abnormal events and supervision resource information into a multi-dimensional vector. Specifically, a tensor representation can be used, where one dimension represents time, another represents the abnormal event, and the third represents the supervision resource. For example, a state might be represented as S_t = [E_t, R_t], where E_t is the set of abnormal events at time t, and R_t is the set of available supervision resources at time t. To handle the high dimensionality of the state space, dimensionality reduction techniques such as Principal Component Analysis (PCA) or autoencoders can be used to reduce the dimensionality of the state representation while retaining key information.

[0152] Next, we need to define the basic units of the dispatch strategy, including the allocation of supervisors, work schedules, and task priorities. For example, a basic dispatch unit might be represented as [Supervisor ID, Task ID, Start Time, End Time, Priority]. Based on these basic units, combinatorial generation algorithms can be used to create various different dispatch strategies. One feasible approach is to use a genetic algorithm, encoding each dispatch strategy as a "chromosome" and generating new strategies through crossover and mutation operations. For example, assuming there are 5 supervisors and 10 potential abnormal events, a complete dispatch strategy might be a 5x10 matrix, where each element represents the probability or time at which a supervisor is assigned to a task. The fitness function of the genetic algorithm can be designed based on metrics such as resource utilization and task coverage. Finally, the action space can be represented as a high-dimensional discrete space, where each point represents a feasible dispatch strategy. To improve the efficiency of subsequent optimization, the action space can be preprocessed, such as through clustering or sparse representation, to reduce the size of the search space.

[0153] Next, we construct a supervisory dispatch decision model by combining the state space and action space. This decision model can be based on a Markov Decision Process (MDP), where the state space S represents all possible states of a project, the action space A represents all possible dispatch strategies, the transition function P(s'|s,a) describes the probability of transitioning to a new state s' after taking action a in the current state s, and the reward function R(s,a) defines the immediate reward obtained by taking action a in state s. In specific implementation, a deep neural network can be used to approximate the state-action value function Q(s,a), which is the long-term cumulative reward expectation of taking action a in state s. The network structure can adopt a two-stream architecture, with one stream processing state information and the other stream processing action information, and finally merging to produce a Q-value estimate. For example, the state stream can use a convolutional neural network (CNN) to process spatially relevant project information and a recurrent neural network (RNN) to process time-related anomaly prediction; the action stream can use fully connected layers to process discrete dispatch strategies.

[0154] Taking maximizing the efficiency of supervision resource utilization as the optimization objective, the final and most critical step of the entire scheme is to output the optimal supervision dispatch strategy for the power engineering project in the future time period based on the supervision dispatch decision model and the offline strategy algorithm for iterative optimization. The selection of the offline strategy algorithm is based on the actual situation, because in actual supervision work, direct online learning may bring risks and costs. Commonly used offline strategy algorithms include batch constrained deep Q learning (BCQ) and conservative Q learning (CQL). ​​Taking BCQ as an example, its core idea is to avoid over-extrapolation by limiting the range of action selection. The specific implementation steps are as follows: (1) First, collect historical supervision dispatch data, including state, action and reward. (2) Train a behavior cloning model G(s) to generate the action distribution under a given state. (3) Train a state-action value function Q(s,a) to evaluate the long-term value of the action. (4) During inference, select the action by the following formula: a*=argmax_a(Q(s,a)+λlog(G(a|s))), where λ is a balance parameter. This formula considers both the long-term value of actions and consistency with historical data. The optimization process is iterative, with each iteration including three stages: data collection, model update, and policy evaluation. To improve optimization efficiency, a priority experience replay technique can be used, prioritizing the learning of samples with high temporal difference errors. Simultaneously, to handle large-scale state and action spaces, function approximation techniques, such as deep neural networks, can be used to represent the Q-function. During optimization, special attention must be paid to avoiding overfitting and policy degradation. Regularization techniques and early stopping strategies can be used to enhance the model's generalization ability. The final output optimal supervisor assignment policy should be a dynamic decision function adaptable to different project states, rather than a static assignment table. For example, for a given project state s, the optimal policy π*(s) can output a probability distribution over the action space, representing the optimality of various assignment decisions in that state. In practical applications, this distribution can be used to sample or select the action with the highest probability. The effect of implementing this method is that it can generate an intelligent and adaptive supervision assignment strategy, which can not only maximize the utilization efficiency of supervision resources, but also flexibly respond to various uncertainties and changes in the project, thereby significantly improving the supervision quality and efficiency of power engineering projects.

[0155] The present invention also discloses an intelligent supervision and management platform based on multi-terminal interaction and data fusion, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the intelligent supervision and management method based on multi-terminal interaction and data fusion described in any of the above embodiments.

[0156] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it in this regard.

[0157] The memory can be an internal storage unit of a computer device, such as a hard disk or RAM, or an external storage device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) provided on the computer device. Furthermore, the memory can be a combination of internal storage units and external storage devices of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0158] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0159] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.

Claims

1. An intelligent supervision and management method based on multi-terminal interaction and data fusion, characterized in that, Includes the following steps: Obtain standardized project supervision documents and available supervision resources for power engineering projects; Deep learning-enhanced natural language processing technology is used to extract key entities of power projects from standardized project supervision documents and identify the entity relationships between key entities. Key entities of power projects include power project process entities, power project resource entities, and project process dependency entities. Entity relationships include entity dependency relationships and entity coreference relationships. The initial graph ontology model is designed based on multiple pre-defined hierarchical concepts; A tree structure of key entities for all power projects is constructed by combining entity dependency and entity coreference. Entity nodes in the entity tree structure represent key entities of power projects, and the parent-child relationship between entity nodes in the entity tree structure represents entity dependency. Entity nodes with entity coreference belong to the same level of the tree structure in the entity tree structure. Count the number of tree structure layers in the entity tree structure and the number of structural branches in each tree structure layer of the entity tree structure; Combining the number of tree structure layers and the number of structural branches, each layer of the entity tree structure is divided into macro level, meso level and micro level in the order from the root node to the leaf node, and node level tags with corresponding level concepts are added to all entity nodes. Traverse each parent-child relationship in the entity tree structure. If the parent node and child node connected by the parent-child relationship have the same node level label, then add a relationship level label with the same level concept as the parent node or child node to the parent-child relationship. If the parent and child nodes connected by the parent-child structure do not have the same node hierarchy label, then add a cross-level relationship label to the parent-child structure. For each layer of the graph model in the initial graph ontology model, target entity nodes and target parent-child structure relationships belonging to the same level of concepts as the graph model are selected based on node hierarchy labels and relationship hierarchy labels. Graph embedding technology is used to generate low-dimensional vector representations for each target entity node and target parent-child structure relationship in the graph model, thereby constructing the project knowledge graph corresponding to the graph model. Once the project knowledge graphs of all levels of concepts are constructed, the vertical connection relationships between project knowledge graphs of different levels are generated based on the parent-child structure relationships with cross-level relationship tags, resulting in a multi-level project knowledge graph for the power engineering project. Based on a multi-level project knowledge graph, a heterogeneous graph neural network inference engine for power engineering projects is constructed by combining a graph neural network model. Collect multi-dimensional project supervision data of power engineering projects in the current time period through various intelligent supervision terminals; Multi-dimensional project supervision data is integrated into multi-modal supervision data. An adaptive cross-modal attention mechanism is used to process the multi-modal supervision data to generate a representation of the causes of project events in the power engineering project in the current time period. Based on the project event cause representation, multi-level reasoning is performed in a heterogeneous graph neural network inference engine by improving the graph traversal algorithm, and the abnormal event result set caused by the project event cause in the preset future time period of the power engineering project is predicted. By combining available supervision resources and abnormal event result sets, a reinforcement learning optimization algorithm is used to generate the optimal supervision assignment strategy for power engineering projects in the future time period.

2. The intelligent supervision and management method based on multi-terminal interaction and data fusion according to claim 1, characterized in that, The process of extracting key entities of the power project from the standardized project supervision documents using deep learning-enhanced natural language processing technology and identifying the entity relationships between these key entities includes the following steps: The project supervision standardization documents were embedded using a pre-trained BERT model with context-sensitive word embeddings. Based on the word embedding results, a named entity recognition model combining bidirectional long short-term memory network and conditional random field is used to extract key entities of the power project from the project supervision standardization documents. The key entities of the power project include power project process entities, power project resource entities, and project process dependency entities. Based on the contextual relationships of the key entities in the power project in the project supervision standardization documents, the entity dependency relationships between different key entities in the power project are identified. Based on the key entities of the power project and using the coreference resolution model to traverse the text of the standardized project supervision documents in reverse, the reference relationships of all the key entities of the power project in the standardized project supervision documents are identified and parsed to obtain the entity coreference relationships between different key entities of the power project.

3. The intelligent supervision and management method based on multi-terminal interaction and data fusion according to claim 2, characterized in that, The method further includes the following steps: The knowledge of the pre-trained large language model is transferred to the BERT model, the named entity recognition model, the coreference resolution model, and the heterogeneous graph neural network inference engine using knowledge distillation technology.

4. The intelligent supervision and management method based on multi-terminal interaction and data fusion according to claim 1, characterized in that, The project supervision data includes project visual data, project audio data, and project multi-sensor data. The process of fusing this multi-dimensional project supervision data into multimodal supervision data, and then using an adaptive cross-modal attention mechanism to process the multimodal supervision data to generate a representation of the project event causes for the power engineering project in the current time period includes the following steps: The project visual features are extracted from the project visual data using a pre-trained convolutional neural network. The acoustic features of the project are extracted from the project's audio data by combining Mel frequency cepstral coefficients and short-time Fourier transform. A sensor fusion algorithm is used to fuse multi-sensor data of the project into comprehensive sensor data, and wavelet transform method is used to extract sensor data features from the comprehensive sensor data; For any one of the project's visual features, acoustic features, and sensor data features, a feature attention score is calculated between the target feature and the other two features based on an adaptive cross-modal attention mechanism. Based on the feature attention score, the project's visual features, acoustic features, and sensor data features are integrated into a comprehensive feature representation of the power engineering project in the current time period; By mapping the comprehensive feature representation to a fixed-dimensional vector space using a multilayer perceptron, the cause representation of the project events of the power engineering project in the current time period is obtained.

5. The intelligent supervision and management method based on multi-terminal interaction and data fusion according to claim 1, characterized in that, The heterogeneous graph neural network inference engine includes a heterogeneous graph attention module, a multi-head attention mechanism module, a gated loop module, a skip connection structure, and a batch normalization module. The heterogeneous graph attention module is used to process the graph nodes and graph node edges of different levels of the project knowledge graph in the multi-level project knowledge graph. The multi-head attention mechanism module is used to capture the information transmission between different levels of project knowledge graphs. The loss function of the heterogeneous graph neural network inference engine is constructed based on a contrastive learning strategy.

6. The intelligent supervision and management method based on multi-terminal interaction and data fusion according to claim 5, characterized in that, The step of predicting the set of abnormal events caused by the project event causes in the power engineering project within a preset future time period, based on the project event cause representation and through multi-level reasoning in the heterogeneous graph neural network inference engine using an improved graph traversal algorithm, includes the following steps: Based on the project event cause representation, the reasoning starting node in the multi-level project knowledge graph is determined, and the project event cause representation is replaced with the node feature vector of the reasoning starting node; The graph is iteratively traversed from the inference starting node using the heterogeneous graph neural network inference engine and a depth-first search algorithm. The depth-first search algorithm incorporates heuristic rules based on knowledge of the power engineering project domain. The number of iterations of the graph iterative traversal is set according to the time interval between a preset future time period and the current time period. During the graph iterative traversal, for each target graph node that is traversed and visited, the heterogeneous graph attention module aggregates the neighbor node information of all neighbor nodes of the target graph node in the same level of the project knowledge graph, and dynamically updates the node features and all node edge features of the target graph node based on the neighbor node information. Once any of the target graph nodes has completed information aggregation and feature dynamic update, the Monte Carlo tree search method is used to select the next graph node to be visited in the project knowledge graph at the level where the target graph node is located. For any level of project knowledge graph, the node update information of all target graph nodes in the project knowledge graph is aggregated, and the aggregated node update information is transmitted to all other levels of project knowledge graph through the multi-head attention mechanism module. After the final round of graph iteration traversal, multiple abnormal event chains are obtained in the multi-level project knowledge graph. The corresponding abnormal event result representations are extracted from each of the abnormal event chains, and all the abnormal event result representations are integrated into the abnormal event result set of the power engineering project in the future time period.

7. The intelligent supervision and management method based on multi-terminal interaction and data fusion according to claim 6, characterized in that, The step of combining the available supervision resources and the abnormal event result set to generate the optimal supervision assignment strategy for the power engineering project in the future time period using a reinforcement learning optimization algorithm includes the following steps: A state space is constructed by combining the abnormal event result set and the available supervision resources; Based on the available supervision resources, a variety of different supervision assignment strategies are generated, and an action space is constructed using all the supervision assignment strategies. A supervisory dispatch decision model is constructed by combining the state space and the action space; With maximizing the efficiency of supervision resource utilization as the optimization objective, the optimal supervision dispatch strategy for the power engineering project in the future time period is output based on the supervision dispatch decision model and an offline strategy algorithm for iterative optimization.

8. An intelligent supervision and management platform based on multi-terminal interaction and data fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the intelligent supervision and management method based on multi-terminal interaction and data fusion as described in any one of claims 1 to 7.

Images