Intelligent supervision management method and platform based on multi-terminal interaction and data fusion

Abstract

The invention provides an intelligent supervision management method and platform based on multi-terminal interaction and data fusion, and the method comprises the following steps: obtaining a supervision standardized file and a to-be-used supervision resource of an electric power engineering project, extracting a key entity through a natural language processing technology, and recognizing an entity relationship; based on the entities and the incidence relation thereof, a multi-level project knowledge graph is constructed, and a heterogeneous graph neural network inference engine is formed in combination with a graph neural network model. Multi-dimensional supervision data are collected through an intelligent supervision terminal, the data are processed by using a self-adaptive cross-modal attention mechanism, and a project event cause representation is generated. Based on the representation, multi-level reasoning is carried out in a reasoning engine through an improved graph traversal algorithm, and an abnormal event result set caused by project event causes in a future time period is predicted. And in combination with the to-be-used supervision resources and the abnormal event result set, adopting a reinforcement learning optimization algorithm to generate an optimal supervision dispatching strategy. According to the invention, efficient supervision and resource optimization configuration can be realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of supervision data management, and specifically relates to an intelligent supervision management method and platform based on multi-terminal interaction and data fusion. Background Art

[0002] Power engineering projects are usually large-scale, complex, and long-term, involving a large number of processes, resources, and dependencies. For example, a large substation construction project may include multiple stages such as civil construction, equipment installation, and commissioning. Each stage contains multiple sub-processes, and there are complex dependencies between these processes. In addition, power engineering projects also require the coordination of multiple resources, including manpower, equipment, materials, etc. The allocation and use of these resources directly affect the progress and quality of the project.

[0003] With the rapid development of intelligent technology, especially the advancement of deep learning and natural language processing technology, intelligent supervision and management methods have emerged. In traditional intelligent supervision and management methods, it mainly relies on manual experience and simple rule processing. This method is difficult to fully capture the key information in the project when facing complex power engineering projects. For example, the complex dependencies between processes, the dynamic allocation requirements of resources, and the changes in environmental factors cannot be effectively handled by simple rules. Secondly, traditional technologies lack the ability to deeply analyze and integrate multi-dimensional data. Power engineering projects involve a variety of data types, including process progress, resource usage, environmental monitoring data, etc. There are complex correlations between these data, which makes it difficult for traditional technologies to fully explore the potential information in these data when facing complex engineering projects, and ultimately leads to the generated automatic dispatching strategy, which is prone to waste resources and poor efficiency. Summary of the invention

[0004] The present invention provides an intelligent supervision management method and platform based on multi-terminal interaction and data fusion to solve the problem that the automatic dispatching strategy generated by the existing intelligent supervision management method is prone to waste resources and has low efficiency.

[0005] In a first aspect, the present invention provides an intelligent supervision management method based on multi-terminal interaction and data fusion, the method comprising the following steps:

[0006] Obtain standardized project supervision documents and available supervision resources for power engineering projects;

[0007] Extracting key entities of the power project from the standardized project supervision documents using natural language processing technology enhanced by deep learning, and identifying entity association relationships between the key entities of the power project, wherein the key entities of the power project include power project process entities, power project resource entities, and project process dependency entities, and the entity association relationships include entity dependency relationships and entity co-reference relationships;

[0008] Constructing a multi-level project knowledge graph of the power engineering project based on the key entities of the power project and according to the entity association relationships;

[0009] Based on the multi-level project knowledge graph and in combination with the graph neural network model, a heterogeneous graph neural network inference engine for the power engineering project is constructed;

[0010] Collect multi-dimensional project supervision data of the power engineering project in the current time period through a variety of intelligent supervision terminals;

[0011] The multi-dimensional project supervision data is integrated into multi-modal supervision data, and the multi-modal supervision data is processed using an adaptive cross-modal attention mechanism to generate a project event cause representation of the power engineering project in the current time period;

[0012] Based on the project event cause representation, multi-level reasoning is performed in the heterogeneous graph neural network inference engine by improving the graph traversal algorithm to predict a set of abnormal event results caused by the project event cause in a preset future time period of the power engineering project;

[0013] In combination with the standby supervision resources and the abnormal event result set, a reinforcement learning optimization algorithm is used to generate an optimal supervision dispatching strategy for the power engineering project in the future time period.

[0014] Optionally, the extracting key entities of the power project from the project supervision standardization document using the deep learning enhanced natural language processing technology and identifying the entity association relationship between the key entities of the power project comprises the following steps:

[0015] Use the pre-trained BERT model to perform context-sensitive word embedding on the project supervision standardization document;

[0016] Based on the word embedding results, a named entity recognition model combining a bidirectional long short-term memory network and a conditional random field is used to extract key entities of the power project from the project supervision standardization document. The key entities of the power project include power project process entities, power project resource entities and project process dependency entities.

[0017] According to the contextual association relationship of the key entities of the power project in the project supervision standardization document, identifying the entity dependency relationship between different key entities of the power project;

[0018] Based on the key entities of the power project and using the co-reference resolution model, the file text of the project supervision standardization document is reversely traversed, the reference relationship of all the key entities of the power project in the project supervision standardization document is identified and parsed, and the entity co-reference relationship between different key entities of the power project is obtained.

[0019] Optionally, the method further comprises the following steps:

[0020] The knowledge distillation technology is used to migrate the model knowledge of the pre-trained large language model to the BERT model, the named entity recognition model, the coreference resolution model and the heterogeneous graph neural network inference engine respectively.

[0021] Optionally, constructing a multi-level project knowledge graph of the power engineering project based on the key entities of the power project and according to the entity association relationships comprises the following steps:

[0022] Designing an initial graph ontology model according to a plurality of preset hierarchical concepts, wherein the hierarchical concepts include a macro level, a meso level, and a micro level;

[0023] Combining the entity dependency relationship and the entity co-referencing relationship, constructing an entity tree structure of all the key entities of the power project, the entity nodes in the entity tree structure represent the key entities of the power project, the parent-child structure relationship between the entity nodes in the entity tree structure represents the entity dependency relationship, and the entity nodes with the entity co-referencing relationship belong to the same level of tree structure in the entity tree structure;

[0024] Counting the number of tree structure layers of the entity tree structure and the number of structural branches of each tree structure layer in the entity tree structure;

[0025] The entity tree structure is split into different hierarchical concepts in combination with the number of tree structure layers and the number of structure branches, and the entity tree structure is mapped to the initial graph ontology model based on the hierarchical concepts to form a multi-level project knowledge graph of the power engineering project.

[0026] Optionally, the step of splitting the entity tree structure into different hierarchical concepts by combining the number of tree structure layers and the number of structure branches, and mapping the entity tree structure to the initial graph ontology model based on the hierarchical concepts to form a multi-level project knowledge graph of the power engineering project includes the following steps:

[0027] In combination with the number of tree structure layers and the number of structure branches, each layer of the tree structure in the entity tree structure is divided into the macro level, the meso level and the micro level in order from the root node to the leaf node, and node level tags corresponding to the level concepts are added to all the entity nodes;

[0028] Traversing each pair of the parent-child structural relationships in the entity tree structure, if the parent node and the child node connected by the parent-child structural relationship have the same node hierarchy mark, adding a relationship hierarchy mark of the same hierarchy concept as the parent node or the child node to the parent-child structural relationship;

[0029] If the parent node and the child node connected by the parent-child structural relationship do not have the same node hierarchy mark, adding a cross-layer relationship mark to the parent-child structural relationship;

[0030] For each layer of the graph model in the initial graph ontology model, based on the node hierarchy labels and the relationship hierarchy labels, the target entity nodes and target parent-child structural relationships belonging to the same hierarchical concept as the graph model are screened out, and a low-dimensional vector representation is generated in the graph model for each target entity node and the target parent-child structural relationship using graph embedding technology, so as to construct a project knowledge graph corresponding to the graph model;

[0031] When the project knowledge graph to which all the hierarchical concepts belong is constructed, a graph vertical connection relationship between project knowledge graphs at different levels is generated based on the parent-child structural relationship with the cross-layer relationship mark to obtain a multi-level project knowledge graph for the power engineering project.

[0032] Optionally, the project supervision data includes project visual data, project audio data and project multi-sensor data, and the multi-dimensional project supervision data is fused into multi-modal supervision data, and the multi-modal supervision data is processed using an adaptive cross-modal attention mechanism to generate a project event cause representation of the power engineering project in the current time period, including the following steps:

[0033] Extracting project visual features from the project visual data using a pre-trained convolutional neural network;

[0034] Extracting project acoustic features from the project audio data by combining Mel-frequency cepstral coefficients and short-time Fourier transform;

[0035] A sensor fusion algorithm is used to fuse the multi-sensor data of the project into sensor comprehensive data, and a sensor data feature is extracted from the sensor comprehensive data by a wavelet transform method;

[0036] For any target feature among the project visual feature, the project acoustic feature, and the sensor data feature, calculating a feature attention score between the target feature and the other two features based on an adaptive cross-modal attention mechanism;

[0037] Integrating the project visual features, the project acoustic features and the sensor data features into a comprehensive feature representation of the power engineering project in the current time period based on the feature attention score;

[0038] The comprehensive feature representation is mapped to a vector space of fixed dimension through a multi-layer perceptron to obtain a project event cause representation of the power engineering project in the current time period.

[0039] Optionally, the heterogeneous graph neural network inference engine includes a heterogeneous graph attention module, a multi-head attention mechanism module, a gated loop module, a skip connection structure and a batch normalization module. The heterogeneous graph attention module is used to process graph nodes and graph node edges of project knowledge graphs at different levels in the multi-level project knowledge graph. The multi-head attention mechanism module is used to capture information transfer between project knowledge graphs at different levels. The loss function of the heterogeneous graph neural network inference engine is constructed based on a contrastive learning strategy.

[0040] The method of performing multi-level reasoning in the heterogeneous graph neural network inference engine by improving the graph traversal algorithm based on the project event cause representation to predict the abnormal event result set caused by the project event cause in the preset future time period of the power engineering project includes the following steps:

[0041] Determine the inference starting node in the multi-level project knowledge graph based on the project event cause representation, and replace the project event cause representation with a node feature vector of the inference starting node;

[0042] The graph is iteratively traversed starting from the inference starting node by using the heterogeneous graph neural network inference engine and a depth-first search algorithm, wherein the depth-first search algorithm incorporates heuristic rules based on the domain knowledge of power engineering projects, and the number of iterations of the graph iterative traversal is set according to the time interval between a preset future time period and the current time period;

[0043] During the iterative traversal of the graph, for each target graph node visited by the traversal, the neighbor node information of all neighbor nodes of the target graph node in the project knowledge graph at the same level is aggregated through the heterogeneous graph attention module, and the node features of the target graph node and all node edge features of the target graph node are dynamically updated according to the neighbor node information;

[0044] When any of the target graph nodes completes information aggregation and dynamic feature update, a Monte Carlo tree search method is used to select the next graph node to be accessed in the project knowledge graph at the level where the target graph node is located;

[0045] For any level of the project knowledge graph, aggregate the node update information of all the target graph nodes in the project knowledge graph, and transmit the aggregated node update information to all other levels of the project knowledge graph through the multi-head attention mechanism module;

[0046] After the last round of graph iteration traversal is completed, multiple abnormal event chains in the multi-level project knowledge graph are obtained, and the corresponding abnormal event result representation is extracted from each of the abnormal event chains, and all the abnormal event result representations are integrated into an abnormal event result set for the power engineering project in the future time period.

[0047] Optionally, the combining the standby supervision resources and the abnormal event result set to generate the optimal supervision dispatching strategy for the power engineering project in the future time period using a reinforcement learning optimization algorithm comprises the following steps:

[0048] Constructing a state space by combining the abnormal event result set and the standby supervision resources;

[0049] generating a plurality of different supervisory work assignment strategies according to the standby supervisory resources, and constructing an action space with all the supervisory work assignment strategies;

[0050] Combining the state space and the action space to construct a supervisory work assignment decision model;

[0051] Taking maximizing the utilization efficiency of supervision resources as the optimization goal, based on the supervision dispatch decision model and using an offline strategy algorithm, iterative optimization is performed to output the optimal supervision dispatch strategy for the power engineering project in the future time period.

[0052] In the second aspect, the present invention also provides an intelligent supervision and management platform based on multi-terminal interaction and data fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program, it implements the intelligent supervision and management method based on multi-terminal interaction and data fusion as described in the first aspect.

[0053] The beneficial effects of the present invention are:

[0054] The present invention obtains the supervision standardization documents and standby supervision resources of the power engineering project, and uses natural language processing technology to extract the key entities of the power project and their associations, which can more comprehensively and accurately identify the processes, resources and dependencies in the project, and solves the problem of incomplete and inaccurate information extraction caused by the traditional method due to reliance on artificial experience and simple rules, thereby providing a more reliable data basis for subsequent supervision management. Secondly, the present invention constructs a multi-level project knowledge graph based on the extracted key entities and their associations, and combines the graph neural network model to form a heterogeneous graph neural network reasoning engine, which can more deeply mine the complex relationships and multi-level structures in the project, overcome the defect that the traditional method is difficult to reflect the complexity of the project due to limited knowledge representation ability, and provide more comprehensive knowledge support for supervision management. In addition, the present invention collects multi-dimensional project supervision data through a variety of intelligent supervision terminals, and merges it into multimodal data, and uses an adaptive cross-modal attention mechanism to process data, which can more efficiently integrate and analyze multi-source heterogeneous data, and solves the problem that the traditional method cannot make full use of multi-dimensional information due to insufficient data processing capabilities, thereby improving the depth and breadth of data analysis. Furthermore, the present invention improves the graph traversal algorithm to perform multi-level reasoning in the reasoning engine, which can more accurately predict the abnormal event result set that may be caused by the cause of the project event in the future time period, overcome the deficiency of the traditional method that is difficult to predict potential risks due to limited reasoning ability, and provide a more scientific decision-making basis for supervision management. Finally, the present invention combines the standby supervision resources and the abnormal event result set, and uses the reinforcement learning optimization algorithm to generate the optimal supervision dispatching strategy, which can allocate resources more reasonably, reduce resource waste, and improve supervision efficiency, solving the problem of unreasonable resource allocation and low efficiency caused by insufficient optimization ability of the traditional method. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flow chart of an intelligent supervision management method based on multi-terminal interaction and data fusion in one of the implementation modes of the present application.

[0056] Figure 2 A schematic diagram of the relationship between the entity tree structure and the initial graph ontology model in one embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0058] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0059] Figure 1 FIG. 1 is a flow chart of an intelligent supervision management method based on multi-terminal interaction and data fusion in one embodiment. It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the above method may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps. Figure 1 As shown, the intelligent supervision management method based on multi-terminal interaction and data fusion disclosed in the present invention specifically includes the following steps:

[0060] S101. Obtain standardized project supervision documents and standby supervision resources for power engineering projects.

[0061] Among them, in the initial stage of the power engineering project, a distributed document management system is first established to store and manage various standardized documents. These documents include but are not limited to national and industry standards such as "Power Construction Engineering Construction Supervision Specifications", "Power Engineering Construction Quality Acceptance and Assessment Regulations", "Technical Specifications for Construction and Acceptance of Overhead Transmission Lines of Transmission and Transformation Engineering", as well as project-specific technical specifications and management systems. The document management system supports multiple formats, such as PDF, Word, CAD drawings, etc., and uses OCR technology to process scanned files and extract text information. For drawings, computer vision technology is used to identify graphic elements and extract key information. File storage uses distributed storage technology to ensure high availability and fast access. Each file generates a unique digital fingerprint for version control and integrity verification. The system integrates a semantic search engine and supports intelligent retrieval based on natural language. For example, by entering "substation grounding requirements", relevant clauses can be quickly located. At the same time, the system automatically extracts key information of the file, generates summaries and tags, and establishes an associated index between files. For example, the construction specifications are associated with the corresponding quality acceptance standards to facilitate subsequent cross-references and consistency checks.

[0062] For the supervision resources to be used, a real-time updated resource management database is established. This database contains all available human resources (such as supervision engineers, professional and technical personnel) and material resources (such as testing equipment, safety equipment). Each resource item has a detailed attribute description, such as the professional qualifications, work experience, current working status of the supervision personnel, the model, performance parameters, and usage records of the equipment. The resource management system adopts a dynamic update mechanism to reflect the availability and status changes of resources in real time. For example, when a supervision engineer is assigned to a project, the system automatically updates its status to ensure the accuracy of resource allocation. In addition, the system also contains resource scheduling history records for analyzing resource utilization efficiency and optimizing future resource allocation. Through this comprehensive and intelligent file and resource management method, not only a solid data foundation is laid for subsequent deep learning and natural language processing, but also the standardization, comprehensiveness and efficient use of resources of project supervision work are ensured. The implementation of this step can significantly improve the efficiency of the initial stage of the project, reduce information retrieval time, avoid resource conflicts, and provide strong support for the smooth progress of the entire project.

[0063] S102. Use deep learning enhanced natural language processing technology to extract key entities of power projects from project supervision standardization documents and identify entity association relationships between key entities of power projects.

[0064] Among them, in this step, the key entities of the power project include the power project process entity, the power project resource entity and the project process dependency entity, and the entity association relationship includes entity dependency and entity co-reference relationship. Specifically, a deep learning model based on BERT (Bidirectional Encoder Representations from Transformers) is used to process the supervision standardization documents of power engineering projects. First, the BERT model fine-tuned for the power field is used to encode the text and generate context-related word embeddings. For example, for a phrase such as "installing high-voltage cables", the model can understand the specific meaning of "high voltage" in the power context. Then, the named entity recognition (NER) technology is used to identify the key entities in the text, such as process names (such as "transformer installation"), resource types (such as "500kV transformer"), etc. The NER model adopts the BiLSTM-CRF (bidirectional long short-term memory network and conditional random field) structure, which can effectively capture long-distance dependencies. For the identified entities, the dependency between them is extracted using a relation extraction model. Here, the graph attention network (GAT) is used for relation extraction, such as identifying the dependency that "transformer installation" must be carried out after "basic engineering is completed". In order to handle the professional terms and abbreviations unique to power engineering, domain-specific dictionaries and rules are introduced to enhance the processing effect. For example, "GIS" is correctly parsed as "gas insulated switchgear". In addition, image processing technology is used to analyze engineering drawings, extract spatial information such as equipment layout and line direction, and merge them with text information. Finally, knowledge graph completion technology is used to infer potential implicit relationships based on the extracted information. Through this series of deep learning-enhanced natural language processing technologies, process information (such as "substation main transformer installation", "overhead line erection", etc.), resource information (such as "500kV transformer", "insulator", etc.) of power engineering projects and their dependencies (such as "cable laying" must be after "cable trench construction") can be accurately extracted from complex supervision documents, laying the foundation for subsequent knowledge graph construction.

[0065] S103. Construct a multi-level project knowledge graph for power engineering projects based on the key entities of power projects and according to the entity association relationships.

[0066] Among them, a hierarchical ontology model is designed to define entity types and relationship types at the macro, meso and micro levels. The macro level includes top-level project categories such as "transmission network construction" and "substation construction"; the meso level includes specific sub-projects such as "500kV substation main transformer installation" and "220kV transmission line erection"; the micro level is refined to specific process steps such as "transformer foundation pouring" and "insulator installation". Then, a hierarchical clustering algorithm (such as AGNES) is used to hierarchically process the entities and relationships extracted from S2. For example, "transformer installation" is classified into the meso level, while "transformer oil tank sealing inspection" is classified into the micro level. During the construction process, special attention is paid to the particularity of power engineering, such as considering factors such as voltage level and power equipment type for classification. The inter-layer relationship is established through edges such as "composition" and "refinement". For example, the "substation main transformer installation" at the meso level is connected to multiple specific steps at the micro level through the "refinement" edge. In order to handle the complex dependencies in power engineering, temporal logic reasoning is introduced to ensure that constraints such as "high-voltage equipment inflation" must be carried out after "vacuum test is qualified" are correctly represented. At the same time, graph embedding technology (such as TransE) is used to generate low-dimensional dense vector representations for each node and edge, which is convenient for subsequent similarity calculation and reasoning. In addition, a knowledge graph completion algorithm is introduced to automatically infer and add potential relationships based on knowledge in the power engineering field, such as automatically inferring the required number of main transformers based on the substation capacity. Finally, a graph database (such as Neo4j) is used to store the constructed knowledge graph to support efficient graph query and update operations. Through this multi-level knowledge graph, not only can the complex structure of power engineering projects be fully represented, but also project analysis and decision-making from different granularities can be supported, laying the foundation for subsequent intelligent supervision management.

[0067] S104. Based on the multi-level project knowledge graph and combined with the graph neural network model, a heterogeneous graph neural network inference engine for power engineering projects is constructed.

[0068] Among them, a heterogeneous graph attention module is first designed to handle different types of nodes (such as substation equipment, line components, human resources, etc.) and edges (such as dependencies, temporal relationships, spatial relationships, etc.) in power engineering. For each type of edge r, a transformation matrix W_r and an attention vector a_r are defined. For example, for edges of the "dependency" type, a higher attention weight is given because it directly affects the order of the process. The attention calculation formula is: α_ij^r=softmax(LeakyReLU(a_r^T[W_rh_i||W_rh_j])), where h_i and h_j are the feature vectors of nodes i and j, respectively. Next, a multi-head attention mechanism module is constructed to capture the information transfer between project knowledge graphs at different levels, and the node feature update formula is: h_i'=||_k=1^Kσ(Σ_j∈N_i^rα_ij^(r,k)W_r^kh_j). In order to handle long-term dependencies in power engineering (such as the entire process from project launch to final grid connection), a gated recurrent module (GRU) is introduced. The update process of GRU takes into account the historical information of the nodes and is particularly suitable for processing long-term processes such as installation, commissioning, and operation of power equipment. In order to alleviate the problem of gradient vanishing in deep networks, a jump connection structure is designed to ensure that important features of the bottom layer can be directly passed to the upper layer. Considering the complexity of power engineering data, batch normalization technology is used to improve the training stability of the model. Finally, the model's ability to represent graph structures, especially the understanding of the topological relationship between key components in the power system, is enhanced through contrastive learning strategies. Through this heterogeneous graph neural network inference engine, it is possible to effectively handle complex relationships and dependencies in power engineering projects, providing strong support for subsequent intelligent decision-making and risk prediction.

[0069] S105. Collect multi-dimensional project supervision data in the power engineering project in the current time period through various intelligent supervision terminals.

[0070] Among them, a variety of intelligent supervision terminals include high-definition camera systems, audio acquisition systems and multi-dimensional sensor networks. The high-definition camera system includes fixed cameras and movable drone cameras. Fixed cameras are installed in key locations, such as the main transformer area of ​​the substation, the GIS equipment area, etc., to provide 24-hour uninterrupted monitoring. Drone cameras are used to inspect the construction of transmission lines and towers. These cameras use 4K resolution, a frame rate of 60fps, and are equipped with infrared night vision to ensure that clear images can be captured under various lighting conditions. The all-round audio acquisition system includes a high-sensitivity microphone array installed in key areas such as substations, with a sampling rate of 48kHz and 24-bit quantization, which can capture sounds from low-frequency transformer hum to high-frequency corona discharge. These microphones use beamforming technology to accurately locate the sound source, which helps to detect abnormal equipment in a timely manner.

[0071] The multidimensional sensor network includes electrical parameter sensors (such as voltage, current, and power factor monitors), environmental parameter sensors (such as temperature, humidity, and wind speed sensors), and structural health monitoring sensors (such as strain gauges and vibration sensors). Partial discharge sensors and infrared thermal imagers are installed around high-voltage equipment, especially for early fault detection. These sensors are designed with industrial grade, have strong anti-interference ability, and can achieve a sampling frequency of up to 1kHz. All collected data are transmitted to the central processing unit in real time through the 5G network to ensure the real-time and integrity of the data. In addition, edge computing technology is introduced to perform preliminary data processing and analysis locally to reduce the transmission bandwidth requirements. For example, a lightweight target detection algorithm can be run on the edge device, and only the detected abnormal events are transmitted. To ensure data security, end-to-end encryption technology is used, and strict access control is implemented. Through this all-round, multi-modal data acquisition system, all aspects of the power engineering project can be monitored in real time and comprehensively, providing a rich and reliable data foundation for subsequent intelligent analysis and decision-making.

[0072] S106. The multi-dimensional project supervision data is integrated into multi-modal supervision data, and the multi-modal supervision data is processed using an adaptive cross-modal attention mechanism to generate a representation of the cause of project events for the power engineering project in the current time period.

[0073] Among them, in power engineering projects, multi-dimensional project supervision data includes visual data (such as high-definition images and videos), audio data (such as equipment operation sound) and various sensor data (such as temperature, humidity, vibration, etc.). First, these heterogeneous data are preprocessed and feature extracted. For visual data, a pre-trained convolutional neural network (such as ResNet-50) is used to extract visual features; for audio data, Mel frequency cepstral coefficients (MFCC) and short-time Fourier transform (STFT) are used to extract acoustic features; for sensor data, wavelet transform is used to extract time-frequency features. Next, an adaptive cross-modal attention mechanism is designed to dynamically adjust the weights between different modalities. Specifically, for each modality m, its attention score with other modalities n is calculated. This mechanism allows the model to dynamically adjust the attention to different modal information according to the current situation. For example, when detecting transformer anomalies, more attention may be paid to audio and thermal imaging data; while when monitoring the construction of transmission lines, more emphasis may be placed on visual data. Finally, a multi-layer perceptron is used to map the fused features to a fixed-dimensional vector space, which is the "project event cause representation".

[0074] For example, for an ongoing transformer installation process, the project event cause representation may be a 300-dimensional vector, in which different dimensions encode information such as equipment status, environmental conditions, and operating steps. Specifically, the first 100 dimensions of the vector may represent visual information (such as whether the equipment is in the correct position and whether the worker's operation is standardized), the middle 100 dimensions represent audio information (such as whether the equipment is running normally and whether the alarm sound exists), and the last 100 dimensions represent sensor data (such as temperature, humidity, vibration intensity, etc.). Each element value of this vector is between 0 and 1, indicating the possibility of a certain state or event. In this way, complex multimodal information is compressed into a unified and easy-to-process representation. This representation not only captures the key information of the current time period, but also retains the relationship between different modes, providing a strong foundation for subsequent event reasoning and prediction. The implementation of this method can greatly improve the ability to understand complex power engineering environments and achieve early identification and early warning of potential problems.

[0075] S107. Based on the representation of the cause of the project event, multi-level reasoning is performed in the heterogeneous graph neural network inference engine through the improved graph traversal algorithm to predict the result set of abnormal events caused by the cause of the project event in the preset future time period of the power engineering project.

[0076] Among them, the project event cause representation is used as the feature vector of the starting node. For example, if the event cause representation shows that the transformer temperature is abnormally high, this will become the starting point of reasoning. Then, the improved depth-first search (DFS) algorithm is used for graph traversal. The traditional DFS algorithm is modified to adapt to the particularity of power engineering, and heuristic rules based on domain knowledge are added. The heuristic function guides the algorithm to preferentially explore paths that are more likely to lead to serious consequences. During the traversal process, the characteristics of nodes and edges are dynamically updated using a heterogeneous graph neural network inference engine. For each visited node, its state is updated by aggregating the information of its neighboring nodes. This dynamic update mechanism enables the reasoning process to adapt to complex power system state changes. In order to deal with the uncertainty in power engineering, Monte Carlo Tree Search (MCTS) is introduced. Each time the next node to be explored is selected, multiple simulations are performed to evaluate the possible results, and then the most promising path is selected. This helps to find the most likely abnormal event chain in a large-scale state space.

[0077] During the multi-level reasoning process, the algorithm jumps between different levels of the knowledge graph. For example, the "abnormal transformer temperature" at the micro level may rise to the "abnormal substation operation" at the meso level, and then to the "power grid stability threat" at the macro level. This multi-level reasoning can comprehensively evaluate the potential impact of the event. Finally, the abnormal event result set is generated based on the traversal results. This result set is a probability distribution that represents the probability of occurrence of different abnormal events within a preset future time period. For example, {("transformer overheating", 0.7, "within 24 hours"), ("insulation failure", 0.4, "within 48 hours"), ("local power outage", 0.2, "within 72 hours")}, where each tuple contains the abnormal event type, probability of occurrence, and expected time of occurrence. Through this improved graph traversal algorithm and multi-level reasoning, the system can comprehensively analyze various abnormal situations that may be caused by the cause of the project event, and provide predictions in the time dimension, providing an important basis for subsequent risk management and decision-making. The implementation of this method can significantly improve the risk prediction ability of power engineering projects and achieve early warning and timely intervention of potential problems.

[0078] S108. Combining the standby supervision resources and the abnormal event result set, a reinforcement learning optimization algorithm is used to generate the optimal supervision dispatching strategy for the power engineering project in the future time period.

[0079] Among them, a Markov decision process (MDP) model considering multiple objectives can be constructed. The state space S includes the current resource status (such as available supervisors and equipment), predicted abnormal events (type, probability, expected time of occurrence), and project progress. The action space A includes different dispatching decisions, such as assigning specific supervisors to specific work areas, dispatching detection equipment, etc. The reward function R comprehensively considers multiple objectives, including risk minimization, resource utilization efficiency, project progress, etc. The Soft Actor-Critic (SAC) algorithm is used as the core of reinforcement learning. The SAC algorithm strikes a balance between exploration and exploitation by maximizing the weighted sum of expected returns and policy entropy. The policy network π(a|s) uses Gaussian distribution to output action probabilities, and the value function Q(s,a) evaluates the value of the state-action pair. Through this reinforcement learning optimization algorithm, the system is able to generate an adaptable and globally optimal supervisor dispatching strategy. Specifically, the generated strategies include the following: 1) According to the predicted "transformer overheating" event, 3 additional supervisory engineers with transformer professional background will be dispatched to the relevant substations in the next 24 hours, and 2 infrared thermal imagers will be deployed for continuous monitoring. 2) In view of the possible "insulation failure" problem, 5 insulation resistance tests will be arranged in the next 48 hours, with the test time at the 12th, 24th, 36th, 42nd and 47th hours, each test lasting 2 hours, and supervisors with high-voltage equipment inspection qualifications will be assigned to perform. 3) Considering the potential risk of "local power outage", a comprehensive line inspection will be carried out every 8 hours in the next 72 hours, focusing on the weak links that may cause power outages, such as aging insulators, loose wire clamps, etc. 4) Adjust the original routine inspection plan, reduce the inspection frequency of non-critical areas by 50%, and reallocate the saved human resources to high-risk areas. 5) During the predicted high-risk period (such as the 48th-60th hour), arrange 24-hour uninterrupted monitoring and prepare an emergency response team to ensure that immediate action can be taken when any abnormal situation occurs.

[0080] This dispatching strategy not only solves potential problems in a targeted manner, but also optimizes resource allocation and ensures quality control at critical time points. Through dynamic adjustment and real-time optimization, the system can continuously adjust strategies according to environmental changes and new forecast results to maintain the efficiency and pertinence of supervision work. The implementation of this method can significantly improve the management efficiency and safety of power engineering projects, reduce human errors, optimize resource utilization, and improve the ability to respond to emergencies. Ultimately, it will help reduce project risks, improve project quality, and ensure the smooth completion and long-term stable operation of power engineering projects.

[0081] In one embodiment, extracting key entities of a power project from a standardized project supervision document using natural language processing technology enhanced by deep learning and identifying entity association relationships between key entities of the power project includes the following steps:

[0082] Use the pre-trained BERT model to perform context-sensitive word embedding on project supervision standardization documents;

[0083] Based on the word embedding results, the named entity recognition model combining bidirectional long short-term memory network and conditional random field is used to extract the key entities of power projects from the project supervision standardization documents. The key entities of power projects include power project process entities, power project resource entities and project process dependency entities.

[0084] According to the contextual association relationship of key entities of power projects in the standardized documents of project supervision, the entity dependency relationship between key entities of different power projects is identified;

[0085] Based on the key entities of power projects and using the co-reference resolution model, the document text of project supervision standardization documents is reversely traversed to identify and parse the reference relationship of all key entities of power projects in the project supervision standardization documents, and the entity co-reference relationship between different key entities of power projects is obtained.

[0086] In this embodiment, the pre-trained BERT model is used to perform context-related word embedding on the project supervision standardization document. BERT, namely Bidirectional Encoder Representations from Transformers, is a deep learning model based on the Transformer architecture. It can capture the deep semantic information of words in the text by pre-training on large-scale text data. In this step, the BERT model first receives the text input of the project supervision standardization document, which usually contains various specifications, processes and standards in the power engineering project. The BERT model performs layer-by-layer semantic encoding of the input text through multiple Transformer encoder layers inside it. Each layer of encoding takes into account the contextual information of the word, that is, the meaning of a word is affected by the words before and after it. In this way, the BERT model can generate high-dimensional vector representations of each word, which not only contain the semantics of the word itself, but also its meaning in a specific context. The result of this word embedding provides a rich semantic basis for subsequent entity recognition and relationship extraction. For example, in a power engineering project, the word "transformer" may refer to different entities in different contexts, such as equipment entities or process entities. The BERT model can generate different vector representations based on the context, thereby providing accurate semantic information for subsequent steps.

[0087] Next, based on the word embedding results generated by the BERT model, the named entity recognition model combining the bidirectional long short-term memory network (BiLSTM) and the conditional random field (CRF) is used to extract the key entities of the power project from the project supervision standardization document. BiLSTM is a special recurrent neural network (RNN) that can simultaneously consider the forward and backward information of the text sequence, so as to better capture the long-distance dependencies in the text. CRF is a statistical model for sequence labeling, which can predict the most likely output label sequence given the input sequence. In this step, BiLSTM first receives the word embedding vector generated by the BERT model as input, and then processes the text sequence word by word through its internal LSTM unit. Each LSTM unit outputs a hidden state, which contains information about the current word and its context. Next, the CRF model receives the hidden state sequence output by BiLSTM and labels each word according to a predefined label set (such as power project process entity, power project resource entity, and project process dependency entity). In this way, the model can accurately identify the key entities in the text. For example, the phrase "transformer installation" is identified in the text and annotated as a power project process entity. This entity recognition method can not only handle complex text structures, but also effectively deal with the nesting and overlap problems between entities, thereby improving the accuracy and robustness of entity recognition.

[0088] Next, according to the contextual associations of the key entities of power projects in the standardized documents of project supervision, the entity dependencies between the key entities of different power projects are identified. Entity dependencies refer to the semantic dependencies between entities. For example, a process entity may depend on a resource entity. In this step, a dependency parsing model needs to be built first. The model usually represents the relationship between entities based on a graph structure or a tree structure. The model receives the key entities identified in the previous step and their contextual information as input, and then constructs a dependency graph between entities by analyzing the grammatical and semantic relationships between entities. For example, when the sentence "transformer installation requires transformer equipment" is identified in the text, the model will analyze that the process entity "transformer installation" depends on the resource entity "transformer equipment". The identification of this dependency relationship not only depends on the surface form of words, but also requires a deep understanding of the semantic structure of the text. In this way, the model can accurately capture the complex relationships between entities, thus providing a basis for the subsequent construction of the knowledge graph.

[0089] Next, based on the key entities of the power project and using the coreference resolution model, the document text of the project supervision standardization document is reversely traversed to identify and parse the reference relationship of all key entities of the power project in the project supervision standardization document, and the entity coreference relationship between different key entities of the power project is obtained. The coreference relationship refers to the phenomenon that different words or phrases in the text point to the same entity, for example, "transformer" and "the equipment" may point to the same entity. In this step, the coreference resolution model first receives the key entities identified in the previous step and their context information as input, and then identifies all words or phrases pointing to the same entity by analyzing the reference phenomena such as pronouns and demonstratives in the text. For example, when the sentence "Transformer installation requires transformer equipment, which must comply with the standard" is identified in the text, the model will analyze that "the equipment" points to the entity "transformer equipment". The coreference resolution model is usually implemented based on rules, statistics or deep learning methods, among which the deep learning model can better handle complex reference phenomena. In this way, the model can accurately identify the coreference relationship in the text, thereby providing more complete and accurate semantic information for the subsequent knowledge graph construction. The identification of this co-reference relationship can not only improve the integrity of the knowledge graph, but also enhance the reasoning ability of the knowledge graph, thereby providing more powerful support for the intelligent supervision and management of power engineering projects.

[0090] In one embodiment, the method further comprises the steps of:

[0091] The knowledge distillation technology is used to migrate the model knowledge of the pre-trained large language model to the BERT model, named entity recognition model, coreference resolution model and heterogeneous graph neural network inference engine respectively.

[0092] In this embodiment, knowledge distillation is a technology of model compression and knowledge transfer, the core idea of ​​which is to guide the training process of a relatively simple model (usually called a student model) through a trained complex model (usually called a teacher model). The teacher model usually has stronger expressive power and higher accuracy, while the student model needs to minimize the use of computing resources and storage space while maintaining high performance. In this step, you first need to select a pre-trained large language model as the teacher model, such as GPT-3 or T5. These large language models are usually pre-trained on large-scale text data and can capture rich language knowledge and semantic information.

[0093] Next, the knowledge of the teacher model is transferred to the BERT model, named entity recognition model, coreference resolution model and heterogeneous graph neural network inference engine through knowledge distillation technology. Specifically, the process of knowledge distillation usually includes two stages: soft label generation and student model training. In the soft label generation stage, the teacher model processes the input text and generates a probability distribution for each word or phrase. These probability distributions are called soft labels. The soft label not only contains the teacher model's prediction results for each word or phrase, but also contains the teacher model's confidence information on these prediction results. In the student model training stage, the student model receives the same input text and is trained by minimizing the difference between its output and the soft label generated by the teacher model. This difference is usually measured by the cross entropy loss function, which is calculated as follows: L = -∑(y i *log(p i )), where y i is the soft label generated by the teacher model, p i is the predicted probability of the student model.

[0094] In this way, the student model can learn the knowledge of the teacher model, thereby reducing the model complexity and consumption of computing resources while maintaining high performance. For example, in the training process of the BERT model, through the knowledge distillation technology, the BERT model can learn the deep understanding of the contextual semantics of the large language model, thereby improving its performance in specific tasks. In the training process of the named entity recognition model, the knowledge distillation technology can help the model better capture the entity boundaries and category information in the text, thereby improving the accuracy of entity recognition. In the training process of the coreference resolution model, the knowledge distillation technology can help the model better understand the reference relationship in the text, thereby improving the accuracy of coreference resolution. In the training process of the heterogeneous graph neural network reasoning engine, the knowledge distillation technology can help the model better capture the complex relationship in the knowledge graph, thereby improving the accuracy and efficiency of reasoning. Through the knowledge distillation technology, these models can not only perform well in specific tasks, but also maintain high performance and efficiency under limited computing resources and storage space. This knowledge transfer method not only improves the generalization ability of the model, but also enhances the adaptability and robustness of the model in practical applications, thereby providing more reliable and efficient technical support for the intelligent supervision and management of power engineering projects.

[0095] In one implementation, constructing a multi-level project knowledge graph of a power engineering project based on key entities of the power project and according to entity association relationships includes the following steps:

[0096] Design an initial graph ontology model based on multiple preset hierarchical concepts, including macro level, meso level and micro level;

[0097] Combining entity dependency and entity co-reference, an entity tree structure of all key entities of power projects is constructed. The entity nodes in the entity tree structure represent key entities of power projects. The parent-child structure relationship between entity nodes in the entity tree structure represents entity dependency. Entity nodes with entity co-reference belong to the same level of tree structure in the entity tree structure.

[0098] Count the number of tree structure layers of the entity tree structure and the number of structural branches of each tree structure layer in the entity tree structure;

[0099] The entity tree structure is split into different hierarchical concepts based on the number of tree structure layers and the number of structural branches, and the entity tree structure is mapped to the initial graph ontology model based on the hierarchical concepts to form a multi-level project knowledge graph for power engineering projects.

[0100] In this embodiment, the graph ontology model is a structured model for representing knowledge, which describes knowledge in a specific field by defining entities, attributes and the relationships between them. In this step, it is first necessary to clarify the definition and scope of each level concept. The macro level usually represents the high-level concepts and overall framework in the power engineering project, such as the overall goals, main stages and key milestones of the project. The meso level focuses on the middle-level concepts in the project, such as specific processes, resource allocation and task dependencies. The micro level goes deep into the details of the project, such as specific operating steps, equipment parameters and personnel division of labor. When designing the initial graph ontology model, it is necessary to define the corresponding entity type, attribute type and relationship type for each level concept. For example, at the macro level, the "project stage" entity type can be defined, and its attributes include "stage name", "start time" and "end time"; at the meso level, the "process" entity type can be defined, and its attributes include "process name", "required resources" and "dependent processes"; at the micro level, the "operation step" entity type can be defined, and its attributes include "step name", "executor" and "equipment parameters". In this way, the initial graph ontology model can provide a clear structural framework for the subsequent construction of the entity tree structure and the formation of a multi-level knowledge graph.

[0101] Next, the entity tree structure of all key entities of the power project is constructed by combining entity dependencies and entity co-reference relationships. The entity tree structure is a tree-like data structure used to represent the hierarchical relationship and dependency relationship between entities. In this step, the key entities of the power project need to be represented as nodes in the entity tree structure. Each node represents an entity, such as the "transformer installation" process entity or the "transformer equipment" resource entity. The parent-child structure relationship between entity nodes represents the entity dependency relationship. For example, the "transformer installation" process entity depends on the "transformer equipment" resource entity, so in the entity tree structure, the "transformer equipment" node is the parent node of the "transformer installation" node. Entity nodes with entity co-reference relationships belong to the same level of tree structure in the entity tree structure. For example, "transformer equipment" and "the equipment" point to the same entity, so in the entity tree structure, these two nodes belong to the same level of tree structure. In this way, the entity tree structure can clearly represent the hierarchical relationship and dependency relationship between the key entities of the power project, thereby providing a basis for the subsequent construction of a multi-level knowledge graph.

[0102] Next, count the number of tree structure layers of the entity tree structure and the number of structural branches in each tree structure layer in the entity tree structure. The number of tree structure layers refers to the maximum depth of the entity tree structure from the root node to the leaf node, and the number of structural branches refers to the number of branches of nodes in each tree structure layer. In this step, first traverse the entity tree structure, starting from the root node, and count the number of nodes in each layer layer by layer. Figure 2 As shown in the figure, in the entity tree structure, the root node is "power engineering project", and its child nodes include "project phase 1", "project phase 2" and "project phase 3", and these child nodes have their own child nodes, such as "process 1", "process 2" and "process 3". In this way, the total number of layers of the entity tree structure and the number of nodes in each layer can be counted. Finally, based on the number of layers of the tree structure and the number of branches in each layer, a logical rule can be introduced to dynamically divide the entity tree structure into macro level, meso level and micro level.

[0103] In one implementation, the entity tree structure is split into different hierarchical concepts based on the number of tree structure layers and the number of structure branches, and the entity tree structure is mapped to the initial graph ontology model based on the hierarchical concepts to form a multi-level project knowledge graph of the power engineering project, including the following steps:

[0104] According to the number of tree structure layers and the number of structural branches, each layer of the tree structure in the entity tree structure is divided into macro level, meso level and micro level in order from the root node to the leaf node, and node level tags of corresponding level concepts are added to all entity nodes;

[0105] Traverse each pair of parent-child structural relationships in the entity tree structure. If the parent node and child node connected by the parent-child structural relationship have the same node hierarchy mark, add a relationship hierarchy mark of the same hierarchy concept as the parent node or child node to the parent-child structural relationship;

[0106] If the parent node and the child node connected by the parent-child structural relationship do not have the same node hierarchy mark, a cross-layer relationship mark is added to the parent-child structural relationship;

[0107] For each layer of the graph model in the initial graph ontology model, the target entity nodes and target parent-child structural relationships belonging to the same hierarchical concept as the graph model are screened out based on the node hierarchical labels and relationship hierarchical labels. The graph embedding technology is used to generate a low-dimensional vector representation for each target entity node and target parent-child structural relationship in the graph model, and the project knowledge graph corresponding to the graph model is constructed.

[0108] When the project knowledge graphs to which all hierarchical concepts belong are constructed, the graph vertical connection relationships between project knowledge graphs at different levels are generated based on the parent-child structural relationship with cross-layer relationship markers to obtain a multi-level project knowledge graph for the power engineering project.

[0109] In this implementation, each layer of the tree structure in the entity tree structure is divided into macro level, meso level and micro level in order from the root node to the leaf node in combination with the number of tree structure layers and the number of structural branches. The core idea of ​​the hierarchical division rule is to divide the tree structure from the root node to the leaf node into macro level, meso level and micro level in order according to the number of layers of the entity tree structure and the number of branches in each layer. The specific rules are as follows:

[0110] Macro level: Contains the first N levels of the entity tree structure, where the value of N is determined by the total number of levels and branches in the tree structure. The macro level usually represents high-level concepts of the power engineering project, such as the overall project goals, major phases, and key milestones.

[0111] Meso-level: Contains the middle M layers of the entity tree structure, where the value of M is determined by the total number of layers and branches in the tree structure. The meso-level usually represents the middle-level concepts of the project, such as specific processes, resource allocation, and task dependencies.

[0112] Micro-level: Contains the remaining layers of the entity tree structure, usually representing the detailed level of the project, such as specific operating steps, equipment parameters, and personnel division of labor.

[0113] Assume that the total number of layers in the tree structure is L, and the average number of branches in each layer is B. The specific division formula is as follows:

[0114] The number of layers at the macro level N=ceil(log_B(L)), where ceil represents rounding up, and log_B represents the logarithm with B as the base.

[0115] The number of layers at the mesoscopic level M=ceil(log_B(L / 2)).

[0116] The number of layers at the microscopic level is the number of remaining layers, i.e., LNM.

[0117] Next, add the corresponding level mark for each node. Figure 2 As shown in the figure, the "Power Engineering Project" node at the first layer and the "Project Phase 1" node at the second layer are marked as the macro level, the "Process 1" node at the third layer is marked as the meso level, and the "Operation Step 1" node at the fourth layer and the "Equipment Parameter 1" node at the fifth layer are marked as the micro level. In this way, each node in the entity tree structure is accurately divided into the corresponding hierarchical concept, thereby providing a clear hierarchical structure for the subsequent knowledge graph construction. Then traverse each pair of parent-child structural relationships in the entity tree structure. If the parent node and child node connected by the parent-child structural relationship have the same node hierarchical mark, add a relationship hierarchical mark of the same hierarchical concept as the parent node or child node to the parent-child structural relationship. The core goal of this step is to identify and mark the intra-hierarchical relationships in the entity tree structure, that is, the relationships between nodes at the same level. First, it is necessary to traverse all parent-child structural relationships in the entity tree structure. As Figure 2 As shown in the figure, the parent node is "Project Phase 1" (macro level) and the child node is "Project Phase 2" (macro level). This pair of parent-child structural relationships belongs to the same level. Next, add a relationship level tag with the same level concept as the parent node or child node to this type of relationship. Figure 2 As shown in the figure, a macro-level relationship hierarchy mark is added to the relationship between "Project Phase 1" and "Project Phase 2". In this way, the intra-level relationships in the entity tree structure are accurately marked, thereby providing clear intra-level relationship information for the subsequent knowledge graph construction. This intra-level relationship mark can not only improve the structured degree of the knowledge graph, but also enhance the query and reasoning efficiency of the knowledge graph.

[0118] If the parent node and child node connected by the parent-child structural relationship do not have the same node hierarchy mark, then add a cross-layer relationship mark to the parent-child structural relationship. The core goal of this step is to identify and mark the cross-layer relationship in the entity tree structure, that is, the relationship between nodes at different levels. First, it is necessary to traverse all parent-child structural relationships in the entity tree structure. Figure 2As shown in the figure, the parent node is "Project Phase 1" (macro level) and the child node is "Process 1" (meso level), then this pair of parent-child structural relationships is a cross-level relationship. Next, add cross-level relationship tags for such relationships. For example, add a cross-level relationship tag for the relationship between "Project Phase 1" and "Process 1". In this way, the cross-level relationships in the entity tree structure are accurately marked, thereby providing clear cross-level relationship information for the subsequent knowledge graph construction. This cross-level relationship tagging can not only improve the integrity of the knowledge graph, but also enhance the multi-level reasoning ability of the knowledge graph.

[0119] Next, the nodes and relationships in the entity tree structure are mapped to the initial graph ontology model, and a low-dimensional vector representation is generated to construct a multi-level project knowledge graph. First, the target entity nodes and target parent-child structural relationships that belong to the same hierarchical concept as the graph model need to be screened out based on the node hierarchical labeling and the relationship hierarchical labeling. For example, in the macro-level graph model, all nodes and relationships marked as macro-level are screened out. Next, a low-dimensional vector representation is generated for each target entity node and target parent-child structural relationship using graph embedding technology. Graph embedding technology is a method of mapping nodes and relationships in a graph structure to a low-dimensional vector space. Commonly used graph embedding algorithms include Node2Vec, GraphSAGE, etc. Through graph embedding technology, each node and relationship is represented as a low-dimensional vector, which can facilitate subsequent queries and reasoning. In this way, each layer of the graph model is constructed as a corresponding project knowledge graph.

[0120] When the project knowledge graphs to which all hierarchical concepts belong are constructed, the project knowledge graphs at different levels are connected through cross-level relationships to form a complete multi-level knowledge graph. First, it is necessary to traverse all parent-child structural relationships marked with cross-level relationships. For example, if there is a cross-level relationship between the "Project Phase 1" node in the macro-level graph model and the "Process 1" node in the meso-level graph model, the "Project Phase 1" node and the "Process 1" node are connected through a cross-level relationship. In this way, the project knowledge graphs at the macro level, meso level, and micro level are connected into a complete multi-level knowledge graph. This multi-level knowledge graph can not only clearly represent the hierarchical relationships and dependencies between concepts at different levels in the power engineering project, but also support efficient query and reasoning.

[0121] In one implementation manner, the project supervision data includes project visual data, project audio data, and project multi-sensor data. The multi-dimensional project supervision data is fused into multi-modal supervision data. The multi-modal supervision data is processed using an adaptive cross-modal attention mechanism. The generation of a project event cause representation of a power engineering project in a current time period includes the following steps:

[0122] Use pre-trained convolutional neural networks to extract project visual features from project visual data;

[0123] The project acoustic features are extracted from the project audio data by combining Mel frequency cepstral coefficients and short-time Fourier transform;

[0124] The sensor fusion algorithm is used to fuse the multi-sensor data of the project into sensor comprehensive data, and the sensor data features are extracted from the sensor comprehensive data through the wavelet transform method;

[0125] For any target feature among the project visual features, project acoustic features and sensor data features, a feature attention score between the target feature and the other two features is calculated based on an adaptive cross-modal attention mechanism;

[0126] Based on the feature attention score, the project visual features, project acoustic features and sensor data features are integrated into a comprehensive feature representation of the power engineering project in the current time period;

[0127] The comprehensive feature representation is mapped to a vector space of fixed dimension through a multi-layer perceptron to obtain the cause representation of project events of the power engineering project in the current time period.

[0128] In this embodiment, the convolutional neural network is a deep learning model specifically used for processing image data, and its core idea is to extract multi-level feature representations from images through structures such as convolutional layers, pooling layers, and fully connected layers. Pre-trained CNN models are usually trained on large-scale image data sets (such as ImageNet) and can capture common features in images, such as edges, textures, and shapes. In this step, the project visual data (such as images or video frames of the construction site) are first input into the pre-trained CNN model. The CNN model processes the image layer by layer through its internal convolutional layers, and each layer extracts features of different levels. For example, the low-level convolutional layer may extract the edge and texture features of the image, while the high-level convolutional layer may extract the semantic features of the image, such as the type of equipment or the construction status. In this way, the CNN model can generate high-dimensional feature vectors of the image, which contain rich information about the image. For example, in a power engineering project, the CNN model can extract visual features such as the location of the transformer, the operating status of the equipment, and the activities of the construction personnel from the image of the construction site.

[0129] Next, the project acoustic features are extracted from the project audio data by combining Mel frequency cepstral coefficients (MFCC) and short-time Fourier transform (STFT). Mel frequency cepstral coefficients are a commonly used audio feature extraction method. It converts audio signals into a series of coefficients that reflect the spectral characteristics of sound by simulating the human ear's perception of sound. Short-time Fourier transform is a method of converting time-domain audio signals into frequency-domain representations, which can capture the changes in the frequency components of audio signals over time. In this step, the project audio data (such as the sound of equipment operation or the conversation of construction workers) is first preprocessed, including operations such as framing, windowing, and denoising. Next, the Mel frequency cepstral coefficients and short-time Fourier transform are calculated for each frame of audio signal. The calculation process of Mel frequency cepstral coefficients includes the steps of Fourier transforming the audio signal, converting the frequency scale to Mel scale, taking the logarithm and performing discrete cosine transform. Short-time Fourier transform generates its frequency domain representation by Fourier transforming each frame of audio signal. In this way, the audio data is converted into a series of coefficients and spectrograms that reflect the characteristics of the sound. For example, in power engineering projects, MFCC and STFT can extract the vibration frequency, operating status and abnormal sound characteristics of the equipment from the sound of equipment operation.

[0130] Next, the sensor fusion algorithm is used to fuse the project multi-sensor data into sensor comprehensive data, and the sensor data features are extracted from the sensor comprehensive data by wavelet transform method. The sensor fusion algorithm is a method of integrating and processing data from multiple sensors, which can improve the accuracy and reliability of the data. Wavelet transform is a method of decomposing a signal into different frequency components, which can capture the local features and transient changes in the signal. In this step, the project multi-sensor data (such as temperature, humidity, vibration and pressure, etc.) are first preprocessed, including data alignment, denoising and normalization. Next, the sensor fusion algorithm is used to fuse the multi-sensor data into sensor comprehensive data. Common sensor fusion algorithms include Kalman filtering, weighted averaging and principal component analysis. For example, the Kalman filtering algorithm can be used to fuse the data of the temperature sensor and the humidity sensor to generate comprehensive data reflecting the environmental state. Then, the wavelet transform method is used to process the sensor comprehensive data and extract the sensor data features. Wavelet transform generates coefficients that reflect the local features of the signal by multi-scale decomposition of the signal. For example, the wavelet transform can extract the vibration frequency and abnormal vibration mode of the device from the data of the vibration sensor.

[0131] Next, the project visual features, project acoustic features, and sensor data features are input into the adaptive cross-modal attention mechanism. For any target feature, the feature attention score between it and the other two features is calculated. The calculation of the feature attention score is usually based on the similarity of feature vectors, and the correlation between features can be measured by dot product or cosine similarity. For example, assuming that the target feature is the project visual feature, calculate the feature attention score between it and the project acoustic feature and sensor data feature. The calculation formula of the feature attention score is: Attention score = softmax((Q*K T ) / sqrt(d k ), where Q is the query vector of the target feature, K is the key vector of other features, and d k is the dimension of the vector. In this way, the adaptive cross-modal attention mechanism can dynamically adjust the weight of each modal feature to capture the correlation between multimodal data. Each modal feature is then weighted according to the feature attention score. For example, for the project visual feature, its weight is the feature attention score between the project acoustic feature and the sensor data feature. Next, the weighted multimodal features are summed to generate a comprehensive feature representation. In this way, the comprehensive feature representation can reflect multi-dimensional information such as the operating status of the equipment, environmental conditions, and construction progress.

[0132] Finally, the comprehensive feature representation is mapped to a vector space of fixed dimension through a multi-layer perceptron (MLP) to obtain the project event cause representation of the power engineering project in the current time period. The multi-layer perceptron is a commonly used neural network model. Its core idea is to map the input features to the target space through multiple fully connected layers and nonlinear activation functions. In this step, the comprehensive feature representation is first input into the multi-layer perceptron. The multi-layer perceptron processes the input features layer by layer through its internal fully connected layers, and each layer performs a nonlinear transformation on the features. For example, the first fully connected layer maps the input features to the hidden layer space, and the second fully connected layer maps the hidden layer features to the output space. In this way, the multi-layer perceptron can map the comprehensive feature representation to a vector space of fixed dimension, thereby effectively converting the multimodal data of the power engineering project into the event cause representation.

[0133] In one embodiment, based on the project event cause representation, multi-level reasoning is performed in a heterogeneous graph neural network inference engine by improving the graph traversal algorithm to predict the abnormal event result set caused by the project event cause in a preset future time period of the power engineering project, including the following steps:

[0134] Determine the inference starting node in the multi-level project knowledge graph based on the project event cause representation, and replace the project event cause representation with the node feature vector of the inference starting node;

[0135] Through the heterogeneous graph neural network inference engine and the depth-first search algorithm, the graph is iteratively traversed from the inference starting node. The depth-first search algorithm incorporates heuristic rules based on the domain knowledge of power engineering projects. The number of iterations of the graph iterative traversal is set according to the time interval between the preset future time period and the current time period.

[0136] During the graph iterative traversal, for each target graph node visited by the traversal, the neighbor node information of all neighbor nodes of the target graph node in the same level project knowledge graph is aggregated through the heterogeneous graph attention module, and the node features of the target graph node and all node edge features of the target graph node are dynamically updated according to the neighbor node information;

[0137] When any target graph node completes information aggregation and dynamic feature update, the Monte Carlo tree search method is used to select the next graph node to be visited in the project knowledge graph at the level where the target graph node is located;

[0138] For any level of the project knowledge graph, aggregate the node update information of all target graph nodes in the project knowledge graph, and pass the aggregated node update information to all other levels of the project knowledge graph through the multi-head attention mechanism module;

[0139] After the last round of graph iteration traversal, multiple abnormal event chains in the multi-level project knowledge graph are obtained. The corresponding abnormal event result representations are extracted from each abnormal event chain, and all abnormal event result representations are integrated into an abnormal event result set for the power engineering project in the future time period.

[0140] In this embodiment, first, the inference starting node in the multi-level project knowledge graph is determined based on the project event cause representation, and the project event cause representation is replaced with the node feature vector of the inference starting node. The specific implementation principle of this step is to calculate the similarity between the project event cause representation and the feature vectors of each node in the multi-level project knowledge graph, and select the node with the highest similarity as the inference starting node. The similarity calculation can adopt the cosine similarity method, that is, the project event cause representation and the node feature vector are regarded as vectors in a multidimensional space, and the cosine value of the angle between them is calculated. The calculation formula of cosine similarity is: cos(θ)=(A·B) / (||A||||B||), where A and B represent the project event cause representation and the node feature vector respectively, and ||A|| and ||B|| represent the modulus length of the vector. For example, assuming that the cause of the project event is represented as [0.5, 0.3, 0.2], and the feature vector of a node is [0.4, 0.4, 0.2], then their cosine similarity is (0.5*0.4+0.3*0.4+0.2*0.2) / (sqrt(0.5^2+0.3^2+0.2^2)*sqrt(0.4^2+0.4^2+0.2^2))≈0.989. After calculating the inference starting node, the project event cause representation is directly assigned to the feature vector of the node to achieve feature replacement. The effect of this step is to determine the appropriate starting point for the subsequent graph traversal process, ensuring that reasoning starts from the node most relevant to the cause of the project event, thereby improving the accuracy and efficiency of reasoning.

[0141] Next, the graph is iterated and traversed from the inference starting node by using the heterogeneous graph neural network inference engine and the depth-first search algorithm. The basic principle of the depth-first search (DFS) algorithm is to start from the starting node and search as deeply as possible along the edge of the graph until it can no longer move forward and then backtrack to the previous node and continue searching other unvisited adjacent nodes. In this scheme, the DFS algorithm is optimized to adapt to the characteristics of power engineering projects. The specific implementation methods include: (1) setting a search depth limit to prevent falling into too deep a search; (2) introducing heuristic rules based on domain knowledge, such as giving priority to visiting edges that have a strong correlation with the current node, and the correlation can be represented by a predefined weight matrix; (3) dynamically adjusting the search strategy during the search process to decide whether to continue the in-depth search based on the importance of the node (which can be measured by indicators such as the degree centrality of the node). The method for setting the number of iterations can adopt a linear mapping, that is, the number of iterations = α*(preset future time period - current time period) + β, where α and β are parameters adjusted according to actual conditions. For example, if the preset future time period is 30 days later, the current time period is today, α = 0.5, β = 10, then the number of iterations is 0.5*30+10 = 25 times. This setting method ensures that the time span of reasoning matches the actual project cycle. The implementation effect of this step is that it can effectively explore possible abnormal event propagation paths in the multi-level project knowledge graph, and at the same time, by integrating domain knowledge, it improves the pertinence and efficiency of the search.

[0142] During the graph iterative traversal, for each target graph node visited by the traversal, the heterogeneous graph attention module aggregates the neighbor node information of all neighbor nodes of the target graph node in the same level project knowledge graph, and dynamically updates the node features of the target graph node and all node edge features of the target graph node based on the neighbor node information. The core idea of ​​the heterogeneous graph attention module is to selectively aggregate the information of different types of neighbor nodes by calculating the attention weights. The specific implementation method can be described as the following steps: (1) For the target node v, first collect its neighbor node set N(v) of all types; (2) For each type of neighbor node u∈N(v), calculate the attention coefficient α(v,u)=softmax(LeakyReLU(a^T[Wh_v||Wh_u])), where a is a learnable attention vector, W is a linear transformation matrix, h_v and h_u are the feature vectors of nodes v and u respectively; (3) Aggregate neighbor information according to the attention coefficient: h'_v=σ(∑(α(v,u)*Wh_u)), where σ is a nonlinear activation function such as ReLU; (4) Update the target node feature: h_v^new=h_v+h'_v; (5) For the edge connecting the target node and its neighbor node, update the edge feature e(v,u)^new=e(v,u)+MLP([h_v^new||h_u^new]), where MLP is a multi-layer perceptron. For example, suppose there is a target node representing a certain power equipment, and its neighbor nodes include maintenance records, operating status, and related personnel. By calculating the attention weights, it may be found that maintenance records and operating status have a greater impact on the current equipment status, so these two types of nodes will be given higher weights when aggregating information. The implementation effect of this method is that it can capture the complex relationship between different types of nodes in a heterogeneous graph, improving the accuracy of node representation and the accuracy of abnormal event prediction.

[0143] When any target graph node completes information aggregation and dynamic feature update, the Monte Carlo Tree Search (MCTS) method is used to select the next graph node to be visited in the project knowledge graph at the level of the target graph node. MCTS is a heuristic search algorithm that is particularly suitable for problems with large state spaces. In this scheme, the implementation principle of MCTS includes four main steps: selection, expansion, simulation, and backtracking. The specific implementation method is as follows: (1) Selection: Starting from the root node (the current target graph node), use the UpperConfidenceBound (UCB) formula to recursively select the most promising child node until a leaf node is reached. The UCB formula is: UCB = X_j + C*sqrt(ln(n) / n_j), where X_j is the average return of node j, n is the number of visits to the parent node, n_j is the number of visits to node j, and C is the exploration parameter; (2) Expansion: If the selected leaf node is not a terminal state and has not been fully expanded, a random unexplored action is selected to expand the tree; (3) Simulation: Starting from the newly expanded node, simulation is performed using a random strategy until the terminal state is reached. In power engineering projects, these simulations can be performed based on predefined rules or simplified project models; (4) Backtracking: The simulation results are transmitted back along the selected path to update the statistical information of each node. For example, suppose that at a certain power equipment node, MCTS considers three actions: "maintenance", "replacement of parts", and "continued operation". Through multiple simulations, it is found that the "maintenance" action can reduce the failure risk by 60% on average, "replacement of parts" can reduce it by 80%, and "continued operation" may lead to a 20% increase in the failure rate. MCTS will give priority to the action of "replacing parts" because it has the highest average return. The implementation effect of this method is that it can effectively balance exploration and utilization in complex decision spaces, find the path most likely to lead to abnormal events, and improve the accuracy and comprehensiveness of abnormal event prediction.

[0144] For any level of the project knowledge graph, aggregate the node update information of all target graph nodes in the project knowledge graph, and pass the aggregated node update information to all other levels of the project knowledge graph through the multi-head attention mechanism module. The implementation principle of this step is based on the multi-head attention mechanism, which allows the model to focus on different representation subspaces of information at the same time. The specific implementation can be described as the following steps: (1) For each attention head, first perform a linear transformation on the input to obtain the query Q, key K and value V: Q = W_Q*X, K = W_K*X, V = W_V*X, where X is the input, W_Q, W_K, W_V are learnable weight matrices; (2) Calculate the attention score: Attention(Q,K,V) = softmax(QK^T / sqrt(d_k))V, where d_k is the dimension of the key; (3) Concatenate the outputs of multiple attention heads and perform a linear transformation: MultiHead(Q,K,V) = Concat(head_1,...,head_h)W_O, where h is the number of attention heads and W_O is the output projection matrix; (4) For cross-layer information transfer, graph representations of different levels can be used as inputs of Q, K, and V to achieve inter-layer information interaction. For example, suppose there are three levels of knowledge graphs: macro, meso, and micro, and each level of the graph is represented as a matrix. Through the multi-head attention mechanism, information at the macro level can selectively focus on important features at the meso and micro levels, thereby capturing detailed information while maintaining high-level abstraction. Specifically, if there is a node representing "grid stability" at the macro level, through the multi-head attention mechanism, it may focus on both the "substation operating status" at the meso level and the "key equipment parameters" at the micro level, thereby forming a more comprehensive understanding. The implementation effect of this method is that it can achieve effective information interaction between knowledge graphs at different levels, improve the model's ability to understand complex systems, and enhance the accuracy and interpretability of abnormal event predictions.

[0145] After the last round of graph iteration traversal, multiple abnormal event chains in the multi-level project knowledge graph are obtained. The corresponding abnormal event result representations are extracted from each abnormal event chain, and all abnormal event result representations are integrated into the abnormal event result set of the power engineering project in the future time period. The implementation principle of this step is to identify and extract the key node sequence that may cause abnormal events by analyzing the path formed during the graph traversal process. The specific implementation method can be described as the following steps: (1) Score each traversal path. The scoring criteria may include factors such as path length, node importance, and edge weight. The scoring function can be expressed as: Score = ∑ (w_i*f_i), where w_i is the weight of each factor and f_i is the corresponding feature function; (2) Select the top-k paths as abnormal event chains according to the score; (3) For each abnormal event chain, extract the key nodes (such as nodes with abnormal probability exceeding the threshold) and their attributes to form an abnormal event result representation. The representation can be a vector containing information such as event type, occurrence probability, and impact range; (4) Aggregate all abnormal event result representations into a set, and it may be necessary to remove duplicate or highly similar events.

[0146] In one implementation, combining the standby supervision resources and the abnormal event result set, using the reinforcement learning optimization algorithm to generate the optimal supervision dispatching strategy for the power engineering project in the future time period includes the following steps:

[0147] Construct the state space by combining the abnormal event result set and the standby supervision resources;

[0148] Generate a variety of different supervisor assignment strategies based on the available supervisor resources, and construct an action space with all supervisor assignment strategies;

[0149] Combining state space and action space to build a supervisor dispatch decision model;

[0150] Taking maximizing the utilization efficiency of supervision resources as the optimization goal, based on the supervision dispatch decision model and using the offline strategy algorithm, iterative optimization is performed to output the optimal supervision dispatch strategy for the power engineering project in the future time period.

[0151] In this implementation, first, the abnormal event result set needs to be quantified and encoded. Each abnormal event can be represented by a vector, including attributes such as event type, probability of occurrence, impact range, and expected time of occurrence. At the same time, the standby supervision resources also need to be quantified, which can include attributes such as the number of supervisors, professional type, and experience level. For example, a supervision resource may be represented as [3,2,4], representing 3 supervisors in electrical engineering, 2 supervisors in civil engineering, and 4 supervisors in equipment. The construction of the state space is to combine these quantified abnormal events and supervision resource information into a multidimensional vector. The specific implementation can be represented by a tensor, in which one dimension represents time, another dimension represents abnormal events, and the third dimension represents supervision resources. For example, a state may be represented as S_t=[E_t,R_t], where E_t is the set of abnormal events at time t, and R_t is the set of available supervision resources at time t. In order to deal with the high-dimensional characteristics of the state space, dimensionality reduction techniques such as principal component analysis (PCA) or autoencoders can be used to reduce the dimensionality of the state representation while retaining key information.

[0152] Next, we need to define the basic units of the dispatch strategy, including the allocation of supervisors, work schedule, task priority, etc. For example, a basic dispatch unit may be represented as [supervisor ID, task ID, start time, end time, priority]. Based on these basic units, a combinatorial generation algorithm can be used to create a variety of different dispatch strategies. One feasible method is to use a genetic algorithm to encode each dispatch strategy as a "chromosome" and generate new strategies through crossover and mutation operations. For example, assuming there are 5 supervisors and 10 potential abnormal events, a complete dispatch strategy may be a 5x10 matrix, where each element represents the probability or time that a supervisor is assigned to a task. The fitness function of the genetic algorithm can be designed based on indicators such as resource utilization and task coverage. Ultimately, the action space can be represented as a high-dimensional discrete space, where each point represents a feasible dispatch strategy. In order to improve the efficiency of subsequent optimization, the action space can be preprocessed, such as clustering or sparse representation, to reduce the size of the search space.

[0153] Next, a supervisor dispatch decision model is constructed by combining the state space and action space. This decision model can be constructed based on the Markov decision process (MDP), where the state space S represents the various possible states of the project, the action space A represents all possible dispatch strategies, the transition function P(s'|s,a) describes the probability of transferring to the new state s' after taking action a in the current state s, and the reward function R(s,a) defines the immediate reward obtained by taking action a in state s. In specific implementation, a deep neural network can be used to approximate the state-action value function Q(s,a), that is, the long-term cumulative reward expectation of taking action a in state s. The network structure can adopt a two-stream architecture, one stream processes state information, the other stream processes action information, and finally merges to generate a Q value estimate. For example, the state stream can use a convolutional neural network (CNN) to process spatially related project information, and a recurrent neural network (RNN) to process time-related abnormal event prediction; the action stream can use a fully connected layer to process discrete dispatch strategies.

[0154] Taking maximizing the efficiency of supervision resource utilization as the optimization goal, iterative optimization is performed based on the supervision dispatch decision model and the offline strategy algorithm is used to output the optimal supervision dispatch strategy for the power engineering project in the future time period. This is the last step of the entire scheme and the most critical step. The selection of the offline strategy algorithm is based on actual considerations, because in actual supervision work, direct online learning may bring risks and costs. Commonly used offline strategy algorithms include batch constrained deep Q learning (BCQ) and conservative Q learning (CQL). ​​Taking BCQ as an example, its core idea is to avoid excessive extrapolation by limiting the range of action selection. The specific implementation steps are as follows: (1) First, collect historical supervision dispatch data, including state, action and reward. (2) Train a behavior cloning model G(s) to generate the action distribution under a given state. (3) Train a state-action value function Q(s,a) to evaluate the long-term value of the action. (4) During reasoning, select the action by the following formula: a*=argmax_a(Q(s,a)+λlog(G(a|s))), where λ is a balance parameter. This formula takes into account both the long-term value of the action and the consistency with historical data. The optimization process is iterative, and each iteration includes three stages: data collection, model update, and strategy evaluation. In order to improve the optimization efficiency, the priority experience replay technology can be used to give priority to learning samples with high temporal difference errors. At the same time, in order to handle large-scale state and action spaces, function approximation techniques such as deep neural networks can be used to represent the Q function. During the optimization process, special attention should be paid to avoiding overfitting and strategy degradation problems. The generalization ability of the model can be enhanced through regularization techniques, early stopping strategies, and other methods. The optimal supervision dispatching strategy outputted in the end should be a dynamic decision function that can adapt to different project states, rather than a static dispatching table. For example, for a given project state s, the optimal strategy π*(s) can output a probability distribution over the action space, indicating the preference of various dispatching decisions in this state. In practical applications, the highest probability action can be sampled or selected based on this distribution. The implementation effect of this method is that it can generate an intelligent and adaptive supervision dispatching strategy, which can not only maximize the utilization efficiency of supervision resources, but also flexibly respond to various uncertainties and changes in the project, thereby significantly improving the supervision quality and efficiency of power engineering projects.

[0155] The present invention also discloses an intelligent supervision and management platform based on multi-terminal interaction and data fusion, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the intelligent supervision and management method based on multi-terminal interaction and data fusion described in any one of the above-mentioned embodiments is implemented.

[0156] Among them, the processor can adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be adopted. The general-purpose processor can adopt a microprocessor or any conventional processor, etc., and this application does not impose any restrictions on this.

[0157] Among them, the memory can be an internal storage unit of a computer device, such as a hard disk or memory of a computer device, or an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD) or a flash memory card (FC) equipped on the computer device, etc., and the memory can also be a combination of an internal storage unit and an external storage device of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or is to be output, and this application does not impose any restrictions on this.

[0158] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as above, which are not provided in detail for the sake of simplicity.

[0159] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the protection scope of the present application.

Claims

1. An intelligent supervision management method based on multi-terminal interaction and data fusion, characterized in that: The steps include: Obtain standardized project supervision documents and available supervision resources for power engineering projects; Extracting key entities of the power project from the project supervision standardization document using natural language processing technology enhanced by deep learning, and identifying entity association relationships between the key entities of the power project, wherein the key entities of the power project include power project process entities, power project resource entities, and project process dependency entities, and the entity association relationships include entity dependency relationships and entity co-reference relationships; Constructing a multi-level project knowledge graph of the power engineering project based on the key entities of the power project and according to the entity association relationships; Based on the multi-level project knowledge graph and in combination with the graph neural network model, a heterogeneous graph neural network inference engine for the power engineering project is constructed; Collect multi-dimensional project supervision data of the power engineering project in the current time period through a variety of intelligent supervision terminals; The multi-dimensional project supervision data is integrated into multi-modal supervision data, and the multi-modal supervision data is processed using an adaptive cross-modal attention mechanism to generate a project event cause representation of the power engineering project in the current time period; Based on the project event cause representation, multi-level reasoning is performed in the heterogeneous graph neural network inference engine by improving the graph traversal algorithm to predict a set of abnormal event results caused by the project event cause in a preset future time period of the power engineering project; In combination with the standby supervision resources and the abnormal event result set, a reinforcement learning optimization algorithm is used to generate an optimal supervision dispatching strategy for the power engineering project in the future time period.

2. The intelligent supervision management method based on multi-terminal interaction and data fusion according to claim 1 is characterized in that: The method of extracting key entities of a power project from the standardized project supervision document using the natural language processing technology enhanced by deep learning and identifying the entity association relationship between the key entities of the power project comprises the following steps: Use the pre-trained BERT model to perform context-sensitive word embedding on the project supervision standardization document; Based on the word embedding results, a named entity recognition model combining a bidirectional long short-term memory network and a conditional random field is used to extract key entities of the power project from the project supervision standardization document. The key entities of the power project include power project process entities, power project resource entities and project process dependency entities. According to the contextual association relationship of the key entities of the power project in the project supervision standardization document, identifying the entity dependency relationship between different key entities of the power project; Based on the key entities of the power project and using the co-reference resolution model, the file text of the project supervision standardization document is reversely traversed, the reference relationship of all the key entities of the power project in the project supervision standardization document is identified and parsed, and the entity co-reference relationship between different key entities of the power project is obtained.

3. The intelligent supervision management method based on multi-terminal interaction and data fusion according to claim 2 is characterized in that: The method further comprises the steps of: The model knowledge of the pre-trained large language model is respectively transferred to the BERT model, the named entity recognition model, the coreference resolution model and the heterogeneous graph neural network inference engine using knowledge distillation technology.

4. The intelligent supervision management method based on multi-terminal interaction and data fusion according to claim 2 is characterized in that: The multi-level project knowledge graph of the power engineering project is constructed based on the key entities of the power project and according to the entity association relationship, and includes the following steps: Designing an initial graph ontology model according to a plurality of preset hierarchical concepts, wherein the hierarchical concepts include a macro level, a meso level, and a micro level; Combining the entity dependency relationship and the entity co-referencing relationship, constructing an entity tree structure of all the key entities of the power project, the entity nodes in the entity tree structure represent the key entities of the power project, the parent-child structure relationship between the entity nodes in the entity tree structure represents the entity dependency relationship, and the entity nodes with the entity co-referencing relationship belong to the same level of tree structure in the entity tree structure; Counting the number of tree structure layers of the entity tree structure and the number of structural branches of each tree structure layer in the entity tree structure; The entity tree structure is split into different hierarchical concepts in combination with the number of tree structure layers and the number of structure branches, and the entity tree structure is mapped to the initial graph ontology model based on the hierarchical concepts to form a multi-level project knowledge graph of the power engineering project.

5. The intelligent supervision management method based on multi-terminal interaction and data fusion according to claim 4 is characterized in that: The step of combining the number of tree structure layers and the number of structure branches to split the entity tree structure into different hierarchical concepts, and mapping the entity tree structure to the initial graph ontology model based on the hierarchical concepts to form a multi-level project knowledge graph of the power engineering project includes the following steps: In combination with the number of tree structure layers and the number of structure branches, each layer of the tree structure in the entity tree structure is divided into the macro level, the meso level and the micro level in order from the root node to the leaf node, and node level tags corresponding to the level concepts are added to all the entity nodes; Traversing each pair of the parent-child structural relationships in the entity tree structure, if the parent node and the child node connected by the parent-child structural relationship have the same node hierarchy mark, adding a relationship hierarchy mark of the same hierarchy concept as the parent node or the child node to the parent-child structural relationship; If the parent node and the child node connected by the parent-child structural relationship do not have the same node hierarchy mark, adding a cross-layer relationship mark to the parent-child structural relationship; For each layer of the graph model in the initial graph ontology model, based on the node hierarchy labels and the relationship hierarchy labels, the target entity nodes and target parent-child structural relationships belonging to the same hierarchical concept as the graph model are screened out, and a low-dimensional vector representation is generated in the graph model for each target entity node and the target parent-child structural relationship using graph embedding technology, so as to construct a project knowledge graph corresponding to the graph model; When the project knowledge graph to which all the hierarchical concepts belong is constructed, a graph vertical connection relationship between project knowledge graphs at different levels is generated based on the parent-child structural relationship with the cross-layer relationship mark to obtain a multi-level project knowledge graph for the power engineering project.

6. The intelligent supervision management method based on multi-terminal interaction and data fusion according to claim 1 is characterized in that: The project supervision data includes project visual data, project audio data and project multi-sensor data. The multi-dimensional project supervision data is fused into multi-modal supervision data, and the multi-modal supervision data is processed using an adaptive cross-modal attention mechanism to generate a project event cause representation of the power engineering project in the current time period, including the following steps: Extracting project visual features from the project visual data using a pre-trained convolutional neural network; Extracting project acoustic features from the project audio data by combining Mel-frequency cepstral coefficients and short-time Fourier transform; A sensor fusion algorithm is used to fuse the multi-sensor data of the project into sensor comprehensive data, and a sensor data feature is extracted from the sensor comprehensive data by a wavelet transform method; For any target feature among the project visual feature, the project acoustic feature, and the sensor data feature, calculating a feature attention score between the target feature and the other two features based on an adaptive cross-modal attention mechanism; Integrating the project visual features, the project acoustic features and the sensor data features into a comprehensive feature representation of the power engineering project in the current time period based on the feature attention score; The comprehensive feature representation is mapped to a vector space of fixed dimension through a multi-layer perceptron to obtain a project event cause representation of the power engineering project in the current time period.

7. The intelligent supervision management method based on multi-terminal interaction and data fusion according to claim 1 is characterized in that: The heterogeneous graph neural network inference engine includes a heterogeneous graph attention module, a multi-head attention mechanism module, a gated loop module, a jump connection structure and a batch normalization module. The heterogeneous graph attention module is used to process the graph nodes and graph node edges of the project knowledge graphs of different levels in the multi-level project knowledge graph. The multi-head attention mechanism module is used to capture the information transfer between the project knowledge graphs of different levels. The loss function of the heterogeneous graph neural network inference engine is constructed based on the contrastive learning strategy.

8. The intelligent supervision management method based on multi-terminal interaction and data fusion according to claim 7 is characterized in that: The method of performing multi-level reasoning in the heterogeneous graph neural network inference engine by improving the graph traversal algorithm based on the project event cause representation to predict the abnormal event result set caused by the project event cause in the preset future time period of the power engineering project includes the following steps: Determine the inference starting node in the multi-level project knowledge graph based on the project event cause representation, and replace the project event cause representation with a node feature vector of the inference starting node; The graph is iteratively traversed starting from the inference starting node by using the heterogeneous graph neural network inference engine and a depth-first search algorithm, wherein the depth-first search algorithm incorporates heuristic rules based on the domain knowledge of power engineering projects, and the number of iterations of the graph iterative traversal is set according to the time interval between a preset future time period and the current time period; During the iterative traversal of the graph, for each target graph node visited by the traversal, the neighbor node information of all neighbor nodes of the target graph node in the project knowledge graph at the same level is aggregated through the heterogeneous graph attention module, and the node features of the target graph node and all node edge features of the target graph node are dynamically updated according to the neighbor node information; When any of the target graph nodes completes information aggregation and dynamic feature update, a Monte Carlo tree search method is used to select the next graph node to be accessed in the project knowledge graph at the level where the target graph node is located; For any level of the project knowledge graph, aggregate the node update information of all the target graph nodes in the project knowledge graph, and transmit the aggregated node update information to all other levels of the project knowledge graph through the multi-head attention mechanism module; After the last round of graph iteration traversal is completed, multiple abnormal event chains in the multi-level project knowledge graph are obtained, and the corresponding abnormal event result representation is extracted from each of the abnormal event chains, and all the abnormal event result representations are integrated into an abnormal event result set for the power engineering project in the future time period.

9. The intelligent supervision management method based on multi-terminal interaction and data fusion according to claim 8 is characterized in that: The step of combining the standby supervision resources and the abnormal event result set and using a reinforcement learning optimization algorithm to generate the optimal supervision dispatching strategy for the power engineering project in the future time period comprises the following steps: Constructing a state space by combining the abnormal event result set and the standby supervision resources; generating a plurality of different supervisory work assignment strategies according to the standby supervisory resources, and constructing an action space with all the supervisory work assignment strategies; Combining the state space and the action space to construct a supervisory work assignment decision model; Taking maximizing the utilization efficiency of supervision resources as the optimization goal, based on the supervision dispatch decision model and using an offline strategy algorithm, iterative optimization is performed to output the optimal supervision dispatch strategy for the power engineering project in the future time period.

10. An intelligent supervision management platform based on multi-terminal interaction and data fusion, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the intelligent supervision management method based on multi-terminal interaction and data fusion as described in any one of claims 1 to 9 is implemented.

Images