Electric power operation risk prediction method based on knowledge graph and large language model

Abstract

The invention provides an electric power operation risk prediction method based on a knowledge graph and a large language model, and relates to the field of electric power operation risk management and control. Constructing a risk knowledge graph based on the multi-dimensional data set; for real-time multi-modal data used for prediction, features of all modal data are extracted in combination with the fine-tuned large oracle model, early fusion is carried out on the features corresponding to part of the modal data, and late fusion is carried out on the result obtained through early fusion and the rest of the modal data to obtain fusion data; inputting the fused data into a risk inference engine based on a deep neural network so as to carry out risk prediction, and carrying out early warning according to a risk prediction result; and performing evaluation feedback on the risk prediction result, and optimizing the risk knowledge graph and the large language model according to the evaluation feedback content. According to the invention, the accuracy, interpretability and intelligent level of electric power operation risk prediction are improved.

Description

Technical Field

[0001] This invention relates to the field of power operation risk management, and in particular to a power operation risk prediction method based on knowledge graphs and large language models. Background Technology

[0002] Currently, the power industry is continuously promoting the transformation of its safety production governance model, but on-site operational safety risk management still faces enormous pressure. Traditional safety supervision and risk pre-control methods heavily rely on the personal experience and sense of responsibility of supervisors to identify and assess risks. However, in actual operations, risk analysis is often fragmented and static. Key information such as work plans, on-site survey records, work permits, and operating procedures are scattered across different independent systems, forming data silos that are difficult to effectively correlate and comprehensively utilize. When conducting pre-event risk identification or in-event supervision, supervisors find it difficult to quickly and comprehensively obtain all historical risk events, similar violation cases, and control measures related to the current work task, environment, and personnel, resulting in insufficient depth and breadth of risk analysis. This manual "puzzle-like" analysis model is not only inefficient but also prone to overlooking risks in complex operational scenarios due to missing information or misjudgment, creating hidden dangers for on-site personnel and equipment safety.

[0003] While current safety management systems incorporate some machine assistance, such as rule-based video analytics systems, the widespread deployment of intelligent sensing devices (e.g., surveillance cameras, recorders, and sensors) at power sites has generated massive amounts of unstructured operational video and image data. However, video analytics systems have limited capabilities, only able to identify limited, clearly defined explicit violations such as "wearing a safety helmet" and "inappropriate clothing." They are ineffective in handling more complex risk scenarios requiring contextual judgment (e.g., violations of work procedures or hidden risk trends in personnel behavior). Therefore, a large amount of video data still relies on manual post-event review, which is labor-intensive and inefficient, failing to achieve real-time risk warnings and proactive intervention.

[0004] Therefore, existing safety management technologies lack semantic understanding of textual knowledge such as operating procedures, making it impossible to reason about implicit risks that violate institutional logic. They are largely limited to numerical superposition of sensor data and threshold judgments, failing to construct a semantic knowledge system to deeply connect heterogeneous elements such as "personnel-equipment-environment-management," resulting in insufficient risk tracing and reasoning capabilities. Secondly, while some technical methods introduce knowledge graphs for knowledge management, they are essentially static retrieval and question-answering systems. Graph construction relies on manual compilation, making it difficult to adaptively learn and evolve from massive amounts of unstructured historical data (such as accident reports and operation logs). Furthermore, they fail to achieve closed-loop integration with natural language processing and visual analysis capabilities, resulting in limited intelligence and an inability to achieve truly cognitive and forward-looking risk prediction. These limitations mean that existing systems still face significant challenges in predictive accuracy, adaptability, and decision support effectiveness when facing the complex and ever-changing environment and high real-time management requirements of power operation sites. Summary of the Invention

[0005] The main objective of this invention is to propose a power operation risk prediction method based on knowledge graphs and large language models, which improves the accuracy, interpretability, and intelligence level of power operation risk prediction.

[0006] This invention is achieved through the following technical solution:

[0007] The power operation risk prediction method based on knowledge graphs and large language models includes the following steps:

[0008] Step S1: Collect multi-source heterogeneous data and preprocess it to construct a multi-dimensional dataset, which includes preprocessed structured data, unstructured text data and real-time non-text data.

[0009] Step S2: Construct a risk knowledge graph based on a multi-dimensional dataset. The knowledge extraction part of the construction process is carried out using a pre-trained big oracle model. Then, the big language model is fine-tuned based on the constructed risk knowledge graph.

[0010] Step S3: For the real-time multimodal data used for prediction, extract the features of each modality data by combining the fine-tuned big oracle model, and perform early fusion of the features corresponding to some modality data. The results of the early fusion are then fused with the remaining modality data in the late stage to obtain the fused data.

[0011] Step S4: Integrate data input into a risk reasoning engine based on a deep neural network to perform risk prediction and issue early warnings based on the risk prediction results;

[0012] Step S5: Evaluate and provide feedback on the risk prediction results, and optimize the risk knowledge graph and large language model based on the evaluation feedback.

[0013] Furthermore, in step S1, the multi-source heterogeneous data includes structured data, unstructured text data, and real-time non-text data; the structured data includes the detailed files of on-site personnel, which are established based on work information, plan information, personnel identity and qualifications, equipment ledgers, training records, personnel qualifications, and historical violation information obtained from the production management system and / or safety risk supervision and control platform; the unstructured text data includes power safety regulations, operating procedures, typical accident case reports, and technical manuals collected from the safety supervision department's document library; the real-time non-text data includes real-time video streams and images generated by on-site deployment balls, smart safety helmets, and recorders accessed through the IoT management platform, as well as time-series data streams uploaded by environmental sensors, including positioning data, alarm data, and communication data.

[0014] Furthermore, in step S1, the preprocessing of the multi-source heterogeneous data includes data cleaning and normalization, text parsing and segmentation, and annotation; cleaning and normalization includes filling or removing missing values ​​and outliers in structured data, and mapping and normalizing data fields from different sources according to a unified standard; text parsing and segmentation includes parsing and extracting text content from PDF and Word format procedural documents, and segmenting them by chapter and paragraph; annotation includes frame extraction, noise reduction, and enhancement processing of the video stream, and annotation of key behaviors in the video stream and images based on historical violation cases.

[0015] Furthermore, in step S2, constructing the risk knowledge graph includes knowledge modeling, knowledge extraction, knowledge fusion, knowledge processing, and knowledge updating. Knowledge modeling enables the mapping of physical entities, entity relationships, and entity attributes. A pre-trained large language model is used to extract entities, relationships, and attributes from unstructured text data, forming knowledge triples. Knowledge fusion employs a referencing disambiguation technique to eliminate fuzzy information and contradictory knowledge in the knowledge triples, resulting in an initial knowledge graph. Knowledge processing performs one-hop and two-hop relation reasoning on the initial knowledge graph using a depth-first traversal approach, and updates the initial knowledge graph based on the reasoning results to obtain the risk knowledge graph. Knowledge updating automatically updates the nodes of the risk knowledge graph and the relationships between nodes and edges through the mapping relationship between new knowledge and entities.

[0016] Furthermore, in step S2, fine-tuning the large oracle model includes data preparation, knowledge injection, and the fine-tuning process. Data preparation includes cleaning the data used for fine-tuning to remove format errors, garbled characters, and duplicate content, and labeling the cleaned data with electricity risk-related information to obtain the fine-tuning dataset. Knowledge injection first extracts electricity risk-related knowledge from the risk knowledge graph using a graph traversal algorithm, and then converts the extracted knowledge into natural language descriptions before integrating it into the fine-tuning dataset. During the fine-tuning process, most parameters of the large language model remain unchanged, and the parameters of the last few layers are updated based on gradient descent using the knowledge-injected dataset.

[0017] Furthermore, in step S3, the multimodal data includes video image data, sensor data, and text data. The video image data refers to the data collected by multiple high-definition cameras deployed at the power operation site during the power operation process; the sensor data refers to the collected power operation site parameters and environmental data; and the text data refers to the data obtained from power operation documents. Convolutional neural networks are used to extract video image features. The sensor data exists in time series form. After normalization processing, the sensor data is used as sensor data features. The fine-tuned large oracle model is used to extract text data features.

[0018] Furthermore, in step S3, the early fusion refers to directly splicing the features of each modality data in the early fusion to obtain new features, and the late fusion refers to using the features of each modality participating in the late fusion to perform risk prediction, and then weighting and fusing the risk prediction results to obtain sub-fused data.

[0019] Furthermore, in step S3, any two of the image / video features, sensor data features, and text data features are first fused in an early stage, and then fused in a late stage with another feature, so that all the sub-fused data are used as the fused data for prediction.

[0020] Furthermore, in step S4, the deep neural network includes a long short-term memory network or a gated recurrent unit; the early warning based on the risk prediction result includes: for high-level risks, simultaneously reminding the operator through sound alarms, SMS notifications, and application interface pop-ups; for medium-level risks, reminding the operator through application interface pop-ups and vibration; for low risks, only displaying prompt information on the application interface, the prompt information including decision suggestions provided based on the risk type and severity level.

[0021] Furthermore, in step S5, a feedback entry is set in the application interface, where operators evaluate and provide feedback on the risk prediction results and decision suggestions, and optimize the risk knowledge graph and large language model based on the evaluation feedback.

[0022] As can be seen from the above description of the present invention, compared with the prior art, the present invention has the following beneficial effects:

[0023] This invention first collects and preprocesses multi-source heterogeneous data to construct a multi-dimensional dataset. A risk knowledge graph is then built based on this dataset, and a large oracle model is fine-tuned based on this knowledge graph. The knowledge extraction process during the construction of the risk knowledge graph utilizes a pre-trained large oracle model. Next, for the real-time multimodal data used for prediction, features of each modality are extracted using the fine-tuned large oracle model. Features corresponding to some modalities are then fused early, and the results of this early fusion are fused late with the remaining modalities to obtain fused data. This fused data is then input into a risk inference engine based on a deep neural network for risk prediction. Early warnings are issued based on the risk prediction results, and the results are evaluated and fed back. The risk knowledge graph and the large language model are then optimized based on the evaluation feedback. By combining the powerful generative capabilities of large language models with the structured knowledge associations of knowledge graphs, semantic understanding and contextual risk reasoning of multi-source heterogeneous data are achieved. This solves the current "data silo" problem, where multi-source heterogeneous information such as textual procedures, video images, and sensor data are isolated from each other and difficult to integrate and utilize, resulting in a single dimension of risk analysis and a lack of a global perspective. It overcomes the shortcomings of existing technologies in lacking deep semantic understanding and contextual reasoning capabilities for complex and hidden risks (such as violations of operational logic and incorrect timing of safety measures). It also solves the problem of poor interpretability of prediction results and difficulty in using them for actual decision-making. This effectively improves the accuracy, interpretability, and intelligence level of power operation risk prediction, achieving an intelligent leap from "human defense" to "technology defense". Attached Figure Description

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Figure 1 This is a flowchart of the present invention.

[0026] Figure 2 This is a schematic diagram of the structure of the present invention.

[0027] Figure 3 A framework diagram for constructing the knowledge graph of this invention.

[0028] Figure 4 This is a flowchart illustrating the knowledge modeling process of this invention.

[0029] Figure 5 This is a flowchart of the knowledge processing of the present invention.

[0030] Figure 6 This is a flowchart illustrating the knowledge update process of this invention.

[0031] Figure 7 This is a flowchart of the recall strategy of the present invention.

[0032] Figure 8 This is an example of the power knowledge search based on knowledge graphs and the Big Prophecy model of this invention.

[0033] Figure 9 This is a flowchart for fine-tuning the grand oracle model of the present invention.

[0034] Figure 10 This is a flowchart of the data fusion process of the present invention.

[0035] Figure 11 This is a flowchart illustrating the feedback optimization process of this invention. Detailed Implementation

[0036] The present invention will be further described below through specific embodiments.

[0037] like Figure 1 and Figure 2 As shown, the power operation risk prediction method based on knowledge graphs and large language models includes the following steps:

[0038] Step S1: Collect multi-source heterogeneous data and preprocess it to construct a multi-dimensional dataset, which includes preprocessed structured data, unstructured text data and real-time non-text data.

[0039] Multi-source heterogeneous data includes structured data, unstructured text data, and real-time non-text data:

[0040] Structured data includes detailed files of on-site personnel, which are established based on operational information, planning information, personnel identity and qualifications, equipment ledgers, training records, personnel qualifications and historical violations obtained from the production management system (PMS) and / or safety risk monitoring and control platform.

[0041] Unstructured text data includes power safety regulations, operating procedures, typical accident case reports, and technical manuals collected from the safety supervision department's document library;

[0042] Real-time non-text data includes real-time video streams and images generated by on-site monitoring balls, smart safety helmets, and recorders accessed through the IoT management platform, as well as time-series data streams uploaded by environmental sensors, including positioning data, alarm data, and communication data.

[0043] Preprocessing of multi-source heterogeneous data includes data cleaning and normalization, text parsing and segmentation, and annotation:

[0044] Cleaning and normalization include filling in or removing missing or outlier values ​​in structured data, and mapping and normalizing data fields from different sources (such as "job type" and "position") according to a unified standard.

[0045] Text parsing and segmentation involves using natural language processing technology to parse PDF and Word format procedure documents, extract text content, and segment them by chapters and paragraphs to prepare for subsequent knowledge extraction.

[0046] The annotation process includes frame extraction, noise reduction, and enhancement of the video stream, and annotation of key behaviors in the video stream and images (such as not wearing a seat belt or insufficient safe distance) based on historical violation cases, forming a high-quality training sample set for the visual model.

[0047] It also established data access and governance standards to form a sustainable data pipeline:

[0048] Establish data access standards: clearly define the format, frequency, interface protocol, and quality requirements for various types of data;

[0049] Building a data lake warehouse: storing processed data in a unified data lake or data warehouse to provide a high-quality, traceable data foundation for knowledge graph construction and large model training;

[0050] Implement data security and privacy protection: De-identify sensitive information (such as personnel ID numbers) during the preprocessing stage and comply with network security regulations to ensure the security of data during collection, transmission, and storage.

[0051] Step S2: Construct a risk knowledge graph based on a multi-dimensional dataset. The knowledge extraction part of the construction process is carried out using a pre-trained big oracle model. Then, the big language model is fine-tuned based on the constructed risk knowledge graph.

[0052] Constructing a risk knowledge graph, such as Figure 3 As shown, it includes knowledge modeling, knowledge extraction, knowledge fusion, knowledge processing, and knowledge updating.

[0053] Knowledge modeling is the process of building the basic model at the schema layer, enabling the mapping of physical entities, entity relationships, and entity attributes, and guiding the construction of the data layer and knowledge graph, such as... Figure 4 As shown;

[0054] A pre-trained large language model is used to extract entities, relations, and attributes from unstructured text data, forming knowledge triples. Entity extraction directly affects the overall performance of the power indicator knowledge graph. Due to the special nature of power indicator data, different indicators require different fields to be extracted, so the extraction of entity information needs to be defined by professionals. At the same time, in order to improve the feature extraction quality of the system and align user needs with the database, a large language model is used for extraction, and the extracted knowledge information is semi-manually reviewed, thereby fully ensuring the quality of the graph knowledge.

[0055] The knowledge triples extracted from knowledge are not yet complete, and their internal logical relationships are still missing. This may lead to knowledge conflicts during retrieval. Knowledge fusion adopts the technique of referential disambiguation to eliminate fuzzy information and contradictory knowledge in the knowledge triples to obtain an initial knowledge graph, thereby improving the quality of the triples and the accuracy of the graph.

[0056] The sheer volume and complexity of power data can easily lead to incomplete knowledge graphs or attribute errors. Therefore, quantitative evaluation of existing knowledge is necessary before adding it to address these errors. The system employs a combination of model recognition and manual review, discarding knowledge with low credibility to ensure the quality of the knowledge graph base. Specifically, knowledge processing includes... Figure 5 As shown, the initial knowledge graph is subjected to one-hop and two-hop relationship reasoning based on depth traversal, and the risk knowledge graph is updated based on the reasoning results.

[0057] Knowledge updates primarily employ a data-driven approach, automatically updating the nodes of the risk knowledge graph and the relationships between nodes and edges by mapping new knowledge to entities. Figure 6 As shown, in order to ensure the consistency of knowledge, when a node changes, the relationships between the nodes connected to it will also change.

[0058] The construction of knowledge subgraphs is a prerequisite for improving the accuracy of the retrieval and analysis system. Therefore, the system employs a specific recall strategy to retrieve knowledge triples and reconstruct the subgraph. Specifically, the indicator names first undergo existence verification. If the indicator exists in the graph, the entity and relation matching rate is checked; otherwise, the system calculates the entity existence rate under different indicators and uses the indicator with the highest probability of existence as the new indicator. When the entity and relation matching rate exceeds a set threshold, entity and relation slots are filled. Finally, data is retrieved and the knowledge subgraph is reconstructed using graph query statements, such as... Figure 7 As shown.

[0059] To improve the accuracy of information extraction, a parameter-based efficient fine-tuning technique is employed to inject knowledge facts from the power sector into the big oracle model, enabling the understanding of user requests and the extraction of structured information. Then, based on the extracted text information, relevant knowledge is retrieved from the background knowledge base. Next, special field information in the knowledge fragments is normalized to accurately recall relevant knowledge fragments. The power index knowledge graph construction module constructs structured knowledge graphs for knowledge information of different structures through knowledge modeling, knowledge extraction, knowledge fusion, knowledge processing, and knowledge updating strategies. Furthermore, to further improve the accuracy of the constructed subgraphs, the system employs a knowledge recall strategy to recall knowledge triples for more accurate subgraph reconstruction. Finally, to better visualize the generated content for users, the report generation module generates customizable response content based on the big oracle model and the professional knowledge information retrieved from the knowledge subgraphs based on user requests. An example of power knowledge search based on knowledge graphs and the big oracle model is shown below. Figure 8 As shown.

[0060] Fine-tuning the big oracle model, such as Figure 9 As shown, the process includes data preparation, knowledge injection, and fine-tuning. Data preparation involves cleaning the data used for fine-tuning to remove formatting errors, garbled characters, and duplicate content. The cleaned data is then labeled with electricity risk-related information to obtain the fine-tuned dataset.

[0061] Specifically, these noises are removed using methods such as regular expressions and data format validation. For example, for data with incorrect date formats, regular expressions are used to match and correct them to the standard date format. Let the original dataset be... The cleaned dataset is The cleaning process can be simply represented as: In the formula, This indicates that a cleaning operation is performed on the data d.

[0062] To enable the model to learn specific knowledge in the power sector, the cleaned data is labeled. Labeling primarily focuses on power risk-related information, such as risk type, risk level, and involved equipment. A combination of manual and semi-automatic labeling tools can be used. For example, using the open-source labeling tool Label Studio, labelers can annotate the text based on their power sector expertise. The labeled dataset can be represented as {( , )},in It is text data. This is the corresponding annotation information.

[0063] During fine-tuning, knowledge injection first extracts electricity risk-related knowledge from the risk knowledge graph using graph traversal algorithms (such as breadth-first search (BFS) or depth-first search (DFS). Then, the extracted knowledge is transformed into natural language descriptions and integrated into the fine-tuning dataset. Specifically, let the knowledge graph be KG, and the extracted and transformed knowledge set be K. The transformation process can be represented as: In the formula, This indicates that relevant knowledge is extracted from a knowledge graph (KG). This involves transforming knowledge triples into natural language descriptions. When integrating them into the fine-tuning dataset, the transformed knowledge is mixed with the original text data in a certain proportion, allowing the model to learn this knowledge during the fine-tuning process. For example, one knowledge description text is inserted for every 10 original text data entries.

[0064] During fine-tuning, most parameters of the large language model remain unchanged, and the parameters of the last few layers are updated based on gradient descent using the knowledge-injected dataset.

[0065] Specifically, the large language model has a multi-layered Transformer structure, which can effectively handle long sequences of text. During fine-tuning, most parameters of the pre-trained model remain unchanged, with adjustments made only to the last few layers. This is based on the idea of ​​transfer learning, utilizing features learned by the pre-trained model on a general language to perform specific optimizations for the power industry. Let the pre-trained model be... The fine-tuned model is The fine-tuning process can be represented as In the formula, This indicates that model M is fine-tuned using dataset D. Stochastic gradient descent (SGD) and its variants (such as Adagrad, Adadelta, Adam, etc.) are used as optimization algorithms to update the model parameters. Taking the Adam algorithm as an example, it adaptively adjusts the learning rate of each parameter during training. Let the model parameters be θ, and the loss function be L(θ). At each iteration t, the parameter update formula is: , , , , In the formula, , , It's the learning rate. .

[0066] Step S3: For the real-time multimodal data used for prediction, extract the features of each modality data by combining the fine-tuned big oracle model, and perform early fusion of the features corresponding to some modality data. The results of the early fusion are then fused with the remaining modality data in the late stage to obtain the fused data.

[0067] Multimodal data includes video image data, sensor data, and text data. Video image data refers to data collected by multiple high-definition cameras deployed at power work sites during power operations. Video image data exists in the form of consecutive frames, and each frame can be represented as a multidimensional matrix. For example, for an RGB image, it can be represented as... , where x and y represent the spatial dimensions of the image, and z represents the color channels (z=3 corresponds to the red, green, and blue channels respectively).

[0068] Sensor data refers to the collected parameters (such as location, electric field, height, etc.) and environmental data (such as humidity, wind speed, etc.) at the power operation site. Sensor data usually exists in the form of a time series. Let the sensor... The data collected at time t is The data set from multiple sensors can be represented as , where n is the number of sensors.

[0069] Text data refers to data obtained from power operation documents (power operation procedures, safety manuals, etc.) and preprocessed (word segmentation, part-of-speech tagging, etc.). Let the text data be T, and the preprocessed text data can be represented as follows: .

[0070] Convolutional neural networks are used to extract video image features. Taking the classic VGG16 network as an example, it extracts image features step by step through multiple convolutional and pooling layers. Let the input image be I. After a series of convolution and pooling operations, the output feature vector is... , can be represented as ;

[0071] Sensor data exists in time series form. After normalization, the sensor data is used as its characteristics. Let the sensor data... The feature vector after feature extraction is ,but ;

[0072] The fine-tuned Big Prophet model is used to extract features from text data. The model outputs feature representations of the text through semantic understanding. Let the preprocessed text data be... The extracted feature vector is ,but .

[0073] Early fusion refers to directly concatenating the features of various modalities into new features. Let the video image feature vector be... Sensor data feature vectors (Merging all sensor features) and text data feature vectors The fused feature vector is for ;

[0074] Late-stage fusion refers to performing risk prediction using the features of each modality participating in late-stage fusion, and then weighting and fusing the results of each risk prediction to obtain sub-fused data. , , and These are the weighted weights, , and This represents the results of risk prediction based on the characteristics of each modality.

[0075] For image / video features, sensor data features, and text data features, any two features are first fused in an early stage, and then fused with the third feature in a later stage. All sub-fused data are used as the fused data for prediction, such as... Figure 10 As shown.

[0076] Step S4: Integrate data input into a risk reasoning engine based on a deep neural network to perform risk prediction and issue early warnings based on the risk prediction results;

[0077] Deep neural networks include Long Short-Term Memory (LSTM) networks or gated recurrent units; using actual risk events as labels y, the fused feature vectors... As input, by minimizing a loss function (such as the cross-entropy loss function) Where N is the number of samples, The model is trained to predict probabilities, and its parameters are updated using backpropagation during training. Let the LSTM model be... The trained model is The training process can be represented as ,

[0078] In practical applications, real-time acquired multimodal data is processed through feature extraction and fusion before being input into a trained model. In this process, real-time risk prediction results for power operations are obtained. .

[0079] Early warnings based on risk prediction results include: for high-level risks, operators will be alerted simultaneously through sound alarms, SMS notifications, and application interface pop-ups; for medium-level risks, operators will be alerted through application interface pop-ups and vibrations; and for low-level risks, only a prompt message will be displayed on the application interface, which includes decision-making suggestions based on the risk type and severity level.

[0080] Risk Information Display: The application interface displays real-time risk information for power operations in an intuitive way. For example, different colored progress bars represent different levels of risk: red for high risk, yellow for medium risk, and green for low risk. Detailed information such as risk description, involved equipment, and probability of occurrence is also listed. Let the risk information set be... ,in This represents a risk information item, displayed in a list format on the interface.

[0081] Visualization: Charts (such as line graphs and bar charts) are used to display the changing trends of risk and the distribution of different types of risk. For example, in a line graph, the horizontal axis represents time t, and the vertical axis represents the risk level. It can be drawn The curve.

[0082] For risk warnings, when the system predicts a risk, it issues warnings in different ways depending on the risk level. For high risk, operators are alerted simultaneously via sound alarms, SMS notifications, and application interface pop-ups; for medium risk, alerts are sent via application interface pop-ups and vibration; for low risk, a prompt message is displayed only on the application interface. Let the risk level be... The early warning method can be represented as: Based on the type and severity of the risk, provide supervisors with corresponding decision-making suggestions. For example, for the risk of personnel operational violations, suggest the correct operating procedures. Let the risk type be... The set of decision recommendations is The decision suggestion generation function can be expressed as: .

[0083] Step S5: Evaluate and provide feedback on the risk prediction results, and optimize the risk knowledge graph and large language model based on the evaluation feedback.

[0084] A feedback entry point is set up in the application interface, allowing operators to evaluate and provide feedback on risk prediction results and decision suggestions. After collecting feedback information, the system analyzes it. If it finds that a certain type of risk is frequently misjudged, the reasons for the misjudgment are analyzed, which may be due to unreasonable model parameter settings or inaccurate related knowledge in the knowledge graph. To address the problem, the model parameters are adjusted or the knowledge graph is updated. The risk knowledge graph and large language model are optimized based on the evaluation feedback. For example, if it is found that the risk of a certain type of operation is frequently overestimated, the weights of features related to that operation type in the model are checked for being too high and adjusted accordingly; or the related knowledge of that operation in the knowledge graph is checked for errors and corrected. Let the model be M and the knowledge graph be KG, the optimization process can be represented as follows: Regularly analyze the system's operational data (including risk prediction results and feedback information) to identify potential problems and areas for improvement. For example, data analysis might reveal low accuracy in risk prediction under certain operational scenarios. Further research into the data characteristics of these scenarios could lead to optimization of feature extraction methods or model structures, continuously improving system functionality. The feedback optimization process is as follows: Figure 11 As shown.

[0085] In this invention, the terms "first," "second," and "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. The use of terms such as "upper," "lower," "left," "right," "front," and "rear" to indicate orientation or positional relationships is based on the orientation or positional relationships shown in the accompanying drawings and is only for the convenience of describing the invention, not to indicate or imply that the device referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation on the scope of protection of this invention. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0086] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0087] The above are merely specific embodiments of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial modifications made to the present invention using this concept shall be considered as infringing upon the protection scope of the present invention.

Claims

1. A power operation risk prediction method based on knowledge graphs and large language models, characterized by: Includes the following steps: Step S1: Collect multi-source heterogeneous data and preprocess it to construct a multi-dimensional dataset, which includes preprocessed structured data, unstructured text data and real-time non-text data. Step S2: Construct a risk knowledge graph based on a multi-dimensional dataset. The knowledge extraction part of the construction process is carried out using a pre-trained big oracle model. Then, the big language model is fine-tuned based on the constructed risk knowledge graph. Step S3: For the real-time multimodal data used for prediction, extract the features of each modality data by combining the fine-tuned big oracle model, and perform early fusion of the features corresponding to some modality data. The results of the early fusion are then fused with the remaining modality data in the late stage to obtain the fused data. Step S4: Integrate data input into a risk reasoning engine based on a deep neural network to perform risk prediction and issue early warnings based on the risk prediction results; Step S5: Evaluate and provide feedback on the risk prediction results, and optimize the risk knowledge graph and large language model based on the evaluation feedback.

2. The power operation risk prediction method based on knowledge graph and large language model according to claim 1, characterized in that: In step S1, the multi-source heterogeneous data includes structured data, unstructured text data, and real-time non-text data. The structured data includes the detailed files of on-site personnel, which are established based on work information, planning information, personnel identity and qualifications, equipment ledgers, training records, personnel qualifications, and historical violation information obtained from the production management system and / or safety risk supervision and control platform. The unstructured text data includes power safety regulations, operating procedures, typical accident case reports, and technical manuals collected from the safety supervision department's document library. The real-time non-text data includes real-time video streams and images generated by on-site surveillance cameras, smart safety helmets, and recorders accessed through the IoT management platform, as well as time-series data streams uploaded by environmental sensors, including positioning data, alarm data, and communication data.

3. The power operation risk prediction method based on knowledge graph and large language model according to claim 2, characterized in that: In step S1, the preprocessing of the multi-source heterogeneous data includes data cleaning and normalization, text parsing and segmentation, and annotation. Cleaning and normalization includes filling or removing missing or outlier values ​​in structured data, and mapping and normalizing data fields from different sources according to a unified standard. Text parsing and segmentation includes parsing and extracting text content from PDF and Word format procedural documents, and segmenting them by chapter and paragraph. Annotation includes frame extraction, noise reduction, and enhancement processing of the video stream, and annotation of key behaviors in the video stream and images based on historical violation cases.

4. The power operation risk prediction method based on knowledge graph and large language model according to claim 1, 2, or 3, characterized in that: In step S2, constructing the risk knowledge graph includes knowledge modeling, knowledge extraction, knowledge fusion, knowledge processing, and knowledge updating; knowledge modeling enables the association mapping of physical entities, entity relationships, and entity attributes. A pre-trained large language model is used to extract entities, relations, and attributes from unstructured text data to form knowledge triples. Knowledge fusion employs a substitution disambiguation technique to eliminate fuzzy information and contradictory knowledge in knowledge triples to obtain an initial knowledge graph. Knowledge processing involves performing one-hop and two-hop relationship reasoning on the initial knowledge graph using a depth-first traversal approach, and updating the initial knowledge graph based on the reasoning results to obtain a risk knowledge graph. Knowledge updates automatically update the nodes of the risk knowledge graph and the relationships between nodes and edges by mapping new knowledge to entities.

5. The power operation risk prediction method based on knowledge graph and large language model according to claim 4, characterized in that: In step S2, fine-tuning the big oracle model includes data preparation, knowledge injection, and fine-tuning processes. Data preparation includes cleaning the data used for fine-tuning to remove format errors, garbled characters, and duplicate content, and labeling the cleaned data with power risk-related information to obtain the fine-tuned dataset; Knowledge injection first extracts electricity risk-related knowledge from the risk knowledge graph using a graph traversal algorithm, and then integrates the extracted knowledge into the fine-tuning dataset after converting it into a natural language description. During fine-tuning, most parameters of the large language model remain unchanged, and the parameters of the last few layers are updated based on gradient descent using the knowledge-injected dataset.

6. The power operation risk prediction method based on knowledge graph and large language model according to claim 1, 2, or 3, characterized in that: In step S3, the multimodal data includes video image data, sensor data, and text data. The video image data refers to the data collected by multiple high-definition cameras deployed at the power operation site during the power operation process; the sensor data refers to the collected power operation site parameters and environmental data; and the text data refers to the data obtained from power operation documents. Convolutional neural networks are used to extract video image features. The sensor data exists in time series form. After normalization processing, the sensor data is used as sensor data features. The fine-tuned large oracle model is used to extract text data features.

7. The power operation risk prediction method based on knowledge graph and large language model according to claim 6, characterized in that: In step S3, early fusion refers to directly splicing the features of each modality data in early fusion to obtain new features, and late fusion refers to using the features of each modality participating in late fusion to perform risk prediction, and weighting and fusing the risk prediction results to obtain sub-fused data.

8. The power operation risk prediction method based on knowledge graph and large language model according to claim 7, characterized in that: In step S3, any two of the image / video features, sensor data features, and text data features are first fused in an early stage, and then fused in a late stage with another feature, so that all the sub-fused data are used as the fused data for prediction.

9. The power operation risk prediction method based on knowledge graph and large language model according to claim 1, 2, or 3, characterized in that: In step S4, the deep neural network includes a long short-term memory network or a gated recurrent unit; the early warning based on the risk prediction results includes: for high-level risks, reminding operators simultaneously through sound alarms, SMS notifications, and application interface pop-ups; for medium-level risks, reminding operators through application interface pop-ups and vibrations; for low-risk risks, only displaying prompt information on the application interface, which includes decision suggestions based on the risk type and severity level.

10. The power operation risk prediction method based on knowledge graph and large language model according to claim 1, 2, or 3, characterized in that: In step S5, a feedback entry is set in the application interface, where operators evaluate and provide feedback on the risk prediction results and decision suggestions, and optimize the risk knowledge graph and large language model based on the evaluation feedback.

Images