Electric power operation safety disclosure intelligent analysis system and method based on multi-source data fusion

Abstract

The invention discloses an electric power operation safety disclosure intelligent analysis system and method based on multi-source data fusion. The system comprises a display layer, a service access layer, a gateway isolation layer, a service application layer and a data persistence layer. The display layer completes a user interaction function; the service access layer is responsible for processing a request from the display layer, interacting with the back-end service layer, receiving and processing the request of the display layer and calling back-end service interface data format conversion and adaptation, and the service access layer serves as a bridge between the display layer and back-end service; the gateway isolation layer is responsible for forwarding request routing, load balancing, authentication and authorization; log recording and monitoring of request routing and load balancing authentication and authorization requests and responses; according to the service application layer, JTA is adopted for transaction control in the service application layer; and the data persistence layer is used for accessing data sources including the relational database and converting the data into Java objects to be called by programs of other layers. According to the invention, the risk identification capability is obviously improved.

Description

Technical Field

[0001] This invention belongs to the field of power grid system technology, and in particular relates to an intelligent analysis system and method for power operation safety briefing based on multi-source data fusion. Background Technology

[0002] Safety briefings for power operations are a crucial step in ensuring the safety of power grid operation and maintenance. However, current on-site safety briefings are often perfunctory, with inadequate risk disclosure, heavy reliance on paper documents and verbal communication, resulting in problems such as delayed information transmission, missed signatures, and significant deviations in execution. While technologies such as artificial intelligence (AI) are developing and systems based on intelligent semantic recognition, voice transcription analysis, and video intelligent analysis are becoming increasingly widespread, current operations still face the following limitations: manual inspections rely on experience-based judgment, resulting in a safety hazard identification accuracy rate of less than 70%; and existing systems cannot dynamically assess the impact of changes in the work environment on safety risks.

[0003] The existing information system for safety management in power construction has achieved intelligent management of some aspects by solidifying work processes and setting technical lock-in points. During construction, the information system enables online transmission and recording of safety briefings, ensuring that the briefing content is complete and traceable. However, it still has the following limitations: the content of the briefing recordings does not match the work content by more than 60%; the data is limited, mainly relying on work ticket texts and personnel location data, and lacks integration of multi-source data such as work videos and environmental parameters.

[0004] To address the various issues arising during safety briefings, improve manual work efficiency, and prevent and reduce safety accidents, achieving safer and more automated power grid operations and further promoting digital transformation have become urgent problems to be solved. Summary of the Invention

[0005] The purpose of this invention is to solve the problems mentioned in the background art and to propose an intelligent analysis system and method for safety briefing of power operations based on multi-source data fusion.

[0006] To achieve the objectives of this invention, this invention provides an intelligent analysis system for safety briefings in power operations based on multi-source data fusion. The system architecture includes five main categories: presentation layer, service access layer, gateway isolation layer, business application layer, and data persistence layer. The technical solutions used at different architecture layers are also quite different.

[0007] The presentation layer, or front-end UI, primarily handles user interaction. Data display uses JavaScript to generate HTML pages, user input uses HTML controls, special operations and page effects are implemented using JavaScript, and CSS is used to style the pages. Data submitted to the server is submitted via AJAX to improve user-friendliness and is then processed by the backend controller class. The Vue.js front-end framework is used to render the pages, and user interaction uses Element controls.

[0008] Service access layer:

[0009] The service access layer is responsible for handling requests from the presentation layer and interacting with the backend service layer. It receives and processes requests from the presentation layer, calls backend service interfaces, and performs data format conversion and adaptation. The service access layer acts as a bridge between the presentation layer and the backend service.

[0010] Gateway isolation layer:

[0011] The gateway isolation layer is responsible for request routing, load balancing, authentication, and authorization; it ensures that requests reach the backend services correctly and securely. This includes request routing, load balancing, authentication, authorization, request and response logging, and monitoring.

[0012] Business application layer:

[0013] Transaction control is implemented at the business application layer using JTA. Each method in the business logic layer constitutes a database transaction. A business logic layer method calls one or more DAO methods, and the DAO operations are performed within a single transaction, using a transaction manager provided by Spring. The business application layer includes the following applications:

[0014] SMS platform management: Used to send warning SMS messages to operators;

[0015] Artificial intelligence platform management: Uses intelligent AI algorithms to process and analyze alarm-related data;

[0016] Safety risk control platform management: Obtain risk control operation plan data;

[0017] OCR Intelligent Recognition Management: Intelligently recognizes image content, analyzes and processes relevant data in the image;

[0018] Job planning management function: The function of displaying and querying job planning related functions is realized through the writing of code logic.

[0019] Personnel qualification management function: Through work plan data, obtain images of personnel's high-altitude operation qualification certificates, integrate OCR intelligent recognition technology, and then remove the relevant content of the qualification certificates.

[0020] Intelligent speech recognition and analysis function: It extracts acoustic features through FBANK and audio spectrogram, signal preprocessing technology, uses Transformer architecture and deep neural network (DNN) for acoustic depth modeling, and combines neural network language model (BERT / GPT) and Viterbi algorithm to improve the accuracy and efficiency of complex sentence recognition.

[0021] Intelligent speech translation analysis function: Normalizes text through common rule templates (regular expressions) or seq2seq models, domain-dependent dictionaries and contextual reasoning, and performs real-time analysis, processing and transcription of speech using natural language processing (NLP) and natural language understanding (NLU).

[0022] The semantic intelligent verification function is as follows: By comparing the speech recognition text with the OCR text in the same scene or the original content text through cross-source comparison, the consistency is judged by the text similarity (such as cosine similarity and BLEU score). When the difference is too large, manual verification is triggered.

[0023] Safety briefing and alarm information display function: Based on a visual dashboard and web interface, alarm data is pushed, summarized and statistically analyzed, and listed for query in real time.

[0024] Data persistence layer:

[0025] The data access layer is responsible for accessing data sources, including relational databases, and converting the data into Java objects for other layers to call. For general data maintenance functions, there is no need for a separate DAO design; the publicly provided data access services can be called directly in the service. For data objects shared by multiple modules or data objects that provide external interfaces, it is recommended to encapsulate them into more easily understandable objects using JavaBeans. MyBatis is used for data persistence operations.

[0026] This application also provides an intelligent analysis method for power operation safety briefing based on multi-source data fusion. The method is implemented based on the system provided in this application and includes: operation plan analysis and statistics, intelligent parsing and feature extraction of multi-source data, multi-dimensional data fusion and verification comparison, and alarm visualization display based on a single power grid map.

[0027] Step 31, Work plan analysis and statistics;

[0028] By calling the power grid data platform interface and using RPA automation tools to obtain the source data of the work plan in the safety management system, and then analyzing, filtering and cleaning it to obtain the final key business data, including safety briefing content, audio files, and images of the qualification certificates of the operators, the key business data is pushed to the source data analysis module in real time, along with the data retrieval timestamp and data integrity verification code to ensure that the source data is traceable.

[0029] Clearly define the data retrieval frequency and the range of key fields, including job ID, job location, planned time, person responsible for briefing, and associated qualification certificate number;

[0030] Step 32, Intelligent parsing and feature extraction of multi-source data:

[0031] The obtained briefing content and work permit image content are processed and obtained through two parallel threads: the audio intelligent transcription thread and the OCR qualification certificate recognition thread.

[0032] Intelligent transcription thread for audio recordings: Automatically calls the speech recognition (ASR) service, enables noise reduction and semantic focus algorithms, and converts the safety briefing recordings into text; subsequently, the natural language processing (NLP) engine performs deep semantic analysis on the text to extract key elements, including work location, safety measures, risk points, and personnel division of labor;

[0033] OCR Certificate Recognition Thread: Automatically calls the OCR recognition service to scan the qualification certificates of operators. It not only recognizes text information, including name, certificate number, job category, and validity period, but also extracts feature information, including the outline of the issuing authority's seal, the pattern of the official watermark, and the pixel features of the photo, for anti-counterfeiting verification.

[0034] Real-time push notifications, with confidence level processing included.

[0035] Step 33, Multidimensional Data Fusion and Verification Comparison:

[0036] Safety briefing content verification: Compare the semantic results of the safety briefing recordings with the written work plans and standard safety procedures in the risk control platform to establish key information matching rules; based on the established rules, extract the corresponding content and perform full matching, set matching, and semantic matching according to business needs;

[0037] Personnel operation qualification certificate compliance verification: The personnel qualification certificate information data extracted by OCR, namely the certificate number and validity period, is connected to a specific special operation qualification certificate database through interface API calls to compare the data in real time, determine whether there is an expiration, forgery, or mismatch between the certificate type and the operation, and generate a compliance conclusion;

[0038] The two verification results are linked with the initial work plan data to form a comprehensive safety assessment report. The conclusion of whether the verification passes or fails is pushed to the alarm display module on the large screen in real time to provide a basis for alarm classification.

[0039] 34. Visualize alarms based on a single power grid diagram:

[0040] Alarm classification: The alarm information is geospatialized; on the power grid geographic information map, the location of the substation and line tower where the current operation plan is located is located, and the alarm is divided into three levels according to the severity of the risk: red is emergency, yellow is alert, and blue is notification. Different levels correspond to different flashing frequencies: red flashes once per second and yellow flashes once every 3 seconds.

[0041] Source tracing interaction: Clicking on any alarm item will take you to the verification details page, which displays the data source, comparison process, and cause of the anomaly. It also supports administrators to export anomaly reports with one click.

[0042] The significant advancement of this invention compared to existing technologies lies in:

[0043] 1. Significantly improved risk identification capabilities:

[0044] By employing data fusion diagnostics, the false positive rate was reduced to 3.2% (a 67% reduction compared to single video analysis).

[0045] It facilitates information retrieval and editing, and improves data processing efficiency.

[0046] 2. Optimize work efficiency:

[0047] The task handover identification is performed using an AI-powered voice model, which improves work efficiency by 10 times compared to traditional manual inspection.

[0048] The response time for handling anomalies is ≤8 seconds, which is 87.5% more efficient than the traditional 70-second manual inspection.

[0049] 3. Reduced safety management costs

[0050] Reduce on-site inspection manpower by 30%.

[0051] To more clearly illustrate the functional characteristics and structural parameters of the present invention, further explanation is provided below in conjunction with the accompanying drawings and specific embodiments. Attached Figure Description

[0052] Figure 1 This is a system architecture diagram provided in the embodiments of this application;

[0053] Figure 2 This is a functional design flowchart provided in the embodiments of this application;

[0054] Figure 3 This is a flowchart of the job plan acquisition process provided in an embodiment of this application;

[0055] Figure 4 This is a flowchart of the process for obtaining personnel qualification image information provided in an embodiment of this application;

[0056] Figure 5This is a personnel qualification classification alarm statistics chart provided in the embodiments of this application;

[0057] Figure 6 This is a flowchart of the security briefing alarm display and push notification provided in the embodiments of this application;

[0058] Figure 7 The diagram shows the ER structure provided in the embodiments of this application. Detailed Implementation

[0059] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] The system architecture diagram provided in this application is shown in Figure 1:

[0061] Part 1: System Architecture Introduction

[0062] The system architecture diagram shows a five-layer structure, from bottom to top: database persistence layer, business application layer, gateway isolation layer, service access layer, and page presentation layer. The database persistence layer consists of components on the local server, providing underlying technical support for the upper layers. The business application layer integrates multi-source data services, such as AI platforms, security risk control platforms, and OCR intelligent systems, to build general data storage and computing capabilities, providing data service support for services. The gateway isolation layer accesses internal applications through configuration, distribution, and monitoring applications. The service access layer, based on the abstraction and classification of business functions, builds services that implement various complex business logics, providing rapid business response capabilities. The page presentation layer provides users with a visual way to interact with the system.

[0063] The specific implementation process of the technical solution is based on the system architecture diagram, and the process is described as follows:

[0064] The presentation layer is used for web-based applications and large-screen visualization services.

[0065] This project utilizes front-end technologies to implement user data interaction functionality. Data display primarily uses JavaScript to generate HTML pages, user input uses HTML controls, and some special operations and page effects are implemented using JavaScript. CSS is used to style the pages. Data submitted to the server is submitted via AJAX to improve user-friendliness and is then processed by the backend controller class. The project uses the Vue.js front-end framework to render the pages, allowing users to query current business data and stay informed about operational status and safety briefings through web browsers and visual dashboards.

[0066] Service Access Layer

[0067] The service access layer is responsible for handling requests from the presentation layer and interacting with the backend service layer. It receives and processes requests from the presentation layer, calls backend service interfaces, and performs data format conversion and adaptation. It acts as a bridge between the presentation layer and the backend services. Components are the smallest unit of presentation, and unified specifications are established for component development, mainly including: development language, component lifecycle, and public methods.

[0068] Gateway isolation layer

[0069] The R&D team deployed an Nginx service to handle request routing, load balancing, authentication, and authorization. This layer ensures that requests reach the backend services correctly and securely. It also includes request routing, load balancing, authentication, authorization, request and response logging, and monitoring.

[0070] Business application layer:

[0071] The Java logic code implements the calls to various multi-source data interfaces, data processing, data analysis, and logical judgments. Data transmission between various business services uses JSON key-value pairs, and transaction control ensures the atomicity and consistency of each interface. It provides unified and stable support for the top-level application. Specific services and functions are as follows:

[0072] Data synchronization service: Provides message synchronization function for interaction between various subsystems in the platform. For example, by calling multiple data sources such as the safety management and risk control system and the power grid data platform system, the operation plan data can be synchronized and stored, which is convenient for operation personnel to manage and query.

[0073] SMS service: The platform uses the SMS system to provide push message functionality, sending relevant alarm data to the mobile phones of relevant personnel in a timely manner for easy access.

[0074] File service: Provides functions such as long-distance transfer, download, viewing, and modification of unstructured files within the platform.

[0075] Log service: Provides a unified log management service for the platform through Spring Boot and AOP aspect technology.

[0076] Map service: By calling the power grid map system, a display platform can be provided for subsequent work site location display, safety briefing and risk warning push of work qualification certificates.

[0077] Job planning management function: The function of displaying and querying job planning related functions is realized through the writing of code logic.

[0078] Personnel qualification management function: Through work plan data, obtain images of personnel's high-altitude operation qualification certificates, integrate OCR intelligent recognition technology, collect and recognize text data in the images, verify the date of the qualification certificate, and summarize the qualification certificates in statuses such as normal, near expiration, expired, and invalid. Personnel qualification details data can be viewed through the list.

[0079] Intelligent speech recognition and analysis function: It extracts acoustic features through FBANK and audio spectrogram, signal preprocessing technology, performs acoustic depth modeling using Transformer architecture and deep neural network (DNN) technology, and improves the accuracy and efficiency of complex sentence recognition by combining neural network language model (BERT / GPT) and Viterbi algorithm.

[0080] Intelligent speech transcription analysis function: The text is normalized by using common rule templates (regular expressions) or seq2seq models, domain-dependent dictionaries and contextual reasoning. Natural language processing (NLP) and natural language understanding (NLU) technologies are used to analyze, process and transcribe speech in real time, and the transcribed data is stored in the database and queried.

[0081] The semantic intelligent verification function is as follows: By comparing the speech-recognized text with the OCR text in the same scene or the original text, the consistency is judged by the text similarity (such as cosine similarity and BLEU score). When the difference is too large, manual verification is triggered and the results are quickly responded to and relevant alarm data is generated.

[0082] Safety briefing alarm information display function: Based on the visualization screen and web terminal, the alarm data with low matching degree between the recording and the work content is queried, statistically displayed, and the power grid map service is called to highlight and push the work location of the alarm on the map.

[0083] Data persistence layer:

[0084] The cache database version is Redis 6.2, deployed in a 3-master, 3-slave configuration to ensure high availability. It is mainly used to store user login information, token data, encrypted signatures, and other data.

[0085] Database type and version: Version: MySQL 8.0

[0086] The basic principles of database design are as follows:

[0087] It uses the utf8mb4 encoding format, which is a format unique to MySQL and is generally considered to be UTF-8.

[0088] The primary key for each table, PrimaryKey, uses an auto-incrementing Long type. For columns requiring uniqueness, a unique index is used to ensure this.

[0089] All data involving character types should use the varchar type, not char. All names and encodings are limited to varchar(255), corresponding to a maximum of 255 characters on the front end.

[0090] The table name and column names are defined using all lowercase letters.

[0091] The data tables in this system are divided into three categories:

[0092] The system's basic permission classes include menu tables, user tables, and role tables.

[0093] System operation tables: including login logs, operation logs, etc.

[0094] Business-related documents: Work plan sheets, safety briefing attachment sheets, intelligent safety equipment sheets, alarm information sheets, operator sheets, personnel qualification information sheets, safety briefing information sheets, work trajectory sheets, equipment ledger basic information sheets, etc.

[0095] like Figure 7 As shown, this is the table structure of the ER part of the current system's database.

[0096] II. Key Technologies at Each Layer of the System:

[0097] 2.1 Important operations in the presentation layer:

[0098] The presentation layer, as the direct entry point for user interaction with the system (web management, visual dashboards), needs to closely align with the specific characteristics of power industry operational scenarios while also ensuring the stability and ease of use of the technical implementation. The following explanation elaborates on this from a technical perspective, using specific scenarios as examples:

[0099] 2.1.1 Power Grid One-Diagram Bottom-Level Access:

[0100] The power grid map serves as the core carrier of the digital twin of the power system, realizing a comprehensive digital mapping of the entire medium and low voltage distribution network, from substation outgoing switches to user meter switches. The underlying access technology is the fundamental support of the system, mainly including multi-source data access and fusion, real-time data rendering, and computational optimization.

[0101] 2.1.1.1 Multi-source data access and fusion

[0102] Automatic Device and Topology Mapping: In data visualization application scenarios, based on the concept of a single power grid map, the system uses IoT sensing technology to achieve automatic device registration and automatic topology generation. When the outgoing switch of a substation is closed to supply power, the medium and low voltage node devices will be energized step by step, and the device registration information and high-precision geographic coordinates will be automatically uploaded, and geographic maps and single-line maps will be automatically generated. This method completely changes the traditional method of relying on manual drawing, manual configuration of point tables and set parameters, and solves the problems of low efficiency, high cost and easy error in map generation caused by the large number of distribution network points, complex network structure, frequent line switching and equipment changes.

[0103] Multi-source heterogeneous data integration: The system needs to integrate data from the Equipment Asset Lean Management System (PMS3.0), the Safety Risk Control System Platform, the Distribution Management System (DMS), and the Business Middle Platform System. The data includes equipment operation and maintenance data, work plan data, substation and line power monitoring and equipment operation information, power grid equipment ledger information, asset information, defect and fault information, maintenance information, and real-time and historical operation data. Through topology fusion, the system integrates the power grid's internal topology and interconnections, the physical connections between users and the power grid, and the specific installation locations of various sensors and data acquisition devices in the power grid, thus connecting the topological connections between power grids of different voltage levels and forming a single power grid topology map.

[0104] 2.1.1.2 Real-time data rendering and computation optimization:

[0105] Distributed rendering and load balancing: In real-time page rendering and display applications, a single power grid map involves massive data visualization, requiring distributed rendering to ensure system smoothness. The system distributes rendering tasks across multiple computing nodes, using load balancing algorithms to avoid single-point overload. For large-scale power grid data, a dynamic hierarchical loading strategy is adopted, automatically loading data of different precision based on the view zoom level and region to ensure user experience. When users browse the panorama, the system displays a simplified version of the power grid topology. When users focus on a specific area, the system automatically loads detailed equipment information and connection relationships for that area.

[0106] Spatiotemporal Data Management: Based on graph databases and graph computing, the system constructs a spatiotemporal data management system. It leverages the structural consistency, natural expression, and intuitive display characteristics of graph data structures with the actual power grid. The system integrates the overall topology of the power grid, including the internal topology of all links such as transmission, transformation, and distribution, as well as the connection relationships between them, to form a single graph of the power grid topology. It also integrates data from different management perspectives, organizational methods, time and space dimensions, and scales from various business departments of power grid operation, monitoring, maintenance, production, and marketing.

[0107] Computational optimization strategies: When dealing with a large amount of data from distribution network equipment operation, the system employs multiple computational optimization strategies, including: 1. Hierarchical data storage: storing hot, warm, and cold data on storage media with different performance levels; 2. Incremental computing: only updating changed data to avoid the overhead of full-scale computing; 3. Edge computing: processing data near the data source to reduce data transmission latency and bandwidth pressure; 4. Pre-computation and caching: pre-computing and caching frequently used query and analysis results to accelerate data retrieval speed.

[0108] 2.1.2 Secure encryption of page data interaction:

[0109] In multi-source power grid data fusion and sharing application scenarios, security and privacy protection are crucial considerations. The system needs to employ multi-layered and comprehensive security encryption technologies to ensure the security of data during transmission, storage, and access, especially for sensitive business content such as security briefings.

[0110] 2.1.2.1 Transport layer security encryption mechanism:

[0111] Attribute-Based Encryption-Based Secure Data Sharing Algorithm: In scenarios involving access to specific security briefing content in work plans, the system employs an Attribute-Based Encryption (ABE)-based secure data sharing algorithm (AEDS). This algorithm enhances data security through fuzzy access strategies and utilizes a linear key-sharing strategy, supporting arbitrarily monotonic access structures and thus improving the expressibility of access policies. Attribute encryption algorithms are particularly suitable for smart grid environments because they enable fine-grained access control, allowing only users meeting specific attribute conditions to decrypt and access data. For example, only users with attributes such as "Security Administrator," "Region = East China," and "Position = Maintenance Work Supervisor" can access specific briefing content.

[0112] Hybrid Encryption Mechanism: In access control log application scenarios, the system employs a hybrid encryption mechanism, combining the advantages of symmetric and asymmetric encryption. First, a symmetric encryption algorithm (such as AES-256) is used to encrypt the power data. Then, the symmetric key is encrypted using an attribute encryption algorithm, and the access policy and keywords are encrypted and uploaded to the consortium blockchain together. This ensures both the performance of big data encryption and flexible access control. Only data users who meet the access policy and match the keywords can access the power data.

[0113] Sensitive data transmission protection: In terms of security briefing recording data processing, the AES-256-GCM algorithm is used for end-to-end encryption, and the key is dynamically generated through Diffie-Hellman key exchange; in terms of spatial coordinate data processing, coordinate offset encryption is implemented, and the data is automatically decrypted and restored when displayed on the front end; in business data transmission, field-level encryption is used, and fields including operator information and equipment parameters are encrypted and transmitted separately.

[0114] 2.1.2.2 Data storage and interaction security:

[0115] Client-side data storage: Local data encryption is implemented using a Web Crypto API, with keys generated and stored via a hardware security module; a tiered data storage strategy is implemented.

[0116] Highly sensitive data (such as complete voice recordings): not stored locally, but temporarily cached in memory and automatically cleared after the session ends;

[0117] Sensitive data (such as disclosure confirmation records): Encrypted storage and set to automatically destroy after 7 days;

[0118] Low-sensitivity data (such as the unit to which the workers belong): can be cached for a long time but contains watermark information;

[0119] Interactive operation security protection: In critical operations such as confirmation of information, intelligent voice parsing, and early warning risk query, a two-stage encryption verification is adopted, requiring the input of a dynamic password (generated through a special power token); a unique nonce value is generated for each interactive request, with a lifespan of 30 seconds (in cryptography, a nonce is an arbitrary or non-repeating random value that is used only once), and it expires immediately after the server verifies it.

[0120] Front-end code protection: The encrypted data in the system, including equipment ledgers, work plan information, work location, and safety briefing data, adopts code obfuscation and dynamic loading technology to prevent reverse engineering to obtain the encryption algorithm; the code is detected by hash value comparison to detect whether it has been tampered with, and a security alarm is triggered immediately when an anomaly is found to achieve runtime integrity verification.

[0121] 2.1.3 Business relevance of page security briefing:

[0122] Safety briefings are a critical safety step in power operation procedures, aiming to improve workers' safety awareness and risk prevention capabilities through the dissemination of work content. The intelligent safety briefing system based on a unified power grid map organically combines intelligent voice recognition, spatial positioning analysis, and risk early warning technologies, achieving visualization, traceability, and analyzability of safety briefing content.

[0123] 2.1.3.1 Mapping of safety briefing content with power grid elements:

[0124] Space and Business Dual Association Mechanism: The system establishes a connection between safety briefing content and specific graphic elements on the power grid map through a dual association mechanism of spatial location and business attributes; based on high-precision geographic coordinate information, the system automatically associates safety briefing content with operating equipment or areas; at the same time, through power grid topology analysis, the system automatically identifies the scope of operation impact and associates safety briefing content with relevant equipment affected by the operation, ensuring that safety measures cover all affected areas;

[0125] Dynamic Operation Safety Boundary Analysis: Based on the power grid topology connection relationship, the system dynamically analyzes the operation safety boundary and automatically generates a safety measure layout plan; for line operations, the system automatically determines the areas that need to be de-energized, the locations that need to be grounded, and the warning ranges that need to be set according to the topology analysis, and displays them visually on a power grid map; the system supports simulating the effects of safety measures, discovering potential safety blind spots in advance, and optimizing the safety measure layout plan.

[0126] Visualization of briefing effectiveness evaluation: The briefing quality index calculation model is designed, including indicators such as comprehensive coverage, accuracy, and detail. The briefing quality of different areas is displayed on the map using a heat map. The briefing situation of the same area at different times is overlaid on the map, and the time axis can be slid to view the trend of change, so as to realize trend comparison.

[0127] 2.1.4 Summary of Technological Innovations in the Presentation Layer:

[0128] This presentation layer technical solution revolves around the core carrier of a unified power grid map. Through innovative multi-source data access technology, multi-layered security encryption mechanisms, and deeply integrated business association technology, it achieves seamless integration of voice recognition and spatial information. This provides an intuitive, efficient, and secure visualization solution for safety briefings in power operations. The core innovation of the solution lies in constructing a "voice-space-business" ternary fusion model. Through dynamic association algorithms and security encryption mechanisms, it solves the problem of real-time data fusion and display in the special environment of the power industry. At the same time, through multi-terminal differentiation technology, it meets the usage needs of different scenarios, providing strong technical support for the safety management of power operations.

[0129] 2.2 Key Operations at the Business Application Layer:

[0130] The business application layer integrates multi-source power grid data and utilizes large-scale artificial intelligence models, OCR recognition, and voice analysis to intelligently parse safety briefing content, automatically verify personnel qualifications and the consistency of briefing recordings, and achieve accurate identification and real-time early warning of operational risks. This information is then visualized on a single power grid map. The following is a step-by-step analysis of the technical methods used in the business layer:

[0131] 2.2.1 Intelligent Recognition and Analysis of Disclosure Content Based on the Guangming Big Data Model:

[0132] In the scenario of intelligent identification of safety briefing for work plans, intelligent technology is used to automate, standardize and deeply analyze the briefing content, covering four core business dimensions: work task matching, completeness of risk point identification, compliance of prevention and control measures and accuracy of equipment association, providing a basis for judging the compliance of work plans.

[0133] 2.2.1.1 Synchronous Analysis of Work Plan Data:

[0134] By using the power grid data platform and automated RPA tools, the source data of the work plan in the safety management system is obtained, analyzed, filtered, and cleaned to obtain the final business data (such as the core safety briefing content), which provides data support for subsequent intelligent analysis.

[0135] 2.2.1.2 Corpus Construction and Analysis of Disclosure Content:

[0136] Collect power grid safety regulations, work instructions, historical briefing documents, accident case reports, and electronic briefing documents related to work plans, including work location, voltage level, work type, risk point description, prevention and control measures, and associated equipment IDs; to form a dedicated corpus for training and fine-tuning the Guangming large model.

[0137] 2.2.1.3 Business Element Extraction and Model Training:

[0138] The system uses a pre-trained Guangming large model based on the Transformer architecture as its foundation, and specializes the power grid through two methods: domain-adaptive pre-training and downstream task fine-tuning.

[0139] Key element extraction and training: The training model identifies entities in the text, including work location (110kVXX substation), work type (high-altitude work, cable laying), safety measures (voltage testing, grounding wire installation), personnel roles (work supervisor, dedicated supervisor), risk point types (electric shock, fall from height, equipment damage, etc., 18 categories), risk level (Level 1 / Level 2 / Level 3), prevention and control measures (such as "voltage testing before work", "wearing double-hook safety belt"), equipment status requirements (such as "circuit breaker open" "grounding switch closed"), etc., using BiLSTM-CRF or Transformer-based sequence labeling models;

[0140] 2.2.1.4 Multi-dimensional business compliance verification:

[0141] Task matching analysis: Compare the extracted basic task information with the task information in the task planning system to verify whether the task corresponding to the briefing content is consistent with the approved task plan. Check whether the voltage level and task location match (avoid mismatches such as "planned task is a 10kV line, briefing content is a 35kV line"). The matching accuracy rate should be above 99.5%.

[0142] Risk Point Identification Completeness Analysis: Based on the risk point association reasoning capability of the Guangming Big Data Model, and through training with historical violation cases, a mapping relationship between work type and risk points is established. For example, "high-altitude tower work must be associated with 'fall from height' and 'object strike' risks," determining whether the briefing content has omitted any mandatory risk points. For instance, for 220kV transmission line maintenance work, the model automatically retrieves the standard risk point list (6 items in total). If the briefing document only mentions 4 items, it marks "risk point missing" and outputs the name of the missing risk point and the relevant safety regulations clause.

[0143] Equipment association accuracy: Associate the equipment ID in the briefing with the equipment ledger in the power grid GIS system (Geographic Information System) to verify whether the equipment exists (avoid "associating with decommissioned equipment") and whether the equipment status requirements match the actual site conditions (e.g., "requires the circuit breaker to be open" but the GIS system shows that the circuit breaker is in the closed state, then mark "equipment status requirement conflict").

[0144] 2.2.2 Intelligent Verification of Operator Qualifications Based on OCR:

[0145] In power grid high-altitude operations, workers must hold a valid "High-Altitude Work Permit." Traditional qualification verification relies on manual checking of permit photos, validity periods, and driving license types, which presents three major pain points: low efficiency, high error rates, and difficulty in traceability (paper verification records are easily lost and cannot be linked to work plans). The business layer needs to utilize OCR technology to automate the extraction, real-time verification, and traceability of qualification information, ensuring "person and document match, qualification valid, and driving license compatible," thereby mitigating the risk of personnel qualification violations from the source.

[0146] 2.2.2.1 Collection and Preprocessing of Qualification Documents:

[0147] Image acquisition adaptation: Supports two acquisition methods: mobile devices (mobile phone shooting at the work site) and fixed terminals (scanning in the business hall, downloading from the official website of the special operation qualification certificate). Addressing issues such as uneven lighting at power grid sites (e.g., strong indoor light in substations, backlighting at transmission line sites), document creases (wear and tear during handling by workers), and background interference (e.g., equipment logos in the background when holding the document), the following technologies are employed:

[0148] Lighting correction: Based on the Retinex algorithm, uneven lighting is eliminated, and the standard deviation of brightness in the document area is controlled within 5 (to ensure text clarity).

[0149] Image denoising: An adaptive denoising algorithm based on wavelet transform is used to filter out equipment operating noise (such as image jitter and blur caused by transformer hum).

[0150] Edge detection and cropping: The Canny algorithm is used to identify the edges of the document and automatically crop the background area to ensure that only the valid content of the document is retained (avoiding background text from interfering with the recognition).

[0151] 2.2.2.2 Key Information Extraction (OCR):

[0152] For the application scenario of collecting qualification information of special operation personnel in the power grid, a high-precision OCR engine based on CNN+CTC (such as PaddleOCR, EasyOCR or self-developed model) is used to perform text recognition on the corrected image; the focus is on collecting name, certificate number, issuing authority, validity period and operation category;

[0153] 2.2.2.3 OCR-based intelligent recognition and verification:

[0154] Name matching: The certificate holder's name extracted by OCR is precisely matched with the operator's name in the power grid safety management and control platform system. If there is no match, an alert for mismatch between person and certificate is triggered.

[0155] Validity period verification: The start and end dates of the validity period are converted into timestamps using a date parsing algorithm and compared with the planned work dates in the work plan. If the planned date is greater than the end date of the validity period, an expiration warning is triggered.

[0156] Job Category Adaptation: Based on the risk level of power grid operations (e.g., "500kV transmission line high-altitude operation" belongs to "Level III high-altitude operation"), verify whether the "job category" extracted by OCR contains the corresponding level. If it does not, trigger a "qualification mismatch" warning.

[0157] Issuing authority compliance: The database of compliant issuing authorities is accessed, and a fuzzy match is performed on the issuing authorities extracted by OCR (e.g., "XX Provincial Emergency Management Department" is compliant, "XX Training Company" is non-compliant). If the rules are not met, an invalid certificate warning is triggered.

[0158] 2.2.2.4 Results Linkage and Integrated Display in a Single Chart:

[0159] Based on the power grid map, the verification results are linked to the operator's ID. When the mouse hovers over the operator's avatar in the work area of ​​the power grid map, the qualification verification status is displayed: green for qualified and red for unqualified. Clicking on the red warning will show the specific reason for the unqualified status and the OCR-recognized document screenshot, supporting real-time traceability by management personnel.

[0160] 2.2.3 Analysis of Disclosure Recordings Based on Intelligent Speech Recognition

[0161] In the application scenario of safety briefings for verification operations, there are business issues such as "inconsistencies between written and oral briefings," "lack of record-keeping for verbally supplemented risk points," and "lack of traceability for the briefing process" (e.g., the written briefing does not mention "temporary adjustments to the work sequence," but this is stated verbally, making it impossible to trace the process if an accident occurs later). Intelligent speech recognition and analysis technology, through recording noise reduction, speech-to-text conversion, semantic analysis, and information completion, transforms the oral briefing content into analyzable and traceable structured data, while supplementing information not covered by the written briefing, forming a complete record of both written and oral briefings, laying the foundation for subsequent verification and comparison;

[0162] 2.2.3.1 Analysis of Pre-processing Technology for Handover Recording (Solving On-site Noise Interference)

[0163] Noise type identification: Power grid site noise includes equipment operation noise (such as circuit breaker operation sound, fan sound, frequency 100-500Hz), environmental noise (such as wind sound, people talking sound, frequency 200-2000Hz), and electromagnetic interference noise (such as transformer electromagnetic field noise, frequency 50-100Hz). The dominant noise type in the current recording is automatically identified by a noise classification algorithm based on Mel-frequency cepstral coefficients (MFCC).

[0164] Targeted noise reduction analysis and processing:

[0165] (1) Equipment noise reduction: An adaptive filtering-based noise reduction algorithm is adopted, using the equipment operation sound without disclosure as the reference signal to cancel the equipment noise in the recording. The signal-to-noise ratio is improved by ≥15dB after noise reduction.

[0166] (2) Environmental noise reduction: Wavelet threshold noise reduction algorithm is used to suppress high-frequency environmental noise (such as wind noise) and preserve the amplitude characteristics of speech signal;

[0167] Speech segmentation technology processing: Based on the speech activity detection (VAD) algorithm, the speech of the person giving the briefing, the response speech of the operator, and silent segments are identified. Silent segments (duration < 0.5 seconds) are automatically removed, and the speech segments are labeled according to the interaction order of the person giving the briefing and the operator (e.g., “Segment 1: Person giving the briefing (00:01-00:30)”, “Segment 2: Operator (00:31-00:45)”).

[0168] 2.2.3.2 Power Grid-Specific Speech-to-Text (ASR) and Semantic Analysis:

[0169] Optimization of ASR Model in the Power Grid Sector: A specialized terminology dictionary was constructed, incorporating terminology specific to power grid safety briefings (such as "grounding wire installation location," "insulation shielding range," "two-ticket, three-system implementation requirements," and "step voltage protection"). These terms were embedded into the language model (LM) of the ASR model, increasing the accuracy of specialized terminology recognition from 85% in general ASR to over 98%. Regarding accent recognition, considering the wide geographical distribution of power grid workers, 500 hours of voice samples were collected from power grid personnel in Northeast, North, and Southwest China. The ASR model was then fine-tuned to address accent-related recognition biases (such as accurately recognizing the dialectal pronunciation of "insulating gloves").

[0170] Speech-to-Text (STT) output analysis: Generate structured text by segmenting speech, and label the speaker type (informer / operator) and timestamp (accurate to the second) for each segment.

[0171] Intelligent semantic analysis (NLP):

[0172] Intent recognition analysis: Using a BERT-based intent classification model, the business intent of each text segment is identified, such as "risk point notification to the person in charge", "question and inquiry from the operator", "response to the measures provided by the person in charge", and "confirmation and response from the operator". The intent recognition accuracy rate is ≥95%.

[0173] Key information extraction: The Named Entity Recognition (NER) model is used to extract job tasks, risk points, prevention and control measures, safety requirements, and points of doubt from the transcribed text; and it is aligned with the entity database of the written briefing content (e.g., extracting "risk points: accidental contact with live wires, falls from heights, and tool drops causing injury").

[0174] 2.2.3.3 Results Presentation and Document Completion Analysis:

[0175] Verbal information supplementation: Information not mentioned in the written briefing (such as questions from workers or measures temporarily added by the briefer) in the transcript of the audio recording will be automatically added to the supplementary notes section of the written briefing document to form a complete record of written and oral briefing.

[0176] Escaped text display: In the work area early warning module of the power grid map, click on the briefing recording analysis to view the transcribed text and semantic analysis results, and support jumping to the corresponding recording segment by timestamp (click "segment 1" to play the recording from 00:01 to 00:30), realizing text and voice linkage traceability;

[0177] 2.2.4 Analysis of Intelligent Verification and Comparison Technology Between Disclosure Content and Recordings:

[0178] In traditional safety briefing verification and comparison scenarios, the core risk is "inconsistency between written content and oral statements" (e.g., the written statement says "install 2 sets of grounding wires," while the oral statement says "install 1 set of grounding wires"). Manual verification requires comparing the text and recording word by word, which is time-consuming and prone to omissions. The core objective of intelligent verification and comparison technology is to automatically compare the written briefing content with the transcribed text of the recording from three business dimensions: consistency of key information, completeness of content, and accuracy of expression. It identifies discrepancies and triggers warnings to ensure that the written and oral briefing information are consistent and without deviation, while also providing early warning basis for the power grid's integrated map system.

[0179] 2.2.4.1 Define comparison dimensions and business rules

[0180] Consistency of key information (core comparison dimensions):

[0181] Business definition: Are the descriptions of core fields such as work tasks, risk point list, prevention and control measures, safety requirements, work time and location consistent between written instructions and audio transcripts?

[0182] Comparison rules:

[0183] Job task: Must be completely matched (e.g., if written "Replace insulators on towers #5-#8 of 110kV XX line" and audio transcription "Replace insulators on tower #5 of 110kV XX line", it will be judged as inconsistent due to the difference in the scope of the work);

[0184] Risk Point List: Set matching is used. The risk points listed in the written form and the risk points in the audio transcription must completely overlap, although different orders are allowed. For example, if the written form is "Risk 1: Fall from height, Risk 2: Accidental contact with live electricity", and the audio transcription is "Risk 1: Accidental contact with live electricity, Risk 2: Fall from height", they are considered to be consistent. If there are 3 risk points in the written form and 2 in the audio transcription, they are considered to be inconsistent.

[0185] Prevention and control measures: Semantic matching is adopted, and synonyms are allowed (e.g., written "wear insulated gloves" and audio transcription "wear insulated gloves" are considered consistent; written "install grounding wire" and audio transcription "do not install grounding wire" are considered inconsistent).

[0186] Safety requirements: Key values ​​must be fully matched (e.g., the written statement "Workers must hold a valid high-altitude work permit" is inconsistent with the audio transcription "Workers may hold an expired permit"; the written statement "Work must stop when the wind speed is ≥10m / s" is inconsistent with the audio transcription "Work must stop when the wind speed is ≥12m / s").

[0187] Accuracy of expression (error correction and comparison dimension)

[0188] Business definition: Whether there are contradictions in the written and audio-transcribed texts (e.g., the written text says "install 2 sets of grounding wires", while the audio-transcribed text says "install 1 set of grounding wires") or errors in professional terminology. For example, the written text says "insulation shielding", while the audio-transcribed text says "insulation covering", which are synonyms and are not judged as errors; the written text says "grounding wire", while the audio-transcribed text says "electric wire", which is judged as an error.

[0189] Comparison rules: Automatically identify terminology errors and numerical inconsistencies through a power grid terminology dictionary and business rule base.

[0190] 2.2.4.2 Intelligent Comparison Algorithm Design and Analysis

[0191] Key information consistency comparison algorithm:

[0192] Entity-level matching: A BERT-based entity alignment algorithm is used. First, entities such as tasks, risk points, and prevention and control measures are extracted from written text and audio-transcribed text (using the NER model). Then, the cosine similarity of the entity vectors is calculated (the threshold is set to 0.85). If the similarity is ≥0.85, they are considered to be consistent, and if it is <0.85, they are considered to be inconsistent.

[0193] Numerical matching: For numerical fields including operation time, wind speed threshold, and number of grounding wires, exact matching plus range verification is used. For example, if the written "operation time 8:00-12:00" is inconsistent with the audio transcription "operation time 8:00-11:30" (exceeding the allowable deviation ±15 minutes), it is determined to be inconsistent.

[0194] Content integrity comparison algorithm:

[0195] Completeness of written content: Convert the core essential content of the written content into a set of keywords (such as {"fall prevention measures from heights", "accidental contact with live wires prevention measures"}), and check whether the transcript of the audio recording contains all the keywords. If any are missing, it is determined that the audio recording does not cover the written content.

[0196] Completeness of audio recording content: The topic extraction based on topic model LDA is used to analyze whether the topic of the audio transcription text contains core topics not mentioned in the written text (e.g., if the audio mentions "temporarily added anti-inductive electric current measures", and the written text does not mention it, it is determined that "the written text does not cover the audio content").

[0197] Algorithm for comparing the accuracy of statements:

[0198] Terminology error identification: Construct a database of correct and incorrect mappings of power grid professional terms (such as "grounding wire - power line" and "insulation shielding - insulation covering (error)"). Identify terminology errors through string matching and semantic similarity calculation.

[0199] Conflict detection: A conflict detection algorithm based on logical rules is adopted. For example, in the "Number of Installations" field, if the difference between the written and transcribed values ​​is greater than 0, it is determined to be a conflict; in the "Work Scope" field, if the transcribed scope is smaller than the written scope (and "temporary adjustment" is not noted), it is determined to be a conflict.

[0200] 2.2.4.3 Analysis of the linkage between verification results and early warning system of the power grid:

[0201] Risk level outcome classification:

[0202] Warning (red): Inconsistent key information (such as missing risk points or contradictory prevention and control measures) or incomplete core content (such as failure to mention two or more risk points in writing).

[0203] Warning (yellow): Inconsistent auxiliary information (such as differences in emergency contact phone numbers), minor errors in wording (such as errors in non-core terminology);

[0204] Normal (green): Key information is consistent, core content is complete, and there are no contradictory statements.

[0205] Large screen display:

[0206] Spatial association: Mark the verification results in the working area (such as around tower #5) on the power grid map with the color (red / yellow / green). Hover the mouse over the area to display the specific details of inconsistencies, incompleteness, and errors (such as "Inconsistent risk points: 3 in written form, 2 in audio transcription, missing 'tool falling and injuring people'").

[0207] Source tracing: Clicking on the warning content will take you to the verification details page, which displays the written briefing text, the corresponding audio transcript, and the audio timestamp (supports playing the corresponding audio transcript), making it easier for managers to trace the reasons for the discrepancies;

[0208] The business-level innovations in this application include:

[0209] Deep fusion of multimodal data: For the first time, text (OCR), speech (ASR), image (CV), and spatial data (GIS) are deeply integrated and cross-verified in the field of power grid safety briefing, which transcends the limitations of single-modal analysis.

[0210] Domain knowledge-driven AI model: Through domain-adaptive training and the application of power grid safety knowledge graph, the general AI model (the power grid's light model) has profound industry insights, and the analysis results are accurate and reliable.

[0211] Dynamic, real-time, and proactive: Shifting from "post-event review" to "real-time analysis and early warning during the event," the security management checkpoint is moved forward to proactively identify and intercept potential risks.

[0212] Spatial visualization and context awareness: This innovative approach binds abstract safety briefing content with concrete spatial locations on a "power grid map," giving safety risks spatial attributes and greatly improving managers' context awareness and decision-making efficiency.

[0213] Three: The process of multi-source data fusion in the system:

[0214] The process of multi-source data fusion includes work plan analysis and statistics, intelligent parsing and feature extraction of multi-source data, multi-dimensional data fusion and verification comparison, and alarm visualization display based on a single power grid map.

[0215] 31. Work plan analysis and statistics, such as... Figure 3 As shown:

[0216] By calling the power grid data platform interface and using RPA automation tools to obtain the source data of the work plan in the safety management system, and then analyzing, filtering and cleaning it to obtain the final key business data, including safety briefing content, audio files, and images of the qualification certificates of the operators, the key business data is pushed to the source data analysis module in real time, along with the data retrieval timestamp and data integrity verification code to ensure that the source data is traceable.

[0217] Before using RPA automation tools, you need to configure RPA data retrieval rules in advance: specify the data retrieval frequency (e.g., every 15 minutes / time, matching the job plan update rhythm) and the range of key fields, including job ID, job location, planned time, responsible person for handover, and associated qualification certificate number, to avoid redundant data interfering with subsequent analysis.

[0218] 32. Intelligent parsing and feature extraction of multi-source data: such as Figure 4 As shown:

[0219] The obtained briefing content and work permit image content are processed and obtained through two parallel threads: the audio intelligent transcription thread and the OCR qualification certificate recognition thread.

[0220] The intelligent transcription thread automatically calls the speech recognition (ASR) service, enables noise reduction and semantic focus algorithms, and converts the safety briefing recording into text. Subsequently, the natural language processing (NLP) engine performs deep semantic analysis on the text to extract key elements, including the work location, safety measures, risk points, and personnel division of labor.

[0221] OCR Certificate Recognition Thread: Automatically calls the OCR recognition service to scan the qualification certificates of operators. It not only recognizes text information, including name, certificate number, job category, and validity period, but also extracts feature information, including the outline of the issuing authority's seal, the pattern of the official watermark, and the pixel features of the photo, for anti-counterfeiting verification.

[0222] The current stage is the core of intelligence. By extracting information from audio-to-text transcription and OCR-recognized images, the data needs to be pushed to the dual-verification module in real time, along with confidence level processing (e.g., 95% confidence level for audio transcription and 88% confidence level for OCR validity period recognition), to provide a basis for the verification module to make judgments.

[0223] 33. Multidimensional data fusion and verification comparison:

[0224] Safety briefing content verification: The semantic results of the safety briefing recordings are compared with the written work plans and standard safety procedures in the risk control platform to establish key information matching rules; such as: work tasks, risk point list, prevention and control measures, safety requirements, work time, etc. According to the established rules, the corresponding content is extracted and subjected to full matching, set matching, and semantic matching according to business needs.

[0225] Personnel qualification certificate compliance verification: The personnel qualification certificate information data extracted by OCR, namely the certificate number and validity period, is connected to a specific special operation qualification certificate database through interface API calls to compare data in real time and determine whether there are three types of violations: expired, forged, and certificate type not matching the operation (such as using a low-voltage electrician certificate for high-voltage operation), and generate a compliance conclusion.

[0226] The two verification results are linked with the initial work plan data to form a comprehensive safety assessment report. The conclusion of whether the verification passes or fails (including specific differences / violation details) is pushed to the alarm display module on the large screen in real time to provide a basis for alarm classification.

[0227] 34. Visualize alarms based on a single power grid diagram:

[0228] Alarm classification: The alarm information is geospatialized; on the power grid geographic information map (GIS map), the location of the substation and line tower where the work plan is located is located, and the alarm is divided into three levels according to the severity of the risk: red emergency (expired qualification certificate, mismatch of core briefing content), yellow reminder (non-core information difference, qualification certificate about to expire), and blue notification (verification passed). Different levels correspond to different flashing frequencies: red once per second and yellow once every 3 seconds.

[0229] Source tracing interaction: Clicking on any alarm item will take you to the verification details page, which displays the data source (such as RPA data retrieval time, OCR recognition screenshot), comparison process, and cause of the anomaly. It also supports administrators to export anomaly reports with one click.

[0230] The innovations of the system provided in this application in the process of multi-source data fusion include:

[0231] Multi-source data integration and visualization: Based on a single power grid map, it realizes automatic mapping of equipment topology and fusion of multi-source heterogeneous data. Combined with distributed rendering and dynamic loading, it solves the problems of low efficiency and error-proneness of traditional manual drawing, and improves the smoothness of visualization and decision-making efficiency.

[0232] Data access control security: Employing the ABE algorithm, hybrid encryption, and hierarchical storage, coupled with dynamic password secondary verification, we build multi-layered protection to achieve fine-grained access control and full lifecycle security management of sensitive data.

[0233] Deep fusion of multimodal data: For the first time, text (OCR), speech (ASR), image (CV), and spatial data (GIS) are deeply integrated and cross-verified in the field of power grid safety briefing, which transcends the limitations of single-modal analysis.

[0234] Domain-Knowledge-Driven AI Models: Through domain-adaptive training and the application of power grid safety knowledge graphs, a general-purpose AI model (the power grid's "brightness model") possesses profound industry insights, resulting in accurate and reliable analysis results.

[0235] IV. The following principles were followed during the system design and implementation process:

[0236] Normative principles:

[0237] The system code design follows Sun's unified Java specification, which facilitates later code maintenance. The architecture follows the MVC layered concept to ensure the clarity of the system architecture.

[0238] Scalability principle:

[0239] The system ensures that the implemented platform has good scalability during hardware configuration, solution design, system development, and system implementation; it has flexible configuration for business processing, can be flexibly reorganized and adjusted as business functions change, and provides standard open interfaces.

[0240] Safety principles:

[0241] The system invention possesses high security and reliability, and ensures the safe and stable operation of the system by adopting multiple security mechanisms and technical means, thus meeting the requirements for the safe operation of networks and information systems.

[0242] 5. Implementation, Deployment, and Testing of the Invention

[0243] 1. Researchers need to prepare the software for the deployment of the invention, including installing JDK 1.8, deploying Nginx 1.14 service, Tomcat 8.5 service, Redis 6.2 caching service, MySQL 8.0 database service, and the system service of the current invention, "Intelligent Analysis System and Method for Power Operation Safety Disclosure Based on Multi-Source Data Fusion".

[0244] 2. After the invention was launched, the following effects were achieved through actual testing by the operators:

[0245] Risk identification capabilities have been significantly improved:

[0246] The task handover identification is performed using an AI-powered voice model, which improves work efficiency by 10 times compared to traditional manual inspection.

[0247] The response time for handling anomalies is ≤8 seconds, which is 87.5% more efficient than the traditional 70-second manual inspection.

[0248] Safety management costs have been reduced, with labor costs decreasing by about 30% compared to before, which has played a role in reducing costs and increasing efficiency.

[0249] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A smart analysis system for safety briefings in power operations based on multi-source data fusion, characterized in that, The system includes: Presentation layer, service access layer, gateway isolation layer, business application layer, data persistence layer; The presentation layer, or front-end UI, handles user interaction. Data display uses JavaScript to generate HTML pages, user input uses HTML controls, special operations and page effects are implemented using JavaScript, and CSS is used to style the pages. Data submitted to the server is submitted via AJAX and processed by the back-end controller class. The Vue.js front-end framework is used to render the page, and user interaction uses Element controls. The service access layer is responsible for handling requests from the presentation layer and interacting with the backend service layer. It receives and processes requests from the presentation layer, calls backend service interfaces, and performs data format conversion and adaptation. The service access layer acts as a bridge between the presentation layer and the backend service. The gateway isolation layer is responsible for routing, load balancing, authentication, and authorization of requests; logging and monitoring of request routing, load balancing, authentication, authorization requests, and responses; Business application layer: A method in the business application layer is equivalent to a database transaction. A method in the business application layer calls one or more DAO methods, and the DAO operations are performed within a single transaction. The data persistence layer is responsible for accessing data sources, including relational databases, and converting the data into Java objects for other layers to call. For general data maintenance functions, the data access service is called directly in the service. For data objects shared by multiple modules or data objects that provide external interfaces, it is recommended to encapsulate them as objects using JavaBeans. MyBatis is used for data persistence operations.

2. The system according to claim 1, characterized in that, The business application layer includes the following applications: SMS platform management: Used to send warning SMS messages to operators; Artificial intelligence platform management: Uses intelligent AI algorithms to process and analyze alarm-related data; Safety risk control platform management: Obtain risk control operation plan data; OCR Intelligent Recognition Management: Intelligently recognizes image content, analyzes and processes relevant data in the image; Job planning management function: Implements the display and query of job planning-related functions through code logic; Personnel qualification management function: Through work plan data, obtain images of personnel's high-altitude operation qualification certificates, integrate OCR intelligent recognition technology, and then remove the relevant content of the qualification certificates; Intelligent speech recognition and analysis function: It extracts acoustic features through FBANK and audio spectrogram and signal preprocessing technology, uses Transformer architecture and deep neural network (DNN) for acoustic depth modeling, and combines neural network language model and Viterbi algorithm to improve the accuracy and efficiency of complex sentence recognition. Intelligent speech translation analysis function: Normalizes text through common rule templates or seq2seq models, domain-dependent dictionaries and contextual reasoning, and uses natural language processing (NLP) and natural language understanding (NLU) to perform real-time analysis, real-time processing and real-time transcription of speech; The semantic intelligent verification function of the disclosure: By comparing the speech recognition text with the OCR text in the same scene or the original content text through cross-source comparison, the consistency is judged by the text similarity. When the difference is too large, manual verification is triggered. Safety briefing and alarm information display function: Based on a visual dashboard and web interface, alarm data is pushed, summarized and statistically analyzed, and listed for query in real time.

3. The system according to claim 1, characterized in that, Key operations at the presentation layer include: access to the underlying layer of the power grid map, secure encryption of page data interaction, and business associations for secure page disclosure; 2.1.1, where the bottom-level access of the power grid map includes multi-source data access and fusion, real-time data rendering and computational optimization; 2.1.1.1 Multi-source data access and fusion: Automatic device and topology mapping: In data visualization application scenarios, based on the concept of a single power grid map, the system uses IoT sensing technology to achieve automatic device registration and automatic topology generation; when the substation outgoing switch is closed to supply power, the medium and low voltage node devices will be energized step by step, and the device registration information and high-precision geographic coordinates will be automatically uploaded, and the geographic map and single-line map will be automatically generated accordingly. Multi-source heterogeneous data integration: The system needs to integrate data from the equipment asset lean management system, safety risk control system platform, power distribution management system, and business middleware system; the data includes equipment operation and maintenance data, work plan data, substation and line power monitoring and equipment operation information, power grid equipment ledger information, asset information, defect and fault information, maintenance information, and real-time and historical operation data; through topology fusion, the system integrates the power grid's internal topology and inter-connection relationships, the physical connection between users and the power grid, and the specific installation locations of various sensors and data acquisition devices in the power grid, etc., to connect the topological connections between power grids of different voltage levels and form a single power grid topology map; 2.1.1.2 Real-time data rendering and computation optimization: Distributed rendering and load balancing: In real-time page rendering and display application scenarios, distributed rendering is adopted; the system distributes rendering tasks to multiple computing nodes and avoids single-point overload through load balancing algorithms; for large-scale power grid data, a dynamic hierarchical loading strategy is adopted, automatically loading data of different precision according to the view zoom level and region to ensure user experience; when users browse the panorama, the system displays a simplified version of the power grid topology; when users focus on a specific area, the system automatically loads detailed equipment information and connection relationships for that area; Spatiotemporal Data Management: Based on graph databases and graph computing, the system constructs a spatiotemporal data management system, utilizing the structural consistency between graph data structures and the actual power grid. The system integrates the overall topology of the power grid, including the internal topology of all links such as transmission, transformation, and distribution, as well as the connection relationships between them, to form a single topology graph of the power grid. It also integrates data from different management perspectives, organizational methods, time and space dimensions, and scales from various business departments of power grid operation, monitoring, maintenance, production, and marketing. Computational optimization strategies: When dealing with a large amount of data from distribution network equipment operation, the system employs multiple computational optimization strategies, including:

1. Hierarchical data storage: storing hot, warm, and cold data on storage media with different performance levels; 2. Incremental computing: only updating changed data to avoid the overhead of full-scale computing; 3. Edge computing: processing data near the data source to reduce data transmission latency and bandwidth pressure; 4. Pre-computation and caching: pre-computing and caching frequently used query and analysis results to accelerate data retrieval speed. 2.1.2 Security Encryption of Page Data Interaction: The system adopts multi-layered and comprehensive security encryption technology to ensure the security of data during transmission, storage and access; 2.1.2.1 Transport layer security encryption mechanism: Attribute-based encryption-based secure data sharing algorithm: In scenarios where the system accesses specific security disclosure content of the work plan, it adopts an attribute-based encryption-based secure data sharing algorithm. The algorithm improves the security of accessed data through a fuzzy access strategy; it uses a linear key sharing strategy to support any monotonic access structure, thereby enhancing the expressibility of the access strategy; only users who meet specific attribute conditions can decrypt and access the data. Hybrid encryption mechanism: In access control log application scenarios, the system adopts a hybrid encryption mechanism that combines the advantages of symmetric and asymmetric encryption. First, the power data is encrypted using a symmetric encryption algorithm. Then, the symmetric key is encrypted using an attribute encryption algorithm. The access policy and keywords are also encrypted and uploaded to the consortium blockchain. Sensitive data transmission protection: In terms of security briefing recording data processing, the AES-256-GCM algorithm is used for end-to-end encryption, and the key is dynamically generated through Diffie-Hellman key exchange; in terms of spatial coordinate data processing, coordinate offset encryption is implemented, and the data is automatically decrypted and restored when displayed on the front end; in business data transmission, field-level encryption is used, and fields including operator information and equipment parameters are encrypted and transmitted separately. 2.1.2.2 Data storage and interaction security: Client-side data storage: Local data encryption is implemented using a Web Crypto API, with keys generated and stored via a hardware security module; a tiered data storage strategy is implemented. Highly sensitive data: not stored locally, but temporarily cached in memory and automatically cleared after the session ends; Sensitive data: Encrypted storage and set to automatically destroy after 7 days; Low-sensitivity data can be cached for a long time but contains watermark information; Interactive operation security protection: In critical operations such as confirmation of disclosure, intelligent voice parsing, and early warning risk query, a two-stage encryption verification is adopted, requiring the input of a dynamic password; a unique nonce value is generated for each interactive request, with a lifespan of 30 seconds, which expires immediately after the server verifies the nonce. Front-end code protection: The encrypted data in the system, including equipment ledgers, work plan information, work location, and safety briefing data, adopts code obfuscation and dynamic loading technology to prevent reverse engineering to obtain the encryption algorithm; the code is detected by hash value comparison to detect whether it has been tampered with, and a security alarm is triggered immediately when an anomaly is detected to achieve runtime integrity verification; 2.1.3 Business Relationships in Page Security Disclosure: 2.1.3.1 Mapping of safety briefing content with power grid elements: Space and Business Dual Association Mechanism: The system establishes a connection between safety briefing content and specific graphic elements on the power grid map through a dual association mechanism of spatial location and business attributes; based on high-precision geographic coordinate information, the system automatically associates safety briefing content with operating equipment or areas; at the same time, through power grid topology analysis, the system automatically identifies the scope of operation impact and associates safety briefing content with relevant equipment affected by the operation, ensuring that safety measures cover all affected areas; Dynamic Operation Safety Boundary Analysis: Based on the power grid topology connection relationship, the system dynamically analyzes the operation safety boundary and automatically generates a safety measure layout plan; for line operations, the system automatically determines the areas that need to be de-energized, the locations that need to be grounded, and the warning ranges that need to be set according to the topology analysis, and displays them visually on a power grid map; the system supports simulating the effects of safety measures, discovering potential safety blind spots in advance, and optimizing the safety measure layout plan. Visualization of briefing effectiveness evaluation: The briefing quality index calculation model is designed, including indicators such as comprehensive coverage, accuracy, and detail. The briefing quality of different areas is displayed on the map using a heat map. The briefing situation of the same area at different times is overlaid on the map, and the time axis can be slid to view the trend of change, so as to realize trend comparison.

4. The system according to claim 2, characterized in that, Key operations at the business application layer include: intelligent identification and analysis of briefing content based on the Guangming big data model, intelligent verification of operator qualifications based on OCR, analysis of briefing recordings based on intelligent speech recognition, and intelligent verification and comparison technology analysis of briefing content and recordings. The business application layer integrates multi-source data from the power grid, utilizes large-scale artificial intelligence models, OCR recognition, and voice analysis to intelligently parse safety briefing content, automatically verify personnel qualifications and the consistency of briefing recordings, achieve accurate identification and real-time early warning of operational risks, and visualize the results on a single map of the power grid.

5. The system according to claim 4, characterized in that, Intelligent identification and analysis of disclosure content based on the Guangming Big Data Model, including: In the intelligent identification scenario of safety briefings for work plans, intelligent technology is used to automate, standardize, and deeply analyze the briefing content, covering four core business dimensions: work task matching, completeness of risk point identification, compliance of prevention and control measures, and accuracy of equipment association, providing a basis for determining the compliance of work plans; specifically including: 2.2.1.1 Synchronous Analysis of Work Plan Data: By acquiring source data of work plans from the safety management and control system through the power grid data platform and automated RPA tools, and then analyzing, filtering, and cleaning the data to obtain the final business data, the data will provide data support for subsequent intelligent analysis. 2.2.1.2 Corpus Construction and Analysis of Disclosure Content: Collect power grid safety regulations, work instructions, historical briefing documents, accident case reports, and electronic briefing documents related to work plans, including work location, voltage level, work type, risk point description, prevention and control measures, and associated equipment IDs; to form a dedicated corpus for training and fine-tuning the Guangming large model; 2.2.1.3 Business Element Extraction and Model Training: The system uses a pre-trained Guangming large model with a Transformer architecture as its foundation, and specializes the power grid through two methods: domain-adaptive pre-training and downstream task fine-tuning. Key element extraction and training: The training model identifies entities in the text, including work location, work type, safety measures, personnel roles, risk point types, risk levels, prevention and control measures, and equipment status requirements. This is accomplished using BiLSTM-CRF or a Transformer-based sequence labeling model. 2.2.1.4 Multi-dimensional business compliance verification: Task matching analysis: The extracted basic task information is compared with the task information in the task planning system to verify whether "the task corresponding to the briefing content is consistent with the approved task plan" and to check whether the voltage level and task location match. Risk point identification completeness analysis: Based on the risk point association reasoning ability of the Guangming big model, through training with historical violation cases, establish the mapping relationship between operation type and risk point, and determine whether the briefing content has omitted any mandatory risk points; Equipment association accuracy: Associate the equipment ID in the briefing with the equipment ledger in the power grid GIS system to verify whether the equipment exists and whether the equipment status requirements meet the actual site conditions.

6. The system according to claim 4, characterized in that, Intelligent verification of operator qualifications based on OCR includes: 2.2.2.1 Collection and Preprocessing of Qualification Documents: Image acquisition adaptation: Supports both mobile and fixed terminal acquisition methods. Addressing issues such as uneven lighting, document creases, and background interference at power grid sites, the following technologies are employed: Lighting correction: Based on the Retinex algorithm, uneven lighting is eliminated, and the standard deviation of brightness in the document area is controlled within 5. Image denoising: An adaptive denoising algorithm based on wavelet transform is used to filter out equipment operating noise; Edge detection and cropping: The Canny algorithm is used to identify the edges of the document and automatically crop the background area to ensure that only the valid content of the document is retained; 2.2.2.2 Key Information Extraction: For the application scenario of collecting qualification information of special operation personnel in the power grid, a high-precision OCR engine based on CNN+CTC is used to perform text recognition on the corrected image; the key information collected is name, certificate number, issuing authority, validity period, and job category. 2.2.2.3 OCR-based intelligent recognition and verification: Name matching: The certificate holder's name extracted by OCR is precisely matched with the operator's name in the power grid safety management and control platform system. If there is no match, an alert for mismatch between person and certificate is triggered. Validity period verification: The start and end dates of the validity period are converted into timestamps using a date parsing algorithm and compared with the planned work dates in the work plan. If the planned date is greater than the end date of the validity period, an expiration warning is triggered. Job Category Adaptation: Based on the risk level of power grid operations, verify whether the "job category" extracted by OCR contains the corresponding level. If not, trigger a "qualification mismatch" warning. Issuing authority compliance: The database of compliant issuing authorities is accessed, and a fuzzy match is performed on the issuing authorities extracted by OCR. If the match does not meet the rules, an invalid certificate warning is triggered. 2.2.2.4 Results Linkage and Integrated Display in a Single Chart: Based on the power grid map, the verification results are linked to the operator's ID. When the mouse hovers over the operator's avatar in the work area of ​​the power grid map, the qualification verification status is displayed: green for qualified and red for unqualified. Clicking on the red warning will show the specific reason for the unqualified status and the OCR-recognized document screenshot, supporting real-time traceability by management personnel.

7. The system according to claim 4, characterized in that, Analysis of handover recordings based on intelligent speech recognition includes: Intelligent speech recognition and analysis technology transforms oral briefing content into analyzable and traceable structured data through recording noise reduction, speech-to-text conversion, semantic analysis, and information completion. At the same time, it supplements information not covered by written briefings, forming a complete record of both written and oral briefings. 2.2.3.1 Analysis of Preprocessing Technology for Disclosure Recordings Noise type identification: Power grid site noise includes equipment operation noise, environmental noise, and electromagnetic interference noise. The dominant noise type in the current recording is automatically identified by a noise classification algorithm based on Mel frequency cepstral coefficients (MFCC). Targeted noise reduction analysis and processing: (1) Equipment noise reduction: An adaptive filtering-based noise reduction algorithm is adopted, using the equipment operation sound without disclosure as the reference signal to cancel the equipment noise in the recording; (2) Environmental noise reduction: The wavelet threshold noise reduction algorithm is used to suppress high-frequency environmental noise and preserve the amplitude characteristics of the speech signal; Voice segmentation technology processing: Based on the Voice Activity Detection (VAD) algorithm, the speech of the person giving the instructions, the response speech of the operator, and silent segments are identified. Silent segments are automatically removed, and speech segments are labeled according to the interaction order of the person giving the instructions and the operator. 2.2.3.2 Power Grid-Specific Speech-to-Text ASR and Semantic Analysis: Optimization of ASR model in the power grid field: Construction of a professional terminology dictionary, including special terms for power grid safety briefings, and embedding the terms into the language model (LM) of the ASR model. In terms of accent recognition, considering the wide geographical distribution of power grid workers, voice samples of power grid workers from Northeast, North, and Southwest China were collected to fine-tune the ASR model based on accents and solve the recognition bias caused by dialect accents. Speech-to-text output analysis: Generate structured text by segmenting speech, and label the speaker type (instructor / operator) and timestamp of each segment; Intelligent semantic analysis: Intent recognition and analysis: Identify the business intent of each text segment using a BERT-based intent classification model; Key information extraction: Using the Named Entity Recognition (NER) model, the task, risk points, prevention and control measures, safety requirements, and points of doubt are extracted from the transcribed text; and aligned with the entity database of the written briefing content. 2.2.3.3 Results Presentation and Document Completion Analysis: Verbal information supplementation: Information not mentioned in the written briefing in the transcript of the audio recording is automatically added to the supplementary explanation column of the written briefing document to form a complete record of written and oral briefings; Escaped text display: In the work area early warning module of the power grid map, click on the briefing recording analysis to view the transcribed text and semantic analysis results, and support jumping to the corresponding recording segment by timestamp to realize text and voice linkage traceability.

8. The system according to claim 4, characterized in that, The intelligent verification and comparison technology analysis of the briefing content and the recording includes: From three key business dimensions—consistency of critical information, completeness of content, and accuracy of expression—the system automatically compares written briefing content with audio-transcribed text, identifies discrepancies, and triggers alerts to ensure that written and oral briefing information are consistent and unbiased, while also providing early warning evidence for the power grid's integrated mapping system. Specifically, this includes: 2.2.4.1 Define comparison dimensions and business rules: Consistency of key information: Business definition: Are the descriptions of core fields such as work tasks, risk point list, prevention and control measures, safety requirements, work time and location consistent between written instructions and audio transcripts? Comparison rules: The matching degree varies for different tasks; Job task: Must be a complete match; Risk Point List: Set matching is used. The risk points listed in writing must completely overlap with the risk points in the audio transcription, although different orders are allowed. Prevention and control measures: Semantic matching is used, and synonyms are allowed; Security requirement: Must perfectly match key values; Accuracy of expression: Business definition: Whether there are contradictions in expression or errors in technical terminology in the written and audio transcription texts; Comparison rules: Automatically identify terminology errors and numerical inconsistencies through a power grid terminology dictionary and business rule base; 2.2.4.2 Intelligent Comparison Algorithm Design Analysis: Key information consistency comparison algorithm: Entity-level matching: Employing a BERT-based entity alignment algorithm, we first extract entities such as task assignments, risk points, and prevention and control measures from the written text and the transcribed text, and then calculate the cosine similarity of the entity vectors. Numerical matching: For numerical fields including operation time, wind speed threshold, and number of grounding wires, exact matching plus range validation is used; Content integrity comparison algorithm: Completeness of written content: Convert the core essential content of the written content into a set of keywords, check whether the transcript of the audio recording contains all the keywords, and if any are missing, it is determined that the audio recording does not cover the written content. Completeness of audio recording content: The topic extraction based on the topic model LDA is used to analyze whether the topic of the audio transcription text contains core topics not mentioned in the written text; Algorithm for comparing the accuracy of statements: Terminology error identification: Construct a mapping library of correct and incorrect power grid professional terms, and identify terminology errors through string matching and semantic similarity calculation; Conflict detection: A logic rule-based conflict detection algorithm is used; 2.2.4.3 Analysis of the linkage between verification results and early warning system of the power grid: Risk level classification: Warning: Inconsistent key information, incomplete core content; Note: Inconsistent supporting information, minor errors in description; Normal: Key information is consistent, core content is complete, and there are no contradictions in the expression; Large screen display: Spatial association: Mark the verification results by color in the work area of ​​the power grid map, and display the specific content of inconsistencies, incompleteness and errors when the mouse hovers over them; Source tracing: Clicking on the warning content will take you to the verification details page, which displays the written disclosure text, the corresponding audio transcription, and the audio timestamp.

9. A smart analysis method for safety briefings in power operations based on multi-source data fusion, characterized in that, The method includes: work plan analysis and statistics, intelligent parsing and feature extraction of multi-source data, multi-dimensional data fusion and verification comparison, and alarm visualization display based on a single power grid map; Step 31, Work plan analysis and statistics; By calling the power grid data platform interface and using RPA automation tools to obtain the source data of the work plan in the safety management system, and then analyzing, filtering and cleaning it to obtain the final key business data, including safety briefing content, audio files, and images of the qualification certificates of the operators, the key business data is pushed to the source data analysis module in real time, along with the data retrieval timestamp and data integrity verification code to ensure that the source data is traceable. Clearly define the data retrieval frequency and the range of key fields, including job ID, job location, planned time, person responsible for briefing, and associated qualification certificate number; Step 32, Intelligent parsing and feature extraction of multi-source data: The obtained briefing content and work permit image content are processed and obtained through two parallel threads: the audio intelligent transcription thread and the OCR qualification certificate recognition thread. Intelligent transcription thread for audio recordings: Automatically calls the speech recognition (ASR) service, enables noise reduction and semantic focus algorithms, and converts the safety briefing recordings into text; subsequently, the natural language processing (NLP) engine performs deep semantic analysis on the text to extract key elements, including work location, safety measures, risk points, and personnel division of labor; OCR Certificate Recognition Thread: Automatically calls the OCR recognition service to scan the qualification certificates of operators. It not only recognizes text information, including name, certificate number, job category, and validity period, but also extracts feature information, including the outline of the issuing authority's seal, the pattern of the official watermark, and the pixel features of the photo, for anti-counterfeiting verification. Real-time push notifications, with confidence level processing included. Step 33, Multidimensional Data Fusion and Verification Comparison: Safety briefing content verification: Compare the semantic results of the safety briefing recordings with the written work plans and standard safety procedures in the risk control platform to establish key information matching rules; based on the established rules, extract the corresponding content and perform full matching, set matching, and semantic matching according to business needs; Personnel operation qualification certificate compliance verification: The personnel qualification certificate information data extracted by OCR, namely the certificate number and validity period, is connected to a specific special operation qualification certificate database through interface API calls to compare the data in real time, determine whether there is an expiration, forgery, or mismatch between the certificate type and the operation, and generate a compliance conclusion; The two verification results are linked with the initial work plan data to form a comprehensive safety assessment report. The conclusion of whether the verification passes or fails is pushed to the alarm display module on the large screen in real time to provide a basis for alarm classification. Step 34: Visualize alarms based on a single power grid diagram: Alarm classification: The alarm information is geospatialized; on the power grid geographic information map, the location of the substation and line tower where the current operation plan is located is located, and the alarm is divided into three levels according to the severity of the risk: red is emergency, yellow is alert, and blue is notification. Different levels correspond to different flashing frequencies: red flashes once per second and yellow flashes once every 3 seconds. Source tracing interaction: Clicking on any alarm item will take you to the verification details page, which displays the data source, comparison process, and cause of the anomaly. It also supports administrators to export anomaly reports with one click.

Images