Local anti-misoperation system and method applied to switching of transformer substation

Abstract

The invention discloses a local anti-misoperation system and method applied to switching of a transformer substation, the system comprises an AI glasses end and an edge calculation box end which are in communication connection with each other, and the AI glasses end comprises a data acquisition module and a man-machine interaction module; the edge calculation box end comprises a knowledge graph module which is used for storing and loading a time sequence knowledge graph of an operation order; the visual identification module is used for receiving the equipment image and identifying field equipment information; and the inference engine module is used for receiving the equipment information and the eyeball movement data, performing inference based on the time sequence knowledge graph so as to verify the compliance triggered by the current operation and the next operation, and outputting corresponding operation prompt or alarm information based on a verification result. According to the invention, localized, real-time and intelligent safety management and control of the switching operation of the transformer substation are realized through the steps of determining the knowledge graph, collecting data, identifying equipment, reasoning and checking, outputting information and the like.

Description

Technical Field

[0001] This invention relates to the field of substation operation technology, and in particular to a local anti-misoperation operating system and method for substation switching. Background Technology

[0002] In power system operation and maintenance, substation switching operations are a core component in ensuring the safe and stable operation of the power grid. The accuracy of these operations directly determines equipment safety, personnel safety, and power supply reliability. As power systems develop towards intelligence and automation, substation operation and maintenance models are gradually shifting from "manned operation + regular inspections" to "unmanned operation + remote monitoring," which places higher demands on the prevention and control of errors in switching operations.

[0003] Traditional substation switching operations rely primarily on an "operation ticket system + manual verification" model for error prevention and control: upon receiving an operation task from the dispatch center, the operator drafts an operation ticket and verifies the equipment number, operation sequence, and equipment status one by one according to the operation ticket. Only after confirming that everything is correct is the operation executed. With technological advancements, some substations have introduced rule-based error prevention systems. These systems convert power safety regulations (such as the "five preventions" rules: preventing accidental opening and closing of circuit breakers, preventing the opening and closing of disconnecting switches under load, preventing the connection of grounding wires while the circuit is energized, preventing the closing of circuit breakers with grounding wires connected, and preventing accidental entry into energized compartments) into fixed code logic, performing simple sequential verification of the operation steps.

[0004] Existing rule-engine-based substation switching operation error prevention systems, while reducing the risk of misoperation to some extent, still have the following key defects and shortcomings: (1) The rules are fixed and cannot adapt to complex logic and dynamic scenarios: The existing system’s error prevention rules are fixed with “if-else” code logic, which can only handle the linear sequential relationship of “operation A → operation B” and cannot parse the complex logic hidden in the operation ticket, such as “voltage testing must be done before grounding and voltage testing requires voltage tester function check”, “if the equipment is under maintenance, the disconnector status check can be skipped”, “when the equipment is under maintenance, the corresponding protection pressure plate and control pressure plate of the equipment can be put on or taken off”; when the substation equipment is updated or the operation process is adjusted, the code needs to be rewritten and deployed, which has extremely poor adaptability.

[0005] (2) Reliance on network or main control room, lack of real-time performance and reliability: The main control room deployed system needs to interact with SCADA through the network to obtain the equipment status. If the network is interrupted or delayed, the system will not be able to perform normal verification. Although the mobile terminal auxiliary system supports local verification, it needs to rely on the operation ticket data pre-issued by the main control room. It cannot update the dynamic status of the equipment in real time, which can easily lead to the failure of the verification results.

[0006] (3) Lack of multi-dimensional monitoring, unable to avoid human negligence: The existing system only focuses on "whether the operation steps are in the correct order", without monitoring the operator's visual attention and operational details. For example, if the operator fills out the operation ticket incorrectly, the operator may perform the operation incorrectly, posing a risk of misoperation of the equipment; if the five-proof lock fails on site, the operator may mistakenly open the knife switch, ground switch, etc.; during the activation and deactivation of protection, the position of the protection cabinet may not be checked, resulting in incorrect operation; the supervisor may not supervise properly, and the operator may switch the handle position inconsistent with the operation ticket; on-site personnel may violate regulations and not operate in the order of the operation ticket. These human negligences may all lead to misoperation, but the existing system cannot detect and warn in time. Summary of the Invention

[0007] In order to overcome the above-mentioned shortcomings of the prior art, the purpose of this invention is to provide a local anti-misoperation operating system and method for substation switching.

[0008] The technical solution adopted by this invention to solve its technical problem is: a local anti-misoperation operating system for substation switching, including an AI glasses terminal and an edge computing box terminal that are interconnected. The AI ​​glasses include: The data acquisition module is used to collect images of on-site equipment and eye movement data of operators; The human-computer interaction module is used to output operation prompts or alarm information to the operator; The edge computing box includes: The knowledge graph module is used to store and load the temporal knowledge graph of operation tickets; the temporal knowledge graph includes operation task nodes, temporal relationship edges that define the order and association between operation tasks, and security condition attributes associated with operation tasks. The visual recognition module is used to receive images of the equipment and identify on-site equipment information; The inference engine module is used to receive device information and eye movement data, and perform inference based on the time-series knowledge graph to verify the compliance of the current operation and the next operation. Based on the verification result, it outputs corresponding operation prompts or alarm information.

[0009] As a further improvement of the present invention: the data acquisition module includes an image acquisition module and an eye-tracking module, the human-computer interaction module includes a voice interaction module and a display module, the voice interaction module is used to play operation prompts or alarm information and receive voice commands from the operator; the display module is used to display operation steps, navigation prompts, alarm information and eye-tracking feedback.

[0010] As a further improvement of the present invention: the inference engine module verifies the compliance of the current operation, specifically including: Verify whether the security conditions associated with the current operation task node are met; Verify whether the operation steps of the current task node meet the sequence requirements; If any safety condition is not met or any operation step does not meet the sequence requirements, it is determined to be a potential misoperation and an alarm message is generated; If all safety conditions are met and all operations comply with the sequence requirements, the current operation is deemed compliant, and the status of the current operation task node in the time sequence knowledge graph is updated.

[0011] As a further improvement of the present invention: the inference engine module verifies the compliance of the next operation specifically includes: In response to the trigger command for the next operation, traverse the prerequisites for the next operation in the time-series knowledge graph and at least verify whether the following prerequisite conditions are met; The status of the pre-operation task nodes is completed, and all safety conditions are met. The time interval between the completion of the preceding operation task node and the triggering of the next operation meets the interval threshold. Retrieve attention verification records to verify that the operator's attention to the on-site equipment meets the effective gaze standard; If all prerequisites are met, the next operation instruction and safety conditions will be output; if any prerequisite is not met, an alarm message or operation prompt will be output.

[0012] As a further improvement of the present invention: the inference engine module is configured to call historical eye movement data and initiate the attention verification process, specifically as follows: Based on the key regions defined in the device image, extract the operator's gaze data on the key regions; To determine whether the operator's attention to the operation of the on-site equipment meets the effective gaze criteria, the effective gaze criteria include a gaze duration of a preset time and a gaze point falling within a key area at a set threshold. If the conditions are met, the operator's attention check is deemed passed; If the operator's attention is not met, the operator's attention check is deemed to have failed, an operation prompt indicating the valid gaze standard is output, and the attention check is restarted.

[0013] As a further improvement of the present invention: the device information includes a device number and status, and the visual recognition module includes: The device identification unit is configured to locate and identify device number characters from the image to confirm the device identity; The device status recognition unit is configured to analyze the device status indication area in the image to determine the operating status of the device.

[0014] As a further improvement of the present invention: the inference engine module is further configured as follows: In response to the end signal of the last step, the final verification process is initiated as follows: Based on the final image of the equipment after the operation is completed, confirm that the final state of the equipment is qualified. Confirm the state update of the last step operation task node in the time-series knowledge graph; Confirm the upload of the entire operation record and output a prompt indicating that the operation is complete and the user is ready to leave. Receive confirmation instructions from operators indicating completion of operations and record the response time.

[0015] As a further improvement of the present invention, the edge computing box also includes a voice processing module, which is used to convert the operator's voice commands into text and to convert alarm information or operation prompt text into voice signals.

[0016] This invention also provides a local anti-misoperation method for substation switching, comprising the following steps: Determine the temporal knowledge graph corresponding to the current operation ticket; the temporal knowledge graph is defined with operation task nodes, temporal relationship edges, and security condition attributes. Collect image data from on-site equipment and eye-tracking data from operators; Based on the image data, the information of the field equipment is identified; Based on device information and eye movement data, and based on the temporal knowledge graph, reasoning is performed to verify the compliance of the current operation and the next operation triggered. Based on the verification results, output corresponding operation prompts or alarm information.

[0017] As a further improvement of the present invention: verifying the compliance of the next operation trigger specifically includes: In response to the trigger command for the next operation, traverse the prerequisites for the next operation in the time-series knowledge graph and at least verify whether the following prerequisite conditions are met; The status of the pre-operation task nodes is completed, and all safety conditions are met. The time interval between the completion of the preceding operation task node and the triggering of the next operation meets the interval threshold. Retrieve attention verification records to verify that the operator's attention to the on-site equipment meets the effective gaze standard; If all prerequisites are met, the next operation instruction and safety conditions will be output; if any prerequisite is not met, an alarm message or operation prompt will be output.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This invention constructs a system combining AI glasses and an edge computing box, and utilizes temporal knowledge graphs for real-time reasoning that integrates visual and behavioral data. This effectively overcomes the shortcomings of traditional error prevention systems, such as rigid rules, reliance on networks, and lack of personnel monitoring. By leveraging localized knowledge graph reasoning, it eliminates dependence on the main control room network, improving system reliability and real-time performance in network-constrained scenarios such as unattended stations. By introducing eye-tracking data for attention verification and deeply binding it to the safety conditions of operational steps, it achieves multi-dimensional proactive prevention and control of the key risk factor of "human negligence," significantly reducing the risk of human error caused by misidentifying equipment or reversing steps, and overall improving the safety and intelligence level of substation operations. Attached Figure Description

[0019] Figure 1 This is a logical diagram of the system modules of the present invention.

[0020] Figure 2 This is a schematic diagram of the initialization of the temporal knowledge graph and the loading of operation tickets in this invention.

[0021] Figure 3 This is a schematic diagram of the image acquisition and visual recognition of the device of the present invention.

[0022] Figure 4 This is a schematic diagram of the voltage testing operation monitoring and safety condition verification in an embodiment of the present invention.

[0023] Figure 5 This is a schematic diagram illustrating the grounding operation triggering and final verification in an embodiment of the present invention.

[0024] Figure 6 This is a flowchart illustrating an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0026] In order to solve the technical problems in the prior art, the present invention will now be further described in conjunction with the accompanying drawings and embodiments: like Figures 1 to 3 As shown, this invention discloses a local anti-misoperation operating system for substation switching, including an AI glasses terminal and an edge computing box terminal that are interconnected. The AI ​​glasses include: The data acquisition module is used to collect images of on-site equipment and eye movement data of operators; The human-computer interaction module is used to output operation prompts or alarm information to the operator; The edge computing box includes: The knowledge graph module is used to store and load the temporal knowledge graph of operation tickets; the temporal knowledge graph includes operation task nodes, temporal relationship edges that define the order and association between operation tasks, and security condition attributes associated with operation tasks. The visual recognition module is used to receive the device image and identify on-site device information, including device number and status; The inference engine module is used to receive device number, device status and eye movement data, and perform inference based on the time-series knowledge graph to verify the compliance of the current operation and the next operation. Based on the verification result, it outputs the corresponding operation prompts or alarm information.

[0027] Furthermore, the data acquisition module includes an image acquisition module and an eye-tracking module, and the human-computer interaction module includes a voice interaction module and a display module. The voice interaction module is used to play operation prompts or alarm information and receive voice commands from the operator; the display module is used to display operation steps, navigation prompts, alarm information and eye-tracking feedback.

[0028] The image acquisition unit can use a high-definition camera module to acquire images of the field equipment. The display module can be a HUD display module.

[0029] In some implementations, the inference engine module verifies the compliance of the current operation, specifically including: Verify whether the security conditions associated with the current operation task node are met; Verify whether the operation steps of the current task node meet the sequence requirements; If any safety condition is not met or any operation step does not meet the sequence requirements, it is determined to be a potential misoperation and an alarm message is generated; If all safety conditions are met and all operations comply with the sequence requirements, the current operation is deemed compliant, and the status of the current operation task node in the time sequence knowledge graph is updated.

[0030] In some implementations, the inference engine module verifies the compliance of the next operation specifically by including: In response to the trigger command for the next operation, traverse the prerequisites for the next operation in the time-series knowledge graph and at least verify whether the following prerequisite conditions are met; The status of the pre-operation task nodes is completed, and all safety conditions are met. The time interval between the completion of the preceding operation task node and the triggering of the next operation meets the interval threshold. Retrieve attention verification records to verify that the operator's attention to the on-site equipment meets the effective gaze standard; If all prerequisites are met, the next operation instruction and safety conditions will be output; if any prerequisite is not met, an alarm message or operation prompt will be output.

[0031] In some implementations, the inference engine module is configured to call historical eye movement data to initiate the attention verification process, specifically as follows: Based on the key regions defined in the device image, extract the operator's gaze data on the key regions; To determine whether the operator's attention to the operation of the on-site equipment meets the effective gaze criteria, the effective gaze criteria include a gaze duration of a preset time and a gaze point falling within a key area at a set threshold. If the conditions are met, the operator's attention check is deemed passed; If the operator's attention is not met, the operator's attention check is deemed to have failed, an operation prompt indicating the valid gaze standard is output, and the attention check is restarted.

[0032] In some embodiments, the visual recognition module includes: The device identification unit is configured to locate and identify device number characters from the image to confirm the device identity; The device status recognition unit is configured to analyze the device status indication area in the image to determine the operating status of the device.

[0033] In some implementations, the inference engine module is further configured to: In response to the end signal of the last step, the final verification process is initiated as follows: Based on the final image of the equipment after the operation is completed, confirm that the final state of the equipment is qualified. Confirm the state update of the last step operation task node in the time-series knowledge graph; Confirm the upload of the entire operation record and output a prompt indicating that the operation is complete and the user is ready to leave. Receive confirmation instructions from operators indicating completion of operations and record the response time.

[0034] As a further improvement of the present invention, the edge computing box also includes a voice processing module, which is used to convert the operator's voice commands into text and to convert alarm information or operation prompt text into voice signals.

[0035] This invention also provides a local anti-misoperation method for substation switching, comprising the following steps: Determine the temporal knowledge graph corresponding to the current operation ticket; the temporal knowledge graph is defined with operation task nodes, temporal relationship edges, and security condition attributes. Collect image data from on-site equipment and eye-tracking data from operators; Based on the image data, the information of the field equipment is identified, including the equipment number and status; Based on the device number, device status, and eye movement data, and using the aforementioned temporal knowledge graph for reasoning, the compliance of the current operation and the next operation is verified. Based on the verification results, output corresponding operation prompts or alarm information.

[0036] This invention overcomes the limitations of traditional rule engines in parsing complex logic by constructing a three-in-one operational step timing knowledge graph of "operational tasks - temporal relationships - safety conditions." It can flexibly adapt to substations of different voltage levels and switching tasks of different operation types. When power safety regulations are revised or equipment is updated, only the graph nodes and conditions need to be adjusted through a visual interface, without modifying the code, significantly reducing maintenance costs. Adopting a local deployment architecture of "AI glasses + edge computing box," all data collection, recognition, and inference are completed locally, completely eliminating dependence on the main control room network, even in unattended stations or network outage areas. Under these conditions, it can still achieve a real-time verification response of ≤50ms, ensuring operational continuity; by integrating visual recognition, eye tracking and voice interaction technologies, it forms a closed-loop monitoring system from three dimensions: equipment identity, operational status and personnel attention. This can effectively identify potential risks such as equipment misidentification, reversed steps, and negligence, reducing the misjudgment rate of switching operations to below 0.1%; at the same time, the entire process operation data is automatically recorded and uploaded to the edge computing box, providing complete evidence for operation and maintenance traceability, and comprehensively improving the safety, reliability and intelligence level of substation switching operations. This has important practical significance for promoting the unmanned transformation of power system operation and maintenance.

[0037] Implementation Case 1: This invention discloses a local anti-misoperation method for substation switching operations, which is applicable to intelligent anti-misoperation scenarios that rely on edge computing to realize real-time verification of operation steps and combine AI glasses for visual recognition and eye tracking. It can meet the safety management and control requirements of switching operations in unattended substations or network-restricted environments.

[0038] AI glasses are smart glasses that can collect image, voice, and eye-tracking data to assist operators in equipment recognition and operation prompts. Edge computing boxes are local computing devices used to process and analyze data, reducing reliance on networks and improving system real-time performance and reliability.

[0039] like Figure 1As shown, the system modules in this embodiment are divided into two main modules: the AI ​​glasses terminal and the edge computing box terminal. Considering the network data security requirements in substation production operations and the timeliness requirements of inter-module device communication, the two establish a real-time data connection through the configuration of APN cards to realize data interaction and collaborative work. The specific module composition and cooperation relationship are as follows: The AI ​​glasses integrate three sub-modules: "data acquisition," "interaction output," and "attention monitoring." These modules are responsible for data acquisition and user interaction during operation. Specifically, they include: High-definition camera module: Equipped with a high-pixel sensor, it supports high-definition video acquisition and has image enhancement function. It can clearly capture equipment images in complex environments such as strong light and low light in substations, providing a basis for subsequent equipment numbering and status identification.

[0040] Voice interaction module: Includes a microphone and a speaker. The microphone is used to collect the operator's voice commands, and the speaker is used to play system alarms and operation prompts, enabling voice interaction between the operator and the system.

[0041] HUD display module: Employs a high-definition miniature display screen, supporting high brightness, and clearly displays operation steps, equipment navigation prompts, alarm information, and eye-tracking feedback, facilitating real-time viewing by operators. HUD stands for Heads-Up Display, a display technology that displays information in front of the user's field of vision, making it convenient for operators to view operation steps and prompts.

[0042] Eye-tracking module: Using near-infrared corneal reflection technology, it can collect the operator's eye movement data in real time and determine whether the operator is effectively focusing on key areas such as equipment number markings, safety warning signs, and status indicator lights.

[0043] The edge computing box uses an industrial-grade processor to handle data processing, knowledge graph management, and reasoning decision-making. Its modules specifically include: Visual recognition module: Deploys a lightweight YOLOv5s model and a MobileNet device status classification model. After receiving device images transmitted from the AI ​​glasses, it first identifies the device number using the YOLOv5s model, then classifies the device status using the MobileNet model, and finally outputs the correspondence between the device number and the status.

[0044] It should be noted that YOLOv5s stands for You Only Look Once version 5, a fast object detection algorithm used to quickly identify objects in images, such as device IDs. MobileNet is a lightweight convolutional neural network and is existing technology. In this embodiment, the YOLOv5s model and MobileNet are used as examples to identify device IDs and status, respectively. Other existing models or algorithms can be used to accomplish the same task.

[0045] The knowledge graph module is built upon a graph database and includes three core elements: "operation task nodes," "temporal relationship edges," and "security condition attributes." Operation task nodes record basic information about the operations, temporal relationship edges define the order and association between operations, and security condition attributes specify the security requirements that each operation task must meet. This module supports a visual interface, allowing for quick addition, deletion, or modification of nodes, relationships, and conditions.

[0046] The inference engine module employs a forward-chain inference algorithm. It receives device information from the visual recognition module and attention data from the eye-tracking module, and updates the status of operation task nodes in the knowledge graph in real time (e.g., "not started," "in progress," "completed," "abnormal"). When the operator triggers the next operation, the inference engine module traverses the knowledge graph to verify the completion status of the preceding task, the satisfaction of safety conditions, and the effectiveness of attention. If any condition is not met, it is judged as a "potential misoperation."

[0047] Voice processing module: Deploys a lightweight speech recognition model and a TTS synthesis model (existing technology). The speech recognition model converts the operator's voice commands into text, and the TTS model converts system alarms or prompts into speech signals, which are then output through the AI ​​glasses' speaker.

[0048] The workflow for coordination between modules includes: 1. The edge computing box module first loads the time-series knowledge graph corresponding to the current operation ticket through the knowledge graph module, and sends the initial operation steps to the HUD display module of the AI ​​glasses for display; 2. The high-definition camera module on the AI ​​glasses captures images of the device and transmits them to the visual recognition module on the edge computing box to identify the device number and status; at the same time, the eye-tracking module collects the operator's attention data and transmits it to the inference engine module on the edge computing box. 3. The inference engine module combines "visual recognition results" and "attention data" to update the state of the current task node in the knowledge graph; 4. If the operator triggers the next operation via voice command, the inference engine module verifies the prerequisites. If the prerequisites are met, the next step is sent to the HUD display module. If the prerequisites are not met, the voice processing module generates an alarm voice, which is played by the speaker in the AI ​​glasses and the missed step is highlighted in the HUD display module.

[0049] like Figures 2 to 6 As shown, taking "the 220kV Yongliang I line being switched from cold standby to maintenance" as an example, the core steps of its operation ticket are: 1. Check that the Yongliang I line bay disconnector is in the open position → 2. Conduct voltage testing → 3. Install the grounding wire. The implementation steps of this embodiment are explained in detail: like Figure 2 As shown, step 1: Knowledge graph initialization and operation ticket loading, specifically as follows: 1.1 Template Selection and Parameter Configuration: Operators select the "500kV Pingyongjia Line from Cold Standby to Maintenance" template from the "Substation Switching Operation Map Library" on the 10.1-inch industrial touchscreen of the edge computing box's visual interface. The system automatically loads a pre-built map instance containing three core elements: Operation task nodes: There are 3 nodes, namely T001 (check that the Yongliang I-line interval switch is in the open position), T002 (voltage test), and T003 (connect the grounding wire). Each node carries the following attributes: device ID (e.g., T003 corresponds to "YL-JD"), operation type ("grounding operation"), and expected state threshold (e.g., grounding wire connection image detection). Timing relationship edges: Edge E001 (T001→T002) has the attribute "strong dependency - must precede", with no time interval limit, but it must satisfy "T002 can only start after T001 is completed"; Edge E002 (T002→T003) has the attribute "strong dependency - must precede", with no time interval limit, but it must satisfy "T002 safety conditions are fully met, i.e., the voltage test is passed"; Safety condition attribute: T002 is associated with 3 conditions C001 (voltage detection duration is greater than or equal to 3 seconds), C002 (no voltage detector audible and visual alarm is detected), and C003 (operator looks at voltage detector alarm light for ≥1.5 seconds). The condition judgment logic is "C001∧C002∧C003" (simultaneous satisfaction).

[0050] 1.2 Operation Ticket Data Import and Knowledge Graph Binding: The system retrieves the 220kV Yongliang I line operation ticket numbered "2500xxx" from the power grid management platform. Based on image recognition and OCR recognition functions, it extracts data such as operation task details, equipment parameters, and equipment target status. It then maps the fields of the operation task nodes in the knowledge graph (e.g., "A20322 disconnector and A20331 disconnector" in the operation ticket correspond to T001 equipment ID), thus completing the binding of the knowledge graph with the actual operation ticket. The binding result is simultaneously displayed on the HUD display module as "Knowledge graph loading completed, operation ticket binding successful".

[0051] 1.3 Initial Step Push and Interaction Confirmation: The knowledge graph module encapsulates the operation command of T001 ("Check that the 220kV Yongliang I-line bay switch is in the open position") into a structured command and transmits it to the AI ​​glasses via APN. The HUD display module displays the command in a "blue title bar + white text" format, while the voice module plays a prompt tone at a speed of 150 words / minute. The operator replies "Received step 1" via voice, and the system's voice processing module recognizes the command and responds with "Step 1 confirmed, you can start the operation," completing the initial interaction.

[0052] like Figure 3 As shown, step 2: Device image acquisition and visual recognition, specifically as follows: The operator, wearing AI glasses, walks to the 220kV Yongliang I line bay switch and stops 1.5 meters away from the switch mechanism box (the HUD display module indicates "current distance 1.5 meters, meeting recognition requirements" via the distance sensor). The high-definition camera module on the AI ​​glasses automatically starts, capturing equipment images (including equipment name and number) at a frame rate of 30 frames per second. Each frame is processed by the ISP module for noise reduction (removing interference from metallic reflections on substation equipment) and sharpening (enhancing the edges of equipment number characters) before being transmitted in real time to the edge computing box via the Internet of Things. The visual recognition module processes the data in the following two steps: Equipment number recognition: The YOLOv5s model performs slice processing on the equipment image. For example, it extracts features in the 80×80 pixel equipment number area (located in the upper left corner of the circuit breaker panel) and outputs the equipment number "A20322" with a confidence level of 99.2%. The system determines that "the equipment identity matches the equipment corresponding to T001". Initial state recognition: The MobileNet model focuses on the disconnector status indication area and calculates that the current disconnector status indication area is open. The disconnector robotic arm is vertical, and it is determined that "the current state is open, which is the same as the expected state of T001". The recognition result is fed back to the inference engine module, and at the same time, the HUD display module displays "Current state: open, the check state meets the requirements".

[0053] Step 3: Eye tracking and attention verification, as detailed below: The inference engine module calls historical data from the eye-tracking module to initiate the attention verification process: Key Area Definition and Data Extraction: In the equipment image acquired in step 2, the system marks two key areas: ROI1 (the area containing the equipment number "A20322", with an area of ​​200×50 pixels) and ROI2 (the area containing the tripping indicator, with an area of ​​50×50 pixels). The system then extracts the gaze data of the operator towards the two ROIs during the operation period (09:12:20-09:12:35). Effective gaze determination: Based on safety regulations regarding personnel's attention during on-site equipment operation, the "effective gaze" standard is set as follows: gaze duration ≥ 1 second and the proportion of gaze points falling within the ROI ≥ 80%. Data shows: ROI1 gaze duration 1.8 seconds (meets the standard), ROI2 gaze duration 0.5 seconds (meets the standard); Warning and Re-verification Processing: The system issues a prompt via the voice module: "Attention! Safety warning sign not effectively confirmed. Please look at the ROI2 area for more than 1 second." Simultaneously, the HUD display module highlights the ROI2 location with a red box and displays "Please confirm the safety sign here." After the operator looks at the ROI2 area for 1 second as prompted, the eye-tracking module re-collects data and determines "ROI2 gaze meets the standard," indicating the attention verification has passed. The system records "Attention verification completed, no abnormalities."

[0054] Step 4: The next operation triggers and verifies the prerequisites, as follows: The operator triggers the next step of the process by using the voice command "Start Step 2 Voltage Testing Operation" (i.e., T002). The inference engine module then initiates a forward chain verification, traversing the pre-dependencies of T002 in the graph: Task completion status verification: If the status of node T001 is "completed" and all expected statuses meet the threshold, it is determined that "the previous task completion verification has passed". Time interval verification: Read the completion time of T001 (09:12:35) and the current trigger time of T002 (09:12:40), calculate the time difference as 5 seconds, which is less than the time interval threshold (1 minute) of the E001 edge, and determine "time interval verification passed"; Attention verification result reuse: retrieve the attention verification record from step 3, confirm that ROI1-ROI2 have all met the effective fixation requirements, and determine "attention preconditions passed".

[0055] After all three checks pass, the inference engine module extracts the operation instructions and safety conditions of T002 from the knowledge graph and encapsulates them as the instruction "Electrical testing operation: Perform electrical testing on the outgoing side of Yongliang I line, which must meet the requirements of continuous testing for ≥3 seconds, no light on the voltage tester, and no sound." This instruction is then transmitted to the AI ​​glasses via the APN IoT. The HUD module displays the instruction with a "green title bar + white text," while the voice module plays "Step 2 electrical testing operation initiated. Please bring a multimeter to the outgoing side of Yongliang I line and perform the electrical testing according to the specifications." The operator replies "Step 2 received" via voice, completing the trigger confirmation.

[0056] like Figure 4 As shown, step 5: Monitoring and verifying the voltage testing operation and safety conditions, is detailed below: 5.1 Identification and Preparation Confirmation of Voltage Detector Equipment: The operator carries a high-voltage voltage detector (model: GDY-III) to the outgoing side of the Yongliang I line. The high-definition camera module at the AI ​​glasses end acquires images of the voltage detector in real time and transmits them to the visual recognition module at the edge computing box end. The YOLOv5s model performs feature recognition on the voltage detector, extracting features such as the length of the insulating rod, the shape of the voltage detection contact, and the position of the display screen. These features are matched with the "Qualified Voltage Detector Feature Library" pre-stored in the image library to confirm that the model of the voltage detector currently used meets the voltage detection requirements of the Yongliang I line (220kV), and that the insulating rod is undamaged and the voltage detection contact is not oxidized, thus determining that the "voltage detector equipment is qualified". At the same time, the eye-tracking module monitors whether the operator is looking at the "test button" area of ​​the voltage detector. If the operator presses the test button (triggering the voltage detector self-test), the voltage detector will light up and make a sound, indicating that the audible and visual alarm functions are normal, and the HUD will display "Voltage detector self-test normal, voltage detection can begin".

[0057] 5.2 Real-time monitoring of the voltage testing process: The operator places the voltage tester contacts the conductive part of the Yongliang I-line output side. The high-definition camera module on the AI ​​glasses focuses on the voltage tester display and the contact point of the contacts, acquiring images at a frame rate of 30 frames per second. At the same time, the eye-tracking module starts continuous monitoring. Voltage testing duration statistics: The inference engine module starts timing from the moment the contact touches the conductive part (the time of the frame image of "contact contacting the wire" detected by the visual recognition module: 09:15:10), and generates a "voltage testing duration timer", which is displayed in real time on the HUD (e.g., "voltage tested for 1 second, 2 seconds to go"). Voltage data recognition and judgment: The visual recognition module performs color recognition on the illuminated area of ​​the voltage detector, and the voice interaction module recognizes the alarm sound of the voltage detector. The recognition information is transmitted to the inference engine module. The inference engine module compares the recognized voltage value with the safety condition C002 (no audible and visual alarm) of T002, and updates the comparison result every 0.5 seconds. The HUD terminal simultaneously displays "Currently no power, meets safety requirements"; Real-time attention monitoring: The eye-tracking module sets the electroscope indicator light as the key area ROI4 (100×80 pixels) and collects the operator's gaze point data in real time. If the gaze point deviates from ROI4 for more than 1 second, a red warning box will immediately pop up on the HUD and display "Please look at the electroscope indicator light" until the gaze point returns to ROI4 and meets the duration requirement.

[0058] 5.3 Comprehensive Safety Condition Judgment: When the voltage detection duration timer displays "3 seconds" (time: 09:15:13), the inference engine module initiates a comprehensive safety condition judgment: C001 (Electrical Testing Duration): Actual duration 3 seconds, equal to the threshold, is judged as "compliant"; C002 (Audible and Visual Alarm): The voltage tester does not sound an alarm and the alarm light does not turn red, indicating that the standard is met. C003 (Attention Duration): According to the eye-tracking module statistics, during the period from 09:15:10 to 09:15:13, the operator's effective gaze duration on ROI4 was 3.5 seconds, which is greater than the threshold of 1.5 seconds, and is judged as "meeting the standard".

[0059] Once all three conditions are met, the inference engine module updates the status of node T002 to "completed" and marks the completion time "09:15:13" in the graph. At the same time, the HUD pushes "Step 2 voltage testing operation completed, all safety conditions met" and the voice module plays "Voltage testing qualified, ready to perform grounding wire connection operation".

[0060] like Figure 5 As shown, step 6: Grounding wire connection operation triggering and final verification, specifically as follows: 6.1 Pre-grounding wire connection verification: The operator triggers the process via voice command "Start step 3 grounding wire connection operation". The inference engine module traverses the pre-dependencies of T003 in the graph and starts the verification: Pre-task status verification: Query the node status of T001 (completed, 09:12:35) and T002 (completed, 09:15:13), confirm that both are "completed", and T002 meets all three security conditions, so it is determined that "pre-task verification passed". Grounding wire equipment compliance verification: The operator carries a sign indicating that the grounding wire is a 220kV grounding wire and runs to the outgoing end of the 220kV Yongdong I line. The high-definition camera module of the AI ​​glasses collects images of the grounding wire, and the visual recognition module identifies the appearance of the grounding wire and the material of the grounding clamp. It is confirmed that the grounding wire meets the grounding requirements of the 220kV line, and that the grounding wire has no broken strands and the grounding clamp has no rust. The grounding wire is then judged to be "compliant". Operating environment safety verification: The visual recognition module performs image analysis on the environment around the outgoing end of the 220kV Yongdong I line to confirm that no unauthorized personnel have entered the operating area (safe distance ≥ 1.5 meters) and no other live equipment is close to the grounding operating point, thus determining that "the operating environment is safe".

[0061] 6.2 Monitoring of the grounding wire connection process: Operators shall perform the grounding wire connection operation in accordance with the specification of "connecting the grounding terminal first, then the line terminal". The AI ​​glasses terminal shall monitor the process collaboratively using multiple modules. Grounding terminal connection monitoring: The high-definition camera module focuses on the connection process between the grounding terminal and the grounding electrode of the grounding wire. The visual recognition module identifies whether the grounding clamp is fully clamped to the grounding electrode (judgment criteria: the contact area between the grounding clamp and the grounding electrode is ≥90%, with no signs of loosening). It confirms that the operator uses a wrench / or manually tightens the grounding clamp bolts, and the HUD displays "Grounding terminal connection is firm". Line-end connection monitoring: When the operator connects the grounding wire line-end clamp to the outgoing end of the 220kV Yongdong I line, the visual recognition module monitors the contact status between the clamp and the conductor in real time to confirm that the clamp completely covers the conductor and there is no poor contact. At the same time, the eye-tracking module monitors whether the operator is looking at the "line-end connection point" to ensure that the operator confirms that the connection is in place. Operation sequence verification: The inference engine module records the "grounding terminal connection completion time (09:16:40)" and the "line terminal connection start time (09:16:55)", confirming that the grounding terminal connection is first and the line terminal connection is second, which complies with the safety requirement of "connecting the grounding terminal first", and determines that "the operation sequence is compliant".

[0062] 6.3 Final Status Confirmation and Process Closure: After the operator completes the grounding wire connection operation, a voice notification will announce "Step 3 operation completed," and the inference engine module will initiate the final verification. Grounding wire status confirmation: The visual recognition module collects a complete grounding wire connection image, confirms that the grounding end and line end are firmly connected, and that the grounding wire is neither loose nor tangled, and determines that "the grounding wire installation is qualified"; Knowledge graph status update: Update the status of node T003 to "Completed", mark the completion time as "09:17:20", and generate a node in the graph that says "220kV Yongdong I Line Cold Standby to Maintenance Operation Completed", and associate it with the completion records and verification data of T001-T003; Operation result feedback: The edge computing box automatically uploads the entire operation record (step completion time, verification result, and abnormal warning record) to the substation operation and maintenance management platform (local deployment version). At the same time, the HUD module of the AI ​​glasses displays the summary information "The 220kV Yongdong I line cold standby to maintenance operation is completed and the operation is compliant". The voice module plays "Operation successful. All steps meet the safety requirements. Please confirm that there are no tools left on site before leaving the operation area".

[0063] After the operator confirms that no tools have been left on site, they will reply via voice, "Operation completed, site cleanup finished." The system will record the reply time, thus completing the closed loop of the entire switching operation process.

[0064] Compared to existing rule engine-based error prevention systems, this embodiment has the following advantages: Construct a three-in-one operation step sequence knowledge graph of "operation task - temporal relationship - safety conditions", transform power safety regulations, "five prevention" rules and operation ticket templates into a structured graph model, break through the limitation of fixed rule engine in parsing complex logic, and realize flexible reasoning; The local deployment architecture of "AI glasses + edge computing box" is adopted. The knowledge graph inference engine and lightweight visual recognition model are deployed on the edge computing box. Combined with the high-definition camera and eye-tracking sensor of AI glasses, the device information is locally recognized, the operation steps are locally verified, and the visual attention is locally monitored, completely eliminating the dependence on the network or the main control room and ensuring real-time performance and reliability. Integrating multi-dimensional control through "visual recognition + eye tracking + voice interaction," the YOLOv5 model automatically identifies device numbers and status, eye tracking sensors monitor key information and gaze behavior, and the voice module provides real-time alarms and navigation, forming a closed-loop control system of "pre-operation verification - in-operation monitoring - misoperation warning." This effectively avoids risks caused by human negligence. At the same time, the structured characteristics of the knowledge graph enable rapid updates of rules and templates, reducing maintenance costs.

[0065] In summary, after reading this invention document, those skilled in the art can make various other corresponding modifications to the technical solutions and concepts based on this invention without creative mental effort, and all of these modifications fall within the scope of protection of this invention.

Claims

1. A local misoperation prevention system applied to a switching of a substation, characterized in that, An AI glasses end and an edge computing box end connected in mutual communication, The AI glasses end comprises: a data acquisition module for acquiring images of field devices and eye movement data of an operator; a human-computer interaction module for outputting operation prompts or alarm information to the operator; The edge computing box end comprises: a knowledge graph module for storing and loading a time sequence knowledge graph; the time sequence knowledge graph comprises operation task nodes, time sequence relationship edges defining the sequence and correlation between operation tasks, and safety condition attributes associated with the operation tasks; a visual recognition module for receiving the device images and identifying field device information; an inference engine module for receiving the device information and the eye movement data, and performing inference based on the time sequence knowledge graph to check the compliance of the current operation and the triggering of the next operation, and outputting corresponding operation prompts or alarm information based on the checking result.

2. The local misoperation prevention system for substation switching according to claim 1, wherein The data acquisition module comprises an image acquisition module and an eye tracking module, The human-computer interaction module comprises a voice interaction module and a display module, The voice interaction module is used to play operation prompts or alarm information and receive voice instructions of the operator; The display module is used to display operation steps, navigation prompts, alarm information and eye tracking feedback.

3. The local misoperation prevention system for substation switching according to claim 1, characterized in that, The inference engine module checks the compliance of the current operation, specifically including: verifying whether the safety conditions associated with the current operation task node meet the requirements; verifying whether the operation steps of the current operation task node meet the sequence requirements; if any safety condition does not meet the requirements or any operation step does not meet the sequence requirements, determining that there is a potential misoperation and generating an alarm information; if all safety conditions meet the requirements and all operation steps meet the sequence requirements, determining that the current operation is compliant, and updating the state of the current operation task node in the time sequence knowledge graph.

4. The local misoperation prevention system for substation switching according to claim 1, wherein, The inference engine module checks the compliance of the next operation, specifically including: in response to a triggering instruction of the next operation, traversing the preconditions of the next operation in the time sequence knowledge graph, and checking whether the following preconditions are met; querying whether the state of the pre-operation task node is completed and whether all safety conditions meet the requirements; verifying whether the time interval between the completion of the pre-operation task node and the triggering of the next operation meets the interval threshold; calling attention checking records to review whether the operator's attention to the field device operation meets the effective gaze standard; if all preconditions are met, outputting the next operation instruction and safety conditions; if any precondition is not met, outputting an alarm information or an operation prompt.

5. The local misoperation prevention system applied to the switching of a transformer substation according to claim 1 or 4, characterized in that, The inference engine module is configured to call eye movement historical data to start an attention checking process, specifically including: extracting the gaze data of the operator on the key region based on the key region defined in the device images; judging whether the operator's attention to the field device operation meets the effective gaze standard, which includes that the gaze time length reaches a preset time and the proportion of the gaze point falling within the key region reaches a set threshold; if it meets, it is determined that the operator's attention checking is passed; if it does not meet, it is determined that the operator's attention checking is not passed, an operation prompt of the effective gaze standard is outputted, and the attention checking is restarted.

6. The local misoperation prevention system for substation switching according to claim 1, wherein The device information includes a device number and a state, and the visual recognition module includes: A device identity recognition unit configured to locate and recognize device number characters from the image to confirm device identity; A device state recognition unit configured to analyze a device state indication area in the image to determine the running state of the device.

7. The local misoperation prevention system for substation switching according to claim 1, characterized in that, The inference engine module is further configured to: In response to an end signal of the last step operation, start a final verification process as follows: Based on the final image of the device after the operation is completed, confirm that the final state of the device is qualified; Confirm the state update of the last step operation task node in the timing knowledge graph; Confirm the upload of the full-process operation record and output the operation completion departure prompt; Receive the operation completion confirmation instruction of the operator and record the reply time.

8. The local misoperation prevention system for substation switching according to claim 1, characterized in that, The edge computing box end further includes a voice processing module, which is used to convert the voice instruction of the operator into text and to convert the alarm information or operation prompt text into a voice signal.

9. A local misoperation prevention method applied to a substation switching, characterized in that, The method comprises the following steps: Determine the timing knowledge graph corresponding to the current operation ticket; Collect image data of the on-site device and eye movement data of the operator; Based on the image data, identify the on-site device information; Based on the device information and eye movement data, and based on the timing knowledge graph, perform inference to verify the compliance of the current operation and the triggering of the next operation; Based on the verification result, output the corresponding operation prompt or alarm information.

10. The local misoperation prevention method for substation switching according to claim 9, characterized in that, The verification of the compliance of the triggering of the next operation specifically includes: In response to the triggering instruction of the next operation, traverse the preconditions of the next operation in the timing knowledge graph, and verify whether the following preconditions are met; Query the pre-operation task node state to be completed, and the safety conditions to be up to standard; The time interval between the completion of the pre-operation task node and the triggering of the next operation meets the interval threshold; Call the attention verification record to review whether the operator's attention to the on-site device operation meets the effective gaze standard; If all preconditions are met, output the next operation instruction and safety conditions; if any precondition is not met, output the alarm information or operation prompt.

Images