Power operation order compliance automatic identification system and method

Abstract

The invention relates to the technical field of electrical operation supervision, in particular to a power operation order compliance automatic identification system and method. The power operation ticket compliance automatic identification system comprises a data import module which is used for receiving and analyzing structured operation ticket text data and associating and importing a video file of a corresponding operation execution process; the multi-modal data processing module comprises a voice recognition unit and a video analysis unit; the compliance analysis module is used for comparing and analyzing the output results of the voice recognition unit and the video analysis unit with the operation ticket text data and judging the compliance of the operation process; and the report generation module is used for generating a structured compliance review report. Through cooperative work of the multi-modal data processing module and the compliance analysis module, automatic analysis of an operation order execution process is realized, the problems of missed judgment and misjudgment caused by subjective factors are avoided, and objectivity and fairness of an examination result are ensured.

Description

Technical Field

[0001] This invention relates to the field of electrical operation supervision technology, and in particular to an automatic identification system and method for the compliance of power operation tickets. Background Technology

[0002] In the power industry, the "operation ticket" system is a core management tool to ensure the safe and standardized execution of electrical operations, playing an irreplaceable role, especially in high-risk operations such as switching operations. According to regulations, operators must strictly follow the approved operation ticket and adhere to the "supervisor-repeated system," whereby the supervisor reads the ticket aloud, the operator repeats it, and the supervisor confirms its accuracy before the operator can execute the operation. This multi-layered control over the operation process prevents accidents caused by misoperation.

[0003] Currently, the supervision and verification of the execution process of operation tickets mainly relies on manual review after the fact, using on-site video recordings and audio recordings. Reviewers must repeatedly watch the videos and listen to the audio recordings, verifying each step of the operation to ensure consistency with the ticket content, the completeness of the monitoring and repetition process, and the adequacy of safety measures. This method has the following significant drawbacks: Low efficiency: Manual review is time-consuming and difficult to adapt to the needs of large-scale, high-frequency operation supervision, especially under heavy power grid operation tasks, where review work is severely delayed. High subjectivity: Review results are highly dependent on the experience and focus of the inspectors, easily leading to missed or incorrect judgments due to fatigue, negligence, or varying levels of expertise. Lack of real-time intervention: The existing method is post-event supervision, unable to promptly detect and stop violations during the operation, lacking in-process control capabilities. Difficulty in evidence integration: The comparison and analysis of video, audio, and ticket content lacks systematic integration, and the process of locating, collecting evidence, and statistically analyzing violations is cumbersome.

[0004] Therefore, there is an urgent need for an intelligent system that can automatically identify and analyze operational compliance in order to improve the technical level and execution efficiency of power operation safety management. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the difficulties in achieving efficient, objective and systematic supervision of the operation ticket execution process in the prior art.

[0006] To address the aforementioned technical problems, this invention provides an automatic identification system for the compliance of power operation tickets, comprising: The data import module is used to receive and parse structured operation ticket text data, and to import the corresponding video files of the operation execution process. The multimodal data processing module includes a speech recognition unit and a video analysis unit. The speech recognition unit is connected to the data import module and is used to separate the audio stream from the video file and perform speech recognition, converting the speech content into text with timestamps and role labels. The video analysis unit is used to perform image capture and behavior recognition on the video file and output analysis results with timestamps. The compliance analysis module, connected to the multimodal data processing module, is used to compare and analyze the output results of the speech recognition unit and the video analysis unit with the operation ticket text data according to the preset compliance rule base, and determine the compliance of the operation process. The report generation module, connected to the compliance analysis module, is used to generate a structured compliance review report based on the judgment results of the compliance analysis module.

[0007] Preferably, the voice recognition unit distinguishes the voice roles of the guardian and the operator through voiceprint recognition technology; and identifies and marks the voice content and occurrence time of key instructions such as calling out the ballot, repeating, and confirming.

[0008] Preferably, the video analysis unit uses a computer vision action recognition model to identify the specific operational behavior of the operator and record the time of occurrence; it automatically captures images at preset operation nodes and uses an image classification model to identify the wearing and use of safety tools, including safety helmets and insulating gloves; and it verifies the logical sequence of operation steps by analyzing continuous video frames.

[0009] Preferably, the pre-built compliance rule base includes: Rules for the integrity of the monitoring and recitation system, rules for consistency of operational content, and rules for compliance with safety regulations; The integrity rule of the monitoring and recitation system is used to verify whether each operation step has a complete chain of voice and behavioral evidence, including voting, recitation, confirmation, and operation. The operation content consistency rule is used to verify whether the operation content recognized by speech is consistent with the operation ticket text content through semantic similarity calculation. The aforementioned safety compliance rules are used to verify whether personnel wear and use safety protective equipment correctly during key operational steps.

[0010] Preferably, the structured compliance review report includes: The execution overview statistics include the total number of steps, the number of compliant steps, and the number of various violations; A detailed list of violations, with each violation associated with the time point of the violation, the type of violation, and the corresponding rule clause; Video screenshots and speech-to-text transcripts showing the time of the violation, serving as evidence.

[0011] Preferably, it also includes: a real-time early warning module, used to issue an immediate warning to the operator through the prompt module box integrated in the on-site smart safety helmet when a serious violation is identified in real-time monitoring mode.

[0012] This invention also provides a method for automatically identifying the compliance of power operation tickets, characterized by comprising: Import structured operation ticket text data and its associated operation execution video file; The video file is subjected to speech recognition to generate speech text with timestamps and character annotations; the video file is subjected to image capture and behavior analysis to generate behavior and state recognition results with timestamps. The voice text and the behavior and state recognition results are compared with a preset compliance rule base to determine the compliance of each operation step; Based on the judgment results, a structured compliance review report is generated.

[0013] Preferably, the step of comparing the voice text and the behavior and state recognition results with a preset compliance rule base to determine the compliance of each operation step includes: For each operation step, within a preset time window, check whether there is a chain of voice and behavioral evidence that are closely connected in chronological order for the voting, recitation, confirmation, and operation. If any link is missing or the chronological logic is incorrect, it is judged as a violation. The semantic similarity between the operation item text identified by speech recognition and the corresponding steps in the operation ticket text is calculated using a natural language processing model. When the similarity is lower than a preset threshold, it is determined to be a content inconsistency violation. At critical operational points such as voltage testing and grounding, check whether the operator is wearing a safety helmet and insulated gloves correctly in the image recognition results. If the operator is not identified as wearing them correctly, it is considered a violation.

[0014] Preferably, it further includes: The audio segments identified as violations and their corresponding video image segments, along with their corresponding violation tags, constitute training sample pairs. Using the training sample pairs, incremental training is performed on the acoustic model and language model in the speech recognition engine, as well as the image classification model and action recognition model in the video analysis unit, to optimize the recognition accuracy of the models.

[0015] Preferably, it further includes: Based on historical operation ticket texts and corresponding compliance review results, an operation ticket knowledge base is constructed and updated; the knowledge base includes frequently violated steps, standard operation video templates for specific equipment, and risk points for different task types; When making core compliance logic judgments, the system calls upon knowledge base information as an auxiliary basis for judgment, summarizes newly generated review results, and automatically updates the knowledge base.

[0016] The technical solution of the present invention has the following advantages compared with the prior art: The automatic identification system and method for compliance of power operation tickets described in this invention, through the collaborative work of a multimodal data processing module and a compliance analysis module, achieves automated and streamlined analysis of the operation ticket execution process. It can automatically parse the ticket content, recognize voice commands, capture key behavioral images, and perform logical judgments, reducing the manual review process, which originally required hours, to minutes, greatly improving review efficiency, freeing up manpower, and providing a technical foundation for large-scale, routine compliance screening of operation tickets. Based on a pre-set compliance rule base, the system ensures standardized rules, completely avoiding omissions and misjudgments caused by subjective factors such as varying levels of expertise among reviewers, fatigue, and negligence. This guarantees the objectivity and fairness of the review results, ensuring consistent and reliable quality for each review. Attached Figure Description

[0017] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein: Figure 1 This is a structural diagram of an automatic identification system for compliance of power operation tickets provided by the present invention; Figure 2 This is a flowchart of an automatic identification method for the compliance of power operation tickets provided by the present invention. Detailed Implementation

[0018] The core of this invention is to provide an automatic identification system for the compliance of power operation tickets. Through the collaborative work of a multimodal data processing module and a compliance analysis module, it realizes automated and pipelined analysis of the operation ticket execution process, which greatly improves the review efficiency.

[0019] To enable those skilled in the art to better understand the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Please refer to Figure 1. Figure 1 The logical relationship diagram of the automatic identification system for compliance of power operation tickets provided by the present invention is as follows: The specific operation steps are as follows: The data import module is used to receive and parse structured operation ticket text data, and to import the corresponding video files of the operation execution process. In one embodiment, the system backend receives and parses the imported, approved operation ticket text data in a structured format. This data includes at least key fields such as "Operation Task," "Operation Step Number," and "Operation Item." Examples include "Open 110kV Rain Copper Wire Circuit Breaker DL151" and "Check that 110kV Rain Copper Wire Circuit Breaker DL151 trips normally." Operators wear smart safety helmets integrated with high-definition cameras and noise-canceling microphones to record the entire operation process. After the task is completed, operators upload the video file of the operation along with the corresponding operation ticket text through the system's web interface.

[0021] The multimodal data processing module includes a speech recognition unit and a video analysis unit. The speech recognition unit is connected to the data import module and is used to separate the audio stream from the video file and perform speech recognition, converting the speech content into text with timestamps and role labels. The video analysis unit is used to perform image capture and behavior recognition on the video file and output analysis results with timestamps. Specifically, the voice recognition unit uses voiceprint recognition technology to distinguish the voice roles of the guardian and the operator; it identifies and marks the voice content and time of occurrence of key instructions such as calling out votes, repeating, and confirming.

[0022] The video analysis unit uses a computer vision action recognition model to identify the operator's specific actions and record the time of occurrence; it automatically captures images at preset operation nodes and uses an image classification model to identify the wearing and use of safety equipment, including safety helmets and insulating gloves; and it verifies the logical sequence of operation steps by analyzing continuous video frames.

[0023] In one embodiment, speech stream processing and recognition: The system separates the audio track from the video stream and calls a speech recognition engine (e.g., a deep learning-based ASR automatic speech recognition model) to convert it into text. The engine performs voiceprint separation and role labeling to distinguish between the voices of "guardian" and "operator." The recognized text is then associated with a timestamp.

[0024] For example: [16:15:30] Guardian: "Now proceed to step 10, open the 110kV Yutong wire circuit breaker DL151." (This is "calling out the vote"); [16:15:40] Operator: "Execute step 10, open the 110kV Yutong wire circuit breaker DL151" (This is a "repeated"); [16:15:50] Guardian: "Correct, execute!" (This is a "confirmation" instruction); Video Stream Processing and Image Capture Analysis: Behavior and Sequence Verification: The system uses computer vision algorithms to analyze video frames and links them with speech recognition results. Upon recognizing the "Execute!" command, the system focuses on the operator's hand movements, using a motion recognition model to determine if the "pull the switch" action was performed and recording the timing of that action. By analyzing consecutive video frames, the system can verify the execution sequence of operation steps. For example, it must recognize the "switch is off" state before it can begin detecting the "hang the sign" action, thus verifying the operational logic.

[0025] Intelligent Safety Standard Recognition: Safety Tool Recognition: The system automatically captures images at key operational nodes (such as voltage testing and grounding wire installation), and uses an image classification model to determine whether the operator is correctly wearing a safety helmet and insulated gloves, and whether the tools used (such as voltage testers and grounding wires) comply with regulations. Equipment Status Recognition: The system can recognize captured equipment images to assist in verifying operational results (such as determining whether a switch is off by recognizing indicator light status).

[0026] The compliance analysis module, connected to the multimodal data processing module, is used to compare and analyze the output results of the speech recognition unit and the video analysis unit with the operation ticket text data according to the preset compliance rule base, and determine the compliance of the operation process. Specifically, the pre-built compliance rule base includes: Rules for the integrity of the monitoring and recitation system, rules for consistency of operational content, and rules for compliance with safety regulations; The integrity rule of the monitoring and recitation system is used to verify whether each operation step has a complete chain of voice and behavioral evidence, including voting, recitation, confirmation, and operation. The operation content consistency rule is used to verify whether the operation content recognized by speech is consistent with the operation ticket text content through semantic similarity calculation. The aforementioned safety compliance rules are used to verify whether personnel wear and use safety protective equipment correctly during key operational steps.

[0027] In one embodiment, the system's multimodal compliance analysis engine begins to operate, integrating voice text, behavioral timing, and image information, and automatically comparing it with a pre-built rule base: Rule R1 (Guardian Repetition Integrity Rule): For each operation step, the engine checks for the existence of audio and behavioral evidence of a tightly sequential “vote announcement → repetition → confirmation → operation”. If any step is missing (e.g., only “vote announcement” without “repetition”, or no operation for a long time after the “confirmation” command), the step is considered to be in violation.

[0028] Rule R2 (Operation Content Consistency Rule): The engine calculates the semantic similarity between the "operation item" recognized by speech and the corresponding steps in the operation ticket text. For example, if the repeated "open the DL151 switch" is consistent with the content on the ticket, it is compliant; if the repeated "close the DL151 switch" is correct, it is considered a serious violation.

[0029] Rule R3 (Safety Compliance Rule): The engine checks whether personnel are wearing appropriate safety protective equipment in images captured during critical operational steps. For example, if image recognition detects that an operator is not wearing insulated gloves while checking for electricity, then that step is deemed a violation.

[0030] The report generation module, connected to the compliance analysis module, is used to generate a structured compliance review report based on the judgment results of the compliance analysis module.

[0031] Specifically, structured compliance review reports include: The execution overview statistics include the total number of steps, the number of compliant steps, and the number of various violations; A detailed list of violations, with each violation associated with the time point of the violation, the type of violation, and the corresponding rule clause; Video screenshots and speech-to-text transcripts showing the time of the violation, serving as evidence.

[0032] The real-time early warning module is used to issue immediate warnings to operators through the alert module box integrated into the on-site smart safety helmet when serious violations are identified in real-time monitoring mode.

[0033] In one embodiment, the report generation module summarizes all analysis results to generate an in-depth review report with multimedia evidence. This report not only includes textual conclusions but can also embed video clips and speech-to-text transcripts from the time of the violation as direct evidence. The report content is presented in a structured manner: Execution Overview: Total number of steps, number of compliant steps, and statistics on the number of various violations.

[0034] Detailed description of the violations: Time [16:15:40]: No operator repetition was detected, violating the supervised repetition rule (R1).

[0035] Time [XX:XX:XX]: Image recognition shows that the operator was not wearing insulated gloves when performing the voltage testing operation (rule RX).

[0036] Compliance score: Provides an overall compliance score for this operation.

[0037] Real-time alerts: In real-time monitoring mode, the system can make real-time judgments on serious violations (such as inconsistent content or incorrect sequence). Once identified, it can issue an immediate warning to on-site personnel through the prompt module integrated into the smart safety helmet (such as flashing LED lights, slight vibration, or audio prompts), thereby enabling in-process intervention.

[0038] This embodiment provides an automatic recognition system for the compliance of power operation tickets. Through the collaborative work of a multimodal data processing module and a compliance analysis module, it achieves automated and streamlined analysis of the operation ticket execution process. It can automatically parse the ticket content, recognize voice commands, capture key behavioral images, and perform logical judgments, reducing the manual review process, which originally required hours, to minutes. This significantly improves review efficiency, frees up manpower, and provides a technical foundation for large-scale, routine compliance screening of operation tickets. Judgments are based on a pre-set compliance rule base, with standardized rules that completely avoid omissions and misjudgments caused by subjective factors such as varying levels of expertise among reviewers, fatigue, or negligence. This ensures the objectivity and fairness of the review results, making the quality of each review consistently reliable.

[0039] like Figure 2 As shown, Figure 2 The present invention provides an automatic identification method for the compliance of power operation tickets, which is as follows: Step S201: Import the structured operation ticket text data and its associated operation execution video file; Step S202: Perform speech recognition on the video file to generate speech text with timestamps and role labels; perform image capture and behavior analysis on the video file to generate behavior and state recognition results with timestamps; Step S203: Compare the voice text and the behavior and state recognition results with the preset compliance rule base to determine the compliance of each operation step; Step S204: Based on the judgment results, generate a structured compliance review report.

[0040] Based on the above embodiments, this embodiment will provide a detailed description of step S203: Specifically, for each operation step, within a preset time window, it is checked whether there is a chain of voice and behavioral evidence that are closely connected in chronological order for voting, recitation, confirmation and operation. If any link is missing or the chronological logic is incorrect, it is judged as a violation. The semantic similarity between the operation item text identified by speech recognition and the corresponding steps in the operation ticket text is calculated using a natural language processing model. When the similarity is lower than a preset threshold, it is determined to be a content inconsistency violation. At critical operational points such as voltage testing and grounding, check whether the operator is wearing a safety helmet and insulated gloves correctly in the image recognition results. If the operator is not identified as wearing them correctly, it is considered a violation.

[0041] The automatic identification method for compliance of power operation tickets also includes: The audio segments identified as violations and their corresponding video image segments, along with their corresponding violation tags, constitute training sample pairs. Using the training sample pairs, incremental training is performed on the acoustic model and language model in the speech recognition engine, as well as the image classification model and action recognition model in the video analysis unit, to optimize the recognition accuracy of the models.

[0042] Also includes: Based on historical operation ticket texts and corresponding compliance review results, an operation ticket knowledge base is constructed and updated; the knowledge base includes frequently violated steps, standard operation video templates for specific equipment, and risk points for different task types; When making core compliance logic judgments, the system calls upon knowledge base information as an auxiliary basis for judgment, summarizes newly generated review results, and automatically updates the knowledge base.

[0043] This embodiment provides an automatic identification method for the compliance of power operation tickets. Through the collaborative work of a multimodal data processing module and a compliance analysis module, it achieves automated and pipelined analysis of the operation ticket execution process. It can automatically parse the ticket content, recognize voice commands, capture key behavioral images, and perform logical judgments, reducing the manual review process, which originally required hours, to minutes. This significantly improves review efficiency, frees up manpower, and provides a technical foundation for large-scale, routine compliance screening of operation tickets. Judgments are based on a pre-set compliance rule base, with standardized rules that completely avoid omissions and misjudgments caused by subjective factors such as varying levels of expertise among reviewers, fatigue, or negligence. This ensures the objectivity and fairness of the review results, making the quality of each review consistently reliable.

[0044] Based on the above embodiments, this embodiment provides a detailed description of an automatic identification system for the compliance of power operation tickets, as follows: This embodiment uses the typical step of "opening the 110kV Yutong wire circuit breaker DL151" in a substation switching operation as an example to illustrate the complete workflow of this system.

[0045] Step 1: System Initialization and Data Import Operation Ticket Text Import: The system backend receives and parses the approved operation ticket text data imported in a structured format (such as Excel). This data contains key fields such as "Operation Task," "Operation Step Number," and "Operation Item." (See attached...) Figure 1In the screenshot of the operation ticket, the system parsed the following: "Open the 110kV Yutong wire circuit breaker DL151"; "The DL151 circuit breaker of the 110kV Yutong line was checked and found to be operating normally." Video import process: Operators wear smart safety helmets (not shown in the image) integrated with a high-definition camera and noise-canceling microphone to record the entire operation process. After the task is completed, operators upload the video file of this operation along with the aforementioned operation ticket text through the system's web interface.

[0046] Step 2: Parallel processing and analysis of multimodal data: Speech stream processing and recognition: The system separates the audio track from the video stream and calls a deep learning-based speech recognition (ASR) engine to convert it into text.

[0047] The engine uses voiceprint separation technology to distinguish the voice roles of the "guardian" and the "operator," and associates the recognized text with precise timestamps. For example, the system identified the following key dialogue: [16:15:30] Guardian: "Now proceed to step 10, open the 110kV Yutong wire circuit breaker DL151." (This sentence is marked as "voting"). [16:15:40] Operator: "Execute step 10, open the 110kV Yutong wire circuit breaker DL151" (This sentence is marked as "repeated"); [16:15:50] Guardian: "Correct, execute!" (This sentence is marked as "confirmed"); Video stream processing and image capture analysis: Behavior and Sequence Verification: After the system recognizes the "Execute!" command at [16:15:50], the video analysis unit immediately focuses on the operator's hand area. Through the pre-trained action recognition model, the system successfully recognizes the typical action of "extending the arm and pulling the switch" at [16:16:00] and records the time of this action.

[0048] Intelligent safety compliance recognition: The system automatically captures multiple frames of on-site images before and after the operator performs the "pull away" action. Analysis using an image classification model confirms that the operator is correctly wearing a safety helmet but not insulated gloves. This recognition result is recorded along with a timestamp.

[0049] Step 3: Core Judgment of Compliance Logic: The system's multimodal compliance analysis engine begins to work, integrating all the above information and automatically comparing it with a pre-built rule base: Application rule R1 (Guardian Recitation System Integrity Rule): For the data within the engine's inspection time window [16:15:30] to [16:16:00], a complete evidence chain of "voting → recitation → confirmation → operation" was found, and the sequence was tight. Therefore, this step is deemed compliant in terms of process integrity.

[0050] Application rule R2 (Operation Content Consistency Rule): The engine calculates the semantic similarity between the operator's repeated text "Open the 110kV Yutong Line Circuit Breaker DL151" and the text on the operation ticket. If the calculation result exceeds the preset 95% threshold, it indicates that the content is completely consistent. Therefore, this step is deemed compliant in terms of operation content.

[0051] Rule R3 (Safety Compliance Rule): The engine retrieved image recognition results captured at the time of operation and detected that the operator was not wearing insulated gloves. According to the rule base definition, "opening the circuit breaker" is a critical operation that requires the wearing of insulated gloves. Therefore, this step is deemed a violation of safety regulations.

[0052] Step 4: Intelligent Report Generation and Early Warning Report Generation: The report generation module summarizes all analysis results and generates a structured multimedia review report. The report is presented as follows: Execution Overview: (Overall System Statistics) Detailed description of the violations: Time [16:16:00]: When executing "open 110kV Yutong wire circuit breaker DL151", the image recognition showed that the operator was not wearing insulated gloves, which violated the safety compliance rule (Rule R3).

[0053] Real-time alert: In another embodiment, the system operates in real-time mode. When the video analysis unit identifies the moment "operation without wearing insulating gloves" at [16:16:00], the compliance analysis engine immediately determines it as a serious violation (rule R3) and immediately triggers the vibration motor and red LED light on the smart safety helmet of the on-site operator through IoT commands, issuing a real-time warning to prompt the operator to immediately stop the operation and correct the violation.

[0054] The above embodiments clearly demonstrate the complete technical chain of this invention from data input and intelligent analysis to result output, proving that the system can effectively and automatically identify various compliance issues in the execution process of operation tickets, and has the potential to upgrade from post-event review to in-process intervention.

[0055] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. An automatic identification system for the compliance of power operation tickets, characterized in that, include: The data import module is used to receive and parse structured operation ticket text data, and to import the corresponding video files of the operation execution process. The multimodal data processing module includes a speech recognition unit and a video analysis unit. The speech recognition unit is connected to the data import module and is used to separate the audio stream from the video file and perform speech recognition, converting the speech content into text with timestamps and role labels. The video analysis unit is used to perform image capture and behavior recognition on the video file and output analysis results with timestamps. The compliance analysis module, connected to the multimodal data processing module, is used to compare and analyze the output results of the speech recognition unit and the video analysis unit with the operation ticket text data according to the preset compliance rule base, and determine the compliance of the operation process. The report generation module, connected to the compliance analysis module, is used to generate a structured compliance review report based on the judgment results of the compliance analysis module.

2. The automatic identification system for compliance of power operation tickets according to claim 1, characterized in that, The voice recognition unit uses voiceprint recognition technology to distinguish the voice roles of the guardian and the operator; it identifies and marks the voice content and time of occurrence of key instructions such as calling out votes, repeating, and confirming.

3. The automatic identification system for compliance of power operation tickets according to claim 1, characterized in that, The video analysis unit uses a computer vision action recognition model to identify the operator's specific actions and record the time of occurrence; it automatically captures images at preset operation nodes and uses an image classification model to identify the wearing and use of safety equipment, including safety helmets and insulating gloves; and it verifies the logical sequence of operation steps by analyzing continuous video frames.

4. The automatic identification system for compliance of power operation tickets according to claim 1, characterized in that, The pre-built compliance rule base includes: Rules for the integrity of the monitoring and recitation system, rules for consistency of operational content, and rules for compliance with safety regulations; The integrity rule of the monitoring and recitation system is used to verify whether each operation step has a complete chain of voice and behavioral evidence, including voting, recitation, confirmation, and operation. The operation content consistency rule is used to verify whether the operation content recognized by speech is consistent with the operation ticket text content through semantic similarity calculation. The aforementioned safety compliance rules are used to verify whether personnel wear and use safety protective equipment correctly during key operational steps.

5. The automatic identification system for compliance of power operation tickets according to claim 1, characterized in that, The structured compliance review report includes: The execution overview statistics include the total number of steps, the number of compliant steps, and the number of various violations; A detailed list of violations, with each violation associated with the time point of the violation, the type of violation, and the corresponding rule clause; Video screenshots and speech-to-text transcripts showing the time of the violation, serving as evidence.

6. The automatic identification system for compliance of power operation tickets according to claim 1, characterized in that, Also includes: The real-time early warning module is used to issue immediate warnings to operators through the alert module box integrated into the on-site smart safety helmet when serious violations are identified in real-time monitoring mode.

7. A method for automatically identifying the compliance of power operation tickets, characterized in that, include: Import structured operation ticket text data and its associated operation execution video file; The video file is subjected to speech recognition to generate speech text with timestamps and character annotations; Perform image capture and behavior analysis on video files to generate behavior and state recognition results with timestamps; The voice text and the behavior and state recognition results are compared with a preset compliance rule base to determine the compliance of each operation step; Based on the judgment results, a structured compliance review report is generated.

8. The automatic identification method for compliance of power operation tickets according to claim 7, characterized in that, The step of comparing the voice text and the behavior and state recognition results with a pre-set compliance rule base to determine the compliance of each operation step includes: For each operation step, within a preset time window, check whether there is a chain of voice and behavioral evidence that is closely connected in chronological order for the voting, recitation, confirmation, and operation. If any link is missing or the chronological logic is incorrect, it is judged as a violation. The semantic similarity between the operation item text identified by speech recognition and the corresponding steps in the operation ticket text is calculated using a natural language processing model. When the similarity is lower than a preset threshold, it is determined to be a content inconsistency violation. At critical operational points such as voltage testing and grounding, check whether the operator is wearing a safety helmet and insulated gloves correctly in the image recognition results. If the operator is not identified as wearing them correctly, it is considered a violation.

9. The automatic identification method for compliance of power operation tickets according to claim 7, characterized in that, Also includes: The audio segments identified as violations and their corresponding video image segments, along with their corresponding violation tags, constitute training sample pairs. Using the training sample pairs, incremental training is performed on the acoustic model and language model in the speech recognition engine, as well as the image classification model and action recognition model in the video analysis unit, to optimize the recognition accuracy of the models.

10. The automatic identification method for compliance of power operation tickets according to claim 7, characterized in that, Also includes: Based on historical operation ticket texts and corresponding compliance review results, an operation ticket knowledge base is constructed and updated; the knowledge base includes frequently violated steps, standard operation video templates for specific equipment, and risk points for different task types; When making core compliance logic judgments, the system calls upon knowledge base information as an auxiliary basis for judgment, summarizes newly generated review results, and automatically updates the knowledge base.

Images