Intelligent supervision method and system for electric power operation site
By deeply integrating AR glasses with the intelligent supervision system, the entire process of power operation site digital closed-loop management has been realized, solving the problems of low efficiency, insufficient standardization and poor environmental adaptability of traditional supervision methods, and improving supervision efficiency and safety.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-12
- Publication Date
- 2026-04-03
AI Technical Summary
Traditional on-site supervision methods for power operations are inefficient, rely on manual inspections which are prone to errors, have scarce expert resources and insufficient remote support, have cumbersome inspection records with poor post-event traceability, are difficult to implement in a standardized manner, and existing AR devices perform poorly in complex environments and lack knowledge interaction and data integration capabilities.
By deeply integrating AR terminals with intelligent supervision systems, spatial modeling and positioning are achieved through AR glasses, and violation identification is performed by combining lightweight AI models. Remote expert collaborative processing is supported, and data is stored and analyzed in real time, forming a closed-loop digital management system for the entire process.
It realizes the full-process digital closed-loop management of power operation site supervision tasks, improves supervision efficiency and safety, supports real-time knowledge query and expert collaboration, and is suitable for complex environments such as substation, transmission line maintenance and power infrastructure construction.
Smart Images
Figure CN121787702A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of power operation safety supervision technology, specifically relating to an intelligent supervision method and system for power operation sites. Background Technology
[0002] In power operation sites, such as substation construction, transmission line inspection, and power infrastructure projects, safety supervision is a crucial link in ensuring the safety of power production. Traditional supervision methods mainly rely on manual on-site inspections, which typically have the following problems: 1. Information acquisition is inefficient and prone to errors. Inspectors need to carry a large number of paper drawings, operating procedures, checklists and other documents with them. On-site review and searching are not only time-consuming and laborious, but also inefficient. Furthermore, relying on manual verification can easily lead to the omission of key information such as equipment parameters and safety regulations, increasing the risk of misjudgment.
[0003] 2. Scarcity of expert resources and insufficient remote support methods. When faced with complex violations or technical problems, traditional telephone and video communication cannot intuitively reproduce the scene, which can easily lead to information bias and low communication efficiency. If experts need to be on-site, there are problems of high cost and long time consumption, making it difficult to respond quickly to urgent needs.
[0004] 3. Inspection records are cumbersome and lack post-incident traceability. Inspection results rely on manual entry, and the correlation between photos and videos and specific equipment and work processes is weak. It is difficult to quickly trace the spatial location, time point, and related personnel where violations occurred, which is not conducive to the determination of responsibility and the review of problems.
[0005] 4. Standardization is difficult to implement and relies heavily on experience. The inspection process and quality are greatly affected by the subjective experience of the personnel, which can easily lead to omissions or errors in the order of steps (such as not following the standard procedure of "checking personnel qualifications first and then checking the condition of tools and equipment"), resulting in inconsistent standardization and low level of standardization in inspections.
[0006] With the development of augmented reality (AR) technology, AR glasses, as wearable devices, overlay virtual information onto real-world scenes using optical display technology, and have been gradually applied in industries and other fields. Their core technologies encompass optical displays (such as optical waveguides and freeform surfaces, affecting field of view, brightness, and lightweight design), sensors and spatial perception (such as SLAM technology for environmental modeling and localization), interaction technologies (gesture recognition, voice interaction, etc.), and low-power thermal management.
[0007] Currently, domestic and international brands such as Microsoft HoloLens, Vuzix, Lingban Technology, and Thunderbird have emerged, each with its own characteristics in terms of display accuracy, environmental adaptability, and battery life. However, they still have significant limitations in power operation site supervision scenarios. First, there is insufficient integration with the supervision and inspection business process. The multimodal interaction (voice, gesture, etc.) of the existing AR devices has a low real-time matching degree with the supervision and inspection tasks, making it difficult to efficiently support core links such as violation investigation and task execution, and failing to form a closed loop of "device-business-data". Secondly, the environmental adaptability needs to be optimized. Power operation sites often face complex environments such as high altitude, strong light, and dust. The existing AR equipment's display visibility under strong light, dustproof and waterproof protection level, and long-term operation performance are all difficult to meet the needs of on-site supervision. Third, there is a lack of knowledge interaction and data integration capabilities. The system has failed to deeply integrate resources such as typical violation databases and operating procedure databases required for supervision, and thus cannot achieve "real-time knowledge query + intelligent auxiliary decision-making". Meanwhile, the real-time encrypted storage of evidence of violations and the seamless integration with the intelligent supervision system are not yet perfect, making it difficult to fully realize the value of the data. Summary of the Invention
[0008] To address the problems existing in the prior art, this invention provides an intelligent supervision method and system for power operation sites. By deeply integrating AR technology with the intelligent supervision system, it solves the problems of low efficiency, slow response, and insufficient standardization in traditional power operation site supervision methods.
[0009] To solve the above-mentioned technical problems and achieve the above-mentioned technical effects, the present invention is implemented through the following technical solution: A method for intelligent supervision of power operation sites includes the following steps: Step 1) Supervision task creation: The backend system generates standardized supervision tasks based on the power operation plan. Each supervision task is associated with a corresponding supervision guidance card and matched with a corresponding typical violation database. Step 2) AR terminal initialization: After the inspector wears the AR terminal at the power operation site, the AR terminal first establishes a communication connection with the backend system via Wi-Fi 6 / 5G, and then constructs a 3D spatial model of the power operation site to be inspected to complete spatial positioning calibration. Step 3) Supervision task assignment: The backend system will send the generated supervision task to the designated AR terminal. The supervisor will receive the specific supervision task through the AR terminal. The AR terminal will automatically download the supervision task-related data, including work drawings, procedure documents, personnel access list and tool ledger, and cache it offline. Step 4) On-site AR-assisted supervision: After receiving a specific supervision task, the supervisor uses the AR terminal to conduct compliance checks on personnel and tools and standardization supervision of the operation process at the power operation site. Step 5) Violation identification and collaborative processing: By combining the lightweight AI model of the AR terminal with the backend system, the system can achieve real-time identification, early warning and recording of violations that occur during on-site AR-assisted supervision, and support remote experts to perform spatial annotation and voice guidance through the AR interface. Step 6) Post-inspection data processing: After the inspection task is completed, closed-loop management of violations, statistical analysis of inspection data, and model and database iteration are carried out respectively.
[0010] Furthermore, in the process of creating an inspection task, the power operation plan includes core elements such as work location, task name, risk level, personnel information, and key points of safety control.
[0011] Furthermore, during the creation of the inspection task, the inspection guidance card includes multiple sub-inspection items related to the power operation site scenario to be inspected, in order to clarify the inspection process and standards.
[0012] Furthermore, during the creation of inspection tasks, the typical violation database contains multiple frequently occurring violation clauses related to the power operation site scenarios to be inspected, providing a basis for on-site violation identification.
[0013] Furthermore, the AR terminal is AR glasses.
[0014] Furthermore, during the initialization process of the AR terminal, the AR terminal uses its built-in IMU (Inertial Measurement Unit) and visual SLAM algorithm to construct a 3D spatial model of the work site and perform spatial positioning calibration.
[0015] Furthermore, before the AR terminal is initialized, the AR terminal completes the binding of one AR terminal with one or more inspectors through the built-in management module. The binding supports AR terminal MAC code recognition and permission control, so as to accurately dispatch the inspection task to the corresponding inspector wearing the specified AR terminal and avoid the deviation of manual information transmission.
[0016] Furthermore, during the on-site AR-assisted supervision process, the compliance inspection of personnel and tools includes personnel qualification identification and tool qualification inspection.
[0017] Furthermore, the specific method for personnel qualification identification is as follows: the AR terminal uses its built-in camera to capture the faces of all workers at the power operation site, then calls a face recognition algorithm to match them with the downloaded personnel access list, and overlays the corresponding personnel name and qualification validity period on the matched workers appearing in the AR field of view in real time, while triggering an alert for the unmatched workers appearing in the AR field of view. Furthermore, the specific method for inspecting the quality of tools and equipment is as follows: the AR terminal scans the QR codes on all tools and equipment at the power operation site using its built-in camera, then queries the downloaded tool and equipment ledger to confirm the name, last test time, and next test time of each tool and equipment, and issues a real-time red warning for expired or unqualified tools and equipment appearing in the AR field of view.
[0018] Furthermore, during on-site AR-assisted supervision, the standardized supervision of the work process includes visual guidance, scenario-based risk push notifications, and real-time knowledge queries.
[0019] Furthermore, the specific method of the visualization guidance is as follows: the AR terminal overlays the supervision step guidance in the AR field of view according to the supervision guidance card, and the supervisor conducts item-by-item inspection at the power operation site according to the displayed supervision step guidance. If the supervision step guidance is not followed in the order, a pop-up reminder is displayed in the AR field of view. Furthermore, the specific method for scenario-based risk push is that the AR terminal automatically pushes frequently occurring violation clauses and risk points in the scenario based on the power operation site to be inspected, and overlays them in a prominent position in the AR field of view using AI algorithms. Furthermore, the specific method for real-time knowledge query is as follows: the supervisor asks questions to the AR terminal through voice (microphone acquisition) or gesture invocation function, and the AR terminal responds in real time based on its built-in knowledge question and answer system, and displays the corresponding procedures and clauses in the AR field of view.
[0020] Furthermore, in the process of violation identification and collaborative processing, the specific method for violation identification is as follows: the AR terminal collects audio and video data in real time. For simple violations, the lightweight AI model deployed on the AR terminal directly identifies the violation in real time and overlays it into the AR field of view. For complex violations, the AR terminal first uploads the data to the backend system for real-time analysis and identification, and then the backend system pushes the violation identification results back to the AR terminal in real time and overlays them into the AR field of view. For serious violations involving personal safety, the AR terminal directly triggers an audio-visual warning to remind the inspector to stop the violation immediately. When the inspector discovers a violation through the AR terminal, he first selects the corresponding violation clause in the typical violation database by voice. Then, the AR terminal automatically associates the current spatial location, takes a photo / video, and uploads it to the backend system in real time. Finally, the backend system saves and records the violation, generates an evidence report, and associates it with the solution in the knowledge base.
[0021] Furthermore, in the process of violation identification and collaborative processing, the specific method of collaborative processing is as follows: when inspectors encounter complex technical problems during the inspection, they initiate an "expert collaboration" request through the AR terminal. The AR terminal first transmits the real-time scene image to the backend system, and then the backend system transmits it to the PC / mobile terminal of a remote expert. The remote expert views the real-time scene image through the PC / mobile terminal, achieving first-person perspective sharing. Then, the remote expert performs spatial annotation and voice guidance on the presented real-time scene image through the PC / mobile terminal. The annotation content and guidance voice are transmitted to the AR terminal through the backend system. The annotation content is superimposed on the AR field of view of the AR terminal in real time, and the guidance voice is output to the headset of the AR terminal in real time, achieving two-way interaction. Finally, the video and annotation content of the collaboration process are automatically associated with the current inspection task and stored in the collaboration record library of the backend system, supporting subsequent playback and traceability, and realizing the archiving of collaboration records.
[0022] Furthermore, in the post-inspection data processing, the specific method of the closed-loop management of violations is as follows: after the back-end system receives the violation record uploaded by the AR terminal, it automatically matches the responsible department and project leader, and generates a violation rectification notice; after the rectification is completed, the inspector goes to the corresponding power operation site for re-inspection, and uploads the rectified photos to the back-end system through the AR terminal, and the back-end system completes the closed-loop archiving of violations. Furthermore, in the post-inspection data processing, the specific method for the statistical analysis of the inspection data is as follows: the backend system statistically analyzes the inspection data, including task completion rate, distribution of violation types, and high-frequency violation scenarios, and generates visual reports; based on historical data and AI algorithms, it identifies "high-risk operation links" to provide a basis for subsequent adjustments to the inspection focus; Furthermore, in the post-inspection data processing, the specific method for model and database iteration is as follows: newly added violation cases and inspection records are added to the typical violation database and the knowledge question-and-answer system to improve the accuracy of AI recognition and question-and-answer; based on hardware usage feedback, edge computing and cloud collaboration strategies are optimized; and based on the model and database iteration of inspection data, the system's recognition accuracy and question-and-answer matching degree gradually improve with the number of uses, forming a positive cycle of "data-model-application".
[0023] An intelligent on-site supervision system for power operations is disclosed, which is based on a collaborative architecture of AR terminal, communication network, and back-end system. The communication network is responsible for data transmission between the AR terminal and the backend system; The AR terminal is responsible for constructing a 3D spatial model of the work site and calibrating spatial positioning using a built-in IMU (Inertial Measurement Unit) and visual SLAM algorithm; receiving supervision tasks from the backend system and downloading relevant data; capturing images of workers' faces using its built-in camera, matching them with the downloaded personnel access list using a facial recognition algorithm, and overlaying the corresponding names and qualification validity periods onto matched workers in the AR field of view in real time, while triggering warnings for unmatched workers; scanning QR codes on all tools and equipment using its built-in camera, querying the downloaded tool and equipment ledger to confirm the name, last test time, and next test time of each tool and equipment, and issuing real-time red warnings for expired or unqualified tools and equipment appearing in the AR field of view; and overlaying supervision step instructions onto the AR field of view according to the supervision guidance card, and issuing pop-up reminders in the AR field of view for actions not performed in the order of the supervision step instructions. This system is responsible for automatically pushing frequently occurring violations and risk points in the power operation scene to be inspected using AI algorithms, and overlaying them in a prominent position in the AR view; identifying questions asked through voice or gesture calls, providing real-time responses based on the built-in knowledge-based question-and-answer system, and displaying the corresponding regulations in the AR view; identifying simple violations using a deployed lightweight AI model; overlaying the identification results of simple, complex, and serious violations in the AR view, and providing audio-visual warnings for serious violations; identifying the content of corresponding violation clauses in the typical violation database selected through voice calls, and automatically taking photos / videos after associating them with the current spatial location; initiating "expert collaboration" requests and transmitting the on-site footage to the remote expert's PC / mobile device via the backend system; overlaying the remote expert's annotations in the AR view in real-time, and outputting the remote expert's guidance voice in the headset in real-time; and conducting follow-up inspections after violation rectification. The backend system is responsible for generating standardized inspection tasks based on the power operation plan, and associating them with corresponding inspection guidance cards and matching them with a database of typical violations; dispatching the generated inspection tasks to the AR terminal; analyzing and identifying complex violations transmitted back by the AR terminal, and pushing the identification results to the AR terminal in real time; saving and recording the violation images uploaded by the AR terminal, generating evidence reports and associating them with solutions in the knowledge base; transmitting real-time on-site images captured by the AR terminal to the PC / mobile terminal of a remote expert for viewing, spatial annotation, and voice guidance, and transmitting the annotation content and guidance voice to the AR terminal; storing the video and annotation content of the collaboration process in the collaboration record database; and receiving the data uploaded by the AR terminal. After a violation is recorded, the system automatically matches the responsible department and project leader, and generates a violation rectification notice; it is responsible for receiving rectification photos uploaded by AR terminals and archiving violations in a closed loop; it is responsible for statistically analyzing inspection data, including task completion rate, violation type distribution, and high-frequency violation scenarios, and generating visual reports; it is responsible for identifying "high-risk operation links" based on historical data and AI algorithms, providing a basis for subsequent adjustments to inspection priorities; it is responsible for supplementing newly added violation cases and inspection records into the typical violation database and the knowledge question-and-answer system to improve the accuracy of AI recognition and question-and-answer; it is responsible for optimizing edge computing and cloud collaboration strategies based on hardware usage feedback; and it is responsible for iterating the model and database based on inspection data, so that the system's recognition accuracy and question-and-answer matching degree gradually improve with the number of uses, forming a positive cycle of "data-model-application".
[0024] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention achieves a fully digital closed-loop management of on-site power operation supervision tasks by executing the processes of "supervision task creation - supervision task assignment - on-site execution - violation tracking - data statistics - system optimization". Its backend system can link work plans, a typical violation database, and supervision guidance cards, accurately assigning standardized tasks to supervisors wearing AR glasses. Violation photography can be triggered by voice, with AR glasses collecting audio and video data in real time. By identifying anomalies (such as equipment malfunctions or operational violations), the system links to the typical violation database, generates an evidence report, and associates solutions from the knowledge base. The backend system automatically calculates the completion rate of supervision tasks and the distribution of violation types, providing a basis for adjusting subsequent supervision priorities. Simultaneously, it adds new violation cases to the typical violation database, achieving continuous optimization of "data, process, and standards".
[0025] This invention breaks through the limitations of traditional single-modal interaction. Inspectors can trigger knowledge queries through voice questions or gesture operations. The system combines gaze point data and environmental perception results to call knowledge graphs to generate answers and present relevant standard documents, operation guides, or expert advice in AR glasses, thereby achieving accurate knowledge queries and real-time guidance.
[0026] This invention adopts a highly efficient processing mode of "intelligent violation recognition + remote expert collaboration," breaking through the limitations of traditional reliance on "inspector experience + on-site expert support." When inspectors encounter complex technical problems, they can initiate an "expert collaboration" request through AR glasses. The AR glasses transmit real-time on-site images with spatial annotations to the backend, allowing experts to view them via PC / mobile devices. Experts can perform spatial annotations and provide voice guidance on the image, with the annotations superimposed on the AR glasses' field of view in real time. The video and annotations of the collaboration process will be automatically associated with the current inspection task and stored in the backend "collaboration record library," supporting subsequent playback and traceability.
[0027] In summary, this invention, through its "edge AI real-time recognition + cloud knowledge base + multi-role collaboration" architecture, effectively solves the problems of low efficiency, slow response, difficulty in obtaining evidence, and lack of standardization in traditional supervision methods. The system supports automatic evidence collection for violations, knowledge-based voice Q&A, and real-time expert annotation, making it highly suitable for various power operation scenarios such as substation operation and maintenance, transmission line repair, distribution network operations, and power infrastructure construction, as well as other complex industrial environments. It can significantly improve supervision efficiency and safety.
[0028] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description
[0029] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating the steps of the intelligent on-site supervision method for power operations according to the present invention.
[0030] Figure 2 This is a system architecture diagram of the intelligent on-site supervision system for power operations according to the present invention. Detailed Implementation
[0031] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings to provide a clearer understanding of the invention's purpose, features, and advantages. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of the invention, but are merely illustrative of the essential spirit of the invention's technical solution.
[0032] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known apparatuses, structures, and techniques associated with this application may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.
[0033] Unless the context requires otherwise, throughout the specification and claims, the word “comprising” and its variations, such as “including” and “having”, shall be understood to have an open, inclusive meaning, that is, to be interpreted as “including, but not limited to”.
[0034] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.
[0035] The singular forms “a” and “the” used in this specification and the appended claims include plural references unless otherwise expressly stated herein. It should be noted that the term “or” is generally used to mean “and / or” unless otherwise expressly stated herein.
[0036] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0037] See Figure 1 As shown, the present invention provides an intelligent supervision method for power operation sites, comprising the following steps: Step 1) Supervision Task Creation: The backend system generates standardized inspection tasks based on the power operation plan. Each inspection task is associated with a corresponding inspection guidance card and matched with a corresponding database of typical violations.
[0038] The power operation plan includes core elements such as work location, task name, risk level, personnel information, and key safety control points.
[0039] The inspection guidance card contains multiple sub-inspection items related to the power operation site scenarios to be inspected. For example, infrastructure construction scenarios need to be associated with sub-inspection items such as "safe distance inspection for ground construction" and "protective measures for glass partition installation" to clarify the inspection process and standards.
[0040] The typical violation database contains several frequently occurring violation clauses related to the power operation site scenarios to be inspected, such as "operating without a ticket" and "not wearing a safety belt while working at height," providing a basis for on-site violation identification.
[0041] Step 2) AR terminal initialization: After inspectors wear AR terminals (such as AR glasses) at the power operation site, the AR glasses first establish a communication connection with the backend system via Wi-Fi 6 / 5G, and then construct a 3D spatial model of the power operation site to be inspected through the built-in IMU (Inertial Measurement Unit) and visual SLAM algorithm, and complete spatial positioning calibration.
[0042] Before the AR terminal is initialized, the AR terminal completes the binding of one AR terminal with one or more inspectors through the built-in management module. The binding supports AR terminal MAC code recognition and permission control, so as to accurately dispatch the inspection tasks to the corresponding inspectors wearing the specified AR terminal and avoid the deviation of information transmission by humans.
[0043] Step 3) Assignment of inspection tasks: The backend system will send the generated inspection task to the designated AR terminal. The inspector will receive the specific inspection task through the AR terminal. The AR terminal will automatically download the relevant data of the inspection task (such as work drawings, procedure documents, personnel access lists and tool ledgers, etc.) and cache them offline.
[0044] Step 4) On-site AR-assisted supervision: Upon receiving a specific inspection task, inspectors use the AR terminal to conduct compliance checks on personnel and tools (including personnel qualification identification and tool qualification checks) and standardized inspections of the operation process at the power operation site (including visual guidance, scenario-based risk push and real-time knowledge query).
[0045] (1) Compliance inspection of personnel and tools: (1.1) Personnel qualification identification: The AR terminal takes pictures of the faces of all workers at the power operation site through its built-in camera, and then calls the face recognition algorithm to match them with the downloaded personnel access list (including face photos, personnel names, qualification certificates, and access permissions). The AR terminal also overlays the corresponding personnel name and qualification validity period (e.g., "Zhang San, special operation certificate valid until 2025-12-31") on the matched workers in the AR field of view in real time. The AR terminal triggers an alarm for the unmatched workers in the AR field of view. (1.2) Inspection of qualified tools and equipment: The AR terminal scans the QR codes on all tools and equipment at the power operation site with its built-in camera, and then queries the downloaded tool and equipment ledger to confirm the name, last test time and next test time of each tool and equipment (e.g., "Safety belt - suspension type, last test 2024-11-01, next test 2025-11-01"), and marks expired or unqualified tools and equipment appearing in the AR field of view in real time with red warning.
[0046] (2) Supervision of standardized work processes: (2.1) Visual guidance: The AR terminal overlays the supervision step guidance (such as "Step 1: Check if the work ticket is complete → Step 2: Check the on-site safety fence setting") in the AR field of view according to the supervision guidance card. The supervisor conducts item-by-item inspections at the power operation site according to the displayed supervision step guidance. If the supervision step guidance is not executed in the order of the supervision step guidance, a pop-up reminder will be displayed in the AR field of view.
[0047] (2.2) Scenario-based risk push: The AR terminal uses AI algorithms to automatically push frequently violated clauses (such as "preventing crush damage during cable laying") and risk points (such as "temporary power grounding inspection") in the scene of the power operation to be inspected (such as "cable laying scene in substation"), and overlays them in a prominent position in the AR field of view.
[0048] (2.3) Real-time knowledge query: Supervisors can ask questions to the AR terminal by voice (microphone acquisition) or gesture (such as voice query "safe distance requirements for high-altitude operations"). The AR terminal will respond in real time based on its built-in knowledge question and answer system and display the corresponding regulations and clauses in the AR field of view.
[0049] Step 5) Violation identification and collaborative processing: By combining the lightweight AI model of the AR terminal with the backend system, the system enables real-time identification, early warning, and recording of violations that occur during on-site AR-assisted supervision, and supports remote experts to perform spatial annotation and voice guidance through the AR interface.
[0050] (1) Violation identification: The AR terminal collects audio and video data in real time. For simple violations (such as "failure to keep safety at high altitudes"), the lightweight AI model deployed by the AR terminal directly identifies them in real time and overlays them in the AR field of view.
[0051] For complex violations (such as "incorrectly filled work tickets"), the AR terminal first uploads the information to the backend system for real-time analysis and identification, and then the backend system pushes the violation identification results to the AR terminal in real time and overlays them in the AR field of view.
[0052] For serious violations involving personal safety (such as "working at height without protection"), the AR terminal will directly trigger an audio-visual warning to remind the inspector to stop it immediately.
[0053] When an inspector discovers a violation through the AR terminal, they first select the corresponding violation clause from the typical violation database via voice (e.g., "voice input: the person in charge of the work is not on site"). Then, the AR terminal automatically associates with the current spatial location, takes a photo / video, and uploads it to the backend system in real time (supporting "upload first, then submit for verification"). Finally, the backend system saves and records the information, generates an evidence report, and associates it with the solutions in the knowledge base.
[0054] Collaborative processing: When inspectors encounter complex technical problems during inspections, they can initiate an "expert collaboration" request through the AR terminal. The AR terminal first transmits the real-time on-site image (including spatial annotations) to the backend system, and then the backend system transmits it to the PC / mobile terminal of the remote expert. The remote expert can view the real-time on-site image through the PC / mobile terminal, realizing first-person perspective sharing.
[0055] Then, remote experts can perform spatial annotations (such as "the grounding bolt is loose here") and provide voice guidance on the real-time on-site scene via PC / mobile. The annotation content and guidance voice are transmitted to the AR terminal through the backend system. The annotation content is superimposed on the AR field of view of the AR terminal in real time, and the guidance voice is output to the AR terminal's headphones in real time, realizing two-way interaction.
[0056] Finally, the video and annotations of the collaboration process are automatically associated with the current inspection task and stored in the collaboration record library of the backend system, supporting subsequent playback and traceability, and realizing the archiving of collaboration records.
[0057] Step 6) Post-inspection data processing: After the inspection is completed, closed-loop management of violations, statistical analysis of inspection data, and model and database iterations will be carried out.
[0058] (1) Closed-loop management of violations: After receiving the violation records uploaded by the AR terminal, the backend system automatically matches the responsible department and project leader, and generates a violation rectification notice.
[0059] After rectification is completed, inspectors go to the corresponding power operation site for follow-up inspection and upload photos of the rectified work to the backend system through the AR terminal. The backend system then completes the closed-loop archiving of violations.
[0060] (2) Supervision data statistics and analysis: The backend system performs statistics on supervision data, including task completion rate, distribution of violation types, and high-frequency violation scenarios, and generates visual reports. Based on historical data and AI algorithms, "high-risk operation links" are identified to provide a basis for subsequent adjustment of supervision focus.
[0061] (3) Model and database iteration: Add new violation cases and inspection records to the typical violation database and the knowledge question-and-answer system to improve the accuracy of AI recognition and question answering. Optimize edge computing and cloud collaboration strategies based on hardware usage feedback (such as AR terminal battery life and communication stability). The model and database iteration based on inspection data gradually improves the system's recognition accuracy and question-and-answer matching degree with the number of uses, forming a positive cycle of "data-model-application".
[0062] Based on the above method, the present invention also provides an intelligent supervision system for power operation sites, which is based on a collaborative architecture of AR terminal-communication network-backend system. Among them, The communication network is responsible for data transmission between the AR terminal and the backend system, and also for data transmission between the backend system and a remote PC / mobile terminal.
[0063] The AR terminal is responsible for constructing a 3D spatial model of the work site and calibrating spatial positioning using a built-in IMU (Inertial Measurement Unit) and visual SLAM algorithm; receiving supervision tasks from the backend system and downloading relevant data; capturing images of workers' faces using its built-in camera, matching them with the downloaded personnel access list using a facial recognition algorithm, and overlaying the corresponding names and qualification validity periods onto matched workers in the AR field of view in real time, while triggering warnings for unmatched workers; scanning QR codes on all tools and equipment using its built-in camera, querying the downloaded tool and equipment ledger to confirm the name, last test time, and next test time of each tool and equipment, and issuing real-time red warnings for expired or unqualified tools and equipment appearing in the AR field of view; and overlaying supervision step instructions onto the AR field of view according to the supervision guidance card, and issuing pop-up reminders in the AR field of view for actions not performed in the order of the supervision step instructions. This system is responsible for automatically pushing frequently occurring violations and risk points in the power operation scene to be inspected using AI algorithms, and overlaying them in a prominent position in the AR view; identifying questions asked through voice or gesture calls, providing real-time responses based on the built-in knowledge-based question-and-answer system, and displaying the corresponding regulations in the AR view; identifying simple violations using a deployed lightweight AI model; overlaying the identification results of simple, complex, and serious violations in the AR view, and providing audio-visual warnings for serious violations; identifying the content of corresponding violation clauses in the typical violation database selected through voice calls, and automatically taking photos / videos after associating them with the current spatial location; initiating "expert collaboration" requests and transmitting the on-site footage to the remote expert's PC / mobile device via the backend system; overlaying the remote expert's annotations in the AR view in real time, and outputting the remote expert's guidance voice in the headset in real time; and conducting follow-up inspections after violation rectification.
[0064] The backend system is responsible for generating standardized inspection tasks based on the power operation plan, and associating them with corresponding inspection guidance cards and matching them with a database of typical violations; dispatching the generated inspection tasks to the AR terminal; analyzing and identifying complex violations transmitted back by the AR terminal, and pushing the identification results to the AR terminal in real time; saving and recording the violation images uploaded by the AR terminal, generating evidence reports and associating them with solutions in the knowledge base; transmitting real-time on-site images captured by the AR terminal to the PC / mobile terminal of a remote expert for viewing, spatial annotation, and voice guidance, and transmitting the annotation content and guidance voice to the AR terminal; storing the video and annotation content of the collaboration process in the collaboration record database; and receiving the data uploaded by the AR terminal. After a violation is recorded, the system automatically matches the responsible department and project leader, and generates a violation rectification notice; it is responsible for receiving rectification photos uploaded by AR terminals and archiving violations in a closed loop; it is responsible for statistically analyzing inspection data, including task completion rate, violation type distribution, and high-frequency violation scenarios, and generating visual reports; it is responsible for identifying "high-risk operation links" based on historical data and AI algorithms, providing a basis for subsequent adjustments to inspection priorities; it is responsible for supplementing newly added violation cases and inspection records into the typical violation database and the knowledge question-and-answer system to improve the accuracy of AI recognition and question-and-answer; it is responsible for optimizing edge computing and cloud collaboration strategies based on hardware usage feedback; and it is responsible for iterating the model and database based on inspection data, so that the system's recognition accuracy and question-and-answer matching degree gradually improve with the number of uses, forming a positive cycle of "data-model-application".
[0065] This invention achieves full-process digital closed-loop management of supervision tasks by establishing a "plan-execution-recording-rectification-optimization" on-site supervision process for power operations. It enables real-time knowledge retrieval and business guidance through the integration of multimodal interaction (voice, gesture, and vision). Furthermore, it enhances the ability to identify violations in complex scenarios by constructing a collaborative violation identification mechanism combining "edge AI real-time recognition + cloud knowledge base." Through AI technology, it supports remote collaboration from a first-person perspective, enabling real-time cross-regional support of expert resources. Finally, it possesses data feedback and model evolution capabilities, continuously improving the system's recognition accuracy and business adaptability.
[0066] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for intelligent supervision of power operation sites, characterized in that, Includes the following steps: Step 1) Supervision task creation: The backend system generates standardized supervision tasks based on the power operation plan. Each supervision task is associated with a corresponding supervision guidance card and matched with a corresponding typical violation database. Step 2) AR terminal initialization: After the inspector wears the AR terminal at the power operation site, the AR terminal first establishes a communication connection with the backend system via Wi-Fi 6 / 5G, and then constructs a 3D spatial model of the power operation site to be inspected to complete spatial positioning calibration. Step 3) Supervision task assignment: The backend system will send the generated supervision task to the designated AR terminal. The supervisor will receive the specific supervision task through the AR terminal. The AR terminal will automatically download the supervision task-related data, including work drawings, procedure documents, personnel access list and tool ledger, and cache it offline. Step 4) On-site AR-assisted supervision: After receiving a specific supervision task, the supervisor uses the AR terminal to conduct compliance checks on personnel and tools and standardization supervision of the operation process at the power operation site. Step 5) Violation identification and collaborative processing: By combining the lightweight AI model of the AR terminal with the backend system, the system can achieve real-time identification, early warning and recording of violations that occur during on-site AR-assisted supervision, and support remote experts to perform spatial annotation and voice guidance through the AR interface. Step 6) Post-inspection data processing: After the inspection task is completed, closed-loop management of violations, statistical analysis of inspection data, and model and database iteration are carried out respectively.
2. The intelligent supervision method for power operation sites according to claim 1, characterized in that: In the process of creating inspection tasks The power operation plan includes core elements such as work location, task name, risk level, personnel information, and key points of safety control. The inspection guidance card contains multiple sub-inspection items related to the power operation site scenarios to be inspected, which are used to clarify the inspection process and standards; The typical violation database contains multiple frequently occurring violation clauses related to the power operation site scenarios to be inspected, providing a basis for on-site violation identification.
3. The intelligent supervision method for power operation sites according to claim 1, characterized in that: During the initialization process of the AR terminal, the AR terminal uses a built-in inertial measurement unit and visual SLAM algorithm to construct a 3D spatial model of the work site and perform spatial positioning calibration.
4. The intelligent supervision method for power operation sites according to claim 1, characterized in that: Before the AR terminal is initialized, the AR terminal completes the binding of one AR terminal with one or more inspectors through the built-in management module. The binding supports AR terminal MAC code recognition and permission control, so as to accurately dispatch the inspection task to the corresponding inspector wearing the specified AR terminal.
5. The intelligent supervision method for power operation sites according to claim 1, characterized in that: During on-site AR-assisted supervision, the compliance checks on personnel and tools include personnel qualification identification and tool conformity inspection; among which... The specific method for personnel qualification identification is as follows: the AR terminal uses its built-in camera to capture the faces of all workers at the power operation site, then calls a face recognition algorithm to match them with the downloaded personnel access list, and overlays the corresponding personnel name and qualification validity period on the matched workers appearing in the AR field of view in real time, and triggers an alert for the unmatched workers appearing in the AR field of view. The specific method for inspecting the quality of tools and equipment is as follows: the AR terminal scans the QR codes on all tools and equipment at the power operation site using its built-in camera, then queries the downloaded tool and equipment ledger to confirm the name, last test time and next test time of each tool and equipment, and provides real-time red alerts for expired or unqualified tools and equipment appearing in the AR field of view.
6. The intelligent supervision method for power operation sites according to claim 1, characterized in that: During on-site AR-assisted supervision, the standardized supervision of the work process includes visual guidance, scenario-based risk alerts, and real-time knowledge queries; among which, The specific method of the visualization guidance is as follows: the AR terminal overlays the supervision step guidance in the AR field of view according to the supervision guidance card. The supervisor conducts item-by-item inspections at the power operation site according to the displayed supervision step guidance. If the supervision step guidance is not followed in the order, a pop-up reminder is displayed in the AR field of view. The specific method of the scenario-based risk push is that the AR terminal uses AI algorithms to automatically push frequently occurring violation clauses and risk points in the scenario of the power operation site to be inspected, and overlays them in a prominent position in the AR field of view. The specific method for real-time knowledge query is as follows: the inspector calls the function by voice or gesture to ask questions to the AR terminal, and the AR terminal responds in real time based on its built-in knowledge question and answer system, and displays the corresponding procedures and clauses in the AR field of view.
7. The intelligent supervision method for power operation sites according to claim 1, characterized in that: In the process of violation identification and collaborative processing, the specific method of violation identification is that the AR terminal collects audio and video data in real time. For simple violations, the lightweight AI model deployed on the AR terminal directly identifies the violations in real time and overlays them in the AR field of view. For complex violations, the AR terminal first uploads the data to the backend system for real-time analysis and identification. Then, the backend system pushes the violation identification results back to the AR terminal in real time and overlays them into the AR field of view. For serious violations involving personal safety, the AR terminal directly triggers an audio-visual warning to remind inspectors to stop the violation immediately. When an inspector discovers a violation through the AR terminal, they first select the corresponding violation clause from the typical violation database via voice. Then, the AR terminal automatically associates the current spatial location, takes a photo / video, and uploads it to the backend system in real time. Finally, the backend system saves and records the data, generates an evidence report, and associates it with solutions in the knowledge base.
8. The intelligent supervision method for power operation sites according to claim 1, characterized in that: In the process of violation identification and collaborative processing, the specific method of collaborative processing is as follows: When inspectors encounter complex technical problems during the inspection, they initiate an "expert collaboration" request through the AR terminal. The AR terminal first transmits the real-time scene image to the backend system, and then the backend system transmits it to the PC / mobile terminal of a remote expert. The remote expert views the real-time scene image through the PC / mobile terminal, achieving first-person perspective sharing. Then, the remote expert performs spatial annotation and voice guidance on the presented real-time scene image through the PC / mobile terminal. The annotation content and guidance voice are transmitted to the AR terminal through the backend system. The annotation content is superimposed on the AR field of view of the AR terminal in real time, and the guidance voice is output to the AR terminal's headphones in real time, achieving two-way interaction. Finally, the video and annotation content of the collaboration process are automatically associated with the current inspection task and stored in the collaboration record library of the backend system, supporting subsequent playback and traceability, and realizing the archiving of collaboration records.
9. The intelligent supervision method for power operation sites according to claim 1, characterized in that: In the post-inspection data processing, the specific method of the closed-loop management of violations is as follows: after the back-end system receives the violation record uploaded by the AR terminal, it automatically matches the responsible department and project leader, and generates a violation rectification notice; after the rectification is completed, the inspector goes to the corresponding power operation site for re-inspection, and uploads the rectified photos to the back-end system through the AR terminal, and the back-end system completes the closed-loop archiving of violations. The specific method for statistical analysis of the inspection data is as follows: the backend system statistically analyzes the inspection data, including task completion rate, distribution of violation types, and high-frequency violation scenarios, and generates visual reports; based on historical data and AI algorithms, it identifies "high-risk operation links" to provide a basis for subsequent adjustments to the focus of inspections. The specific method for iterating the model and database is as follows: adding new violation cases and inspection records to the typical violation database and the knowledge question-and-answer system to improve the accuracy of AI recognition and question answering; optimizing edge computing and cloud collaboration strategies based on hardware usage feedback; and iterating the model and database based on inspection data to gradually improve the system's recognition accuracy and question-and-answer matching degree with the number of uses, forming a positive cycle of "data-model-application".
10. A system for implementing the intelligent on-site supervision method for power operations as described in any one of claims 1-9, characterized in that: Based on a collaborative architecture of AR terminal, communication network, and backend system; among which... The communication network is responsible for data transmission between the AR terminal and the backend system; The AR terminal is responsible for constructing a 3D spatial model of the work site and calibrating spatial positioning using a built-in inertial measurement unit and visual SLAM algorithm; receiving supervision tasks from the backend system and downloading relevant data; capturing images of workers' faces using its built-in camera, matching them with the downloaded personnel access list using a facial recognition algorithm, and overlaying the corresponding names and qualification validity periods onto matched workers in the AR field of view in real time, while triggering warnings for unmatched workers; scanning QR codes on all tools and equipment using its built-in camera, querying the downloaded tool and equipment ledger to confirm the name, last test time, and next test time of each tool and equipment, and issuing real-time red warnings for expired or unqualified tools and equipment appearing in the AR field of view; overlaying supervision step instructions onto the AR field of view according to the supervision guidance card, and issuing pop-up reminders in the AR field of view for actions not performed in the order of the supervision step instructions; and is responsible for... Based on the power operation site scenario to be inspected, the system uses AI algorithms to automatically push frequently occurring violation clauses and risk points in that scenario, and overlays them in a prominent position in the AR view; it is responsible for recognizing the content of questions asked through voice or gesture invocation functions, and then providing real-time responses based on its built-in knowledge question-and-answer system, displaying the corresponding regulations and clauses in the AR view; it is responsible for recognizing simple violations through a deployed lightweight AI model; it is responsible for overlaying the recognition results of simple, complex, and serious violations in the AR view, and providing audio and visual warnings for serious violations; it is responsible for recognizing the content of the corresponding violation clauses in the typical violation database selected through voice invocation functions, and automatically taking photos / videos after associating them with the current spatial location; it is responsible for initiating "expert collaboration" requests and transmitting the on-site images to the PC / mobile terminal of the remote expert via the backend system; it is responsible for overlaying the remote expert's annotations in the AR view in real time, and outputting the remote expert's guidance voice in the headset in real time; and it is responsible for the follow-up inspection after the violation rectification. The backend system is responsible for generating standardized inspection tasks based on the power operation plan, and associating them with corresponding inspection guidance cards and matching them with a database of typical violations; dispatching the generated inspection tasks to the AR terminal; analyzing and identifying complex violations transmitted back by the AR terminal, and pushing the identification results to the AR terminal in real time; saving and recording the violation images uploaded by the AR terminal, generating evidence reports and associating them with solutions in the knowledge base; transmitting real-time on-site images captured by the AR terminal to the PC / mobile terminal of a remote expert for viewing, spatial annotation, and voice guidance, and transmitting the annotation content and guidance voice to the AR terminal; storing the video and annotation content of the collaboration process in the collaboration record database; and receiving the data uploaded by the AR terminal. After a violation is recorded, the system automatically matches the responsible department and project leader, and generates a violation rectification notice; it is responsible for receiving rectification photos uploaded by AR terminals and archiving violations in a closed loop; it is responsible for statistically analyzing inspection data, including task completion rate, violation type distribution, and high-frequency violation scenarios, and generating visual reports; it is responsible for identifying "high-risk operation links" based on historical data and AI algorithms, providing a basis for subsequent adjustments to inspection priorities; it is responsible for supplementing newly added violation cases and inspection records into the typical violation database and the knowledge question-and-answer system to improve the accuracy of AI recognition and question-and-answer; it is responsible for optimizing edge computing and cloud collaboration strategies based on hardware usage feedback; and it is responsible for iterating the model and database based on inspection data, so that the system's recognition accuracy and question-and-answer matching degree gradually improve with the number of uses, forming a positive cycle of "data-model-application".