XR Glasses-Based Electrical Cabinet Inspection System and Method
The power distribution cabinet inspection system based on XR glasses utilizes a visual graphical programming module and a spatial positioning and recognition module on the XR glasses to achieve flexible configuration and real-time guidance for power distribution cabinet inspections. This solves the problems of low efficiency, error-proneness, and poor system flexibility in existing technologies, and improves the accuracy and safety of inspections.
Patent Information
- Application Number
- CN202611115834.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-27
- Publication Date
- 2026-11-17
AI Technical Summary
Existing power distribution cabinet inspection technologies are inefficient and prone to errors. Furthermore, existing XR-assisted inspection solutions lack system flexibility and are difficult to adapt to the actual operation and maintenance needs of a wide variety of power distribution cabinets and frequent changes in inspection requirements.
An XR glasses-based power distribution cabinet inspection system is adopted. Inspection tasks are configured through a visual graphical programming module. Combined with the spatial positioning and recognition module on the XR glasses, the inspection process can be flexibly configured and guided in real time. The data extraction module automatically collects instrument data and provides real-time alerts for anomalies through the anomaly alarm module.
It enables flexible configuration and standardized execution of the inspection process, improves data collection efficiency and accuracy, reduces system maintenance costs, reduces missed inspection rates and human error, and enhances the timeliness and security of inspections.
Smart Images

Figure CN123121983A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of power distribution equipment inspection technology, specifically to a power distribution cabinet inspection system and method based on XR glasses. Background Technology
[0002] Distribution cabinets are critical equipment in power systems, and their operational status directly affects the reliability of the power supply network. To ensure the safe and stable operation of distribution cabinets, power companies need to regularly arrange inspection personnel to check and record the instrument readings, switch status, indicator light signals, and other information inside the cabinets.
[0003] Currently, the following methods are mainly used for the inspection of power distribution cabinets: Inspection personnel carry paper checklists or handheld terminals, checking each distribution cabinet one by one according to a predetermined route. They visually observe instrument readings and determine switch status, manually recording the inspection data. This method has significant drawbacks: with a large number of distribution cabinets and numerous inspection items, inspectors are prone to omissions and errors in the high-intensity, repetitive work; instrument readings require manual reading and transcription, which is not only inefficient but also prone to errors due to eye fatigue or negligence. Furthermore, different teams and personnel often have different inspection focuses, operating methods, and judgment standards for the same model of equipment, resulting in inconsistent inspection quality.
[0004] Inspection personnel use tablets or smartphones to obtain inspection tasks by scanning QR codes or NFC tags, and view operation instructions and enter inspection data on their mobile devices. While this method digitizes inspection information, inspection personnel still need to frequently switch their gaze between the device and the screen, making operation inconvenient; data entry still relies on manual labor, failing to fundamentally solve the problems of low efficiency and human error.
[0005] In recent years, with the development of augmented reality (AR) technology, some companies have begun to explore the application of XR glasses in power line inspections. For example, CN114758426A discloses a power XR real-scene digital inspection system that collects positioning data and real-scene images through XR mobile devices, overlaying the device's location data and status information onto the real-scene images. CN119172501A proposes equipping safety helmets with glasses featuring augmented reality display capabilities, utilizing artificial intelligence algorithms for real-time analysis and processing of device images. CN121836663A achieves spatial correspondence between inspection status and viewing angle through multimodal data acquisition, spatial positioning, and augmented reality overlay. These solutions, to a certain extent, improve the informatization level of inspections, enabling inspection personnel to see overlaid device information within their field of vision.
[0006] However, the existing XR inspection systems generally share a common limitation: the inspection process and business logic are implemented using hard-coded methods. Specifically, the inspection steps, prompts, and data extraction rules for a specific type of distribution cabinet need to be defined by professional software developers writing program code. Once the distribution cabinet model is updated, the inspection rules are adjusted, or new inspection items are added, development resources must be reinvested to modify the code. This development model results in severely insufficient system scalability and flexibility, making it difficult to adapt to the actual operation and maintenance needs of diverse distribution cabinets and frequent changes in inspection requirements, and also limiting the widespread application of the technical solution in more scenarios.
[0007] Regarding inspection task configuration, CN121900803A proposes a method based on a mandatory behavior interface and structured scenario files. It provides functional modules in a graphical designer, allowing users to drag and drop to create inspection flowcharts, which are then written into structured scenario files. This solution binds graphics and logic through a unified interface and uses a single scenario file to connect the entire design, simulation, and deployment process. CN115525266B discloses a visual programming method suitable for power distribution IoT, employing a drag-and-drop programming approach to enable one-click deployment of developed and tested business processes to end-user and edge-side operations.
[0008] While the above solutions enable graphical configuration of the inspection process, none of them address the integration with inspection execution via XR glasses. The configured inspection process still relies on traditional terminal devices (such as robots or edge computing devices) for execution, failing to leverage the advantages of XR technology in virtual-real fusion guidance and intuitive on-site prompts.
[0009] In summary, existing power distribution cabinet inspection technologies have the following shortcomings: traditional manual inspection and mobile terminal inspection rely on manual operation, which is inefficient and prone to errors; although existing XR-assisted inspection solutions have improved the way information is presented, the configuration and modification of the inspection process still rely on professional programmers, resulting in poor system flexibility and high maintenance costs; although visual programming technology has been applied in the field of inspection task configuration, it has not yet been effectively combined with the inspection execution on the XR glasses terminal. Summary of the Invention
[0010] To address the shortcomings of existing technologies, this invention provides a power distribution cabinet inspection system and method based on XR glasses. This system and method enables flexible configuration of the inspection process and utilizes XR technology for real-time guidance and automatic data acquisition during on-site inspections, providing a power distribution cabinet inspection solution.
[0011] This invention provides a power distribution cabinet inspection system based on XR glasses, comprising: The server side includes a visual graphical programming module, which allows managers to configure the inspection tasks of the power distribution cabinet through a graphical interface. The inspection task includes inspection process nodes, prompts for each node, and data extraction rules, and generates the configured inspection task as an inspection script. The XR glasses terminal is communicatively connected to the server terminal, and the XR glasses terminal includes: The spatial positioning and recognition module is used to identify the power distribution cabinet to be inspected and determine the spatial pose of the XR glasses in the current environment. The inspection prompt module is used to receive and parse the inspection script sent by the server, and display the inspection process, specific operation steps and detailed reminders of the current power distribution cabinet on the display screen of the XR glasses in an augmented reality manner according to the inspection process nodes and prompt information defined in the inspection script. The data extraction module is used to acquire real-time images of the power distribution cabinet through the image acquisition device of the XR glasses, and automatically extract the instrument data and equipment status information on the power distribution cabinet according to the data extraction rules defined in the inspection script.
[0012] By setting up a visual graphical programming module, non-professional programmers can configure inspection tasks graphically, solving the problem that existing XR inspection systems require hard coding by professional developers and have poor flexibility. Through the inspection prompt module on the XR glasses, the process and steps are overlaid and displayed in the inspector's field of vision, realizing standardized inspection guidance, reducing the workload of inspectors, and lowering the missed inspection rate. The data extraction module automatically extracts instrument data, eliminating the tedious process of manual meter reading and improving the accuracy and efficiency of data collection.
[0013] Preferably, the visual graphics programming module includes a component library and a canvas area. The component library includes flow control components, prompt information components, and data extraction components. Users can define the execution order and logical relationship between components by dragging and dropping the components to the canvas area and connecting them, thereby generating an inspection process script containing the logical order. The server then sends the inspection process script to the XR glasses.
[0014] By combining the component library with the canvas area, the inspection process can be configured visually. Users can complete the arrangement of the inspection process by simply dragging and dropping and connecting lines, without writing program code. This reduces the technical threshold for configuring the inspection process, enabling business personnel to quickly adjust the inspection process according to on-site needs, shortening the response cycle for changes in inspection rules, and improving the system's flexibility and maintainability.
[0015] Preferably, the inspection prompt module includes a 3D registration unit and a rendering unit. The 3D registration unit is used to spatially align the prompt information with the physical entity of the power distribution cabinet based on the spatial pose calculated by the spatial positioning and recognition module. The rendering unit is used to render indicator arrows, highlight boxes, text descriptions, or operation demonstration animations within the field of view of the inspection personnel to guide the inspection personnel to conduct inspections according to the process defined in the inspection script.
[0016] The 3D registration unit achieves precise spatial alignment between virtual prompts and the physical entity of the distribution cabinet, making the prompts intuitively displayed at the corresponding real component locations. Inspection personnel do not need to switch their view between the equipment and the operation manual. The rendering unit provides guidance in various forms such as arrows, highlighted boxes, and text, enabling inspection personnel to clearly understand the target location and specific actions that need to be performed. This achieves intuitive inspection guidance and reduces reliance on personnel experience.
[0017] Preferably, the data extraction module includes an image preprocessing unit, a target detection unit, and an optical character recognition unit; the target detection unit is used to locate the positions of the instrument panel, indicator light, or switch on the power distribution cabinet; the optical character recognition unit is used to read the readings of digital or pointer instruments; the data extraction module feeds back the extracted data to the XR glasses for display in real time, and simultaneously uploads it to the server.
[0018] The image preprocessing unit performs noise reduction, enhancement, and correction on the images acquired on-site, overcoming interference factors such as uneven lighting, glass reflection, and tilted shooting angles at the power distribution cabinet site. The target detection unit accurately locates the instrument area, improving the accuracy and stability of data extraction. The optical character recognition unit automatically reads the instrument readings, replacing traditional manual visual reading and recording, significantly improving data acquisition efficiency and avoiding human transcription errors.
[0019] Preferably, the XR glasses also include an anomaly alarm module, which is connected to the data extraction module. When the data extracted by the data extraction module exceeds a preset safety threshold, or when the inspection personnel do not perform their operations according to the inspection process, the anomaly alarm module issues a visual and / or auditory alarm on the XR glasses.
[0020] By using the anomaly alarm module to perform real-time threshold judgment on the extracted data, an alarm can be immediately issued to the inspection personnel when the equipment data is abnormal, shortening the time for anomaly detection and response. At the same time, it also promptly reminds the inspection personnel when their operations deviate from the preset procedures, ensuring the standardization and integrity of the inspection operations and improving the safety and reliability of the inspection.
[0021] The present invention also discloses a method for inspecting power distribution cabinets based on the system, comprising the following steps: S1, on the server side, the inspection process, prompts and data extraction rules of the power distribution cabinet are configured through the visual graphical programming module, the inspection script is generated and sent to the XR glasses terminal; S2, the inspection personnel wearing XR glasses arrive at the inspection site, the spatial positioning and recognition module at the XR glasses identifies the target power distribution cabinet and loads the corresponding inspection script; S3, the inspection prompt module on the XR glasses parses the inspection script and gradually overlays the process, operation steps and detailed reminders of the current inspection node in the inspector's field of vision; S4, at nodes where data needs to be recorded, the data extraction module at the XR glasses automatically collects images of the power distribution cabinet and extracts instrument data and status information according to the data extraction rules in the inspection script; S5, after completing all inspection nodes, the XR glasses generate an inspection report and synchronize it to the server.
[0022] Through the complete process from S1 to S5, closed-loop management is achieved, from inspection task configuration to on-site execution and data feedback. In S1, inspection scripts are generated through visual graphical programming, making the inspection process configuration flexible and adjustable; in S2, corresponding tasks are automatically matched through spatial positioning and recognition, eliminating the need for manual selection; in S3, on-site operations are standardized through step-by-step prompts guided by XR; in S4, the workload of inspection personnel is reduced by automatically extracting data; and in S5, the electronic generation and synchronization of reports achieve the digitization and traceability of inspection data.
[0023] Preferably, in step S2, the spatial positioning and recognition module identifies the model and unique identifier of the current power distribution cabinet through synchronous positioning and mapping technology and image feature matching, and calculates the three-dimensional coordinates of the XR glasses relative to the power distribution cabinet.
[0024] By combining synchronous positioning and mapping technology with image feature matching, the automatic identification of the power distribution cabinet model and the accurate calculation of the spatial pose of the glasses were realized. This provided a coordinate reference for the accurate spatial overlay of subsequent augmented reality prompts, ensuring that the virtual guidance information could be accurately presented at the corresponding physical location of the power distribution cabinet, thus improving the accuracy of virtual-real fusion.
[0025] Preferably, in step S4, the data extraction module first performs noise reduction and distortion correction preprocessing on the acquired image, then locates the dashboard area through a target detection algorithm, and finally extracts the instrument reading through optical character recognition. For pointer-type instruments, the reading is calculated by recognizing the dial scale lines and pointer angle.
[0026] By performing noise reduction and distortion correction preprocessing on the acquired images, the influence of interference factors such as changes in ambient lighting at the power distribution cabinet, surface reflections on equipment, and camera distortion is effectively overcome. The target detection algorithm accurately locates the dashboard area, improving the accuracy of subsequent recognition. Different recognition schemes are adopted for digital instruments and pointer instruments, enabling the system to adapt to the reading needs of different types of instruments and improving the system's versatility and practicality.
[0027] Preferably, in step S4, after the data extraction is completed, the extracted data is compared with the preset safety threshold in the inspection script. When the data exceeds the threshold range, the XR glasses issue a visual and / or auditory alarm.
[0028] By comparing the extracted data with preset safety thresholds in real time, visual and / or auditory alarms are immediately issued through the XR glasses when data is abnormal, enabling inspection personnel to be aware of equipment malfunctions at the first moment, shortening the response time and improving the timeliness and safety of inspections.
[0029] Preferably, in step S1, the inspection script is a structured file in JSON or XML format, containing process metadata, a list of components, and the connection relationships between components.
[0030] By compiling the graphical process into a structured script in JSON or XML format, the inspection process is stored and transmitted in a standardized data format, facilitating data interaction between the server and the XR glasses. The structured script is easy to parse and extend. When the inspection process needs to be modified, only a new script file needs to be recompiled and distributed without modifying the application code on the XR glasses, thus improving the system's scalability and maintenance efficiency.
[0031] Compared with existing technologies, this invention has the following advantages: First, by setting up a visual graphical programming module, non-professional programmers can independently configure the inspection process through drag-and-drop and connection methods. Modifications to the inspection process do not require professional software developers, significantly reducing system maintenance costs and the response time for changes in inspection rules. This solves the technical problems of existing XR inspection systems, such as poor flexibility due to hard-coded development, and difficulty in adapting to the diverse types of distribution cabinets and frequent changes in inspection rules. Second, the spatial positioning and recognition module at the XR glasses end automatically identifies the distribution cabinet and loads the corresponding inspection task. The inspection prompt module overlays the inspection process, operation steps, and detailed reminders in an augmented reality manner within the inspector's field of vision, achieving standardized execution and immersive guidance of the inspection process. Inspectors do not need to frequently switch their gaze between equipment and manuals, reducing reliance on personnel experience. The system effectively solves the technical problems of missed inspections, incorrect inspections, and inconsistent standards among different personnel in traditional inspection methods. Third, the data extraction module automatically collects images of the distribution cabinet and extracts instrument data and equipment status information, eliminating the tedious process of manual visual reading and transcription. Combined with the anomaly alarm module, it performs real-time threshold judgment on the extracted data, immediately issuing visual and / or audible alarms when data anomalies occur, significantly improving data collection efficiency, accuracy, and inspection timeliness. Fourth, by compiling the graphical process into a structured inspection script and sending it to the XR glasses for execution, changes in inspection requirements only require reconfiguration and script re-deployment, without modifying the XR glasses' application code, improving system scalability and maintenance efficiency. Simultaneously, inspection reports are automatically generated and synchronized to the server, realizing the electronic and traceable nature of inspection data, facilitating subsequent fault analysis and maintenance decisions. Attached Figure Description
[0032] Figure 1 This is a schematic diagram of the architecture of a power distribution cabinet inspection system based on XR glasses, provided in an embodiment of the present invention.
[0033] Figure 2 This is a flowchart of the power distribution cabinet inspection method provided in an embodiment of the present invention.
[0034] Figure 3 This is a schematic diagram of the server side of the power distribution cabinet inspection system based on XR glasses provided in an embodiment of the present invention.
[0035] Figure 4 This is a schematic diagram of the interface of the visual graphical programming module provided in an embodiment of the present invention.
[0036] Figure 5 This is a schematic diagram illustrating the viewing angle effect of XR glasses-side inspection prompts and data extraction provided in an embodiment of the present invention. Detailed Implementation
[0037] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0038] In this invention, XR stands for Extended Reality, which is a collective term for Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR), and their derivative technologies. The XR glasses described in this invention refer to a head-mounted display device capable of spatially aligning and overlaying computer-generated virtual prompts (including but not limited to text, graphics, arrows, or highlighted frames) with the physical entity of a power distribution cabinet in the real environment. For ease of description, the following embodiments use augmented reality (AR) glasses as an example, but the scope of protection of this invention is not limited to AR glasses.
[0039] like Figure 1 As shown, this embodiment of the invention provides a power distribution cabinet inspection system based on XR glasses, which mainly consists of two parts: a server and an XR glasses terminal.
[0040] The server-side, deployed in the cloud or local data center, is responsible for scheduling inspection tasks, compiling and distributing inspection scripts, centrally storing inspection data, and managing the backend. The server includes a visual graphical programming module, a task management module, a data storage module, and a user management module. The visual graphical programming module allows administrators to configure inspection tasks for power distribution cabinets through a graphical interface. Inspection tasks include inspection process nodes, prompts for each node, and data extraction rules, and generate inspection scripts from the configured tasks. The task management module manages, publishes, and archives the configured inspection tasks. The data storage module stores all instrument data collected during the inspection process, equipment status information, and the final inspection report. A full-process inspection backtracking function is included. The image acquisition unit on the XR glasses simultaneously records a first-person perspective video stream during the inspection process. The video data is encoded, temporarily stored locally, and backed up in real-time before being uploaded to the server. The video stream is simultaneously overlaid with timestamps, current inspection node identifiers, and real-time extracted instrument data, achieving a one-to-one correspondence between the video and the inspection process nodes. After the inspection is completed, the entire process record is automatically archived, supporting multi-dimensional retrieval and playback by inspection task ID, equipment number, abnormal events, and inspection personnel. It provides functions such as playback speed adjustment, key node jumps, and abnormal frame location, for subsequent fault analysis, inspection quality verification, and operator training. The user management module manages the permissions of users with different roles. The server also has a remote collaboration management unit responsible for establishing remote sessions, controlling permissions, and recording the process. Remote assistance can be initiated by inspection personnel via voice / keystrokes, or by backend administrators when abnormal data is detected; audio, video, and annotation data during the assistance process are archived throughout, supporting subsequent review and retrieval.
[0041] like Figure 4As shown, the visual graphical programming module adopts a web-based graphical interface. This interface is developed using the React framework combined with the Ant Design UI component library. Its core is a canvas editor implemented based on HTML5 Canvas and SVG technology, supporting component drag-and-drop, scaling, and connection operations. The server-side visual graphical programming module provides a web-based interactive interface. The left side of the interface is the component library, containing graphical components such as "Start Node," "Identify Distribution Cabinet," "Display Prompt Text," "Highlight Indicator Area," "Extract Instrument Data," "Judgment Threshold," and "End Node." Business personnel only need to drag these components to the central canvas area and define the execution logic between them by connecting them. For example, configuring a "Check Voltmeter" process: First, drag in the "Highlight Indicator Area" component and select the position of the voltmeter on the 3D model of the distribution cabinet; then connect the "Display Prompt Text" component and enter "Please check if the voltage of phase A is normal"; then connect the "Extract Instrument Data" component and configure the extraction rule to "Read digital panel value"; finally, connect the "Judgment Threshold" component and set the normal range to 210V-230V. Once configured, the system automatically compiles the graphical process into an inspection script in JSON or XML format and sends it to the XR glasses.
[0042] Furthermore, the visual graphics programming module includes a component library, a canvas area, a property panel, and a script compiler.
[0043] The component library pre-configures various functional components, each encapsulated as an independent React component, containing both a UI presentation layer and a business logic layer. Components communicate with each other using a publish-subscribe pattern. The library includes flow control components, notification components, and data extraction components. Flow control components define the start, end, and sequence of inspection processes. Notification components define the notification content displayed in the inspector's field of vision. Data extraction components define the data types and extraction methods to be automatically extracted from the distribution cabinet. In addition to these three types of components, the library further includes identification components for identifying distribution cabinets and their components; highlighting components for highlighting specific areas in the field of vision; judgment components for threshold judgments on extracted data; and branching components for implementing conditional branching logic.
[0044] The canvas area provides a visual workflow editing interface. Users drag and drop desired components from the component library onto the canvas area, and the system generates a unique UUID identifier for each component. Users establish connections between components on the canvas by dragging and dropping with the mouse, defining the execution order and logical relationships between components. Internally, the system uses a directed acyclic graph data structure to store and manage these connections, ensuring the logical correctness of the workflow.
[0045] The Properties panel is used to set specific parameters for each component. When a user selects a component in the canvas area, the Properties panel dynamically loads the component's configurable parameters, such as the target coordinate range for identifying the component and the threshold range for data extraction. These parameters are stored in the component's configuration object in the form of key-value pairs.
[0046] The script compiler starts after the user completes the process configuration and clicks the "Compile" button. The script compiler first traverses all components and their connections in the canvas area to build a complete process execution tree; then it performs syntax checks and logic verification, including checking for unconnected isolated components, components without required parameters, and whether the process has clear start and end nodes; after successful verification, it compiles the graphical process into a structured JSON-formatted inspection script. This inspection script contains the process's metadata (such as process name, creation time, applicable device model, etc.), a list of components (type, parameters, and location information for each component), and the connection relationships between components (source component ID, target component ID, and connection type).
[0047] After compilation, the system distributes the inspection script to the designated XR glasses device. The server-side has a built-in inspection process template library covering typical power distribution inspection tasks, including but not limited to routine inspections of low-voltage outgoing line cabinets, special inspections of 10kV incoming line cabinets, bus tie cabinets, and capacitor bank cabinets. Each template has a pre-defined standardized inspection path, a list of mandatory inspection components, data extraction rules, and safety threshold ranges. Administrators can directly call the templates to quickly generate inspection tasks, or customize them using a visual graphical programming module. After adjustment, the template can be saved as a new template or overwritten, enabling process reuse across multiple shifts and scenarios. When a template is instantiated as an inspection task, the system automatically verifies the integrity of the process logic and the compliance of the parameters, ensuring that the generated inspection script can be correctly parsed and executed by the XR glasses.
[0048] The XR glasses are worn by frontline inspection personnel as a terminal for on-site interaction and data acquisition. The XR smart glasses used in this system are head-mounted inspection terminals with core hardware specifications meeting the following requirements: a field of view ≥40°, a single-eye display resolution of 1920×1080; support for three interaction methods: voice interaction, head posture control, and physical buttons, suitable for on-site two-handed operation scenarios; built-in binocular RGB cameras and an inertial measurement unit, with an image acquisition frame rate ≥30fps; equipped with an edge-side AI processor, supporting local inference operation of target detection and character recognition models; the device supports dual-mode wireless communication of 5G and Wi-Fi 6, ensuring stable data transmission. The XR glasses and the server interact bidirectionally via an encrypted communication link. The XR glasses include a spatial positioning and recognition module, an inspection prompt module, a data extraction module, and an anomaly alarm module.
[0049] The spatial positioning and recognition module is used to identify the power distribution cabinet to be inspected and determine the spatial pose of the XR glasses in the current environment.
[0050] Furthermore, the spatial positioning and recognition module adopts a simultaneous positioning and mapping scheme that integrates vision and inertial measurement units, and realizes real-time environmental perception and positioning based on a mature visual SLAM algorithm library; the binocular image acquisition unit built into the XR glasses acquires on-site image sequences, extracts scene feature points and constructs a 3D point cloud map of the environment, and completes pose calculation by combining the motion data of the inertial measurement unit, providing a spatial coordinate reference for augmented reality information overlay.
[0051] In the target recognition stage, this module employs a lightweight deep learning detection model optimized for power distribution cabinet scenarios. The optimization strategies include: adding a small target detection layer at the end of the backbone network to accommodate the recognition needs of small components such as indicator lights and scale markings on the power distribution cabinet surface; and replacing some standard convolutions with depthwise separable convolutions to compress the number of model parameters and adapt to the computational constraints of XR glasses. After pre-training on a general target detection dataset, the model completes transfer learning and fine-tuning using self-collected power distribution cabinet scene image datasets, enabling accurate recognition of different types of power distribution cabinets and key components such as instruments, switches, and indicator lights on the cabinets. To address interference from uneven lighting in the power distribution room, surface reflections, and oil stains, the module incorporates multi-frame temporal fusion and adaptive image enhancement mechanisms to improve recognition accuracy and stability under complex conditions.
[0052] After target recognition is completed, the 3D registration process begins: the module extracts local feature points of the power distribution cabinet to complete feature matching and mismatch removal. Combined with the pose data output by spatial positioning, the pose matrix of the image acquisition unit relative to the power distribution cabinet entity is calculated, realizing the spatial coordinate mapping between virtual prompt information and physical equipment, ensuring accurate alignment and overlay of augmented reality content with the real cabinet.
[0053] The inspection prompt module is used to receive and parse the inspection script sent by the server. Based on the inspection process nodes and prompt information defined in the inspection script, it overlays the inspection process, specific operation steps and detailed reminders of the current power distribution cabinet on the XR glasses display screen in an augmented reality manner.
[0054] Furthermore, the inspection prompt module includes a 3D registration unit and a rendering unit. The 3D registration unit is used to spatially align the prompt information with the physical entity of the distribution cabinet based on the spatial pose calculated by the spatial positioning and recognition module. The rendering unit is used to render indicator arrows, highlight boxes, text descriptions, or operation demonstration animations within the inspector's field of view to guide the inspector to perform the inspection according to the process defined in the inspection script.
[0055] Specifically, the 3D registration unit uses the spatial positioning and recognition module to calculate the spatial pose of the XR glasses and determine the rendering position of each virtual prompt element on the display screen. For each prompt message in the inspection script that needs to be displayed at a specific location on the power distribution cabinet, the 3D registration unit calculates the position of the prompt message in the pixel coordinate system of the display screen through coordinate transformation based on the coordinates of that location in the 3D model of the power distribution cabinet and the current pose of the XR glasses.
[0056] The rendering unit renders the corresponding augmented reality prompts within the inspector's field of view based on the node's component type and parameters. The system uses the OpenGL ES 3.0 graphics API for 3D rendering, overlaying virtual elements such as arrows, highlighted boxes, and text onto the user's field of view using perspective projection. For applications requiring specific areas to be highlighted, the system employs a stencil buffer-based rendering technique. The area to be highlighted is first marked in the stencil buffer, and then a semi-transparent colored overlay is rendered within that area.
[0057] For the "Display Prompt Text" component, the system generates a virtual text label in the space near the target component to display pre-configured prompt text, such as displaying operation prompts next to the circuit breaker that needs to be checked.
[0058] For the "highlighted indicator area" component, the system overlays a highlight box on the corresponding physical location of the distribution cabinet according to the configured target area coordinates, selects the specific components that the inspection personnel need to pay attention to, and generates an indicator arrow near the highlight box to point to the area.
[0059] The rendering of all virtual elements is completed using the display device of XR glasses. Through virtual-real fusion technology, virtual information is superimposed on the real scene, and inspection personnel can obtain operation guidance without shifting their gaze.
[0060] The data extraction module is used to acquire real-time images of the power distribution cabinet through the image acquisition device of the XR glasses, and automatically extract the instrument data and equipment status information on the power distribution cabinet according to the data extraction rules defined in the inspection script.
[0061] Specifically, the data extraction module adopts a multi-stage pipeline processing architecture, including an image preprocessing unit, a target detection unit, and an optical character recognition unit.
[0062] The image preprocessing unit processes the acquired raw images. The system employs image processing algorithms based on the OpenCV library, including Gaussian filtering for noise reduction, histogram equalization for contrast enhancement, and perspective transformation for distortion correction, preparing the data for subsequent extraction. Specifically, for perspective distortion of the target area caused by the shooting angle, the preprocessing unit corrects the target area into a frontal view image using a perspective transformation matrix.
[0063] The target detection unit uses an improved YOLOv5 model to perform target detection on the preprocessed image, locating the positions of dashboards, indicator lights, or switches on the power distribution cabinet. Simultaneously, the target detection unit uses the spatial location information of components recorded in the inspection script to verify and correct the detection results, eliminating false detections.
[0064] The optical character recognition unit (OCR) is used to read the readings of digital or analog instruments. For digital display instruments, the system calls an OCR engine based on a CRNN architecture for digit recognition. This engine combines CNN for image feature extraction, BiLSTM for sequence dependency capture, and CTC loss function for end-to-end training, enabling it to recognize digits in various fonts and display formats. For analog instruments, the system employs a pointer detection algorithm based on Hough transform. First, Canny edge detection is used to extract the edge information of the instrument. Then, Hough linear transform is used to detect the straight line segment of the pointer. Combined with the geometric features of the instrument dial, the rotation angle of the pointer is calculated. Finally, the angle is converted into the actual reading based on the scale range.
[0065] The data extraction module feeds back the extracted data to the XR glasses for display in real time, and simultaneously uploads it to the server.
[0066] The anomaly alarm module is used to issue alarm prompts when the extracted data is abnormal or the inspection operation does not conform to the procedure. The anomaly alarm module is connected to the data extraction module. When the data extracted by the data extraction module exceeds the preset safety threshold, or when the inspection personnel's operation is not performed according to the inspection procedure, the anomaly alarm module issues visual and / or auditory alarm prompts on the XR glasses.
[0067] The abnormal alarm module has preset safety threshold ranges corresponding to the judgment components in the inspection script. These threshold parameters are configured entirely through the server-side visual graphical programming module, requiring no code development. Taking a low-voltage distribution cabinet voltmeter inspection scenario as an example: The administrator drags the "Judgment Threshold" component into the canvas area of the visual graphical programming module, establishing a logical connection between it and the preceding "Extract Instrument Data" component. After selecting this judgment component, the right-hand property panel dynamically loads configurable parameters, including data type dropdown options (voltage, current, temperature, etc.), lower threshold input boxes, upper threshold input boxes, and abnormal trigger action options (voice prompt, highlighted alarm, pause process). The administrator selects "Voltage" as the data type, enters a lower limit of 210V and an upper limit of 230V, and selects "highlight flashing alarm + voice prompt" as the abnormal action. After configuration, the script compiler serializes the above parameters into structured data and embeds it into the corresponding node of the inspection script, which is then sent to the XR glasses along with the script. When the XR glasses reach this node and complete the extraction of instrument data, the abnormal alarm module automatically reads the threshold parameters carried in the inspection script, compares the real-time extracted voltage value with the threshold range of [210V, 230V], and immediately triggers the corresponding bright flashing and voice alarm when the value exceeds the range.
[0068] After the data extraction module completes the data extraction, the anomaly alarm module compares the extracted data with the corresponding safety threshold. When the deviation exceeds the preset tolerance range, it is judged as an anomaly.
[0069] When an anomaly is detected, the anomaly alarm module triggers a multimodal alarm mechanism. Visually, the system displays a prominent red warning sign and anomaly value in the XR glasses' field of vision; auditorily, the system plays preset voice alarm prompts through bone conduction headphones; tactilely, the system controls the XR glasses' vibration motor to generate specific vibration patterns. Simultaneously, the system automatically records the time, location, and data value of the anomaly and generates an anomaly event log. The XR glasses also include a remote assistance unit for first-person perspective remote collaborative guidance. This unit comprises an audio-visual interaction subunit and an annotation rendering subunit: The audio-visual interaction subunit employs a multi-microphone array noise reduction scheme, using beamforming technology to directionally pick up the inspector's voice, suppressing background interference such as equipment operating noise in the power distribution room environment, and improving the clarity of voice communication; at the communication level, a 5G low-latency transmission scheme is used, optimizing audio-visual encoding and decoding strategies and transmission links to ensure end-to-end audio-visual interaction latency ≤50ms, meeting real-time collaboration requirements. The annotation rendering subunit receives annotation instructions from remote experts, and after spatial alignment of the expert's annotation boxes, arrows, and text annotations through the 3D registration unit, overlays and renders them into the inspector's field of view. Annotations support two modes: static spatial locking and dynamic trajectory. Static annotations are anchored to a fixed physical position on the distribution cabinet and do not shift with the inspector's head movement; dynamic annotations synchronize with the expert's operation trajectory in real time, used for step-by-step guidance of complex operations. Annotation spatial locking is achieved through coordinate mapping: based on the camera pose matrix of the current frame, the system converts the expert's annotation coordinates in the 2D image into 3D world coordinates through inverse perspective projection, and associates them with the scene SLAM map points; in subsequent frames, the system continuously tracks the 3D point and reprojects it into the display field of view, ensuring the annotation position is stable and drift-free.
[0070] The communication process between the server and the XR glasses is as follows: The server sends the compiled inspection script to the XR glasses; the XR glasses execute the inspection task according to the script and send the collected instrument data, status information and final inspection report back to the server.
[0071] The system includes a mobile application, which, together with the back-end management system and XR glasses, forms a complete inspection system platform. The mobile app is deployed on the handheld smart devices of inspection personnel, and its core functions include: Task Management: View the list of assigned inspection tasks, support pre-downloading and local caching of inspection scripts, adapt to offline inspection mode in scenarios with poor network signal, and automatically synchronize data after the network is restored; Glasses Connectivity: Establish communication with XR glasses via Bluetooth or Wi-Fi to enable one-click distribution of inspection scripts, device status monitoring, and preview of collected data; Report Management: After the inspection is completed, preview and confirm the inspection report, support electronic signature and one-click submission for archiving, and update the task status synchronously.
[0072] like Figure 2 As shown, this invention implements a standardized inspection process through a preset inspection script. The specific execution steps are as follows: S1 configures the inspection process, prompts, and data extraction rules for the power distribution cabinet on the server side through a visual graphical programming module, generates the inspection script, and sends it to the XR glasses.
[0073] Administrators log in to the server-side visual programming module via a browser, drag and drop required components from the component library onto the canvas area, define execution logic by connecting them, and configure the parameters of each component in the property panel to complete the graphical configuration of the inspection process. Clicking the "Compile" button, the script compiler compiles the graphical process into a structured inspection script, and the user-configured graphical process into a JSON-formatted structured inspection script. The inspection script contains process metadata, a component list, and the connection relationships between components. The system distributes the inspection script to the specified XR glasses device via a RESTful API interface, encrypting the script content using the AES-256 encryption algorithm during transmission.
[0074] S2, the inspection personnel wearing XR glasses arrive at the inspection site, and the spatial positioning and recognition module at the XR glasses identifies the target power distribution cabinet and loads the corresponding inspection script.
[0075] After inspection personnel wearing XR glasses arrive at the inspection site, the spatial positioning and recognition module uses synchronous positioning and mapping technology and image feature matching to identify the model and unique identifier of the current power distribution cabinet, and calculates the three-dimensional coordinates of the XR glasses relative to the power distribution cabinet. Upon successful identification, the system loads the corresponding inspection script based on the power distribution cabinet identifier.
[0076] S3, the inspection prompt module on the XR glasses parses the inspection script and gradually overlays the process, operation steps and detailed reminders of the current inspection node in the inspector's field of vision.
[0077] The inspection prompt module parses the inspection script and executes it sequentially, starting from the first process node. At each node, the rendering unit overlays prompt information in the inspector's field of vision according to the node type. After the inspector completes the inspection operation of the current node according to the augmented reality guidance, they confirm completion through human-computer interaction, and the system automatically advances to the next node.
[0078] S4, at nodes where data needs to be recorded, the data extraction module at the XR glasses automatically collects images of the power distribution cabinet and extracts instrument data and status information according to the data extraction rules in the inspection script.
[0079] When the process reaches a node containing the data extraction component, the data extraction module is automatically activated. The image preprocessing unit performs noise reduction and distortion correction preprocessing on the acquired images; the target detection unit locates the dashboard area using a target detection algorithm; the optical character recognition unit extracts the instrument readings through optical character recognition. For pointer-type instruments, the reading is calculated by recognizing the dial scale lines and pointer angle. The extracted data is displayed in real time in the XR glasses' field of view for inspection personnel to confirm, and is simultaneously uploaded to the server. After data extraction is complete, the extracted data is compared with the preset safety thresholds in the inspection script. When the data exceeds the threshold range, the XR glasses issue visual and / or auditory alarm prompts. S4-1, if complex anomalies or difficult problems are encountered during the inspection process, the inspection personnel can initiate a remote assistance request. Remote experts provide guidance through real-time first-person perspective video with dynamic and static annotations to assist in completing on-site troubleshooting; the entire assistance process is recorded and synchronized to the server.
[0080] S5, after completing all inspection nodes, the XR glasses generate an inspection report and synchronize it to the server.
[0081] After completing all inspection nodes, the XR glasses generate an inspection report. The report generation module uses template-based document generation technology to format and organize all recorded operation steps, extracted data, and abnormal events during the inspection process according to a preset report template. The system uses JSON Schema for structured validation of the report data. The generated inspection report is in PDF format and includes a cover, table of contents, inspection overview, detailed records, anomaly summary, conclusions, and recommendations. The generated inspection report is uploaded to the server via the MQTT protocol and encrypted using the TLS 1.3 protocol during transmission. After receiving the report, the server stores it in the database and updates the inspection task status to "completed".
[0082] In a preferred embodiment of the present invention, the system also supports logical consistency verification between device states.
[0083] The server-side features a logical relationship rule engine, and the visual graphical programming module includes corresponding logical relationship configuration components. These components are a set of pre-defined graphical UI controls used to construct state association rules between power equipment. Business experts can drag and drop these components onto the canvas area and connect them with existing data extraction and judgment components to visually define and deploy complex secondary loop verification logic. This component is responsible for the visual representation and process configuration of the logic. Its configuration results are compiled into a structured rule script by the script compiler and then sent to the server-side logical relationship rule engine for execution.
[0084] Taking the consistency judgment of circuit breaker and indicator light status as an example: when the status of the circuit breaker is identified as "closed", its corresponding closing indicator light must be "green" and the opening indicator light must be "off"; when the status of the circuit breaker is identified as "open", its corresponding opening indicator light must be "red" and the closing indicator light must be "off".
[0085] Similarly, the system also supports the following state consistency checks: Consistency between energy storage status and indicator light: When the circuit breaker's energy storage status is "energy stored", the energy storage indicator light should be "green"; when the energy storage status is "not stored", the energy storage indicator light should be "off" or "yellow".
[0086] Consistent operating status of protection devices: When the protection device is in "normal operation", the operation indicator light should be "solid green" and the alarm indicator light should be "off"; when the protection device malfunctions, the operation indicator light should be "off" or "red" and the alarm indicator light should be "flashing red".
[0087] Consistency between control power supply and device power supply: When the control power air switch is "closed", the protection device it supplies should be in "energized operation" state; when the control power air switch is "open", the protection device should display "power failure" and all relevant indicator lights should be "off".
[0088] The aforementioned consistency judgment rules are configured visually by business experts using the logical relationship configuration component in the visual graphical programming module. After configuration, the script compiler compiles them into machine-executable structured rule scripts. During the inspection process, the device status data collected by the XR glasses is uploaded to the server. The server's logical relationship rule engine then loads and executes the aforementioned rule scripts to complete automatic verification. The verification results are sent to the XR glasses in real time for display or alarm purposes.
[0089] like Figure 5 As shown, after wearing XR glasses, the inspection personnel can see the superimposed image of the physical entity of the power distribution cabinet and the augmented reality virtual information in their field of vision.
[0090] The physical entity of the power distribution cabinet is shown in the center of the image. The power distribution cabinet is equipped with a voltmeter, an ammeter, a circuit breaker, and indicator lights.
[0091] Multiple augmented reality virtual elements are overlaid on the physical entity of the power distribution cabinet.
[0092] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A power distribution cabinet inspection system based on XR glasses, characterized in that, include: The server side includes a visual graphical programming module, which allows managers to configure the inspection tasks of the power distribution cabinet through a graphical interface. The inspection task includes inspection process nodes, prompts for each node, and data extraction rules, and generates the configured inspection task as an inspection script. The XR glasses terminal is communicatively connected to the server terminal, and the XR glasses terminal includes: The spatial positioning and recognition module is used to identify the power distribution cabinet to be inspected and determine the spatial pose of the XR glasses in the current environment. The inspection prompt module is used to receive and parse the inspection script sent by the server, and display the inspection process, specific operation steps and detailed reminders of the current power distribution cabinet on the display screen of the XR glasses in an augmented reality manner according to the inspection process nodes and prompt information defined in the inspection script. The data extraction module is used to acquire real-time images of the power distribution cabinet through the image acquisition device of the XR glasses, and automatically extract the instrument data and equipment status information on the power distribution cabinet according to the data extraction rules defined in the inspection script.
2. The power distribution cabinet inspection system based on XR glasses according to claim 1, characterized in that, The visual graphics programming module includes a component library and a canvas area. The component library contains flow control components, prompt information components, and data extraction components. Users can define the execution order and logical relationship between components by dragging and dropping the components to the canvas area and connecting them, thereby generating an inspection process script containing the logical order. The server then sends the inspection process script to the XR glasses.
3. The power distribution cabinet inspection system based on XR glasses according to claim 1, characterized in that, The inspection prompt module includes a 3D registration unit and a rendering unit. The 3D registration unit is used to spatially align the prompt information with the physical entity of the power distribution cabinet based on the spatial pose calculated by the spatial positioning and recognition module. The rendering unit is used to render indicator arrows, highlight boxes, text descriptions, or operation demonstration animations within the field of view of the inspection personnel to guide the inspection personnel to conduct inspections according to the process defined in the inspection script.
4. The power distribution cabinet inspection system based on XR glasses according to claim 1, characterized in that, The data extraction module includes an image preprocessing unit, a target detection unit, and an optical character recognition unit; the target detection unit is used to locate the positions of instrument panels, indicator lights, or switches on the power distribution cabinet; the optical character recognition unit is used to read the readings of digital or pointer instruments; the data extraction module feeds back the extracted data to the XR glasses for display in real time, and simultaneously uploads it to the server.
5. The power distribution cabinet inspection system based on XR glasses according to claim 1, characterized in that, The XR glasses also include an anomaly alarm module, which is connected to the data extraction module. When the data extracted by the data extraction module exceeds a preset safety threshold, or when the inspection personnel do not follow the inspection process, the anomaly alarm module issues a visual and / or auditory alarm on the XR glasses.
6. A method for inspecting a power distribution cabinet based on the system described in any one of claims 1-5, characterized in that, Includes the following steps: S1, on the server side, the inspection process, prompts and data extraction rules of the power distribution cabinet are configured through the visual graphical programming module, the inspection script is generated and sent to the XR glasses terminal; S2, the inspection personnel wearing XR glasses arrive at the inspection site, the spatial positioning and recognition module at the XR glasses identifies the target power distribution cabinet and loads the corresponding inspection script; S3, the inspection prompt module on the XR glasses parses the inspection script and gradually overlays the process, operation steps and detailed reminders of the current inspection node in the inspector's field of vision; S4, at nodes where data needs to be recorded, the data extraction module at the XR glasses automatically collects images of the power distribution cabinet and extracts instrument data and status information according to the data extraction rules in the inspection script; S5, after completing all inspection nodes, the XR glasses generate an inspection report and synchronize it to the server.
7. The method for inspecting distribution cabinets according to claim 6, characterized in that, In step S2, the spatial positioning and recognition module identifies the model and unique identifier of the current power distribution cabinet through synchronous positioning and mapping technology and image feature matching, and calculates the three-dimensional coordinates of the XR glasses relative to the power distribution cabinet.
8. The method for inspecting distribution cabinets according to claim 6, characterized in that, In step S4, the data extraction module first performs noise reduction and distortion correction preprocessing on the acquired image, then locates the dashboard area through a target detection algorithm, and finally extracts the instrument reading through optical character recognition. For pointer-type instruments, the reading is calculated by recognizing the dial scale lines and pointer angle.
9. The method for inspecting distribution cabinets according to claim 6, characterized in that, In step S4, after the data extraction is completed, the extracted data is compared with the preset safety threshold in the inspection script. When the data exceeds the threshold range, the XR glasses will issue a visual and / or auditory alarm.
10. The method for inspecting distribution cabinets according to claim 6, characterized in that, In step S1, the inspection script is a structured file in JSON or XML format, containing process metadata, a list of components, and the connection relationships between components.
Citation Information
Patent Citations
A visual programming method suitable for power distribution internet of things
CN115525266B
Augmented reality intelligent inspection device and method for power system
CN119172501A
Intelligent inspection method and system for power equipment based on augmented reality
CN121836663A
