Power substation robot dog inspection method, system and device based on active sensing and medium

Abstract

The application discloses a substation robot dog inspection method and system based on active sensing, and relates to the technical field of intelligent inspection of power systems.The system comprises a robot dog mobile carrier, an integrated follow-up holder, a multi-modal sensing module, a sound source positioning module, a vehicle-mounted edge computing module, an inspection strategy scheduling module, an intelligent identification and rechecking module, a multi-modal image fusion and holographic visualization module, a data storage and transmission module, and a fault early warning module; a visible light high-definition image is used as a spatial unified reference to realize pixel-level accurate registration of three-source images; a multi-dimensional information device holographic digital portrait is generated by combining sound source positioning and a deep learning visualization engine; a closed-loop inspection architecture is constructed to support adaptive scheduling and real-time decision-making at the edge.The application effectively solves the problem of multi-modal data fragmentation of existing inspection robots, and improves the autonomous perception, accurate rechecking and visual judgment capabilities of explicit and implicit faults of substation equipment, thereby ensuring the safe operation of power grids.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology for power systems, and in particular to a method, system, equipment, and medium for substation robot dog inspection based on active perception. Background Technology

[0002] As the core hub of the power system, substations undertake the critical functions of power transformation, transmission, and distribution. The safe and stable operation of various high-voltage power equipment within the substation directly affects the power supply reliability and power quality of the regional power grid. With the continuous advancement of the construction of new power systems, power grid operation and maintenance are developing towards intelligence, unmanned operation, precision, and visualization. Traditional manual inspection methods have inherent drawbacks such as low work efficiency, high on-site safety risks, strong subjective judgment, and the inability to achieve continuous operation around the clock. They are gradually being replaced by intelligent robotic inspection methods.

[0003] Quadruped robot dogs, with their strong terrain adaptability, maneuverability, minimal blind spots, and ability to adapt to complex substation environments, have become one of the mainstream equipment for intelligent substation inspection. However, current substation inspection robot dogs on the market still have several core technological shortcomings, making it difficult to meet the operation and maintenance requirements of high precision, visualization, low false alarms and missed alarms, and full-scenario adaptability. Specifically, these shortcomings are manifested in the following aspects:

[0004] First, multimodal sensing data are fragmented and lack a unified fusion benchmark. Existing inspection equipment typically acquires and analyzes visible light, infrared, and ultraviolet images independently. Visible light cameras are only used to identify equipment appearance defects, failing to fully utilize their high-resolution imaging advantages to build a unified spatial benchmark. This results in misalignment of the spatial positions and asynchronous temporal sequences of the three types of images, leading to fragmented multidimensional fault data and an inability to form a complete equipment condition evaluation system.

[0005] Second, the lack of pixel-level image registration technology results in low accuracy of fault fusion analysis. Traditional inspection equipment has not achieved accurate registration of multispectral images. Abnormal infrared temperature areas, ultraviolet discharge points, and abnormal sound source locations cannot be accurately matched in the same view, making it difficult to conduct cross-verification of multi-source data, leading to frequent misjudgments and omissions of faults.

[0006] Third, the fault visualization capabilities are insufficient, failing to support status tracing and trend analysis. Existing inspection equipment can only output single-dimensional detection data and textual conclusions, unable to integrate multi-dimensional fault characteristics such as sound, light, and heat to construct a comprehensive picture of the equipment's status. Maintenance personnel find it difficult to intuitively view the location, scope, and severity of faults, and are also unable to compare historical data or analyze equipment degradation trends, resulting in a significantly weak support capability for refined maintenance.

[0007] Fourth, the data acquisition mode is fixed, resulting in poor environmental adaptability. Traditional inspection robot dogs use a passive acquisition mode with fixed perspective and fixed acquisition parameters, which cannot dynamically adjust the acquisition strategy according to the equipment location, environmental conditions, and suspected abnormal states. When acquiring data at long distances, problems such as image blurring, temperature data deviation, and electromagnetic signal interference are prone to occur, resulting in low reliability of fault detection.

[0008] Fifth, there is a lack of standardized re-inspection mechanisms and power-specific fusion algorithms. Existing equipment often relies solely on single data collection for fault diagnosis, without establishing multi-angle and multi-dimensional verification processes. Furthermore, it lacks multi-modal fusion diagnostic logic adapted to power scenarios, making it difficult to effectively distinguish between real faults and environmental noise, and its ability to identify minor and latent faults is significantly insufficient.

[0009] Sixth, the inspection strategy is rigid and the ability to operate around the clock is weak. Traditional inspection equipment has fixed parameters and cannot dynamically adjust the inspection logic according to changes in day and night light, severe weather such as rain, snow, fog, and dust, as well as fluctuations in equipment operating load. The quality of inspections under extreme conditions drops significantly.

[0010] In summary, existing inspection robot dogs have significant shortcomings in multi-source data fusion, image registration accuracy, fault visualization, re-inspection mechanism construction, latent fault identification, and environmental adaptability, making it difficult to meet the practical application requirements of comprehensive, high-precision, and visualized intelligent inspection of substation equipment. Therefore, there is an urgent need to design an intelligent inspection system and method for substation robot dogs with pixel-level image registration, acoustic-optical-thermal holographic visualization, active perception and precise re-inspection, and full-scene adaptive operation capabilities. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of existing technologies, such as fragmented multimodal data of inspection equipment, lack of pixel-level image registration capability, low degree of fault visualization, lack of standardized and accurate re-inspection mechanism, insufficient fault identification accuracy, and poor adaptability to complex scenarios. This invention provides a substation inspection method and system based on active perception and accurate re-inspection. This invention reconstructs high-definition visible light images into a unified multispectral spatial benchmark, and relies on feature point image matching algorithms to achieve pixel-level accurate registration of visible light, infrared, and ultraviolet images, fundamentally solving the problems of spatial misalignment and temporal fragmentation of multimodal data. Simultaneously, it integrates an array-type microphone sound source localization module, with built-in adaptive environmental noise filtering and equipment abnormal noise feature extraction algorithms to compensate for blind spots in single visual detection. Multimodal fusion and fault identification algorithms are deployed locally through an onboard edge computing module to avoid cloud transmission delays. It adopts a full-domain active perception coarse detection and suspected anomaly triggering mechanism, combined with an integrated dynamic cloud system. The platform's adaptive rotation, zoom, and hovering functions enable precise target locking and multi-view layered re-inspection. Furthermore, leveraging a deep learning-based acoustic-optical-thermal joint visualization engine, sound source localization information is transformed into a dynamic heat map and overlaid onto a multispectral fusion image, generating a holographic digital profile of the equipment status containing multi-dimensional acoustic, optical, and thermal features, supporting historical comparison and degradation trend analysis. Finally, through multi-algorithm weighted fusion decision-making to eliminate environmental interference, precise fault type, location, and severity are determined. A graded differentiated early warning mechanism enables closed-loop management of minor faults (archiving, general pop-ups, and severe audible and visual alarms). This invention constructs a standardized closed-loop inspection architecture of "full-domain coarse inspection—point locking—multi-angle re-inspection—fusion diagnosis," significantly improving the autonomous perception, precise verification, visual judgment, and intelligent grading capabilities for both visible and hidden faults. This comprehensively enhances the accuracy, visualization, and intelligence of substation inspections, effectively ensuring the safe and stable operation of the power grid.

[0012] The solution adopted by this invention to solve its technical problem is as follows:

[0013] A substation robot dog inspection system based on active perception includes:

[0014] The robot dog mobile carrier is used to provide mobile operation support with autonomous walking, intelligent obstacle avoidance and complex terrain adaptation, and is suitable for all-scenario passage operations in the high and low voltage equipment area, busbar area and frame area of ​​substations.

[0015] An integrated follow-up gimbal is fixedly mounted on the top of the robot dog's mobile carrier via its gimbal connector and robot dog mounting flange.

[0016] A multimodal sensing module, which includes a visible light camera, an ultraviolet camera, and an infrared camera;

[0017] The sound source localization module has built-in adaptive environmental noise filtering algorithm and equipment abnormal noise feature extraction algorithm. It includes an array microphone to collect mechanical and electrical abnormal noise signals of substation equipment, locate the source of abnormal noise faults, and make up for the detection blind spots of single visual perception.

[0018] The vehicle-mounted edge computing module has a built-in embedded algorithm chip. Through the embedded algorithm chip, gimbal control algorithms, power equipment fault identification algorithms, multimodal image registration algorithms, sound-light joint visualization algorithms, and multimodal data fusion algorithms are deployed locally to realize real-time processing of inspection data, image fusion modeling, and instant fault decision-making, avoiding cloud transmission delays and network interference.

[0019] The inspection strategy scheduling module has built-in substation equipment ledger, standard inspection path, hierarchical inspection rules and equipment fault threshold standards. It is used to dynamically generate adaptive inspection tasks based on the operation scenario, environmental conditions and equipment operating status.

[0020] The intelligent identification and re-inspection module is communicatively connected to the integrated follow-up gimbal, the multimodal perception module, the sound source localization module, and the vehicle edge computing module. It is used to perform a closed-loop inspection process of full-domain active perception coarse detection, preliminary anomaly judgment, gimbal target locking, multi-angle precise re-inspection, and precise fault classification.

[0021] The multimodal image fusion and holographic visualization module deploys a feature point image matching algorithm and a deep learning acoustic-optical joint visualization engine to complete pixel-level registration and image enhancement optimization of multi-source images. At the same time, it converts sound source localization information into dynamic heat maps and overlays them onto multispectral fusion images to generate a holographic digital portrait of the device status containing sound, light, and heat multidimensional information, supporting anomaly visualization and historical data comparison analysis.

[0022] The data storage and transmission module is used to store the original sensor data of the inspection, image registration and fusion data, holographic digital portrait, re-inspection data and fault diagnosis report, and realize local caching and real-time interaction with the remote backend, remote traceability and status comparison.

[0023] The fault early warning module, which works in conjunction with the vehicle edge computing module and the data storage and transmission module, is used to perform graded early warning push based on the fault classification results, so as to realize real-time management and control of fault risks.

[0024] As another embodiment of the present invention

[0025] The integrated servo gimbal includes a horizontal rotation mechanism, a pitch adjustment mechanism, and a main control cabin.

[0026] The vehicle-mounted edge computing module, the inspection strategy scheduling module, the intelligent identification and re-inspection module, the multimodal image fusion and holographic visualization module, the data storage and transmission module, and the fault early warning module are all integrated in the main control cabin.

[0027] As another embodiment of the present invention

[0028] The multimodal sensing module has an adaptive adjustment function for the acquired parameters;

[0029] The visible light camera supports 4K high-definition imaging, automatic exposure, and intelligent zoom functions. It is used to acquire high-resolution reference images of the device and use them as a unified spatial reference. It relies on feature point image matching algorithms to achieve pixel-level accurate registration of visible light, infrared, and ultraviolet three-source images.

[0030] The infrared camera is used to collect data on the temperature field distribution on the surface of the equipment and to detect potential overheating and abnormal temperature in the equipment.

[0031] The ultraviolet camera is used to capture ultraviolet light signals of insulation discharge in equipment to identify latent insulation faults.

[0032] As another embodiment of the present invention

[0033] The inspection strategy scheduling module supports dynamic adaptive scheduling across all scenarios.

[0034] Based on day and night lighting conditions, severe weather conditions such as rain, snow, fog, and dust, real-time equipment operating load, and historical fault statistics, the pan-tilt-zoom scanning speed, data acquisition accuracy, number of re-inspections, and fault detection threshold are adaptively adjusted.

[0035] In low-light conditions at night, infrared and ultraviolet priority detection modes are automatically activated, and registration is completed based on visible light images to ensure modeling accuracy.

[0036] Enhanced data filtering, increased re-inspection frequency, and image enhancement under adverse weather conditions help resist interference from complex environments.

[0037] As another embodiment of the present invention

[0038] The fault early warning module implements a tiered and differentiated early warning mechanism:

[0039] Minor faults are only archived with holographic profiles and trend records for equipment tracing and degradation trend assessment.

[0040] A common fault triggers a real-time pop-up notification in the background, reminding maintenance personnel to pay attention to the equipment status.

[0041] Serious faults trigger audible and visual alarms on-site and simultaneously upload emergency warning information and a holographic diagnostic profile of the equipment to the remote operation and maintenance backend.

[0042] A substation inspection method based on active perception using a robotic dog includes the following steps:

[0043] S1. Inspection Initialization Calibration: The system receives the inspection task issued by the inspection strategy scheduling module, loads the substation 3D scene map, equipment coordinate ledger, equipment fault identification threshold and exclusive algorithm model, completes the integrated follow-up pan-tilt attitude calibration, multimodal perception module parameter calibration and sound source localization module environmental noise baseline calibration, and ensures that the system is in a standardized standby operation state.

[0044] S2. Full-domain active perception and coarse detection: The robot dog mobile carrier autonomously avoids obstacles along the preset optimal inspection path. The integrated follow-up gimbal opens the full-domain low-speed scanning mode and simultaneously starts the multi-modal perception module and sound source localization module to collect visible light reference images, infrared temperature field data, ultraviolet discharge data and equipment sound field data from all directions. The lightweight power-specific algorithm built into the vehicle edge computing module quickly screens the real-time data to complete the preliminary judgment of equipment anomalies and identify suspected fault targets.

[0045] S3. Abnormal Target Locking and Gimbal Follow-up Positioning: If the coarse detection identifies suspected abnormal data, the robot dog's mobile carrier is immediately controlled to stop moving. The vehicle-mounted edge computing module calculates the spatial coordinates of the abnormal target in real time, generates high-precision gimbal attitude adjustment commands, drives the integrated follow-up gimbal to adaptively rotate, zoom and hover, and accurately locks the suspected abnormal equipment area.

[0046] S4. Multi-angle precise re-inspection operation: The standardized precise re-inspection mechanism is activated, and the integrated follow-up gimbal is controlled to adopt a multi-view layered acquisition strategy to repeatedly collect data from suspected abnormal areas at close range, multiple angles and high resolution. The acquisition parameters are optimized at the same time and environmental interference is filtered out to obtain multi-modal raw data with high consistency and high matching degree, providing high-quality data support for subsequent image registration and holographic modeling.

[0047] S5. Multimodal Image Pixel-Level Registration and Enhancement Processing: Call the multimodal image fusion and holographic visualization module, based on the feature point image matching algorithm, and using the visible light high-definition image as the benchmark, complete the pixel-level accurate registration of visible light, infrared and ultraviolet three-source images, and combine the image enhancement algorithm to eliminate the interference of light, rain, snow and dust environment, and unify the consistency of the spatial dimension and temporal dimension of multi-source images.

[0048] S6. Sound-Light-Heat Joint Holographic Visualization Modeling and Fault Diagnosis: Relying on a deep learning sound-light joint visualization engine, the sound source localization analysis results are transformed into dynamic heat maps, which are dynamically superimposed onto the registered multispectral fusion image to generate a holographic digital portrait of the equipment status containing multidimensional information of sound, light, and heat; combined with a multidimensional power-specific fault identification algorithm, the fault type, location, and level are accurately determined, and historical inspection holographic portrait data are retrieved for comparative analysis to identify the trend of equipment status deterioration;

[0049] S7. Accurate Fault Classification through Multi-Algorithm Fusion: Based on the high-precision re-inspection dataset and holographic digital profile, it integrates multi-dimensional features such as infrared temperature anomaly, ultraviolet insulation discharge, and equipment abnormal noise. Through a multi-modal data weighted fusion strategy, it integrates the detection results, eliminates misjudgment data caused by environmental interference, and accurately completes the classification of fault type, fault location, and fault level.

[0050] S8. Data Archiving and Task Iteration: Completely record the original inspection data, image registration and fusion data, holographic digital portrait, re-inspection data and fault diagnosis results, automatically generate standardized inspection reports and upload them to the remote back-end terminal, control the integrated follow-up PTZ to reset to the initial inspection posture, and the robot dog mobile carrier continues to perform subsequent inspection tasks until the inspection of all equipment in the substation is completed.

[0051] As another embodiment of the present invention

[0052] Step S2 includes:

[0053] S21, the integrated pan-tilt unit adopts a large-area uniform low-speed scanning mode to cover the inspection area on both sides of the robot dog's moving carrier path, and provides no blind spots to cover various power equipment in the substation.

[0054] S22. Acquire high-resolution visible light reference images for subsequent multimodal image registration and modeling;

[0055] Infrared temperature field data is analyzed using a temperature mean comparison algorithm to screen for localized overheating areas and areas with excessive temperature differences in the equipment.

[0056] A pixel threshold determination algorithm was used to analyze ultraviolet imaging data to preliminarily identify insulation discharge anomalies.

[0057] The sound source data was analyzed using a spectral feature comparison algorithm to preliminarily determine the type of abnormal noise fault in the equipment.

[0058] S23. Preset multi-level anomaly initial judgment thresholds, compare the real-time collected infrared, ultraviolet and sound source multi-dimensional data with the equipment standard parameter library in real time. If any dimension of data exceeds the threshold range, it is judged as a suspected anomaly, and the precise re-inspection and holographic visualization modeling instructions are automatically triggered to enter the refined detection process.

[0059] As another embodiment of the present invention

[0060] Step S7 specifically includes:

[0061] S71. Temperature Anomaly Diagnosis:

[0062] The infrared temperature gradient differential algorithm is used to calculate the temperature difference between the equipment body, the connector, and the lead wire. Combined with the real-time ambient temperature correction coefficient, the equipment overheating fault is divided into three risk levels: minor, moderate, and severe.

[0063] S72. Insulation Discharge Diagnosis:

[0064] Using an ultraviolet photon counting algorithm, the number of ultraviolet discharge photons of the equipment per unit time is counted. Combined with the industry standard for discharge level of power equipment, hidden safety hazards such as insulation aging, partial discharge, and flashover are identified.

[0065] S73. Abnormal Noise Troubleshooting:

[0066] Using sound spectrum analysis and feature matching algorithms, we can distinguish between abnormal mechanical vibration noise, abnormal electrical discharge noise, and abnormal noise from loose components, and locate hidden faults that cannot be identified by vision.

[0067] S74. Multimodal fusion decision: A weighted fusion algorithm is used to integrate multidimensional detection results and combine them with the visualization features of holographic digital profiles to set a confidence threshold for fault determination.

[0068] A computer device is characterized by comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a substation robot dog inspection method based on active perception.

[0069] A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, which, when executed by a processor, implements a substation robot dog inspection method based on active perception.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] 1. This invention changes the traditional functional positioning of visible light cameras, using them as a unified spatial registration benchmark for infrared and ultraviolet images. It achieves pixel-level accurate registration of the three types of images through feature point matching algorithms, unifies the spatial and temporal dimensions of multi-source data, fundamentally solves the technical problems of multimodal data fragmentation and spatial misalignment of fault locations in traditional inspections, and lays a solid foundation for multi-dimensional data cross-validation.

[0072] 2. This invention builds a deep learning-based acoustic-optical joint visualization engine, which dynamically converts the sound source localization results into a heat map and overlays it onto a multispectral fusion image in real time, generating a holographic digital portrait of the equipment that includes integrated sound, light, and heat information, intuitively presenting the location of abnormal noises, overheated areas, and discharge points; at the same time, it supports multi-cycle historical portrait comparison analysis, which can accurately identify the trend of equipment condition deterioration and realize early warning of hidden faults.

[0073] 3. This invention abandons the traditional, passive, and extensive inspection model of single-time data collection, and establishes a hierarchical detection mechanism of "comprehensive coarse screening - fixed-point fine verification." Through multi-angle, multi-dimensional re-examination and verification, and multi-algorithm fusion decision-making, it effectively eliminates false alarm data caused by environmental interference. Field tests have shown that this invention achieves a comprehensive fault identification accuracy rate of over 98%, significantly reducing the probability of false alarms and missed faults.

[0074] 4. This invention integrates four-dimensional perception capabilities, including visible light image recognition, infrared temperature measurement, ultraviolet discharge detection, and sound source abnormality location, comprehensively covering various fault types such as abnormal equipment temperature, insulation discharge, mechanical / electrical abnormal noise, and appearance defects. It can simultaneously identify both visible and potential hidden faults, filling the technological gap in the field of multi-dimensional fusion visualization detection in traditional inspection.

[0075] 5. The system of this invention can automatically adjust the pan-tilt scanning speed, data acquisition parameters, re-inspection frequency, image enhancement intensity and fault judgment threshold according to the changes in day and night light, severe weather conditions such as rain, snow, fog and dust, and the real-time operating load of the equipment. It can be specifically adapted to various complex working conditions and effectively ensure the image registration accuracy and fault identification reliability in extreme environments.

[0076] 6. This invention deploys all core algorithms such as image registration, visualization modeling, and fault fusion diagnosis locally on the vehicle-mounted edge computing module, without relying on remote cloud transmission. This effectively avoids problems such as network latency, network outages, and strong electromagnetic interference from substations, resulting in fast inspection response and high overall operational stability.

[0077] 7. This invention realizes fully automated, unmanned operation of the entire process from inspection walking, data acquisition, image registration and fusion, holographic modeling, fault diagnosis, hierarchical early warning to data archiving. It requires no on-site human intervention, which greatly reduces the labor cost of substation operation and maintenance and significantly improves the intelligence level of power grid operation and maintenance. Attached Figure Description

[0078] Figure 1 This is a structural block diagram of a substation robot dog inspection system based on active perception proposed in this invention;

[0079] Figure 2 This is a schematic diagram of the integrated follow-up gimbal installation structure in a substation robot dog inspection system based on active perception proposed in this invention.

[0080] Figure 3 This is a schematic diagram of the inspection process of a substation robot dog inspection method based on active perception proposed in this invention.

[0081] Figure 4 This is a block diagram of the multimodal image registration and acoustic-optical-thermal holographic visualization modeling logic in a substation robot dog inspection method based on active perception proposed in this invention.

[0082] Explanation of reference numerals in the attached figures:

[0083] 1. Mobile carrier for the robot dog; 2. Gimbal connector and robot dog mounting flange; 3. Horizontal rotation mechanism; 4. Pitch adjustment mechanism; 5. Visible light camera; 6. Array microphone; 7. Ultraviolet camera; 8. Infrared camera; 9. Main control cabin; 10. Integrated follow-up gimbal. Detailed Implementation

[0084] The specific embodiments of the present invention are described below with reference to the accompanying drawings and examples:

[0085] It should be noted that the structures, colors, proportions, sizes, etc. shown in the accompanying drawings are only used to complement the content disclosed in the specification, so that those skilled in the art can understand and read them, and are not intended to limit the conditions under which the present invention can be implemented. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0086] Example 1: System composition and module functions.

[0087] like Figures 1-4 As shown, this invention proposes a substation robot dog intelligent inspection system based on active perception and precise re-inspection using a servo-mounted gimbal. The system includes a robot dog mobile carrier 1, an integrated servo-mounted gimbal 10, a multimodal perception module, a sound source localization module, a vehicle-mounted edge computing module, an inspection strategy scheduling module, an intelligent recognition and re-inspection module, a multimodal image fusion and holographic visualization module, a data storage and transmission module, and a fault early warning module. Wherein:

[0088] The robot dog mobile carrier 1 is used to provide mobile operation support with autonomous walking, intelligent obstacle avoidance and complex terrain adaptation, and is suitable for all scenarios of operation in the high and low voltage equipment area, busbar area and frame area of ​​substation.

[0089] The integrated follow-up gimbal 10 is fixedly mounted on the top of the robot dog mobile carrier 1 via the gimbal connector and the robot dog mounting flange 2. The integrated follow-up gimbal 10 includes a horizontal rotation mechanism 3 and a pitch adjustment mechanism 4, which enables it to have 360° horizontal full-range rotation, -30° to +60° vertical adaptive attitude adjustment, target follow-up tracking and fixed-point precise hovering functions, providing perspective support for multimodal data acquisition and visualization modeling.

[0090] The horizontal rotation mechanism 3 can be composed of a servo motor, a worm gear reducer, and an angle encoder. The motor drives the worm gear to rotate the upper housing of the gimbal continuously in the horizontal plane, and the encoder provides real-time angle feedback, enabling arbitrary angle positioning within a 360° range. The pitch adjustment mechanism 4 can be composed of another servo motor, a harmonic reducer, and an angle sensor. The motor drives the pitch frame to rotate around a horizontal axis, and the sensor provides pitch angle feedback, achieving high-precision adjustment within the range of -30° to +60°. The combination of these two mechanisms allows the integrated servo gimbal to automatically adjust its attitude to follow the detected target, completing both fixed-point re-inspection and dynamic tracking.

[0091] The integrated homing gimbal 10 also includes a main control cabin 9, in which an onboard edge computing module, an inspection strategy scheduling module, an intelligent recognition and re-inspection module, a multimodal image fusion and holographic visualization module, a data storage and transmission module, and a fault early warning module are all integrated.

[0092] The integrated servo-driven 10 adopts a high-precision servo drive structure and has the functions of real-time attitude feedback and automatic correction of running deviation. It can autonomously complete actions such as precise target equipment locking, dynamic fine adjustment of detection angle, fixed-point fine re-inspection, and dynamic tracking of fault targets according to the control instructions of the vehicle-mounted edge computing module. It is suitable for inspection operations of all types of high and low voltage equipment in substations.

[0093] The multimodal sensing module is integrated on the integrated servo gimbal 10 and includes a visible light camera 5, an infrared camera 8, and an ultraviolet camera 7. The visible light camera 5 supports 4K high-definition imaging, automatic exposure, and intelligent zoom. It is used to acquire high-definition reference images of the equipment as a unified spatial reference for multi-source image registration. It achieves pixel-level accurate registration of visible light, infrared, and ultraviolet images based on feature point image matching algorithms. The infrared camera 8 is used to acquire surface temperature field distribution data of the equipment to detect fault characteristics such as overheating, abnormal body temperature rise, and uneven temperature distribution. The ultraviolet camera 7 is used to capture ultraviolet light signals generated by insulation discharge of the equipment to identify latent insulation faults such as corona discharge, insulation flashover, and partial discharge.

[0094] In addition, the multimodal sensing module has an adaptive adjustable acquisition parameter function, which can automatically adjust parameters such as exposure, gain, and sampling rate according to ambient light and weather conditions.

[0095] The sound source localization module is integrated into the integrated servo pan-tilt unit 10, including an array microphone 6, used to collect mechanical and electrical abnormal noise signals from substation equipment, locate the source of the abnormal noise fault, and compensate for the detection blind spots of single visual perception. This module incorporates an adaptive environmental noise filtering algorithm and an equipment abnormal noise feature extraction algorithm, and also features an array microphone triangulation localization algorithm, combined with the substation equipment sound field model to complete position correction. It can effectively filter fixed background noise such as substation fans, traffic flow, and environmental wind noise, accurately extract the characteristics of mechanical vibration abnormal noise, electrical discharge abnormal noise, and component loosening abnormal noise, achieving accurate detection of latent faults without visual features and compensating for the detection blind spots of single visual perception.

[0096] The vehicle-mounted edge computing module is used to achieve real-time processing of inspection data, image fusion modeling, and immediate fault decision-making, avoiding cloud transmission delays and network interference. This module has a built-in embedded algorithm chip, which locally deploys gimbal control algorithms, power equipment fault identification algorithms, multimodal image registration algorithms, sound-light joint visualization algorithms, and multimodal data fusion algorithms to achieve real-time processing of inspection data, image fusion modeling, and immediate fault decision-making, avoiding cloud transmission delays and network interference.

[0097] The inspection strategy scheduling module incorporates substation equipment ledgers, standard inspection routes, tiered inspection rules, and equipment fault threshold standards. It dynamically generates adaptive inspection tasks based on the work scenario, environmental conditions, and equipment operating status. This module supports dynamic adaptive scheduling across all scenarios: it adaptively adjusts the pan-tilt-zoom (PTZ) scanning speed, data acquisition accuracy, re-inspection frequency, and fault detection thresholds based on day / night lighting conditions, adverse weather conditions (rain, snow, fog, dust), real-time equipment operating load, and historical fault statistics. In low-light conditions at night, it automatically activates infrared and ultraviolet priority detection modes, relying on visible light images for registration to ensure modeling accuracy. In adverse weather conditions, it enhances data filtering, increases re-inspection frequency, and strengthens image enhancement to resist interference from complex environments.

[0098] The intelligent identification and re-inspection module is communicatively connected to the integrated follow-up gimbal 10, the multimodal perception module, the sound source localization module, and the vehicle-mounted edge computing module, respectively, and is used to perform closed-loop operations such as full-domain active coarse detection, preliminary anomaly judgment, gimbal target locking, multi-angle precise re-inspection, and precise fault classification.

[0099] The multimodal image fusion and holographic visualization module deploys a feature point image matching algorithm and a deep learning acoustic-optical joint visualization engine to complete pixel-level registration of multi-source images and image enhancement optimization. It transforms sound source localization information into a dynamic heat map and overlays it onto a multispectral fusion image to generate a holographic digital portrait of the device status containing multidimensional information of sound, light, and heat, supporting anomaly visualization and historical data comparison analysis.

[0100] The data storage and transmission module is used to store the original sensor data of the inspection, image registration and fusion data, holographic digital portrait, re-inspection data and fault diagnosis report, and realize local caching and real-time interaction with the remote backend, remote traceability and status comparison.

[0101] The fault early warning module works in conjunction with the vehicle-mounted edge computing module and data storage and transmission module to send tiered early warnings based on fault classification results, enabling real-time management of fault risks. The module implements a tiered, differentiated early warning mechanism as follows: minor faults are only archived with a holographic profile and trend records for equipment tracing and degradation trend analysis; general faults trigger real-time pop-up notifications in the background, reminding maintenance personnel to monitor equipment status; severe faults trigger audible and visual alarms locally and simultaneously upload emergency early warning information and a holographic diagnostic profile of the equipment to the remote maintenance backend.

[0102] Example 2: Overall flow of the inspection method.

[0103] A substation robot dog intelligent inspection method based on active perception and precise re-inspection using a servo-panel camera, applied to the inspection system described in Example 1, includes the following steps:

[0104] S1. Inspection Initialization Calibration: The robot dog receives the inspection task issued by the inspection strategy scheduling module, loads the substation 3D scene map, equipment coordinate ledger, equipment fault identification threshold and exclusive algorithm model, and completes the attitude calibration of the integrated follow-up pan-tilt 10, parameter calibration of the multimodal perception module, and environmental noise baseline calibration of the sound source localization module to ensure that the whole machine is in a standardized standby operation state.

[0105] Intelligent strategy scheduling and autonomous movement: The inspection strategy scheduling module combines the daily inspection plan, real-time equipment operating load, and equipment importance level to plan the optimal full-area inspection path; the robot dog activates the intelligent obstacle avoidance mode and moves at a constant speed along the preset path, while the integrated follow-up gimbal 10 activates the low-speed full-area scanning mode.

[0106] S2. Full-Domain Active Perception and Coarse Detection: During the robot dog's movement, the multimodal perception module and sound source localization module are simultaneously activated to collect visible light reference images, infrared temperature field data, ultraviolet discharge data, and equipment sound field data from all directions. The lightweight power-specific algorithm built into the vehicle-mounted edge computing module performs rapid initial screening of the real-time data, completing the preliminary judgment of equipment anomalies and identifying suspected faulty targets, including:

[0107] S21, the integrated follow-up PTZ 10 adopts a large-range uniform low-speed scanning mode, fully covering the inspection area on both sides of the robot dog's path, and covering all kinds of power equipment in the substation without blind spots.

[0108] S22. Acquire high-definition visible light reference images for subsequent multimodal image registration and modeling; use temperature mean comparison algorithm to analyze infrared temperature field data and screen for local overheating and temperature difference exceeding the standard areas of the equipment; use pixel threshold judgment algorithm to analyze ultraviolet imaging data to preliminarily identify insulation discharge anomalies; use spectrum feature comparison algorithm to analyze sound source data and preliminarily determine the type of abnormal noise fault in the equipment.

[0109] S23. Preset multi-level anomaly initial judgment thresholds, compare the real-time collected infrared, ultraviolet and sound source multi-dimensional data with the equipment standard parameter library in real time. If any dimension of data exceeds the threshold range, it is judged as a suspected anomaly, and the precise re-inspection and holographic visualization modeling instructions are automatically triggered to enter the refined detection process.

[0110] S3. Abnormal Target Locking and Gimbal Follow-up Positioning: If the coarse detection identifies suspected abnormal data, the robot dog is immediately controlled to stop moving. The vehicle edge computing module calculates the spatial coordinates of the abnormal target in real time, generates high-precision gimbal attitude adjustment commands, drives the integrated follow-up gimbal 10 to adaptively rotate, zoom, and hover, and accurately locks the suspected abnormal device area.

[0111] S4. Multi-angle Precision Re-inspection: A standardized precision re-inspection mechanism is activated. The integrated PTZ 10 employs a multi-view layered acquisition strategy to repeatedly acquire data from suspected abnormal areas at close range, multiple angles, and high resolution. The re-inspection strategy includes four acquisition modes: close-range zoom high-definition acquisition, horizontal multi-angle surround acquisition, vertical layered scanning acquisition, and local magnification acquisition of abnormal areas. Simultaneously, the camera's frame rate is increased, exposure parameters are optimized to reduce ambient light interference, and the accuracy of sound source signal filtering is enhanced to ensure clear feature points in multi-source images and stable sound field data. This eliminates data deviations and registration errors caused by long-distance acquisition, changes in lighting, electromagnetic interference, and weather interference, providing high-quality data support for subsequent image registration and holographic modeling.

[0112] S5. Multimodal Image Pixel-Level Registration and Enhancement Processing: Call the multimodal image fusion and holographic visualization module, based on the feature point image matching algorithm, and using the visible light high-definition image as the benchmark, complete the pixel-level accurate registration of visible light, infrared and ultraviolet three-source images, and combine the image enhancement algorithm to eliminate the interference of light, rain, snow and dust environment, and unify the consistency of the spatial dimension and temporal dimension of multi-source images.

[0113] S6. Sound-Light-Heat Joint Holographic Visualization Modeling and Fault Diagnosis: Relying on a deep learning sound-light joint visualization engine, the sound source localization analysis results are transformed into dynamic heat maps, which are dynamically superimposed onto the registered multispectral fusion image to generate a holographic digital portrait of the equipment status containing multidimensional information of sound, light, and heat; combined with a multidimensional power-specific fault identification algorithm, the fault type, location, and level are accurately determined, and historical inspection holographic portrait data are retrieved for comparative analysis to identify the trend of equipment status deterioration;

[0114] S7. Accurate Fault Classification through Multi-Algorithm Fusion: Based on high-precision re-inspection datasets and holographic digital profiles, this system integrates multi-dimensional features such as infrared temperature anomalies, ultraviolet insulation discharge, and equipment noise abnormalities. A multi-modal data weighted fusion strategy is used to integrate detection results, eliminating misjudgments caused by environmental interference, and accurately classifying fault type, location, and severity. Specifically, this includes:

[0115] S71. Temperature Anomaly Diagnosis: The infrared temperature gradient differential algorithm is used to calculate the temperature difference between the equipment body, wiring connectors, and leads. Combined with the real-time ambient temperature correction coefficient, the equipment overheating fault is classified into three risk levels: minor, moderate, and severe.

[0116] S72. Insulation discharge diagnosis: Using an ultraviolet photon counting algorithm, the number of ultraviolet discharge photons of the equipment per unit time is counted. Combined with the industry standard for discharge level of power equipment, the hidden safety hazards of insulation aging, partial discharge, and flashover are determined.

[0117] S73. Abnormal Noise Fault Diagnosis: Using sound spectrum analysis and feature matching algorithms, it distinguishes between abnormal noises caused by mechanical vibration, electrical discharge, and loose parts, and locates hidden faults that cannot be identified visually.

[0118] S74. Multimodal fusion decision: A weighted fusion algorithm is used to integrate the multidimensional detection results of temperature, ultraviolet, sound source, and image, and combined with the visualization features of holographic digital portrait, to set the confidence threshold for fault judgment.

[0119] Preferably, step S8, data archiving and task iteration can be added: fully record the original inspection data, image registration and fusion data, holographic digital portrait, re-inspection data, and fault diagnosis results, automatically generate a standardized inspection report and upload it to the remote back-end terminal, control the integrated follow-up PTZ 10 to reset to the initial inspection posture, and the robot dog continues to perform subsequent inspection tasks until the inspection of all equipment in the substation is completed.

[0120] The solutions in Examples 1-2 will be further described in conjunction with the specific work on site.

[0121] Example 3: Intelligent inspection of the entire process of routine substation environment.

[0122] This embodiment is applied to routine substation operations in sunny, well-lit, and interference-free conditions, fully executing the closed-loop inspection process of this invention:

[0123] S1. Equipment Initialization and Calibration: The inspection robot is deployed to the substation work area to be inspected. The robot autonomously loads the substation's 3D scene map, equipment coordinate ledger, various equipment fault identification thresholds, and power-specific algorithm models; it completes the integrated servo gimbal 10 attitude calibration, multi-modal camera imaging parameter and white balance calibration, and sound source localization module environmental noise baseline calibration, and the entire unit enters a standardized standby state. After calibration, the gimbal positioning accuracy, camera imaging parameters, and sound source noise reduction parameters all meet the standards.

[0124] S2, Intelligent Strategy Scheduling and Autonomous Movement: The inspection strategy scheduling module combines the daily inspection plan, real-time equipment operating load, and equipment importance level to plan the optimal full-domain inspection path; the robot dog activates the intelligent obstacle avoidance mode and moves at a constant speed along the preset path, while the integrated follow-up gimbal 10 activates the low-speed full-domain scanning mode.

[0125] S3. Full-Domain Active Perception Coarse Screening: During the robot dog's movement, the multimodal perception module and sound source localization module are activated simultaneously. Visible light camera 5 continuously acquires 4K high-definition images as a reference for subsequent image registration; infrared camera 8 acquires temperature field data of the entire device to screen for local overheating areas; ultraviolet camera 7 captures solar-blind ultraviolet signals to preliminarily identify abnormal insulation discharge; array microphone 6 acquires sound field data, combines noise reduction algorithms to filter background noise, extracts abnormal sound spectrum features, and completes multi-dimensional preliminary fault screening.

[0126] S4. Anomaly Locking and Gimbal Follow-up Positioning: When the coarse detection identifies suspected abnormal signals such as excessive ultraviolet discharge of insulators, abnormal temperature of wiring terminals, or abnormal noise from circuit breakers, the vehicle-mounted edge computing module calculates the spatial coordinates of the abnormal target in real time and outputs high-precision gimbal adjustment commands to drive the integrated follow-up gimbal 10 to rotate rapidly, zoom intelligently, and hover at a fixed point, accurately locking the fault area. The measured average response time for a single target lock by the gimbal is 18ms, and the positioning deviation is 0.35°, meeting the operational requirements.

[0127] S5. Multi-angle Precision Re-inspection: The gimbal switches to re-inspection mode and sequentially performs four operations: close-range zoom magnification, horizontal ±30° surround scan, vertical ±20° layered acquisition, and local area fine acquisition. Simultaneously, camera exposure, sampling frame rate, infrared temperature measurement accuracy, and sound source filtering parameters are optimized to acquire high-definition, low-interference, and highly consistent multimodal raw data.

[0128] S6. Multimodal Image Pixel-Level Registration and Enhancement: A multimodal image registration algorithm based on improved SIFT and edge constraints is adopted to address the difficulty of feature point extraction caused by the different imaging mechanisms of visible light, infrared, and ultraviolet images. The specific implementation is as follows:

[0129] (1) Edge feature extraction: The Sobel operator is used to extract the visible light images of the three images. Infrared images Ultraviolet images The edge outline.

[0130] (2) Feature point localization: The curvature scale space algorithm is used to extract corner points on the edge contour as candidate feature points, and the weak corner points are filtered by adaptive thresholding to retain highly significant feature points.

[0131] (3) Feature description construction: Using the visible light image as the spatial reference, the directional gradient histogram features are extracted in the neighborhood of the feature points; at the same time, the scale-invariant feature transformation algorithm is introduced to construct a 128-dimensional descriptor, and the gradient direction is normalized to enhance the resistance to affine transformation.

[0132] (4) Feature matching and transformation: A fast nearest neighbor search is used for coarse matching, and a random sampling consensus algorithm is used to eliminate mismatched points. The homography transformation matrix H is calculated, which satisfies:

[0133] ;

[0134] (5) Registration accuracy: Experimental verification shows that the registration pixel error of this algorithm is ≤1 pixel under normal working conditions, which meets the accuracy requirements of multi-source data fusion.

[0135] (6) Timing synchronization: Record the acquisition time t of the three cameras to ensure the difference in timestamps of the image groups participating in the fusion. <50ms, unifying the spatial and temporal dimensions of multi-source images. Image enhancement algorithms eliminate minor light and dust interference, highlighting fault characteristics.

[0136] S7. Acoustic-Optical-Thermal Joint Holographic Visualization Modeling and Fault Diagnosis: Relying on a deep learning-based acoustic-optical joint visualization engine, the sound source localization results are transformed into dynamic heat maps and superimposed on multispectral fusion images. The specific algorithm implementation is as follows:

[0137] (1) Sound-light joint visualization mapping: The sound field heat map generation method based on depth prior is adopted to map the intensity S(θ, φ) and spatial coordinates (x, y, z) of abnormal noise signals extracted by array microphones to the RGB color gamut. Red represents strong abnormal noise and blue represents normal noise.

[0138] (2) Perspective projection transformation: The sound field hotspots are transformed by perspective projection. Mapped to image pixel coordinate system:

[0139] ;

[0140] in( , , , ) represents the intrinsic parameter matrix element of the visible light camera 5.

[0141] (3) Holographic image synthesis: The final digital image is generated using a weighted fusion strategy. :

[0142] ;

[0143] Under normal operating conditions, the weights are set as α=0.5, β=0.3, γ=0.1, and δ=0.1. This is an acoustic thermal diagram.

[0144] (4) Historical comparison: Historical inspection holographic image data is retrieved and compared to identify the trend of equipment condition deterioration. In this embodiment, the average sound source positioning error is 0.22m, and the positioning accuracy meets the standard.

[0145] S8. Accurate Fault Classification through Multi-Algorithm Fusion: This step employs a weighted fusion three-branch decision model for comprehensive diagnosis, abandoning single threshold judgment. The specific algorithm is as follows:

[0146] (1) Multi-source evidence fusion: Let the overall confidence level of the fault be C:

[0147] ;

[0148] in:

[0149] The temperature gradient difference function is used to calculate the temperature difference between the device body and the wiring connector based on infrared temperature measurement data, and to assess the degree of overheating by combining the real-time ambient temperature correction coefficient.

[0150] The ultraviolet photon counting function counts the number of ultraviolet discharge photons of the equipment per unit time and determines the insulation discharge level.

[0151] Using the spectrum matching degree function, a spectrum feature comparison algorithm is employed to identify abnormal mechanical vibration noise, abnormal electrical discharge noise, and abnormal noise caused by loose components.

[0152] Confidence level for identifying appearance defects.

[0153] (2) Standard weight configuration: Set in a standard scenario =0.4, =0.3, =0.2, =0.1.

[0154] (3) Setting the confidence threshold for fault determination:

[0155] If C ≥ 0.95, it is determined as a "confirmed fault" and an early warning is triggered;

[0156] If 0.80≤C<0.95, it is judged as "suspected risk" and a second re-inspection is automatically triggered;

[0157] If C < 0.80, it is judged as "environmental interference", and only archives without warning.

[0158] (4) Fault level classification: Based on confidence level C and single-dimensional severity, faults are classified into three levels: minor, moderate and severe.

[0159] In this embodiment, after fusion calculation, the overall confidence level of an overheating fault of a certain terminal block is 98.7%, which is determined to be a valid fault and the level classification is completed.

[0160] S9. Tiered Early Warning and Data Archiving: Differentiated handling is implemented according to the fault level: general overheating faults trigger a background pop-up warning; minor discharge and minor abnormal noise faults only complete holographic image archiving for status tracing. The system synchronously uploads the original data, registered images, holographic images, and diagnostic reports to the remote backend terminal.

[0161] Preferably, step S10, inspection task iteration, can be added: after a single point detection is completed, the gimbal is reset to the full-domain scanning posture, the robot dog continues to move along the path until the full-site inspection is completed, and finally the full-site standardized inspection report is automatically generated.

[0162] Actual test results: Under normal operating conditions, the equipment fault identification accuracy rate is 98.5%. Compared with the traditional single-collection inspection solution, the missed detection rate is reduced by 65%, and the false detection rate is reduced to 1.2%.

[0163] Example 4: Adaptive inspection in low-light environments at night.

[0164] When the inspection strategy scheduling module detects that the ambient light intensity is lower than the preset nighttime threshold Lux< At that time, the system automatically switches to a dedicated nighttime inspection mode. The core of this embodiment lies in the adaptive modal weight adjustment and low-light image enhancement algorithm for low signal-to-noise ratio environments.

[0165] A. Adaptive modal weight adjustment strategy.

[0166] The system's photosensor detects ambient illuminance. When it falls below a preset threshold, the strategy scheduling module dynamically adjusts the weighted fusion weights in S8 to reduce the influence of visible light. The dependence on thermal and corona characteristics is enhanced:

[0167] ;

[0168] The system retains the high-definition imaging function of the visible light camera 5, which continues to serve as the spatial reference for multimodal image registration, while weakening the weight of visible light appearance recognition; the infrared camera 8, ultraviolet camera 7, and sound source localization module are activated at full power, relying on temperature, ultraviolet, and sound data to make up for the shortcomings of nighttime visual imaging.

[0169] B. Low-light image enhancement algorithm based on Retinex theory.

[0170] To address the challenge of feature point extraction due to low brightness in visible light images, a multi-scale retinal enhancement algorithm with color restoration is employed for preprocessing.

[0171] ;

[0172] in This is the image after reflection component enhancement. For the original image, For wrap function, As weight.

[0173] Algorithm function: Enhance details in dark areas at night, ensure the number of SIFT feature points matched with infrared / ultraviolet images, and guarantee registration accuracy of no less than 90% during the day.

[0174] C. Adjustment of nighttime inspection strategy.

[0175] The system automatically reduces the pan-tilt scanning speed by 30% to improve the image acquisition frame rate and sampling accuracy; for all suspected abnormal areas, it automatically adds two multi-angle re-inspection processes to enhance the robustness of image enhancement and feature point matching algorithms, ensuring pixel-level registration accuracy and holographic modeling quality of nighttime images.

[0176] Actual test results: This embodiment effectively solves the problems of missed detection, misjudgment, and lack of visualization support in traditional equipment nighttime inspections by cross-validating multi-dimensional data of sound, light, and heat. The fault identification accuracy rate under nighttime operating conditions is no less than 97.2%.

[0177] Example 5: Adaptive inspection in adverse weather conditions such as rain, snow, fog, and sandstorms.

[0178] When the system detects severe weather conditions such as rain, snow, fog, or sandstorms, it automatically triggers an adaptive inspection strategy for severe weather. The core of this embodiment lies in an image dehazing algorithm based on decomposition convolution and adaptive fusion, as well as a robust feature matching mechanism.

[0179] A. Image dehazing / deraining algorithm.

[0180] To address the challenge of extracting device features under low visibility conditions, an encoder-decoder network architecture based on decomposed convolution is adopted.

[0181] (1) Decomposition of Convolution Enhancement Module: In order to efficiently extract device features under low visibility conditions, the large kernel standard convolution, such as 13×13, is decomposed into a combination of depthwise separable convolution 5×5 and dilated convolution with dilation=3.

[0182] Computational optimization: Number of parameters after decomposition Compared to standard convolution Reduced by approximately 80%, while maintaining the real-time processing capability of the vehicle-mounted edge computing module;

[0183] Receptive field: The equivalent receptive field reaches 13×13, which can capture large-scale contour features under haze.

[0184] (2) Adaptive fusion module: It adopts a dual attention architecture of channel attention mechanism and spatial attention mechanism to automatically learn the feature weights of clear areas and suppress noise interference in foggy areas.

[0185] (3) Contrast regularization: using the foggy image to... and clear image As negative and positive samples, the network output is constrained to tend towards a clear image distribution, restoring the device edges obscured by rain and snow.

[0186] Algorithm effect: After enhancement by this algorithm, the peak signal-to-noise ratio of the image is improved by at least 4dB compared with the original image.

[0187] B. Robust feature matching mechanism.

[0188] Even after image restoration, local distortions may still exist under severe weather conditions. In the S6 registration process, an edge gradient distribution verification mechanism is introduced: only images with high gradient values ​​in both the visible light and infrared edge maps are retained. , The system matches feature points with a preset gradient threshold, eliminates isolated noise points affected by rain and snow, and ensures pixel-level registration accuracy.

[0189] C. Adjustment of strategies for severe weather.

[0190] First, we activated an enhanced multi-dimensional data filtering algorithm to specifically filter image noise, temperature deviations, and sound noise caused by rain and snow obstruction, fog scattering, and sandstorms. Second, we raised the fault determination confidence threshold to 96%, and added two multi-angle re-inspections and verifications to all suspected abnormal areas. Third, we strengthened the image feature point extraction and matching algorithm to ensure pixel-level registration accuracy under harsh conditions. Fourth, we adjusted the fault detection priority, focusing on detecting faults that are prone to occur in severe weather, such as equipment overheating due to water ingress, external insulation discharge, and loose components.

[0191] Actual test results: Under sandstorm and rain conditions, the image registration pixel error is still ≤1 pixel, the sound source localization error is ≤0.28m, the fault identification accuracy is ≥97.0%, and it can stably output high-quality holographic digital images and diagnostic results.

[0192] Example 6: Special inspection conditions for high-load equipment.

[0193] When the inspection strategy scheduling module detects that the real-time operating load of the equipment exceeds the rated threshold, it initiates a high-load special inspection strategy. This embodiment focuses on the quantitative characterization algorithm for ultraviolet discharge and the degradation trend prediction algorithm based on time-series data.

[0194] A. Ultraviolet discharge quantification characterization algorithm: photon counting method.

[0195] Traditional ultraviolet analysis only determines the presence or absence of discharge. This system introduces a solar-blind ultraviolet photon counting algorithm to accurately quantify the discharge intensity.

[0196] (1) Algorithm definition: Suppose that in a certain time period Inside, the total number of photon events in the solar-blind band (240nm-280nm) captured by the UV camera 7 was [number missing]. This wavelength is unaffected by sunlight and can accurately collect ultraviolet signals generated by equipment discharge.

[0197] (2) Grading Standards: Based on the industry standards for partial discharge of power equipment, dynamic grading thresholds are set:

[0198] ;

[0199] in and The settings are dynamically adjusted based on equipment voltage level, ambient humidity, and operating load, rather than fixed thresholds.

[0200] (3) Dynamic correction: Introduce temperature and humidity correction coefficients. Normalize the original photon count:

[0201] ;

[0202] B. Equipment degradation trend prediction algorithm based on Kalman filtering.

[0203] The system retrieves historical holographic image data from the same device. , ..., Compared with current data Conduct status trend analysis:

[0204] (1) Status Extraction – Equipment Comprehensive Health Index: Extracting a single quantitative indicator from the holographic profile – the Equipment Comprehensive Health Index, defined as:

[0205] HI = 1 − C;

[0206] Where C is the overall confidence level of the fault calculated in Example 3. The value of HI ranges from [0,1], the closer it is to 1, the healthier the equipment is, and the closer it is to 0, the higher the risk of fault.

[0207] (2) Kalman Filter State Estimation: The Kalman filter algorithm is used to perform optimal state estimation and short-term prediction of the health index.

[0208] ;

[0209] ;

[0210] in:

[0211] This is a state transition matrix that reflects the natural aging process of the equipment.

[0212] To control the input matrix, External influencing factors such as load fluctuations;

[0213] The process noise covariance matrix;

[0214] P represents the estimation error covariance.

[0215] (3) Kalman filter update measurement update:

[0216] ;

[0217] ;

[0218] ;

[0219] in For Kalman gain, This represents the actual measured value of the current health index. To measure the noise covariance.

[0220] (4) Trend prediction and early warning decision-making: predict the health index value at future time t+n through recursion. If the predicted value is lower than the preset health warning line, such as 0.6, the system will automatically push a "potential deterioration trend" warning even if the serious fault threshold has not been triggered at the current time, so as to achieve predictive maintenance.

[0221] (5) Load-state correlation analysis: By combining historical load data and health index change curves, analyze the performance degradation rate of equipment under high load conditions, predict the risk of equipment aging in advance, and ensure the safe operation of heavy-load equipment.

[0222] C. Adjustment of high-load special strategies.

[0223] Automatically increase the temperature sampling frequency of infrared camera 8, the photon acquisition frequency of ultraviolet camera 7, and the sound source sampling frequency to 1.5 times the normal mode; dynamically narrow the fault temperature threshold, discharge threshold, and abnormal noise threshold, thereby lowering the threshold for triggering judgment; and forcibly add a round of multi-angle re-inspection for high-load equipment, focusing on monitoring three common faults: equipment temperature rise, partial discharge, and vibration abnormal noise.

[0224] Actual results: This embodiment combines ultraviolet photon quantization with Kalman filtering trend prediction, which can detect hidden deterioration trends of equipment 3 to 5 inspection cycles in advance, effectively supporting condition-based maintenance decisions.

[0225] A computer device includes a memory and a processor, which are interconnected and communicate with each other. The memory stores computer instructions, and the processor executes the computer instructions to realize the above-mentioned substation robot dog inspection method based on active perception.

[0226] A computer-readable storage medium in which the methods described above according to embodiments of the present invention can be implemented in hardware, firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and to be stored on a local storage medium after being downloaded over a network, so that the methods described herein can be processed by such software stored on a storage medium using a general-purpose computer, a special-purpose processor, or programmable or special-purpose hardware.

[0227] The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; furthermore, the storage medium can also include combinations of the above types of memory. It is understood that a computer, processor, microprocessor controller, or programmable hardware includes storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0228] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0229] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

[0230] Many other changes and modifications can be made without departing from the concept and scope of this invention. It should be understood that this invention is not limited to the specific embodiments, and the scope of this invention is defined by the appended claims.

Claims

1. A substation robot dog inspection system based on active perception, Its features are, include: The robot dog mobile carrier (1) is used to provide mobile operation support for autonomous walking, intelligent obstacle avoidance and complex terrain adaptation, and is suitable for all-scenario passage operations in the high and low voltage equipment area, busbar area and frame area of ​​substations. An integrated follow-up gimbal (10) is fixedly mounted on the top of the robot dog mobile carrier (1) via a gimbal connector and a robot dog mounting flange (2); The multimodal sensing module includes a visible light camera (5), an ultraviolet camera (7), and an infrared camera (8). The sound source localization module has an integrated adaptive environmental noise filtering algorithm and an equipment abnormal noise feature extraction algorithm, including an array microphone (6) for collecting mechanical and electrical abnormal noise signals of substation equipment, locating the source of abnormal noise faults, and making up for the detection blind spot of single visual perception. The vehicle-mounted edge computing module has a built-in embedded algorithm chip. Through the embedded algorithm chip, gimbal control algorithm, power equipment fault identification algorithm, multimodal image registration algorithm, sound-light joint visualization algorithm, and multimodal data fusion algorithm are deployed locally to realize real-time processing of inspection data, image fusion modeling and instant fault decision-making, and avoid cloud transmission delay and network interference. The inspection strategy scheduling module has built-in substation equipment ledger, standard inspection path, hierarchical inspection rules and equipment fault threshold standards. It is used to dynamically generate adaptive inspection tasks based on the operation scenario, environmental conditions and equipment operating status. The intelligent identification and re-inspection module is connected to the integrated follow-up gimbal (10), the multimodal perception module, the sound source positioning module and the vehicle edge computing module respectively. It is used to perform a closed-loop inspection process of full-domain active perception coarse detection, anomaly preliminary judgment, gimbal target locking, multi-angle precise re-inspection and fault precise classification. The multimodal image fusion and holographic visualization module deploys a feature point image matching algorithm and a deep learning acoustic-optical joint visualization engine to complete pixel-level registration and image enhancement optimization of multi-source images. At the same time, it converts sound source localization information into dynamic heat maps and overlays them onto multispectral fusion images to generate a holographic digital portrait of the device status containing sound, light, and heat multidimensional information, supporting anomaly visualization and historical data comparison analysis. The data storage and transmission module is used to store the original sensor data of the inspection, image registration and fusion data, holographic digital portrait, re-inspection data and fault diagnosis report, and realize local caching and real-time interaction with the remote backend, remote traceability and status comparison. The fault early warning module, which works in conjunction with the vehicle edge computing module and the data storage and transmission module, is used to perform graded early warning push based on the fault classification results, so as to realize real-time management and control of fault risks.

2. The substation robot dog inspection system based on active perception as described in claim 1, Its features are, The integrated servo gimbal (10) includes a horizontal rotation mechanism (3), a pitch adjustment mechanism (4), and a main control cabin (9). The vehicle edge computing module, the inspection strategy scheduling module, the intelligent identification and re-inspection module, the multimodal image fusion and holographic visualization module, the data storage and transmission module, and the fault early warning module are all integrated in the main control cabin (9).

3. The substation robot dog inspection system based on active perception as described in claim 1, Its features are, The multimodal sensing module has an adaptive adjustment function for the acquired parameters; The visible light camera (5) supports 4K high-definition imaging, automatic exposure and intelligent zoom functions, and is used to acquire high-resolution reference images of the device and use them as a unified spatial reference. It relies on feature point image matching algorithm to achieve pixel-level accurate registration of visible light, infrared and ultraviolet three-source images. The infrared camera (8) is used to collect temperature field distribution data on the surface of the equipment and detect potential overheating and abnormal temperature of the equipment. The ultraviolet camera (7) is used to capture ultraviolet light signals of equipment insulation discharge and identify latent insulation faults.

4. A substation robot dog inspection system based on active perception as described in claim 1, Its features are, The inspection strategy scheduling module supports dynamic adaptive scheduling across all scenarios. Based on day and night lighting conditions, severe weather conditions such as rain, snow, fog, and dust, real-time equipment operating load, and historical fault statistics, the pan-tilt-zoom scanning speed, data acquisition accuracy, number of re-inspections, and fault detection threshold are adaptively adjusted. In low-light conditions at night, the infrared and ultraviolet priority detection modes are automatically activated, and registration is completed based on visible light images to ensure modeling accuracy. Enhanced data filtering, increased re-inspection frequency, and image enhancement under adverse weather conditions help resist interference from complex environments.

5. A substation robot dog inspection system based on active perception as described in claim 1, Its features are, The fault early warning module implements a tiered and differentiated early warning mechanism: Minor faults are only archived with holographic profiles and trend records for equipment tracing and degradation trend assessment. A common fault triggers a real-time pop-up notification in the background, reminding maintenance personnel to pay attention to the equipment status. Serious malfunctions trigger audible and visual alarms on-site and simultaneously upload emergency warning information and a holographic diagnostic profile of the equipment to the remote operation and maintenance backend.

6. A substation robot dog inspection method based on active perception, employing a substation robot dog inspection system based on active perception as described in any one of claims 1-5. Its features are, Includes the following steps: S1, Inspection Initialization Calibration: The system receives the inspection task issued by the inspection strategy scheduling module, loads the three-dimensional scene map of the substation, equipment coordinate ledger, equipment fault identification threshold and exclusive algorithm model, completes the attitude calibration of the integrated follow-up pan-tilt (10), parameter calibration of the multimodal perception module and environmental noise baseline calibration of the sound source positioning module, and ensures that the system is in a standardized standby operation state. S2, Full-domain active perception coarse detection: The robot dog mobile carrier (1) autonomously avoids obstacles along the preset optimal inspection path. The integrated follow-up gimbal (10) opens the full-domain low-speed scanning mode and simultaneously starts the multi-modal perception module and sound source positioning module. It collects visible light reference images, infrared temperature field data, ultraviolet discharge data and equipment sound field data in all directions. Through the lightweight power-specific algorithm built into the vehicle edge computing module, it quickly screens the real-time data, completes the preliminary judgment of equipment abnormality, and identifies suspected fault targets. S3, Abnormal Target Locking and Gimbal Follow-up Positioning: If the coarse detection identifies suspected abnormal data, immediately control the robot dog's mobile carrier (1) to stop moving. The vehicle edge computing module calculates the spatial coordinates of the abnormal target in real time, generates a high-precision gimbal attitude adjustment command, drives the integrated follow-up gimbal (10) to adaptively rotate, zoom and hover, and accurately lock the suspected abnormal equipment area. S4. Multi-angle precise re-inspection operation: Start the standardized precise re-inspection mechanism, control the integrated follow-up gimbal (10) to adopt a multi-view layered acquisition strategy, and carry out repeated data acquisition of suspected abnormal areas at close range, multiple angles and high resolution, simultaneously optimize the acquisition parameters and filter environmental interference, and obtain multi-modal raw data with high consistency and high matching degree, so as to provide high-quality data support for subsequent image registration and holographic modeling. S5. Multimodal Image Pixel-Level Registration and Enhancement Processing: Call the multimodal image fusion and holographic visualization module, based on the feature point image matching algorithm, and using the visible light high-definition image as the benchmark, complete the pixel-level accurate registration of visible light, infrared and ultraviolet three-source images, and combine the image enhancement algorithm to eliminate the interference of light, rain, snow and dust environment, and unify the consistency of the spatial dimension and temporal dimension of multi-source images. S6. Sound-light-heat joint holographic visualization modeling and fault diagnosis: Relying on the deep learning sound-light joint visualization engine, the sound source localization analysis results are transformed into dynamic heat maps, which are dynamically superimposed on the registered multispectral fusion image to generate a holographic digital portrait of the equipment status containing sound, light and heat multidimensional information. By combining multi-dimensional power-specific fault identification algorithms, the fault type, location, and level can be accurately determined. At the same time, historical inspection holographic profile data can be retrieved for comparative analysis to identify the trend of equipment condition deterioration. S7. Accurate Fault Classification through Multi-Algorithm Fusion: Based on the high-precision re-inspection dataset and holographic digital profile, it integrates multi-dimensional features such as infrared temperature anomaly, ultraviolet insulation discharge, and equipment abnormal noise. Through a multi-modal data weighted fusion strategy, it integrates the detection results, eliminates misjudgment data caused by environmental interference, and accurately completes the classification of fault type, fault location, and fault level. S8. Data archiving and task iteration: Completely record the original inspection data, image registration and fusion data, holographic digital portrait, re-inspection data and fault diagnosis results, automatically generate standardized inspection reports and upload them to the remote back-end terminal, control the integrated follow-up PTZ (10) to reset to the initial inspection posture, and the robot dog mobile carrier (1) continues to perform subsequent inspection tasks until the substation's full-area equipment inspection is completed.

7. A substation robot dog inspection method based on active perception as described in claim 6. Its features are, Step S2 includes: S21. The integrated PTZ (10) adopts a wide-range uniform low-speed scanning mode to cover the inspection area on both sides of the robot dog's mobile carrier (1) travel path, and covers all kinds of power equipment in the substation without dead angles. S22. Acquire high-resolution visible light reference images for subsequent multimodal image registration and modeling; Infrared temperature field data is analyzed using a temperature mean comparison algorithm to screen for localized overheating areas and areas with excessive temperature differences in the equipment. A pixel threshold determination algorithm was used to analyze ultraviolet imaging data to preliminarily identify insulation discharge anomalies. The sound source data was analyzed using a spectral feature comparison algorithm to preliminarily determine the type of abnormal noise fault in the equipment. S23. Preset multi-level anomaly initial judgment thresholds, compare the real-time collected infrared, ultraviolet and sound source multi-dimensional data with the equipment standard parameter library in real time. If any dimension of data exceeds the threshold range, it is judged as a suspected anomaly, and the precise re-inspection and holographic visualization modeling instructions are automatically triggered to enter the refined detection process.

8. A substation robot dog inspection method based on active perception as described in claim 6. Its features are, Step S7 specifically includes: S71. Temperature Anomaly Diagnosis: The infrared temperature gradient differential algorithm is used to calculate the temperature difference between the equipment body, connectors and leads. Combined with the real-time ambient temperature correction coefficient, the equipment overheating fault is classified into three risk levels: minor, moderate and severe. S72. Insulation Discharge Diagnosis: Using an ultraviolet photon counting algorithm, the number of ultraviolet discharge photons of the equipment per unit time is counted. Combined with the industry standard for discharge level of power equipment, hidden safety hazards such as insulation aging, partial discharge, and flashover are identified. S73. Abnormal Noise Troubleshooting: Using sound spectrum analysis and feature matching algorithms, we can distinguish between abnormal mechanical vibration noise, abnormal electrical discharge noise, and abnormal noise from loose components, and locate hidden faults that cannot be identified by vision. S74. Multimodal fusion decision: A weighted fusion algorithm is used to integrate multidimensional detection results and combine them with the visualization features of holographic digital profiles to set a confidence threshold for fault determination.

9. A computer device, Its features are, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the substation robot dog inspection method based on active perception as described in claim 6.

10. A computer-readable storage medium, Its features are, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the substation robot dog inspection method based on active perception as described in claim 6.

Images