Power distribution room multi-mode sensing inspection robot based on visual navigation

Abstract

The invention discloses a power distribution room multi-mode sensing inspection robot based on visual navigation. The robot comprises a mobile platform; the visual navigation module is arranged on the mobile platform and is used for collecting environmental vision and distance information of the power distribution room and realizing robot positioning and autonomous path planning; the multi-mode sensing module is arranged on the mobile platform and is used for synchronously acquiring the equipment state and environment multi-dimensional data of the power distribution room; and the control module is electrically connected with the mobile platform, the visual navigation module and the multi-mode sensing module, and is used for receiving the positioning path information and the sensing data and controlling the movement of the mobile platform. The method has the beneficial effects that a visual navigation scheme of fusing the binocular camera and the laser radar is adopted; the multi-modal sensing module integrates infrared, sound, gas, temperature and humidity sensing units and other multi-dimensional sensing units, so that the omnibearing detection of the equipment state and the environment is realized, and the fault missing detection caused by a single sensing mode is avoided; the crawler-type moving mechanism is matched with a damping assembly.

Description

Technical Field

[0001] This invention relates to the field of power equipment inspection technology, and in particular to a multimodal perception inspection robot for power distribution rooms based on vision navigation. Background Technology

[0002] As a key component of the power system, the operating status of power distribution rooms directly affects the stability and security of power supply. Traditional power distribution room inspections mainly rely on manual labor, requiring inspectors to be in close contact with high-voltage equipment, facing safety risks such as electric shock and gas poisoning. At the same time, manual inspections are inefficient, easily affected by subjective factors leading to missed faults, and cannot achieve 24-hour uninterrupted inspection.

[0003] Existing technologies include some power distribution room inspection robots, but they have the following drawbacks: 1) Single navigation method, mostly using magnetic track navigation or single vision navigation. Magnetic track navigation requires a preset track and has poor flexibility, while single vision navigation has low positioning accuracy in low light or complex environments; 2) Limited perception mode, mostly only having temperature or image acquisition functions, unable to comprehensively detect various types of faults such as abnormal equipment noise and gas leaks; 3) Weak data processing capability, without the use of data fusion technology, resulting in large perception data errors and low accuracy in anomaly detection; 4) Poor environmental adaptability, the mobile mechanism is difficult to adapt to uneven ground in the power distribution room, and communication is easily interfered with.

[0004] Therefore, it is necessary to propose a vision-navigation-based multimodal perception inspection robot for power distribution rooms to address the above problems. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the present invention aims to provide a vision-navigation-based multimodal perception inspection robot for power distribution rooms.

[0006] A multimodal perception inspection robot for a power distribution room based on visual navigation includes: a mobile platform; and a visual navigation module, which is set on the mobile platform to collect visual and distance information of the power distribution room environment, and realize robot localization and autonomous path planning.

[0007] A multimodal sensing module, installed on the mobile platform, is used to synchronously collect multi-dimensional data on the status of equipment and environment in the power distribution room;

[0008] The control module is electrically connected to the mobile platform, the visual navigation module, and the multimodal perception module, respectively, and is used to receive positioning path information and perception data, control the movement of the mobile platform, and perform fusion processing and anomaly detection on the perception data.

[0009] The communication module, electrically connected to the control module, is used to transmit the processed inspection data and alarm information to the remote monitoring center.

[0010] The visual navigation module includes a binocular camera, a lidar, and a navigation processor. The binocular camera collects environmental image information, the lidar collects environmental distance information, and the navigation processor constructs a 3D map of the power distribution room environment using the SLAM algorithm, and realizes real-time positioning and path planning based on the 3D map.

[0011] The steps of the SLAM algorithm are as follows:

[0012] S1 and ORB feature extraction and descriptor generation;

[0013] Let the image pixels be grayscale value ,by Centered on a circle with a radius of 3 pixels, 16 sampling points are selected. If there are consecutive... The gray values ​​of each point satisfy the following: , ;

[0014] in Here, t represents the sampling point, t represents the grayscale threshold, and p represents the corner point.

[0015] S2. Inter-frame feature matching and outlier removal, assuming frame... Feature point descriptor ,frame Feature point descriptor Matching cost: ;

[0016] For XOR operation

[0017] S3. Attitude estimation: Solve for the essential matrix E, decompose it to obtain (R,t), and use the lidar correction scale.

[0018] Interior point matching pairs satisfy essential matrix constraints

[0019] ;

[0020] in These are pixel coordinates; Matching point coordinates; It is the antisymmetric matrix of the translation vector;

[0021] in Let t be the rotation matrix and t be the translation vector; ;

[0022] S4, 3D map construction and bundle adjustment (BA) optimization; based on adjacent frame pose Triangulation yields the three-dimensional coordinates of the feature points; :

[0023] ;

[0024] in The focal length of the camera; The principal point of the camera;

[0025] S5. Loop closure detection and map correction: A pre-stored feature library of key locations in the power distribution room (e.g., corner points of distribution cabinets, switch markings) is used to calculate the similarity between the current frame and frames in the feature library.

[0026]

[0027] For a bag-of-words dictionary, The term frequency-inverse document frequency weighting is used.

[0028] The multimodal sensing module includes an infrared thermal imaging sensing unit, a sound sensing unit, a gas sensing unit, a temperature and humidity sensing unit, and a synchronization control unit. The infrared thermal imaging sensing unit is used to detect the surface temperature distribution of the power distribution equipment, the sound sensing unit is used to collect abnormal noise signals from the equipment, the gas sensing unit is used to detect the concentration of gases such as SF6, O2, and CO, the temperature and humidity sensing unit is used to collect ambient temperature and humidity data, and the synchronization control unit is used to control each sensing unit to collect data synchronously.

[0029] The algorithm steps for the infrared thermal imaging sensing unit are as follows:

[0030] S11. Preprocessing of raw thermal image data: median filtering for noise reduction. Let the original pixel matrix of the thermal image be... The median filter output is:

[0031] Linear stretching is used to map pixel grayscale values ​​to the [0, 255] range, using the formula:

[0032] ;

[0033] in The minimum grayscale value of a single frame thermal image. The maximum grayscale value for a single frame of thermal image ensures that hotspot areas of the device are clearly identifiable;

[0034] S12. Gray-to-temperature calibration conversion: The gray value output by the infrared sensor has a non-linear relationship with the target temperature. Based on the calibration data of the standard blackbody furnace, a quadratic polynomial fitting is used to achieve gray-to-temperature mapping.

[0035] in For pixels The corresponding actual temperature;

[0036] These are calibration coefficients;

[0037] S13. Power distribution room equipment area segmentation: An adaptive threshold segmentation algorithm is used to separate target equipment such as power distribution cabinets and switches from the environmental background. Adaptive threshold calculation and segmentation threshold are applied.

[0038] in For pixels, Average temperature For variance;

[0039] S14. Temperature feature extraction: Extract key temperature features from the segmented equipment area to provide input for anomaly detection.

[0040] The control module includes a main controller and a data processing unit. The data processing unit uses a Kalman filter algorithm to fuse the multi-dimensional data output by the multimodal sensing module, eliminating data noise interference and improving the accuracy of the sensing data. The main controller has a built-in anomaly detection model, which is trained based on a random forest machine learning algorithm. By comparing the fused data with a preset threshold, it identifies abnormal states such as equipment overheating, gas leakage, abnormal operating noise, and excessive temperature and humidity, and generates alarm information including the location, type, and degree of the anomaly.

[0041] The mobile platform employs a tracked mobility mechanism, including a drive motor, anti-slip tracks, shock absorption components, and a load-bearing chassis. The shock absorption components are spring-damped shock absorption structures used to buffer vibrations generated by uneven ground in the power distribution room. The anti-slip tracks have anti-slip textures on their surface, improving traversal capabilities on complex terrain. The navigation processor is also equipped with a visual feature matching unit. This unit extracts corner and edge features from the environmental image and compares them with a pre-stored database of key location features in the power distribution room to correct the positioning error of the SLAM algorithm, ensuring the robot's positioning accuracy is ≤5cm.

[0042] The gas sensing unit employs an electrochemical sensor array, which includes an SF6 sensor, an O2 sensor, and a CO sensor, with detection accuracies of ±10ppm, ±0.5%VOL, and ±5ppm, respectively, and a response time of ≤3s.

[0043] The communication module adopts a redundant design of wired and wireless communication. The wireless communication module supports dual-mode switching between 5G and Wi-Fi, and the wired communication module is equipped with an Ethernet interface. The communication module also has a built-in data encryption unit that uses the AES-256 encryption algorithm to encrypt the transmitted data.

[0044] It also includes a charging module and a battery management unit. The charging module is located at the bottom of the mobile platform, and the battery management unit is electrically connected to the control module to detect the remaining battery power. When the power is lower than a preset threshold, the control module plans a path through the visual navigation module and controls the mobile platform to move to the preset charging station to achieve autonomous charging.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] 1. It adopts a visual navigation solution that integrates binocular cameras and LiDAR, combined with SLAM algorithm and visual feature matching technology, with a positioning accuracy of ≤5cm. It is suitable for complex environments such as power distribution rooms, does not require preset tracks, and is highly flexible.

[0047] 2. The multimodal sensing module integrates multiple sensing units such as infrared, sound, gas, temperature and humidity to achieve comprehensive detection of equipment status and environment, avoiding missed fault detection caused by a single sensing mode.

[0048] 3. Data fusion is performed using the Kalman filter algorithm to eliminate noise interference, and combined with a machine learning anomaly detection model to improve the accuracy of fault identification;

[0049] 4. The tracked mobile mechanism is equipped with shock absorption components to adapt to uneven ground, and the communication module adopts a dual-mode redundancy design to ensure communication reliability in complex environments.

[0050] 5. It enables autonomous navigation, autonomous inspection, autonomous charging, and automatic fault alarm without human intervention, reducing safety risks and improving inspection efficiency. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the composition of the present invention;

[0052] Figure 2 This is a flowchart of the algorithm steps of the feature extraction unit of the present invention;

[0053] Figure 3 This is a flowchart of the algorithm steps of the model reasoning unit of the present invention. Detailed Implementation

[0054] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0055] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0056] like Figure 1 and combined Figures 2 to 3As shown, a multimodal perception inspection robot for power distribution rooms based on visual navigation includes: a mobile platform; and a visual navigation module, which is set on the mobile platform and is used to collect visual and distance information of the power distribution room environment to realize robot localization and autonomous path planning.

[0057] A multimodal sensing module, installed on the mobile platform, is used to synchronously collect multi-dimensional data on the status of equipment and environment in the power distribution room;

[0058] The control module is electrically connected to the mobile platform, the visual navigation module, and the multimodal perception module, respectively, and is used to receive positioning path information and perception data, control the movement of the mobile platform, and perform fusion processing and anomaly detection on the perception data.

[0059] The communication module, electrically connected to the control module, is used to transmit the processed inspection data and alarm information to the remote monitoring center.

[0060] The visual navigation module includes a binocular camera, a lidar, and a navigation processor. The binocular camera collects environmental image information, the lidar collects environmental distance information, and the navigation processor constructs a 3D map of the power distribution room environment using the SLAM algorithm, and realizes real-time positioning and path planning based on the 3D map.

[0061] The steps of the SLAM algorithm are as follows:

[0062] S1 and ORB feature extraction and descriptor generation;

[0063] Let the image pixels be grayscale value ,by Centered on a circle with a radius of 3 pixels, 16 sampling points are selected. If there are consecutive... The gray values ​​of each point satisfy the following: , ;

[0064] in Here, t represents the sampling point, t represents the grayscale threshold, and p represents the corner point.

[0065] S2. Inter-frame feature matching and outlier removal, assuming frame... Feature point descriptor ,frame Feature point descriptor Matching cost: ;

[0066] For XOR operation

[0067] S3. Attitude estimation: Solve for the essential matrix E, decompose it to obtain (R,t), and use the lidar correction scale.

[0068] Interior point matching pairs satisfy essential matrix constraints

[0069] ;

[0070] in These are pixel coordinates; Matching point coordinates; It is the antisymmetric matrix of the translation vector;

[0071] in Let t be the rotation matrix and t be the translation vector; ;

[0072] S4, 3D map construction and bundle adjustment (BA) optimization; based on adjacent frame pose Triangulation yields the three-dimensional coordinates of the feature points; :

[0073] ;

[0074] in The focal length of the camera; The principal point of the camera;

[0075] S5. Loop closure detection and map correction: A pre-stored feature library of key locations in the power distribution room (e.g., corner points of distribution cabinets, switch markings) is used to calculate the similarity between the current frame and frames in the feature library.

[0076]

[0077] For a bag-of-words dictionary, The term frequency-inverse document frequency weighting is used.

[0078] The multimodal sensing module includes an infrared thermal imaging sensing unit, a sound sensing unit, a gas sensing unit, a temperature and humidity sensing unit, and a synchronization control unit. The infrared thermal imaging sensing unit is used to detect the surface temperature distribution of the power distribution equipment, the sound sensing unit is used to collect abnormal noise signals from the equipment, the gas sensing unit is used to detect the concentration of gases such as SF6, O2, and CO, the temperature and humidity sensing unit is used to collect ambient temperature and humidity data, and the synchronization control unit is used to control each sensing unit to collect data synchronously.

[0079] The algorithm steps for the infrared thermal imaging sensing unit are as follows:

[0080] S11. Preprocessing of raw thermal image data: median filtering for noise reduction. Let the original pixel matrix of the thermal image be... The median filter output is:

[0081] Linear stretching is used to map pixel grayscale values ​​to the [0, 255] range, using the formula:

[0082] ;

[0083] in The minimum grayscale value of a single frame thermal image. The maximum grayscale value for a single frame of thermal image ensures that hotspot areas of the device are clearly identifiable;

[0084] S12. Gray-to-temperature calibration conversion: The gray value output by the infrared sensor has a non-linear relationship with the target temperature. Based on the calibration data of the standard blackbody furnace, a quadratic polynomial fitting is used to achieve gray-to-temperature mapping.

[0085] in For pixels The corresponding actual temperature;

[0086] These are calibration coefficients;

[0087] S13. Power distribution room equipment area segmentation: An adaptive threshold segmentation algorithm is used to separate target equipment such as power distribution cabinets and switches from the environmental background. Adaptive threshold calculation and segmentation threshold are applied.

[0088] in For pixels, Average temperature For variance;

[0089] S14. Temperature feature extraction: Extract key temperature features from the segmented equipment area to provide input for anomaly detection.

[0090] The control module includes a main controller and a data processing unit. The data processing unit uses a Kalman filter algorithm to fuse the multi-dimensional data output by the multimodal sensing module, eliminating data noise interference and improving the accuracy of the sensing data. The main controller has a built-in anomaly detection model, which is trained based on a random forest machine learning algorithm. By comparing the fused data with a preset threshold, it identifies abnormal states such as equipment overheating, gas leakage, abnormal operating noise, and excessive temperature and humidity, and generates alarm information including the location, type, and degree of the anomaly.

[0091] The mobile platform employs a tracked mobility mechanism, including a drive motor, anti-slip tracks, shock absorption components, and a load-bearing chassis. The shock absorption components are spring-damped shock absorption structures used to buffer vibrations generated by uneven ground in the power distribution room. The anti-slip tracks have anti-slip textures on their surface, improving traversal capabilities on complex terrain. The navigation processor is also equipped with a visual feature matching unit. This unit extracts corner and edge features from the environmental image and compares them with a pre-stored database of key location features in the power distribution room to correct the positioning error of the SLAM algorithm, ensuring the robot's positioning accuracy is ≤5cm.

[0092] The gas sensing unit employs an electrochemical sensor array, which includes an SF6 sensor, an O2 sensor, and a CO sensor, with detection accuracies of ±10ppm, ±0.5%VOL, and ±5ppm, respectively, and a response time of ≤3s.

[0093] The communication module adopts a redundant design of wired and wireless communication. The wireless communication module supports dual-mode switching between 5G and Wi-Fi, and the wired communication module is equipped with an Ethernet interface. The communication module also has a built-in data encryption unit that uses the AES-256 encryption algorithm to encrypt the transmitted data.

[0094] It also includes a charging module and a battery management unit. The charging module is located at the bottom of the mobile platform, and the battery management unit is electrically connected to the control module to detect the remaining battery power. When the power is lower than a preset threshold, the control module plans a path through the visual navigation module and controls the mobile platform to move to the preset charging station to achieve autonomous charging.

[0095] Compared with the prior art, the present invention has the following advantages:

[0096] 6. It adopts a visual navigation solution that integrates binocular cameras and LiDAR, combined with SLAM algorithm and visual feature matching technology, with a positioning accuracy of ≤5cm. It is suitable for complex environments such as power distribution rooms, does not require preset tracks, and is highly flexible.

[0097] 7. The multimodal sensing module integrates multiple sensing units such as infrared, sound, gas, temperature and humidity to achieve comprehensive detection of equipment status and environment, avoiding missed fault detection caused by a single sensing mode.

[0098] 8. Use Kalman filtering algorithm for data fusion to eliminate noise interference, and combine it with machine learning anomaly detection model to improve the accuracy of fault identification;

[0099] 9. The tracked mobile mechanism is equipped with shock absorption components to adapt to uneven ground, and the communication module adopts a dual-mode redundancy design to ensure communication reliability in complex environments.

[0100] It enables autonomous navigation, autonomous inspection, autonomous charging, and automatic fault alarm without human intervention, reducing safety risks and improving inspection efficiency.

[0101] Example 1: Hardware Configuration of the Inspection Robot

[0102] Mobile platform: The drive motor model is 57BLDC-200, with a rated power of 200W and a speed of 3000rpm; the anti-slip track is 50mm wide and has a diamond-shaped anti-slip pattern on the surface; the shock absorption component adopts a spring damping structure with a damping coefficient of 0.5N・s / m, and the load-bearing chassis has a maximum load capacity of 10kg.

[0103] Visual navigation module: The binocular camera is model OV9281, with a resolution of 1280×720 and a frame rate of 30fps; the lidar is model RPLIDAR3, with a measurement range of 0.1-12m and an angular resolution of 0.33°; the navigation processor is NVIDIA Jetson Nano, running Ubuntu 18.04 system, and has built-in ORB-SLAM2 algorithm and SIFT feature matching algorithm.

[0104] Multimodal sensing module: The infrared thermal imaging sensor unit is model FLIRLEPTON3.5, with a resolution of 160×120 and a temperature measurement range of -10~400℃; the sound sensor unit is a 4-microphone array with a sampling rate of 48kHz and a signal-to-noise ratio of ≥60dB; the gas sensor unit uses an array composed of MQ-135 and SF6 dedicated sensors, with detection accuracies of SF6±10ppm, O2±0.5%VOL, and CO±5ppm, respectively; the temperature and humidity sensor unit is model DHT22, with a temperature accuracy of ±0.5℃ and a humidity accuracy of ±2%RH; the synchronization control unit uses an STM32F103 microcontroller to control the sampling frequency of each sensor unit to be synchronized at 10Hz.

[0105] Control module: The main controller uses a Xilinx Artix-7 FPGA chip, and the data processing unit integrates an ARM Cortex-A9 processor to run the Kalman filter algorithm and the random forest anomaly detection model (the training samples include 1000 normal data sets and 500 fault data sets).

[0106] Communication module: The wireless communication module adopts Huawei ME909s-8215G module and ESP8266Wi-Fi module, the wired communication module adopts DM9000 Ethernet controller; the encryption unit adopts AES-256 hardware encryption chip.

[0107] Charging module and battery management unit: The charging module outputs 24V and 5A; the battery management unit uses the BQ76952 chip to detect the charge, voltage and temperature of the lithium battery (24V / 10Ah).

[0108] Example 2: Workflow of the Inspection Robot

[0109] Initialization: After the robot starts, the binocular camera and lidar of the visual navigation module collect environmental data of the power distribution room. The navigation processor constructs a 3D map of the environment through the ORB-SLAM2 algorithm and completes the initial positioning. The multimodal perception module performs self-checks to ensure that each sensing unit is working properly. The control module loads the preset inspection path and anomaly detection threshold.

[0110] Autonomous Inspection: The control module controls the movement of the mobile platform according to a preset path and in conjunction with the real-time positioning information from the visual navigation module. The multimodal perception module, under the control of the synchronous control unit, synchronously collects infrared thermal images, operating sounds, gas concentrations, and ambient temperature and humidity data, and transmits them to the control module. Data Processing and Anomaly Detection: The control module's data processing unit fuses multi-dimensional data using a Kalman filter algorithm to eliminate noise. The main controller's anomaly detection model compares the fused data with preset thresholds. If an anomaly is detected (such as equipment temperature exceeding 85℃, SF6 concentration exceeding 1000ppm, or abnormal noises), an alarm is immediately generated, including the anomaly location (based on visual navigation positioning results), type, and severity.

[0111] Data transmission: The communication module encrypts the merged inspection data and abnormal alarm information and transmits them to the remote monitoring center, where the monitoring center can view the inspection status and fault information in real time.

[0112] Autonomous Charging: When the battery management unit detects that the battery level is below 20%, it sends a charging request to the control module. The control module plans a path to the charging station using the visual navigation module, controls the mobile platform to autonomously move to the charging station, and the charging module docks to complete the charging. After charging is completed, the inspection automatically resumes. Example 3: Performance Testing

[0113] Tests were conducted in a 10kV power distribution room with an area of ​​50㎡, containing 10 power distribution cabinets and 5 sets of switchgear. Three simulated faults were set up (local overheating of equipment, slight SF6 leakage, and abnormal noise of equipment): Navigation performance: The robot completed environmental map construction in ≤5min, with positioning accuracy ≤3cm, and the path planning was collision-free. It successfully passed through complex areas such as the edge of cable trenches and uneven ground.

[0114] Sensing performance: The infrared thermal imaging sensor unit accurately identifies local overheating faults (temperature error ≤1℃), the gas sensor unit detects excessive SF6 concentration and alarms (response time ≤2s), and the sound sensor unit captures abnormal noises from the equipment and locates the fault location.

[0115] Anomaly detection accuracy: All three simulated faults were accurately identified, with no missed or false detections, achieving an anomaly detection accuracy of 100%.

[0116] Communication performance: 5G and Wi-Fi dual-mode switching is normal, data transmission rate is ≥10Mbps, and encrypted transmission has no data leakage;

[0117] Battery life: Continuous inspection time ≥ 8 hours on a full charge, with a 100% success rate in autonomous charging and docking. Test results show that the inspection robot described in this invention meets the requirements for automated inspection of power distribution rooms, with precise navigation, comprehensive perception, accurate detection, and strong environmental adaptability, and can effectively replace manual inspection.

[0118] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A vision-navigation-based multimodal perception inspection robot for power distribution rooms, characterized in that: include: Mobile platform; A visual navigation module, located on the mobile platform, is used to collect visual and distance information about the power distribution room environment to enable robot localization and autonomous path planning. A multimodal sensing module, installed on the mobile platform, is used to synchronously collect multi-dimensional data on the status of equipment and environment in the power distribution room; The control module is electrically connected to the mobile platform, the visual navigation module, and the multimodal perception module, respectively, and is used to receive positioning path information and perception data, control the movement of the mobile platform, and perform fusion processing and anomaly detection on the perception data. The communication module, electrically connected to the control module, is used to transmit the processed inspection data and alarm information to the remote monitoring center.

2. The vision-navigation-based multimodal perception inspection robot for power distribution rooms as described in claim 1, characterized in that: The visual navigation module includes a binocular camera, a lidar, and a navigation processor. The binocular camera collects environmental image information, the lidar collects environmental distance information, and the navigation processor constructs a 3D map of the power distribution room environment using the SLAM algorithm, and realizes real-time positioning and path planning based on the 3D map.

3. The vision-navigation-based multimodal perception inspection robot for power distribution rooms as described in claim 1, characterized in that: The steps of the SLAM algorithm are as follows: S1 and ORB feature extraction and descriptor generation; Let the image pixels be grayscale value ,by Centered on a circle with a radius of 3 pixels, 16 sampling points are selected. If there are consecutive... The gray values ​​of each point satisfy the following: , ; in Here, t represents the sampling point, t represents the grayscale threshold, and p represents the corner point. S2. Inter-frame feature matching and outlier removal, assuming frame... Feature point descriptor ,frame Feature point descriptor Matching cost: ; This is the XOR operation; where Indicates Hamming distance; This represents the element at the k-th position in the data at time t; This refers to the element at the k-th position in the data at time t-1. S3. Attitude estimation: Solve for the essential matrix E, decompose it to obtain (R,t), and use the lidar correction scale. Interior point matching pairs satisfy essential matrix constraints , ; in These are pixel coordinates; Matching point coordinates; It is the antisymmetric matrix of the translation vector; Let be the matrix to be decomposed. It is a left singular vector matrix; It is the transpose of the right singular vector matrix; A singular value diagonal matrix, where the diagonal elements are the singular values ​​of E. in Let t be the rotation matrix and t be the translation vector; ; S4, 3D map construction and bundle adjustment optimization; based on adjacent frame pose Triangulation yields the three-dimensional coordinates of the feature points; : ； in The focal length of the camera; The principal point of the camera; These are the coordinates of a point in three-dimensional space. These are the rotation matrix elements for the camera's extrinsic parameters; Translation component of camera extrinsic parameters, These are the pixel coordinates of the final output two-dimensional image; S5. Loop closure detection and map correction: A pre-stored feature library of key locations in the power distribution room is used to calculate the similarity between the current frame and frames in the feature library. , For a bag-of-words dictionary, The term frequency-inverse document frequency weighting; Representing text and Similarity; It is a word In the target text TF-IDF value in; It is a word In the reference text The TF-IDF value in the data.

4. The vision-navigation-based multimodal perception inspection robot for power distribution rooms as described in claim 1, characterized in that: The multimodal sensing module includes an infrared thermal imaging sensing unit, a sound sensing unit, a gas sensing unit, a temperature and humidity sensing unit, and a synchronization control unit. The infrared thermal imaging sensing unit is used to detect the surface temperature distribution of the power distribution equipment, the sound sensing unit is used to collect abnormal noise signals from the equipment, the gas sensing unit is used to detect the concentrations of gases such as SF6, O2, and CO, the temperature and humidity sensing unit is used to collect ambient temperature and humidity data, and the synchronization control unit is used to control each sensing unit to collect data synchronously.

5. The vision-navigation-based multimodal perception inspection robot for power distribution rooms as described in claim 1, characterized in that: The algorithm steps for the infrared thermal imaging sensing unit are as follows: S11. Preprocessing of raw thermal image data, median filtering for noise reduction. Let the original pixel matrix of the thermal image be... ; The median filter output is: Linear stretching is used to map pixel grayscale values ​​to the [0, 255] range, using the formula: ； in The minimum grayscale value of a single frame thermal image. This is the maximum grayscale value for a single frame of thermal image, ensuring that hotspot areas of the device are clearly identifiable. It is the pixel value at pixel coordinates (m, n) in the filtered image; The pixel values ​​of the original image at coordinates m+i, n+j; Indicates taking The 3×3 neighborhood centered on the center; This represents the pixel value at pixel coordinates (m, n) of the image after grayscale stretching; S12. Gray-to-temperature calibration conversion: The gray value output by the infrared sensor has a non-linear relationship with the target temperature. Based on the calibration data of the standard blackbody furnace, a quadratic polynomial fitting is used to achieve gray-to-temperature mapping. ,in For pixels The corresponding actual temperature; These are calibration coefficients; S13. Power distribution room equipment area segmentation: An adaptive threshold segmentation algorithm is used to separate target equipment such as power distribution cabinets and switches from the environmental background. Adaptive threshold calculation and segmentation threshold are applied. ,in The local adaptive threshold at pixel coordinates (m,n); For pixels, Average temperature For variance; For adjustment coefficients; S14. Temperature feature extraction: Extract key temperature features from the segmented equipment area to provide input for anomaly detection.

6. The vision-navigation-based multimodal perception inspection robot for power distribution rooms as described in claim 1, characterized in that: The control module includes a main controller and a data processing unit. The data processing unit uses a Kalman filter algorithm to fuse the multi-dimensional data output by the multimodal sensing module, eliminating data noise interference and improving the accuracy of the sensing data. The main controller has a built-in anomaly detection model, which is trained based on a random forest machine learning algorithm. By comparing the fused data with a preset threshold, it identifies abnormal states such as equipment overheating, gas leakage, abnormal operating noise, and excessive temperature and humidity, and generates alarm information including the location, type, and degree of the anomaly.

7. The vision-navigation-based multimodal perception inspection robot for power distribution rooms as described in claim 1, characterized in that: The mobile platform adopts a tracked mobile mechanism, including a drive motor, anti-slip tracks, shock absorption components and a load-bearing chassis. The shock absorption components are spring-damped shock absorption structures used to buffer vibrations generated by uneven ground in the power distribution room. The anti-slip tracks have anti-slip textures on their surface to improve the ability to travel on complex terrain.

8. The vision-navigation-based multimodal perception inspection robot for power distribution rooms as described in claim 2, characterized in that: The navigation processor is also equipped with a visual feature matching unit, which extracts corner and edge features from the environmental image and compares them with a pre-stored key location feature library of the power distribution room to correct the positioning error of the SLAM algorithm and make the robot positioning accuracy ≤5cm.

9. A vision-navigation-based multimodal perception inspection robot for power distribution rooms as described in claim 3, characterized in that: The gas sensing unit employs an electrochemical sensor array, which includes an SF6 sensor, an O2 sensor, and a CO sensor, with detection accuracies of ±10ppm, ±0.5%VOL, and ±5ppm, respectively, and a response time of ≤3s.

10. A vision-navigation-based multimodal perception inspection robot for power distribution rooms as described in claim 1, characterized in that: The communication module adopts a redundant design of wired and wireless communication. The wireless communication module supports dual-mode switching between 5G and Wi-Fi, and the wired communication module is equipped with an Ethernet interface. The communication module also has a built-in data encryption unit that uses the AES-256 encryption algorithm to encrypt the transmitted data.

Images