Substation equipment operation and maintenance fault positioning and automatic repairing system based on deep learning

Abstract

The invention provides a power transformation equipment operation and maintenance fault positioning and automatic repairing system based on deep learning. The system comprises a multi-modal data acquisition module; a data preprocessing and fusion module; a deep learning fault identification module; a three-dimensional space positioning module; a fault decision and repair scheduling module; a robot automatic repairing module; a visual monitoring and feedback module; according to the invention, fault signals are captured in real time by means of the multi-modal data acquisition module, faults are rapidly identified by combining with the improved YOLOv8 model, the position is locked by the three-dimensional space positioning module, manual troubleshooting is not needed, the traditional positioning time of 2-3 hours is shortened to be less than 10 minutes, the fault response efficiency is greatly improved, and the repair operation is executed through the robot automatic repair module, so that the repair efficiency is greatly improved. And a visual verification module is matched to ensure fault elimination, so that the dependence on artificial experience is remarkably reduced, the field workload of operation and maintenance personnel is greatly reduced, and the safety risks such as electric shock and equipment accidental injury in a high-voltage environment are reduced from the source.

Description

Technical Field

[0001] This invention relates to a deep learning-based fault location and automatic repair system for power equipment operation and maintenance, belonging to the field of power equipment fault location and intelligent repair technology. Background Technology

[0002] Transformer equipment is the core infrastructure of the power system's transmission and distribution links. Its operating status directly determines the stability and security of power supply. With the expansion of the power grid, the number of equipment in large hub substations has surged and their structures have become more complex. Due to their special geographical location, unmanned substations in remote areas are difficult to be inspected frequently by humans. The traditional operation and maintenance mode of transformer equipment has gradually exposed significant shortcomings. Currently, fault location in power equipment mainly relies on manual inspection and single sensor monitoring. Manual inspection requires maintenance personnel to enter the high-voltage area to check one by one. Due to the wide distribution of equipment and the risks of the high-voltage environment, the location time often reaches 2-3 hours, which can easily delay fault handling. A single sensor can only capture one-dimensional information, making it difficult to comprehensively judge the fault type, which can easily lead to misjudgment or missed judgment. In terms of fault location accuracy, traditional methods rely heavily on equipment identification tags and experience estimations, lacking precise correlation with the physical space of the substation. The location error often exceeds 5 meters, resulting in the need to search for the fault point again for subsequent repairs, which is inefficient. The fault repair process relies more on manual operation. Even for simple faults such as loose small parts or poor contact, personnel need to carry tools into the high-voltage area, which not only involves a large workload but also poses safety risks such as electric shock and equipment damage. It is especially unsuitable for the operation and maintenance needs of remote unattended substations. In summary, existing technologies are insufficient to meet the needs of efficient location, accurate identification, and safe repair in the operation and maintenance of substation equipment. There is an urgent need for a technical solution that integrates multi-dimensional perception, intelligent analysis, and autonomous execution to address the pain points of low efficiency, poor accuracy, and high safety risks in traditional operation and maintenance. To this end, a substation equipment operation and maintenance fault location and automatic repair system based on deep learning is proposed. Summary of the Invention

[0003] In view of this, the present invention provides a deep learning-based fault location and automatic repair system for substation equipment operation and maintenance, in order to solve or alleviate the technical problems existing in the prior art, and at least provide a beneficial option.

[0004] The technical solution of the present invention is implemented as follows: a deep learning-based substation equipment operation and maintenance fault location and automatic repair system, including a multimodal data acquisition module, a data preprocessing and fusion module, a deep learning fault identification module, a three-dimensional spatial positioning module, a fault decision and repair scheduling module, a robot automatic repair module, and a visualization monitoring and feedback module that is data connected to the fault decision and repair scheduling module and the robot automatic repair module respectively. The multimodal data acquisition module acquires images of the substation's appearance, temperature distribution data, and operating sound signals through a high-definition camera, infrared thermal imager, and sonar sensor, and performs spatiotemporal synchronization. The data preprocessing and fusion module is used to clean and standardize the data collected by the multimodal data acquisition module, and to fuse visual, infrared, and acoustic features through an attention mechanism. The deep learning fault identification module uses an improved YOLO target detection algorithm to reason about the fused features and identify the fault type and the two-dimensional coordinates of the fault in the image. The three-dimensional spatial positioning module is used to convert the two-dimensional coordinates of the image into world coordinates in conjunction with the three-dimensional spatial model of the substation. The fault decision and repair scheduling module is used to assess the fault level based on the fault type and location results, and to schedule the maintenance robot. The robot automatic repair module is used to control the maintenance robot to carry special tools to the fault location and complete automatic repair through visual guidance. The visualization monitoring and feedback module is used to monitor the repair process in real time and verify the repair effect through secondary detection of multimodal data.

[0005] More preferably, the multimodal data acquisition module is equipped with a high-definition camera array, an infrared thermal imager network, a distributed sonar sensor, and a data synchronization unit. The data synchronization unit realizes the spatiotemporal alignment of multimodal data based on a unified timestamp. In the multimodal data acquisition module, the high-definition camera array has a resolution of 3840×2160 and supports day and night dual-mode imaging. The deployment density is 1 set per 500㎡ substation area. The infrared thermal imager network has a temperature measurement range of -20℃ to 300℃, a temperature measurement accuracy of ±0.5℃, a frame rate of ≥25fps, and supports temperature anomaly threshold early warning. The distributed sonar sensor performs sampling according to the set sampling parameters and has an anti-electromagnetic interference design. The deployment location is ≤5 meters away from the transformer and circuit breaker.

[0006] More preferably, the data synchronization unit in the multimodal data acquisition module uses GPS timing combined with local clock calibration to achieve timestamp synchronization, with a time synchronization accuracy of ≤1ms; The data synchronization unit is also configured with a data caching mechanism, which provides each type of device in the multimodal data acquisition module with an industrial-grade eMMC cache chip. When the network is interrupted, it can cache at least 2 hours of multimodal data locally and automatically retransmit it after the network is restored.

[0007] More preferably, the data preprocessing and fusion module includes a data cleaning unit and a multimodal feature fusion unit. The data cleaning unit performs distortion correction, noise reduction and contrast enhancement on the image data, removes extreme values ​​and standardizes temperature values ​​on the infrared data, and performs Fourier transform on the acoustic data to convert it into frequency domain features. Then, the multimodal feature fusion unit performs dynamic weighted fusion of the preprocessed visual features, infrared features and acoustic features based on an attention mechanism to generate a fusion feature vector that is strongly correlated with the fault. The data cleaning unit uses Gaussian filtering to remove noise from the image data, perspective transformation to correct distortion, and adaptive histogram equalization to enhance contrast. Extreme values ​​were removed from the infrared data using the 3σ criterion, and the temperature values ​​were converted to standardized values ​​in the range of [-1,1] using Z-score standardization. The acoustic data were converted to frequency domain features using short-time Fourier transform, and the Mel frequency cepstral coefficients were extracted as the core parameters of the acoustic features.

[0008] In a further preferred embodiment, the multimodal feature fusion unit assigns weights by calculating the mutual information value between each modal feature and the fault label. The modality with the higher the mutual information value has a larger weight ratio, and residual connections are used during the fusion process to preserve the original feature details of each modality.

[0009] More preferably, the deep learning fault identification module includes an improved YOLO target detection model, which constructs a small target detection branch by adding a 1×1 convolutional layer to the neck of the YOLOv8 base network to capture subtle fault features; the improved YOLO target detection model uses the FocalLoss loss function to solve the problem of class imbalance in fault samples; The deep learning fault identification module is also configured with a fault feature library matching unit. The fault feature library stores typical feature vectors of historical faults. When the improved YOLO target detection model outputs a fault confidence of 85%-98%, the fault type is further verified by feature vector cosine similarity matching.

[0010] More preferably, the three-dimensional spatial positioning module includes a substation three-dimensional modeling unit and a coordinate matching and optimization unit. The substation three-dimensional modeling unit uses a laser point cloud scanner for initial modeling, and subsequently updates the local point cloud of the substation every quarter by using a mobile laser scanning device, and achieves registration of the old and new point clouds through the ICP algorithm. The three-level mapping relationship of the coordinate matching and optimization unit is established through calibration by a calibration board. First, the intrinsic and extrinsic parameters of the high-definition camera are obtained through the calibration board. The intrinsic parameters are the focal length and principal point coordinates, and the extrinsic parameters are the position and attitude relative to the sensor mounting point. The pixel coordinates are then mapped to the sensor physical coordinates. Next, the GPS coordinates of the sensor mounting point are calibrated with the origin of the substation's world coordinate system to establish the sensor physical coordinates to the world coordinates. The number of particles in the particle filtering algorithm is set to 500-1000, and the number of iterations is ≤10.

[0011] In a further preferred embodiment, the fault decision and repair scheduling module has a built-in fault level assessment model. Based on the fault type, the impact range of the fault location on equipment operation, and the risk of fault propagation, the fault is divided into 1-5 levels. Levels 1-3 are simple faults that can be automatically repaired, and levels 4-5 are complex faults that require manual intervention. For faults of levels 1-3, the robot scheduling unit is invoked to plan the optimal movement path based on the fault location and the real-time status of the robot, and the target maintenance robot is scheduled.

[0012] More preferably, the robot's automatic repair module includes a wheeled mobile platform and a magnetic special tool library. The wheeled mobile platform uses a fusion navigation system of LiDAR and visual camera. The LiDAR is used for long-distance obstacle detection, and the visual camera is used for close-range texture recognition. The navigation algorithm introduces a dynamic weight factor, prioritizes paths without high-voltage equipment, has an obstacle avoidance response time ≤100ms, and a maximum moving speed ≤0.5m / s. The magnetic tool library is equipped with a tool identification and positioning unit. The tool identification and positioning unit realizes tool replacement through the mechanical arm and the magnetic connector at the end of the mechanical arm. The tool replacement time is ≤5 seconds. The tool library has a built-in tool wear monitoring sensor. When the tool wear exceeds the threshold, it will automatically trigger tool replacement or alarm.

[0013] More preferably, the visualization monitoring and feedback module includes an AR interaction unit and a repair effect verification unit. The AR interaction unit supports two annotation modes: one is fault information annotation, which overlays information such as fault type, location coordinates, and repair tools on the repair screen; the other is operation guidance annotation, which guides the movement trajectory of the robotic arm through AR lines; and it also supports maintenance personnel to manually draw AR annotations through a remote control terminal, with annotation command transmission delay ≤200ms. The repair effect verification unit adopts a multi-dimensional threshold comparison method. The visual dimension compares the pixel differences of the fault area before and after repair, the infrared dimension compares the temperature value, and the acoustic dimension compares the characteristic frequency energy. When all three indicators meet the threshold requirements, the repair is judged to be successful.

[0014] The embodiments of the present invention have the following advantages due to the adoption of the above technical solutions: This invention significantly reduces positioning time by combining multimodal real-time acquisition with deep learning for rapid identification and precise 3D positioning technology. The multimodal data acquisition module, coupled with a synchronization unit with a synchronization accuracy of ≤1ms, captures fault signals in real time; the improved YOLOv8 model quickly identifies fault types; and the 3D spatial positioning module achieves positioning accuracy of ±0.5 meters, eliminating the need for manual inspection and reducing the traditional 2-3 hour positioning time to less than 10 minutes. The robot's automatic repair module and visualization verification module work together to ensure repair quality. After repair, multi-dimensional verification using visual pixel differences, infrared temperature comparison, and acoustic frequency energy ensures fault elimination, achieving an automatic repair success rate of over 80% for simple faults of levels 1-3. This reduces reliance on human experience, replacing manual labor with automatic repair, and avoiding on-site supervision through remote monitoring, thus reducing personnel exposure. Over 80% of simple faults are automatically handled by the robot without requiring personnel to enter high-voltage areas; only complex faults of levels 4-5 require manual intervention, and the visualization monitoring module supports remote viewing and guidance, significantly reducing the on-site workload of maintenance personnel and reducing safety risks such as electric shock and equipment injury in high-voltage environments from the source.

[0015] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a general system framework diagram of the deep learning-based substation equipment operation and maintenance fault location and automatic repair system of the present invention. Figure 2 This is a sub-flowchart of data acquisition, identification, and location in the deep learning-based fault location and automatic repair system for substation equipment operation and maintenance according to the present invention. Figure 3 This is a sub-flowchart of the robot repair and monitoring verification in the deep learning-based substation equipment operation and maintenance fault location and automatic repair system of the present invention. Detailed Implementation

[0018] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0019] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] like Figure 1-3 As shown, this embodiment of the invention provides a deep learning-based fault location and automatic repair system for substation equipment operation and maintenance, including: The multimodal data acquisition module consists of a high-definition camera array, an infrared thermal imager network, a distributed sonar sensor, and a data synchronization unit. It acquires images of the substation's appearance, temperature distribution data, and operating sound signals through high-definition cameras, infrared thermal imagers, and sonar sensors, and performs spatiotemporal synchronization. The data preprocessing and fusion module consists of a data cleaning unit and a multimodal feature fusion unit. It is used to clean and standardize multimodal data and fuse visual, infrared, and acoustic features through an attention mechanism. The deep learning fault identification module consists of an improved YOLO target detection model and a fault feature library matching unit. Based on the improved YOLO target detection algorithm, it infers the fused features to identify the fault type and the two-dimensional coordinates of the fault in the image. The three-dimensional spatial positioning module consists of a substation three-dimensional modeling unit and a coordinate matching and optimization unit. It is used to combine the substation three-dimensional spatial model to convert the image two-dimensional coordinates into world coordinates, so as to achieve fault location with an accuracy of ±0.5 meters. The fault decision and repair scheduling module is used to assess the fault level based on the fault type and location results, and to schedule maintenance robots for simple faults of levels 1-3. The robot automatic repair module consists of a wheeled mobile platform and a magnetic special tool library. It is used to control the maintenance robot to carry special tools to the fault location and complete automatic repair through visual guidance. The visualization monitoring and feedback module consists of an AR interaction unit and a repair effect verification unit. It is used to monitor the repair process in real time and verify the repair effect through secondary detection of multimodal data.

[0021] In one embodiment, the data synchronization unit achieves spatiotemporal alignment of multimodal data based on a unified timestamp, ensuring temporal consistency and spatial correlation of data acquisition. In the multimodal data acquisition module, the high-definition camera array has a resolution of 3840×2160 and supports day and night dual-mode imaging. The lenses have an IP67 waterproof and dustproof rating, and the deployment density is one set per 500㎡ substation area to ensure no blind spots. The infrared thermal imager network has a temperature measurement range of -20℃ to 300℃, a temperature measurement accuracy of ±0.5℃, a frame rate ≥25fps, and supports temperature anomaly threshold pre-testing. Warning: The distributed sonar sensor has a sampling rate of 44.1kHz and a frequency response range of 20Hz-20kHz. It features an anti-electromagnetic interference design and its deployment location should be ≤5 meters away from sound source devices such as transformers and circuit breakers. The data synchronization unit in the multimodal data acquisition module uses GPS timing combined with local clock calibration to achieve timestamp synchronization with a time synchronization accuracy of ≤1ms. The data synchronization unit is also equipped with a data caching mechanism, which can locally cache at least 2 hours of multimodal data when the network is interrupted and automatically retransmit it after the network is restored to avoid data loss.

[0022] In one embodiment, the data cleaning unit performs distortion correction, denoising, and contrast enhancement on image data; removes extreme values ​​and standardizes temperature values ​​on infrared data; and performs Fourier transform on acoustic data to convert it into frequency domain features. Then, the multimodal feature fusion unit dynamically weights and fuses the preprocessed visual features, infrared features, and acoustic features based on an attention mechanism to generate a fused feature vector strongly correlated with the fault. The data cleaning unit uses Gaussian filtering to denoise the image data, perspective transformation to correct distortion, and adaptive histogram equalization to enhance contrast. For infrared data, the 3σ criterion is used to remove extreme values, and Z-score standardization is used to convert temperature values ​​into standardized values ​​in the [-1,1] range. For acoustic data, short-time Fourier transform is used to convert it into frequency domain features, and Mel-frequency cepstral coefficients are extracted as core parameters of the acoustic features. Gaussian filtering is a linear smoothing filter that reduces noise and preserves edge information by performing a convolution operation on an image. Its core is a two-dimensional Gaussian function. ; Where (x, y) are the pixel coordinates, and σ is the standard deviation, which controls the smoothness. Image I, after being Gaussian filtered, outputs: ; Where k is the convolution kernel radius; Perspective transformation is used to correct geometric distortions in images caused by camera viewpoint. It maps the image from the original coordinate system to a new coordinate system using a 3×3 transformation matrix M. Let the coordinates of a point in the original image be (x, y), and the coordinates after transformation be... Represented using homogeneous coordinates: ; The actual coordinates are obtained after normalization: ; Matrix M is usually solved using known pairs of corresponding points on the calibration plate; The short-time Fourier transform is used to analyze the time-frequency characteristics of non-stationary signals. It is achieved by windowing the signal and performing the Fourier transform in segments. ; The discrete form is: ; Where x[n] is the discrete signal, w[n] is the window function, m is the time frame index, and k is the frequency index.

[0023] In one embodiment, the multimodal feature fusion unit assigns weights by calculating the mutual information value between each modal feature and the fault label. The modality with the higher the mutual information value has a larger weight. Furthermore, residual connections are used during the fusion process to preserve the original feature details of each modality and avoid loss of feature information.

[0024] In one embodiment, the improved YOLO object detection model constructs a small object detection branch by adding a 1×1 convolutional layer to the neck of the YOLOv8 base network to capture subtle fault features such as loose bolts and broken wire strands. The improved YOLO object detection model uses the FocalLoss loss function to solve the problem of class imbalance in fault samples, and the model inference speed is ≥30fps, meeting the requirements for real-time recognition. The fault feature library stores typical feature vectors of historical faults. When the improved YOLO object detection model outputs a fault confidence of 85%-98%, the fault type is further verified by feature vector cosine similarity matching to reduce the false recognition rate.

[0025] In one embodiment, the substation 3D modeling unit uses a laser point cloud scanner for initial modeling, and then uses a mobile laser scanning device to update the local point cloud of the substation every quarter. The registration of the old and new point clouds is achieved through the ICP algorithm to ensure the consistency between the 3D model and the physical substation. The three-level mapping relationship of the coordinate matching and optimization unit is established through calibration by a calibration board. First, the intrinsic parameters (focal length, principal point coordinates) and extrinsic parameters (position and attitude relative to the sensor mounting point) of the high-definition camera are obtained through the calibration board to establish the mapping from pixel coordinates to sensor physical coordinates. Then, the GPS coordinates of the sensor mounting point are calibrated with the origin of the substation's world coordinate system to establish the mapping from sensor physical coordinates to world coordinates. The number of particles in the particle filtering algorithm is set to 500-1000, and the number of iterations is ≤10 to ensure real-time positioning.

[0026] In one embodiment, the fault decision and repair scheduling module has a built-in fault level assessment model. Based on the fault type, the impact range of the fault location on equipment operation, and the risk of fault propagation, the fault is divided into 1-5 levels. Levels 1-3 are simple faults that can be automatically repaired, and levels 4-5 are complex faults that require manual intervention. For faults of levels 1-3, the robot scheduling unit is invoked to plan the optimal movement path based on the fault location and the real-time status of the robot, and the target maintenance robot is scheduled.

[0027] In one embodiment, the wheeled mobile platform uses a fusion navigation system of LiDAR and visual camera. The LiDAR is used for long-distance obstacle detection, and the visual camera is used for close-range texture recognition. The navigation algorithm introduces a dynamic weight factor, prioritizes paths without high-voltage equipment, has an obstacle avoidance response time of ≤100ms, and a maximum moving speed of ≤0.5m / s. The magnetic tool library is equipped with a tool identification and positioning unit. The tool identification and positioning unit realizes tool replacement through the robotic arm and the magnetic connector at the end of the robotic arm. The tool replacement time is ≤5 seconds. The tool library has a built-in tool wear monitoring sensor. When the tool wear exceeds the threshold, it will automatically trigger tool replacement or alarm.

[0028] In one embodiment, the AR interactive unit supports two annotation modes: one is fault information annotation, which overlays information such as fault type, location coordinates, and tools required for repair onto the repair screen; the other is operation guidance annotation, which guides the movement trajectory of the robotic arm through AR lines; and it also supports maintenance personnel to manually draw AR annotations through a remote control terminal, with annotation command transmission latency ≤200ms. AR line guidance is based on the parameter representation and projection transformation of straight lines in three-dimensional space. Let the end effector of the robotic arm start from point... Exercise The straight line can be represented in three-dimensional space as: ; The 3D points are projected onto the AR display plane using the camera projection matrix and extrinsic parameter matrix. ; The two-dimensional points projected into lines form the trajectory lines that guide the movement of the robotic arm in AR. The repair effect verification unit adopts a multi-dimensional threshold comparison method. The visual dimension compares the pixel differences of the fault area before and after repair, the infrared dimension compares the temperature value, and the acoustic dimension compares the characteristic frequency energy. The repair is judged to be successful when all three indicators meet the threshold requirements.

[0029] In one embodiment, a data support module is also included to provide data support for all the above modules. The data support module includes a distributed database unit and a model incremental learning unit. The distributed database unit stores device parameters, historical fault records, repair logs, and multimodal acquisition data. The model incremental learning unit updates and improves the parameters of the YOLO object detection model through transfer learning based on newly acquired unidentified fault data or fault data after repair and verification, thereby continuously improving the model's fault identification generalization ability.

[0030] In one embodiment, for unmanned substations in remote areas, the multimodal data acquisition module and the robot automatic repair module are equipped with a solar power supply unit with a solar panel power of ≥300W and an energy storage battery capacity of ≥100Ah, supporting off-grid operation for ≥72 hours; and the system is equipped with a satellite communication module that supports Beidou + GPS dual-mode, so that when the 4G / 5G network is interrupted, fault alarm information and key repair data are transmitted via satellite.

[0031] In one embodiment, for a large hub substation scenario, the fault decision and repair scheduling module is configured with a multi-robot collaborative scheduling unit, which adopts a load balancing and task splitting strategy: when multiple level 1-3 faults occur simultaneously, tasks are allocated according to robot location, remaining power, and tool configuration, and tool sharing between robots is supported to improve the efficiency of handling multiple faults.

[0032] In operation, this invention deploys: 10 sets of 3840×2160 resolution cameras, one set per 500㎡, with lenses having an IP67 waterproof and dustproof rating, installed on the top of the columns in the substation equipment area, covering key equipment such as main transformers, circuit breakers, and insulators, supporting daytime RGB mode and nighttime infrared mode switching; 8 infrared thermal imagers are deployed, with a temperature measurement range of -20℃ to 300℃, accuracy of ±0.5℃, and a frame rate of 25fps, focusing on covering easily heated parts such as main transformer bushings and circuit breaker joints, with a set temperature anomaly threshold, triggering an early warning when the joint temperature ≥60℃; and 15 sonar sensors are deployed, with a sampling rate of 44.1kHz and a frequency response of 20Hz-20kHz, equipped with EMC Class B... The system boasts strong electromagnetic interference resistance, maintaining a straight-line distance of ≤5 meters from noise sources such as transformers and circuit breakers. It employs a GPS timing module with a synchronization accuracy of ≤1ms, combined with local clock calibration. Each acquisition device is equipped with a 16GB local cache to ensure at least 2 hours of multimodal data can be cached during network interruptions. A laser point cloud scanner with a point cloud density of 120 points / ㎡ and a ranging accuracy of ±2mm is used for initial modeling of the substation, generating a centimeter-level 3D model containing equipment location, dimensions, and topological relationships. Quarterly updates are performed on the area after equipment maintenance using a mobile laser scanning device, with ICP algorithm used to register the old and new point clouds, ensuring consistency between the model and the physical substation. A 1m×1m checkerboard calibration board is used to acquire high-resolution data. Clear the camera's intrinsic parameters (focal length f=3.8mm, principal point coordinates) and extrinsic parameters (position and attitude relative to the sensor mounting column), and establish a pixel coordinate-sensor physical coordinate mapping; calibrate the world coordinate system origin using three GPS reference points within the substation, with an accuracy of ±0.1m, and establish a sensor physical coordinate-world coordinate mapping; configure three maintenance robots, equipped with LiDAR (detection range 0.1-20m, scanning frequency 20Hz) and vision cameras, using an improved AI navigation algorithm (dynamic weighting factor prioritizes avoiding areas with high-voltage equipment above 10kV), obstacle avoidance response time ≤100ms, maximum moving speed 0.5m / s, 8-hour battery life, and a 50kg payload; each robot is equipped with It is equipped with 15 specialized tools, including electric wrenches, sealant guns, and oxide layer cleaning tools. The tool library has a built-in RFID identification unit, and the tool replacement time for a single tool is ≤5 seconds with the magnetic connector at the end of the robotic arm. The tool wear monitoring sensor has a threshold setting, and an alarm is triggered when the torque error of the electric wrench is ≥5%. The improved YOLOv8 model is deployed on the local edge server of the substation. The model training samples include 23 types of substation equipment faults (including insulator damage, loose bolts, broken conductor strands, etc.), and the inference speed is 35fps. A 4K high-definition monitoring terminal is deployed (latency ≤200ms). The AR interaction unit supports fault information annotation (overlaying fault type, coordinates, and tool model) and operation guidance annotation (AR lines guide the movement trajectory of the robotic arm).

[0033] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A deep learning-based fault location and automatic repair system for substation equipment operation and maintenance, characterized in that, It includes a multimodal data acquisition module that sequentially interacts with data, a data preprocessing and fusion module, a deep learning fault identification module, a 3D spatial positioning module, a fault decision and repair scheduling module, a robot automatic repair module, and a visualization monitoring and feedback module that is data-connected to the fault decision and repair scheduling module and the robot automatic repair module, respectively, wherein: The multimodal data acquisition module acquires images of the substation's appearance, temperature distribution data, and operating sound signals through a high-definition camera, infrared thermal imager, and sonar sensor, and performs spatiotemporal synchronization. The data preprocessing and fusion module is used to clean and standardize the data collected by the multimodal data acquisition module, and to fuse visual, infrared, and acoustic features through an attention mechanism. The deep learning fault identification module uses an improved YOLO target detection algorithm to reason about the fused features and identify the fault type and the two-dimensional coordinates of the fault in the image. The three-dimensional spatial positioning module is used to convert the two-dimensional coordinates of the image into world coordinates in conjunction with the three-dimensional spatial model of the substation. The fault decision and repair scheduling module is used to assess the fault level based on the fault type and location results, and to schedule the maintenance robot. The robot automatic repair module is used to control the maintenance robot to carry special tools to the fault location and complete automatic repair through visual guidance. The visualization monitoring and feedback module is used to monitor the repair process in real time and verify the repair effect through secondary detection of multimodal data.

2. The deep learning-based substation equipment operation and maintenance fault location and automatic repair system according to claim 1, characterized in that: The multimodal data acquisition module is equipped with a high-definition camera array, an infrared thermal imager network, a distributed sonar sensor, and a data synchronization unit. The data synchronization unit achieves spatiotemporal alignment of multimodal data based on a unified timestamp. In the multimodal data acquisition module, the high-definition camera array has a resolution of 3840×2160 and supports day and night dual-mode imaging. The deployment density is one set per 500㎡ substation area. The infrared thermal imager network has a temperature measurement range of -20℃ to 300℃, a temperature measurement accuracy of ±0.5℃, a frame rate of ≥25fps, and supports temperature anomaly threshold early warning. The distributed sonar sensor performs sampling according to the set sampling parameters and has an anti-electromagnetic interference design. The deployment location is ≤5 meters away from the transformer and circuit breaker.

3. The deep learning-based substation equipment operation and maintenance fault location and automatic repair system according to claim 1, characterized in that: The data synchronization unit in the multimodal data acquisition module uses GPS timing combined with local clock calibration to achieve timestamp synchronization, with a time synchronization accuracy of ≤1ms. The data synchronization unit is also configured with a data caching mechanism, which provides each type of device in the multimodal data acquisition module with an industrial-grade eMMC cache chip. When the network is interrupted, it can cache at least 2 hours of multimodal data locally and automatically retransmit it after the network is restored.

4. The deep learning-based substation equipment operation and maintenance fault location and automatic repair system according to claim 1, characterized in that: The data preprocessing and fusion module includes a data cleaning unit and a multimodal feature fusion unit. The data cleaning unit performs distortion correction, noise reduction, and contrast enhancement on image data, removes extreme values ​​and standardizes temperature values ​​on infrared data, and performs Fourier transform on acoustic data to convert it into frequency domain features. Then, the multimodal feature fusion unit performs dynamic weighted fusion of the preprocessed visual features, infrared features, and acoustic features based on an attention mechanism to generate a fusion feature vector that is strongly correlated with the fault. The data cleaning unit uses Gaussian filtering to remove noise from the image data, perspective transformation to correct distortion, and adaptive histogram equalization to enhance contrast. Extreme values ​​were removed from the infrared data using the 3σ criterion, and the temperature values ​​were converted to standardized values ​​in the range of [-1,1] using Z-score standardization. The acoustic data were converted to frequency domain features using short-time Fourier transform, and the Mel frequency cepstral coefficients were extracted as the core parameters of the acoustic features.

5. The deep learning-based substation equipment operation and maintenance fault location and automatic repair system according to claim 4, characterized in that: The multimodal feature fusion unit assigns weights by calculating the mutual information value between each modal feature and the fault label. The higher the mutual information value, the greater the weight of the modality. In addition, residual connections are used during the fusion process to preserve the original feature details of each modality.

6. The deep learning-based substation equipment operation and maintenance fault location and automatic repair system according to claim 1, characterized in that: The deep learning fault identification module includes an improved YOLO target detection model. The improved YOLO target detection model constructs a small target detection branch by adding a 1×1 convolutional layer to the neck of the YOLOv8 base network to capture subtle fault features. The improved YOLO target detection model uses the FocalLoss loss function to solve the problem of class imbalance in fault samples. The deep learning fault identification module is also configured with a fault feature library matching unit. The fault feature library stores typical feature vectors of historical faults. When the improved YOLO target detection model outputs a fault confidence of 85%-98%, the fault type is further verified by feature vector cosine similarity matching.

7. The deep learning-based substation equipment operation and maintenance fault location and automatic repair system according to claim 1, characterized in that: The three-dimensional spatial positioning module includes a substation three-dimensional modeling unit and a coordinate matching and optimization unit. The substation three-dimensional modeling unit uses a laser point cloud scanner for initial modeling, and then uses a mobile laser scanning device to update the local point cloud of the substation every quarter, and uses the ICP algorithm to register the old and new point clouds. The three-level mapping relationship of the coordinate matching and optimization unit is established through calibration by a calibration board. First, the intrinsic and extrinsic parameters of the high-definition camera are obtained through the calibration board. The intrinsic parameters are the focal length and principal point coordinates, and the extrinsic parameters are the position and attitude relative to the sensor mounting point. The pixel coordinates are then mapped to the sensor physical coordinates. Next, the GPS coordinates of the sensor mounting point are calibrated with the origin of the substation's world coordinate system to establish the sensor physical coordinates to the world coordinates. The number of particles in the particle filtering algorithm is set to 500-1000, and the number of iterations is ≤10.

8. The deep learning-based substation equipment operation and maintenance fault location and automatic repair system according to claim 1, characterized in that: The fault decision and repair scheduling module has a built-in fault level assessment model. Based on the fault type, the impact range of the fault location on equipment operation, and the risk of fault spread, the fault is divided into 1-5 levels. Levels 1-3 are simple faults that can be automatically repaired, and levels 4-5 are complex faults that require manual intervention. For faults of levels 1-3, the robot scheduling unit is invoked to plan the optimal movement path based on the fault location and the real-time status of the robot, and the target maintenance robot is scheduled.

9. The deep learning-based substation equipment operation and maintenance fault location and automatic repair system according to claim 1, characterized in that: The robot's automatic repair module includes a wheeled mobile platform and a magnetic tool library. The wheeled mobile platform uses a fusion navigation system of LiDAR and a visual camera. The LiDAR is used for long-distance obstacle detection, and the visual camera is used for close-range texture recognition. The navigation algorithm introduces a dynamic weight factor, prioritizes paths without high-voltage equipment, has an obstacle avoidance response time of ≤100ms, and a maximum moving speed of ≤0.5m / s. The magnetic tool library is equipped with a tool identification and positioning unit. The tool identification and positioning unit realizes tool replacement through the mechanical arm and the magnetic connector at the end of the mechanical arm. The tool replacement time is ≤5 seconds. The tool library has a built-in tool wear monitoring sensor. When the tool wear exceeds the threshold, it will automatically trigger tool replacement or alarm.

10. The deep learning-based substation equipment operation and maintenance fault location and automatic repair system according to claim 1, characterized in that: The visualization monitoring and feedback module includes an AR interaction unit and a repair effect verification unit. The AR interaction unit supports two annotation modes: one is fault information annotation, which overlays information such as fault type, location coordinates, and repair tools into the repair screen; the other is operation guidance annotation, which guides the movement trajectory of the robotic arm through AR lines. It also supports maintenance personnel to manually draw AR annotations through a remote control terminal, with annotation command transmission latency ≤200ms. The repair effect verification unit adopts a multi-dimensional threshold comparison method. The visual dimension compares the pixel differences of the fault area before and after repair, the infrared dimension compares the temperature value, and the acoustic dimension compares the characteristic frequency energy. When all three indicators meet the threshold requirements, the repair is judged to be successful.

Images