Converter station intelligent gateway image recognition system and equipment defect detection method

Abstract

The invention discloses a converter station intelligent gateway image recognition system based on a YOLOv3 target detection algorithm, and the system employs a three-stage cooperative processing architecture design, and builds seamless connection of a multispectral image collection layer, an edge calculation gateway layer, and a cloud operation and maintenance management platform layer. The invention further provides an equipment defect detection method based on the system, bimodal image data are collected through the visible light camera and the thermal infrared imager, preprocessing operation is carried out, equipment positioning and defect classification are synchronously executed by utilizing the improved YOLOv3 network, a structured detection result is output, temperature field analysis is carried out on an infrared thermal image, and the equipment defect detection result is obtained. And an abnormal heating area is identified, when defects are detected, multi-level risk response early warning is generated, and a defect diagnosis report is pushed to the cloud operation and maintenance management platform layer. Real-time image analysis of converter station equipment can be realized, the method is suitable for automatic detection of typical fault defects of the equipment, and the operation and maintenance efficiency of a power grid is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent operation and maintenance of power equipment, in particular to a converter station intelligent gateway image recognition system based on a YOLOv3 target detection algorithm and a device defect detection method realized based on the system. BACKGROUND

[0002] With the acceleration of ultra-high voltage power grid construction, the scale of converter station equipment is rapidly growing, and the amount of information collected by intelligent equipment systems in each converter station and intelligent equipment system is increasing by several times, which has already appeared to be out of control. The traditional manual inspection method is facing three major challenges: first, the high density of equipment leads to an increase in blind spots for detection, and operation and maintenance personnel need to climb dozens of meters high equipment for close-range inspection; second, the contradiction between all-weather inspection needs and shortage of human resources is highlighted, especially the lack of emergency detection capability in extreme weather conditions; third, it is difficult to realize the structured management and trend analysis of defect data through manual recording.

[0003] The current mainstream technical solutions can be divided into three categories: Fixed video monitoring system: a network of multi-angle cameras is deployed inside the converter station, and the monitoring pictures are stored through a video streaming server. This solution realizes all-weather monitoring, but has two major limitations: first, it lacks effective intelligent analysis modules, and still requires manual review of massive videos; second, the image stability of ordinary cameras in strong electromagnetic environments is poor, and the accuracy of key frame extraction is low.

[0004] Robot inspection technology: track-type or wheeled robots are used to carry detection instruments and perform inspection tasks according to pre-set paths. This type of system has achieved certain application results in substation scenarios, but the complex electromagnetic environment and equipment layout specific to converter stations limit its applicability. Practice shows that the positioning error of robots in closed spaces such as valve halls can reach centimeters, and the maintenance cost is high.

[0005] Unmanned aerial vehicle inspection system: multi-rotor unmanned aerial vehicles are used for high-altitude photography, and image stitching technology is used to generate panoramic pictures of equipment. This method can cover areas that are difficult for humans to reach, but is subject to three major factors: first, positioning drift caused by GPS signal shielding in converter stations; second, flight stability problems in AC / DC mixed field environments; third, battery endurance limits the duration of a single operation.

[0006] Analysis of key technical bottlenecks: The existing technical system faces four core problems in the context of converter stations: Real-time bottleneck: the processing architecture based on cloud centers results in an end-to-end delay of more than seconds, which cannot meet the millisecond-level response standard required by the Technical Specification for Real-time Dynamic Monitoring of Power Systems. When a sudden fault such as transformer explosion occurs, the response speed of the system directly affects the efficiency of accident disposal.

[0007] Environmental adaptability defects: the strong electromagnetic interference unique to converter stations can reduce the signal-to-noise ratio of image sensors, and conventional noise reduction algorithms can cause key features to be lost. Actual measurement data shows that the image distortion rate of ordinary industrial cameras can reach 15%-20% in the valve hall of a ±800kV converter station.

[0008] Algorithm efficiency problems: the existing deep learning model parameter quantity generally exceeds 50MB, and the inference speed on the edge device is difficult to meet the real-time processing requirements. Taking a typical 1080P video stream as an example, the single-frame processing time of a conventional target detection model exceeds 200ms, which cannot meet the real-time analysis requirement of 25fps.

[0009] Multi-modal data fusion is missing: existing systems mostly use a single spectral data source, and lack a cooperative analysis mechanism for visible light and infrared data. For example, the surface contamination defect of an insulator is not obvious under visible light, but can cause local temperature anomalies, and a single detection mode is easy to cause missed detection. SUMMARY

[0010] The purpose of the present application is to provide a converter station intelligent gateway image recognition system based on a YOLOv3 target detection algorithm, and a device defect detection method realized based on the system, which can realize real-time image analysis of converter station equipment, is suitable for automatic identification and detection of typical faults such as insulator damage, mechanical looseness defect, transformer oil leakage defect, and can significantly improve the efficiency of power grid operation and maintenance.

[0011] In order to achieve the above purpose, the present application adopts the following technical solutions: The present application provides a converter station intelligent gateway image recognition system, which adopts a three-level cooperative processing architecture design, establishes a seamless connection of a multi-spectral image acquisition layer, an edge computing gateway layer and a cloud-based operation and maintenance management platform layer, realizes the spatio-temporal alignment of device state data and video stream, fuses visible light and infrared detection results, and constructs a device health degree evaluation matrix. The multi-spectral image acquisition layer comprises: A heterogeneous sensing unit that integrates a visible light camera and an infrared thermal imager, used for synchronously acquiring dual-mode image data of the converter station equipment and supporting wide dynamic environment adaptability; An intelligent acquisition terminal equipped with a multi-dimensional stable structure and an environmental protection mechanism, used for ensuring reliable data acquisition under complex working conditions; A synchronous control module for realizing the spatio-temporal alignment of multi-source sensors and providing a basic guarantee for multi-modal fusion; The edge computing gateway layer comprises: A heterogeneous computing platform that fuses a general-purpose processor and a special-purpose acceleration unit, used for providing high-performance inference capability; A real-time analysis engine that constructs a multi-path video stream parallel processing pipeline to ensure low-latency response; A data scheduling module equipped with a dynamic cache management mechanism and a network interruption resuming function, which adjusts the data processing strategy adaptively according to the network state to ensure data transmission integrity; An intelligent analysis module integrated with an improved YOLOv3 target detection algorithm framework for real-time processing of image data and output of device state analysis results; A protocol adaptation interface for supporting standardized conversion of various communication protocols in industrial sites.

[0012] The cloud operation and management platform layer includes: A micro-service management platform with core functions such as decoupling device monitoring, defect analysis, and alarm pushing; A knowledge graph engine for establishing an association model of power equipment defect characteristics and disposal strategies; A visual interaction module for mapping physical device states through a three-dimensional digital twin interface and supporting real-time alarm confirmation and interactive operations through mobile applications.

[0013] Further, the improved YOLOv3 target detection algorithm framework integrated by the intelligent analysis module includes: A multi-spectral feature fusion unit for adding a cross-modal feature alignment layer at the end of the backbone network to realize pixel-level fusion of visible light and infrared features, improve detection robustness under complex lighting conditions, and use an attention-guided feature weighting strategy to dynamically adjust the contribution of dual-channel features; A power-specific anchor box optimization unit that re-clusters anchor box sizes based on the geometric characteristics of converter station equipment, designs asymmetric anchor box proportions for typical equipment components such as large components transformers, medium components insulator strings, and small components bolts, introduces a rotation-sensitive convolution layer and enhances the detection capability of inclined equipment; An adaptive computing allocation unit that dynamically switches model depth according to edge node load state, autonomously switches between lightweight and high-precision modes, develops a spatial sampling strategy based on device thermal maps, and prioritizes processing high-fault-probability areas; An incremental knowledge distillation unit for constructing a cloud-edge collaborative training framework, regularly distilling cloud large model knowledge to edge lightweight models, and using adversarial sample enhancement technology to improve small sample scene generalization ability; A multi-scale defect correlation analysis unit for establishing an association rule base of local defects and global device states, realizing progressive research and judgment from pixel-level anomalies to system-level risks, and integrating a defect evolution prediction model with time series information to provide early warning of potential faults; An interpretable output architecture unit is used to embed a feature visualization layer in a detection result, intuitively show a key area of a decision basis, and generate a standardized diagnostic report template in line with power industry specifications.

[0014] Further, the cloud operation and maintenance platform layer can issue image acquisition strategies to the multispectral image acquisition layer through the edge computing gateway layer in the form of cloud instructions, and when the intelligent gateway image recognition system of the converter station determines that an abnormal event exists, the abnormal event triggers local decision of the edge computing gateway layer and linkage response with the cloud operation and maintenance platform layer.

[0015] The application provides a device defect detection method, which is based on the intelligent gateway image recognition system of the converter station and comprises the following steps: S1: The multispectral image acquisition layer synchronously and in real time acquires double-mode image data of the converter station device through a visible light camera and an infrared thermal imager, and transmits the double-mode image data to the edge computing gateway layer by using an RTSP protocol; S2: A preprocessing operation for the double-mode image data is constructed to complete infrared thermal image and visible light image registration, adaptive histogram equalization and noise suppression; S3: An SE attention mechanism is embedded in an improved YOLOv3 network to extract features of the double-mode image data; S4: Different algorithms are combined to identify defects such as broken insulators, transformer leaked oil and mechanical looseness; S5: Multi-level risk response early warning is generated by comprehensively considering parameters such as leakage area and temperature gradient, wherein the risk response early warning comprises a first level response based on a light defect confidence, a second response based on a key device moderate defect confidence and a third response based on a multi-device concurrent serious defect confidence; S6: A defect diagnosis report is pushed to the cloud operation and maintenance platform layer through a WebSocket, and original data and analysis results are synchronously stored in a time series database.

[0016] Further, in the preprocessing operation stage, the double-mode image data is dynamically corrected for color cast by using an adaptive white balance algorithm, the shooting quality in foggy weather is improved by using an improved dark channel prior dehazing algorithm, multi-view image alignment is achieved by using a registration technology based on Harris corner detection, and the interference of specular reflection on a metal surface is suppressed by using a polarized light fusion algorithm.

[0017] Further, in the preprocessing operation stage, the double-mode image data is efficiently purified, and the specific process is as follows: Step 101: adaptive gamma correction is adopted for the visible light video stream to eliminate the effect of backlight, and non-uniformity correction is simultaneously performed on the infrared thermal image, and two-point calibration method is used to compensate for the response difference of the detector, wherein the gamma value of the adaptive gamma correction is dynamically adjusted, and the setting range is 0.8-1.2; Step 102: for dual-spectrum image registration, an improved SURF feature detection algorithm is used to extract key points, and after removing false matches by RANSAC, an affine transformation matrix is calculated to achieve a alignment accuracy of ±0.5 pixels; Step 103: for typical noise sources in the converter station, a cascade filtering process is designed, which is: first, use guided filtering to maintain edge details, then combine wavelet threshold denoising, and the signal-to-noise ratio is improved by more than 12dB. In order to enhance the defect features, the MSRCR Retinex algorithm is applied to the visible light channel based on the hybrid enhancement strategy, and the dynamic histogram projection is used for the infrared channel, which is divided into 8 temperature intervals for mapping. Finally, the output size is normalized to 416x416 tensor data, and the geometric distortion rate is ensured to be less than 0.3% through bilinear interpolation, providing standardized input for the improved YOLOv3 network.

[0018] Further, the improved YOLOv3 network is pre-optimized and a three-level processing architecture is constructed in the feature extraction stage. The optimization process is as follows: Step 201: replace the standard convolution layer with depth separable convolution to reduce the computational complexity; Step 202: delete redundant convolution kernels to compress the model volume; Step 203: perform transfer learning training based on the power equipment defect dataset to improve the model generalization ability; The three-level processing architecture includes: primary feature layer, intermediate feature layer and high-level feature layer. The primary feature layer uses a lightweight MobileNetV3 network to extract basic texture features. The intermediate feature layer uses deformable convolution to adapt to different angles of device shooting. The high-level feature layer introduces channel attention mechanism to strengthen the feature response of abnormal areas.

[0019] Further, for the recognition of insulator damage defects, morphological segmentation is performed to assist in judgment, and the specific recognition process of the insulator damage defect is as follows: Step 301: the system first locates the insulator region through the improved YOLOv3 network, and uses adaptive ROI cropping technology to extract high-resolution target images; Step 302: For the linear characteristics of the insulator string, a direction-sensitive morphological processing flow is designed, which is: first, a binary image is obtained by using dynamic threshold segmentation based on the Otsu algorithm, then the shooting tilt error is eliminated by an angle correction module, a multi-scale morphological operation combination is constructed in the core segmentation stage, a 3x3 circular structural element is used to perform opening operation to eliminate small noise points, and a 12-pixel line structural element is used for closing operation to connect the broken edges; Step 303: To enhance the defect features, a gradient-enhanced morphological reconstruction algorithm is designed, which is: first, the edge gradient field is extracted by the Sobel operator, and then pixel-level fusion is performed with the inflation reconstruction result; Step 304: In the defect quantitative analysis link, a dual-channel feature extraction is designed, which is: in the spatial domain, the area ratio and the aspect ratio of the crack region are calculated, and in the frequency domain, the wavelet transform is used to analyze the texture abnormal energy distribution; Step 305: Finally, a fusion report is output, which includes defect type, position coordinates, morphological feature quantitative index, and three-dimensional reconstruction suggestion for complex defects that need multi-view observation, wherein the defect type includes crack, damage, dirt, etc. Among them, the system integrates an online learning mechanism, when the difference between the morphological segmentation result and the YOLOv3 classification confidence exceeds the threshold value, the sample collection process is automatically triggered to store the controversial image.

[0020] Further, for the recognition of transformer oil leakage defects, motion blur detection is introduced, and the specific recognition process of the transformer oil leakage defect is as follows: Step 401: The system uses a multi-modal fusion detection strategy, first acquires the temperature field distribution of the transformer by an infrared thermal imager, and performs pixel-level registration between the infrared thermal image and the visible light image; Step 402: A dual-stream feature extraction architecture is embedded in the improved YOLOv3 network, the visible light channel uses a Darknet-53 backbone network to extract texture features, and the thermal infrared channel uses a lightweight MobileNetV3 to extract temperature anomaly features; Step 403: For the oil stain diffusion characteristics specific to oil leakage, a dynamic threshold segmentation algorithm is designed, which is: based on the heat conduction model, the expected temperature gradient is calculated, when the measured temperature distribution deviates from the theoretical value by more than 15%, the oil leakage detection process is triggered, in the spatial analysis stage, morphological reconstruction and region growing algorithm are combined, first, the corrosion operation is used to eliminate thermal noise interference, and then the improved flood filling algorithm is used to mark the oil leakage region boundary; Step404: The system integrates the migration learning mechanism, pre-trains the network using synthetic data, fine-tunes the domain adaptation using real seepage oil samples, introduces fuzzy logic reasoning in the decision-making link, comprehensively evaluates indicators such as seepage oil area proportion, temperature gradient change rate and oil stain diffusion speed, outputs five-level seepage oil risk early warning, superimposes display of abnormal areas of the thermal map, seepage oil diffusion direction arrows and recommended disposal measures, and pushes to the cloud operation and management platform layer in real time through the RTSP protocol; Among them, the system automatically performs baseline calibration every 8 hours to eliminate the influence of environmental temperature fluctuations on detection accuracy.

[0021] Further, for the recognition of mechanical loosening defects, it is required to combine temperature gradient analysis, and the specific process of the transformer seepage oil defect is as follows: Step501: The system realizes mechanical loosening detection by fusing time domain analysis and deep learning, adopts optical flow method to calculate device displacement vector between adjacent frames in the video stream processing link, and triggers high frame rate acquisition mode when detecting abnormal vibration mode; Step502: For motion blur features, a double-channel evaluation system is designed, which is: the frequency domain analysis uses improved Radon transform to detect the frequency spectrum attenuation characteristics in the motion direction, and the spatial domain applies LBP-TOP operator to capture the texture dynamic change rule; Step503: In the improved YOLOv3 network, a motion attention module is embedded, the fuzzy feature map and the original feature map are channel weighted and fused, a motion score output head is added at the end of the network, and the device loosening probability is regressed through the fully connected layer; Step504: In order to improve the detection robustness, a multi-scale fuzzy synthetic data set is constructed, a 6-degree-of-freedom motion model is used to simulate the blur effect of different loosening degrees, and an adversarial generative network is used for data enhancement; Step505: The system combines Kalman filter for motion state prediction, when detecting 0.5-5Hz continuous low-frequency vibration with directional blur features, generates a three-level loosening warning, and outputs a result report, the result report includes loosening component positioning frame, motion trajectory heat map, blur score and recommended repair level, and realizes cross-validation by synchronizing vibration monitoring data with the device management system through the OPC UA protocol.

[0022] The beneficial effects of the present application are: The present application can realize real-time image analysis of converter station equipment, support multiple power equipment anomaly detection, and is suitable for automatic identification and detection of typical fault defects such as insulator damage defects, mechanical loosening defects, transformer seepage oil defects, etc., and can automatically generate multi-level risk response early warning, achieving the purpose of multi-level alarm automatic triggering.

[0023] The application reduces the dependence on cloud computing through the localized processing of the edge computing gateway layer, improves the response speed, and reduces the network transmission pressure.

[0024] The application also has the protocol docking capability with the existing power automation system, facilitating centralized monitoring and management by operation and maintenance personnel.

[0025] Compared with the traditional manual inspection, the application can realize all-weather uninterrupted work, timely discover potential fault hidden dangers, greatly improve the operation and maintenance efficiency and safety of the converter station, and provide strong guarantee for the stable operation of the power equipment. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is the flowchart of the device defect detection method in the application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the application will be described below with reference to the drawings.

[0028] The application provides a converter station intelligent gateway image recognition system, which adopts a three-level cooperative processing architecture design, establishes seamless connection of a multispectral image acquisition layer, an edge computing gateway layer and a cloud operation and maintenance platform layer, realizes spatiotemporal alignment of device state data and video stream, fuses visible light and infrared detection results, and constructs a device health degree evaluation matrix. The multispectral image acquisition layer comprises: A heterogeneous sensing unit, which integrates a visible light camera and an infrared thermal imager, is used for synchronously acquiring double-mode image data of the converter station device and supports wide dynamic environment adaptability; An intelligent acquisition terminal, which is equipped with a multidimensional stable structure and an environmental protection mechanism, is used for ensuring reliable data acquisition under complex working conditions; A synchronous control module is used for realizing spatiotemporal alignment of the multi-source sensor and providing basic guarantee for multi-modal fusion; The edge computing gateway layer comprises: A heterogeneous computing platform, which fuses a general-purpose processor and a special-purpose acceleration unit, is used for providing high-performance inference capability; A real-time analysis engine, which constructs a multi-path video stream parallel processing pipeline, is used for ensuring low-delay response; A data scheduling module, which is equipped with a dynamic cache management mechanism and a network interruption continuation function, adaptively adjusts the data processing strategy according to the network state, and guarantees data transmission integrity; An intelligent analysis module, which is integrated with an improved YOLOv3 target detection algorithm framework, is used for real-time processing of image data and output of device state analysis results; A protocol adaptation interface is used to support the standardized conversion of various communication protocols in the industrial field.

[0029] The cloud operation and maintenance platform layer comprises: A micro-service management platform has core functions such as decoupling device monitoring, defect analysis, and alarm pushing. A knowledge graph engine is used to establish an association model of power equipment defect characteristics and treatment strategies. A visual interactive module is used to map the physical device state through a three-dimensional digital twin interface and support real-time alarm confirmation and interactive operation through a mobile application.

[0030] Further, the improved YOLOv3 target detection algorithm framework integrated by the intelligent analysis module comprises: A multi-spectral feature fusion unit is used to add a cross-modal feature alignment layer at the end of the backbone network, realize pixel-level fusion of visible light and infrared features, improve detection robustness under complex lighting conditions, and adopt an attention-guided feature weighting strategy to dynamically adjust the contribution of dual-channel features. A power-specific anchor box optimization unit re-clusters anchor box sizes based on the geometric characteristics of converter station equipment, designs asymmetric anchor box proportions for typical equipment components such as large components (transformers), medium components (insulator strings), and small components (bolts), introduces a rotation-sensitive convolution layer and enhances the detection capability of inclined equipment. An adaptive computing allocation unit dynamically switches model depth according to the load state of the edge node, autonomously switches between lightweight and high-precision modes, develops a spatial sampling strategy based on device heat maps, and prioritizes processing high-fault-probability areas. An incremental knowledge distillation unit is used to build a cloud-edge collaborative training framework, periodically distill cloud large model knowledge to edge lightweight model, and adopt adversarial sample enhancement technology to improve small sample scene generalization ability. A multi-scale defect correlation analysis unit is used to establish an association rule base of local defects and global equipment state, realize progressive research and judgment from pixel-level anomalies to system-level risks, and fuse a defect evolution prediction model with time series information to provide early warning of potential faults. An explainable output architecture unit is used to embed a feature visualization layer in the detection results, intuitively display the key areas of the decision basis, and generate a standardized diagnostic report template that meets the specifications of the power industry.

[0031] Further, the cloud operation and maintenance platform layer can issue image acquisition strategies to the multi-spectral image acquisition layer through the edge computing gateway layer in the form of cloud instructions. When the converter station intelligent gateway image recognition system determines that an abnormal event exists, the abnormal event triggers local decision-making of the edge computing gateway layer and responds in conjunction with the cloud operation and maintenance platform layer.

[0032] Need to explain, visible light camera configuration with global shutter CMOS sensor, can capture device surface details, infrared thermal imager with temperature calibration module, visible light camera and infrared thermal imager combination can capture visible light and thermal radiation information, all cameras are installed on three axis anti shake holder, through RTK positioning technology to ensure the consistency of shooting angle.

[0033] Need to explain, edge computing gateway layer with heterogeneous computing chip set, including GPU acceleration module and FPGA preprocessing unit; The system can configure double channel DDR4 memory to realize data buffering, adopt PCIe 3.0 high speed bus to connect each component, support PoE power supply and lithium battery double backup, establish independent power management.

[0034] Need to explain, in order to improve the processing efficiency, the system adopts memory mapping technology to realize zero copy data transmission, develops heterogeneous computing platform based on OpenCL, can dynamically allocate computing task to GPU's CUDA core and FPGA's DSP unit; In view of the characteristics of limited resources of edge device, realizes model parameter quantization compression technology, compresses 32 bit floating point model into 8 bit integer, under the premise of ensuring accuracy, makes the processing speed improve 3 times; The whole process adopts pipelining design, each processing stage is connected through ring buffer, cooperates with QoS scheduling algorithm to ensure the priority execution of key task, finally outputs standardized feature tensor, for use of upper analysis module.

[0035] It should be noted that the core of the intelligent analysis module is to build a multi-dimensional fusion deep learning inference framework. The improved YOLOv3 target detection algorithm framework is used as the basic detection model in the system. Through transfer learning, the pre-training weight is adapted to the characteristics of power equipment. A special anchor frame parameter set is customized for typical equipment in the converter station. The adaptive image enhancement function is integrated at the network input end. The improved Mosaic data enhancement strategy is adopted to fuse four equipment images under different lighting conditions for joint training. In the feature extraction stage, the Darknet-53 backbone network is used with the SPP module to realize the cross-scale detection of equipment components through the multi-scale feature pyramid. In view of the small target detection problem of power equipment, the fourth level of fine-grained detection is added on the basis of the original three detection layers of YOLOv3, and the minimum detection size is increased to 15*15 pixels. The loss function adopts the CIoU improvement scheme, and the device geometric feature constraint term is introduced, which significantly improves the recognition accuracy of overlapping devices. In the post-processing link, a special screening algorithm for power equipment based on non-maximum suppression is developed. By introducing the prior knowledge of device spatial distribution, the mis-detection problem of densely arranged devices is effectively solved. The system uses the TensorRT acceleration engine to realize model quantization inference during runtime, and achieves a real-time processing performance of 45FPS on the Jetson AGX Xavier edge computing platform. In order to ensure the continuous optimization capability, an online learning mechanism is deployed. By collecting mis-detection samples labeled by operation and maintenance personnel, the model incremental training is automatically triggered, and the knowledge distillation technology is used to maintain the lightweight characteristics. Finally, the structured detection results are output, including device type, position coordinates, confidence and abnormal label, which can also interact with the upper management system in real time through the MQTT protocol.

[0036] It should be noted that the integration scheme of the system adopts an edge-cloud collaborative architecture to realize full-stack deployment. The hardware level is composed of a three-level system of a multispectral image acquisition layer, an edge computing gateway layer and a cloud operation and management platform layer. The edge node selects an industrial Jetson AGX Xavier module, is networked with a camera through PoE power supply, and a built-in hardware encryption chip is used to ensure safe data transmission. The software architecture is designed based on micro-service management, decouples functions such as image acquisition, preprocessing operation, defect analysis and alarm pushing into independent containerized services, and realizes dynamic arrangement through Kubernetes. Network communication adopts a layered protocol stack: the device layer uses the Modbus-TCP protocol to interface with traditional power monitoring devices, the edge layer realizes real-time data uploading through the MQTT protocol, and the cloud layer uses the gRPC framework to support high-concurrency service calls. Data flow processing integrates Apache Kafka message queues to build a distributed stream processing pipeline, supporting stable throughput of 2000+ images per second. The security system of the system realizes threefold protection: the transmission layer uses the national SM2 / SM3 algorithm for encryption, the application layer deploys dynamic token authentication, and the device layer protects model parameters through a TEE trusted execution environment. Integration with existing SCADA systems is realized through an OPC UA standard interface, and a dedicated protocol conversion middleware is developed to map recognition results in real time to device state quantities. The cloud operation and management platform layer is based on Prometheus+Grafana to build visual monitoring dashboards, realizing stereoscopic monitoring from chip temperature to business indicators. To ensure the reliability of the system, a dual-machine hot standby switching mechanism is designed, which can automatically switch to a cloud backup model service when an edge node fails, with a switching delay controlled within 300 ms.

[0037] It should be noted that the cloud operation and management platform layer as the core decision center of the system realizes intelligent operation and management through a three-stage processing process. In the data access stage, the Kafka message queue is used to receive the structured data uploaded by the edge gateway, the structured data includes device coordinates, infrared temperature matrix, time stamp and other fields, the data is cleaned in real time by the Flink stream processing engine to eliminate abnormal values caused by electromagnetic interference, such as sudden ±500℃ invalid data, and the device historical baseline data stored in InfluxDB is associated, including temperature distribution model under normal working condition and device three-dimensional coordinate mapping table. In the intelligent analysis stage, the pre-trained risk assessment model is loaded into the decision engine, which is constructed based on the XGBoost algorithm, and the input features include 12-dimensional parameters such as oil stain diffusion area, temperature rise rate, and coordinate offset of key parts of the device. The device health score is calculated every 5 seconds, and the health score is 0-100 points. When the score is lower than 60 points, the multi-level response mechanism is triggered, and for the score in the interval of 40-60, the warning work order is automatically distributed to the inspection terminal, and for the score in the interval of 30-40, the sound and light alarm device in the station is linked, and lower than 30 points directly sends a short message to the mobile phone of the operation and maintenance personnel. In the visual interaction stage, a three-dimensional device twin body is constructed through WebGL technology, and the YOLOv3 detection frame, heat map and device CAD model are superimposed and displayed, supporting the operation and maintenance personnel to click any component through the Web interface to view the temperature change curve in the last 24 hours and the historical defect record. All data is cached through Redis to accelerate query, ensuring that the interface response delay is not more than 1.5 seconds when 2000 devices are accessed concurrently, and the key operation log is stored on the chain through the Hyperledger Fabric private chain block chain technology, meeting the data audit requirements of the power industry.

[0038] As Figure 1 shown, the application also provides a device defect detection method, which is realized based on the converter station intelligent gateway image recognition system, including the following steps: S1: The multispectral image acquisition layer synchronously and in real time collects the dual-mode image data of the converter station device through the visible light camera and the infrared thermal imager, and transmits the data to the edge computing gateway layer by using the RTSP protocol; S2: Construct a pre-processing operation for the dual-mode image data, complete the infrared thermal image and visible light image registration, adaptive histogram equalization and noise suppression; S3: Embed the SE attention mechanism in the improved YOLOv3 network to extract features from the dual-mode image data; S4: Identify defects such as broken insulator, transformer oil leakage, and mechanical looseness by combining different algorithms; S5: generate a multi-level risk response warning based on parameters such as leakage area and temperature gradient, wherein the risk response warning includes a first level response based on a light defect confidence, a second response based on a key equipment moderate defect confidence, and a third response based on a multi-device concurrent severe defect confidence; S6: push the defect diagnosis report to the cloud operation and maintenance management platform layer through WebSocket, and synchronize the raw data and analysis results to the time series database.

[0039] Further, in the preprocessing operation stage, the dual-mode image data adopts an adaptive white balance algorithm to dynamically correct color cast, an improved dark channel prior dehazing algorithm to improve the quality of foggy shooting, a registration technology based on Harris corner detection to realize multi-view image alignment, and a polarized light fusion algorithm to suppress the specular reflection interference of the metal surface.

[0040] Further, in the preprocessing operation stage, the dual-mode image data is efficiently feature-purified, and the specific process is as follows: Step 101: adaptive gamma correction is used for the visible light video stream to eliminate backlight effect, and non-uniformity correction is performed on the infrared thermal image at the same time, and two-point calibration method is used to compensate for the response difference of the detector, wherein the gamma value of the adaptive gamma correction is dynamically adjusted, and the range is set to 0.8-1.2; Step 102: for dual-spectrum image registration, an improved SURF feature detection algorithm is used to extract key points, and an affine transformation matrix is calculated after removing false matches by RANSAC, achieving a alignment accuracy of ±0.5 pixels; Step 103: for typical noise sources in the converter station, a cascade filtering process is designed, which is: first, use guided filtering to maintain edge details, then combine wavelet threshold denoising, and the signal-to-noise ratio is improved by more than 12dB. To enhance defect features, the MSRCR Retinex algorithm is applied to the visible light channel based on a hybrid enhancement strategy, and the infrared channel uses dynamic histogram projection to divide 8 temperature intervals for mapping. Finally, the output size is normalized to 416x416 tensor data, and the geometric distortion rate is ensured to be less than 0.3% through bilinear interpolation, providing standardized input for the improved YOLOv3 network.

[0041] Further, the improved YOLOv3 network is pre-optimized and a three-level processing architecture is constructed in the feature extraction stage. The optimization process is as follows: Step 201: replace the standard convolution layer with a depth separable convolution to reduce the computational complexity; Step 202: delete redundant convolution kernels to compress the model volume; Step203: Perform transfer learning training based on the power equipment defect dataset to improve the model generalization ability; Among them, the three-level processing architecture includes: primary feature layer, intermediate feature layer and high-level feature layer, the primary feature layer uses lightweight MobileNetV3 network to extract basic texture features, the intermediate feature layer uses deformable convolution to adapt to different angle equipment shooting, and the high-level feature layer introduces channel attention mechanism to strengthen the feature response of abnormal area.

[0042] Further, for the recognition of insulator damage defects, morphological segmentation is performed to assist in judgment, and the specific recognition process of the insulator damage defects is as follows: Step301: The system first locates the insulator region through the improved YOLOv3 network, and extracts high-resolution target images using adaptive ROI cropping technology; Step302: For the linear features of the insulator string, a direction-sensitive morphological processing flow is designed, which is: first, a binary image is obtained by using dynamic threshold segmentation based on Otsu algorithm, then the angle correction module is used to eliminate the shooting tilt error, the multi-scale morphological operation combination is constructed in the core segmentation stage, the 3*3 circular structure element is used to perform open operation to eliminate small noise points, and the 12-pixel line structure element is used to perform close operation to connect the broken edges; Step303: To enhance the defect features, a gradient-enhanced morphological reconstruction algorithm is designed, which is: first, the edge gradient field is extracted by Sobel operator, and then pixel-level fusion is performed with the dilatation reconstruction result; Step304: In the defect quantitative analysis link, a double-channel feature extraction is designed, which is: the area ratio and the length-width ratio of the crack area are calculated in the spatial domain, and the wavelet transform is used to analyze the texture abnormal energy distribution in the frequency domain; Step305: Finally, a fusion report is output, which includes defect type, position coordinates, morphological feature quantification index, and three-dimensional reconstruction suggestion for complex defects that need multi-view observation, wherein the defect type includes crack, damage, contamination, etc. Among them, the system integrates an online learning mechanism, when the difference between the morphological segmentation result and the YOLOv3 classification confidence exceeds the threshold value, the sample collection process is automatically triggered to store the controversial image.

[0043] Further, for the recognition of transformer oil leakage defects, motion blur detection is introduced, and the specific recognition process of the transformer oil leakage defects is as follows: Step401: The system uses a multi-modal fusion detection strategy, first acquires the temperature field distribution of the transformer through an infrared thermal imager, and performs pixel-level registration between the infrared thermal image and the visible light image; Step402: Embed the dual-stream feature extraction architecture in the improved YOLOv3 network, the visible light channel uses the Darknet-53 backbone network to extract texture features, and the thermal infrared channel uses the lightweight MobileNetV3 to extract temperature anomaly features; Step403: For the unique oil stain diffusion characteristics of leaking oil, a dynamic threshold segmentation algorithm is designed, the dynamic threshold segmentation algorithm is: based on the heat conduction model to calculate the expected temperature gradient, when the measured temperature distribution deviates from the theoretical value by more than 15%, trigger the leaking oil detection process, in the spatial analysis stage, combine morphological reconstruction and region growing algorithm, first eliminate thermal noise interference through erosion operation, and then use the improved flood filling algorithm to mark the leaking oil area boundary; Step404: The system integrates the transfer learning mechanism, after pre-training the network using synthetic data, domain adaptation fine-tuning is performed using real leaking oil samples, fuzzy logic reasoning is introduced in the decision-making link, and indicators such as leaking oil area ratio, temperature gradient change rate and oil stain diffusion speed are comprehensively evaluated, five-level leaking oil risk early warning is output, thermal map abnormal area, leaking oil diffusion direction arrow and recommended disposal measures are superimposed and displayed, and real-time push is performed to the cloud operation and management platform layer through the RTSP protocol; Among them, the system automatically performs baseline calibration every 8 hours to eliminate the influence of environmental temperature fluctuations on detection accuracy.

[0044] Further, for the recognition of mechanical looseness defects, it is required to combine temperature gradient analysis, and the specific process of the transformer leaking oil defect is as follows: Step501: The system realizes mechanical looseness detection by fusing time domain analysis and deep learning, in the video stream processing link, the optical flow method is used to calculate the device displacement vector between adjacent frames, and when an abnormal vibration pattern is detected, the high frame rate acquisition mode is triggered; Step502: For motion blur characteristics, a dual-channel evaluation system is designed, the dual-channel evaluation system is: the frequency domain analysis uses the improved Radon transform to detect the frequency spectrum attenuation characteristics in the motion direction, and the spatial domain applies the LBP-TOP operator to capture the texture dynamic change law; Step503: Embed the motion attention module in the improved YOLOv3 network, channel weighted fusion is performed on the blurred feature map and the original feature map, a motion score output head is added at the end of the network, and the device looseness probability is regressed through the fully connected layer; Step504: To improve the detection robustness, a multi-scale blur synthetic data set is constructed, a 6-degree-of-freedom motion model is used to simulate the blur effect of different looseness degrees, and an adversarial generative network is used for data enhancement; Step 505: The system combines Kalman filtering for motion state prediction. When detecting sustained low-frequency vibration of 0.5-5 Hz accompanied by directional ambiguity characteristics, a three-level loosening early warning is generated, and a result report is output, which includes a loosening component positioning frame, a motion trajectory heat map, an ambiguity score, and a recommended maintenance level. Cross-validation is achieved by synchronizing vibration monitoring data in real time with the device management system through OPC UA protocol.

Claims

1. A converter station intelligent gateway image recognition system, characterized in that: The aforementioned converter station intelligent gateway image recognition system adopts a three-level collaborative processing architecture design, establishing a seamless connection between the multispectral image acquisition layer, the edge computing gateway layer, and the cloud operation and maintenance management platform layer, realizing the spatiotemporal alignment of equipment status data and video streams, and fusing visible light and infrared detection results to construct an equipment health assessment matrix; The multispectral image acquisition layer includes: The heterogeneous sensing unit integrates a visible light camera and an infrared thermal imager to simultaneously acquire dual-modal image data of the converter station equipment and supports wide dynamic environment adaptability. The intelligent data acquisition terminal is equipped with a multi-dimensional stabilization structure and environmental protection mechanism to ensure reliable data acquisition under complex working conditions. The synchronization control module is used to achieve spatiotemporal alignment of multi-source sensors, providing a fundamental guarantee for multimodal fusion; The edge computing gateway layer includes: Heterogeneous computing platforms, which integrate general-purpose processors and dedicated acceleration units, are used to provide high-performance inference capabilities; A real-time analytics engine that builds a pipeline for parallel processing of multiple video streams to ensure low-latency response; The data scheduling module is equipped with a dynamic cache management mechanism and a network disconnection resume function. It adaptively adjusts the data processing strategy according to the network status to ensure the integrity of data transmission. The intelligent analysis module integrates an improved YOLOv3 object detection algorithm framework for real-time processing of image data and output of device status analysis results. Protocol adapter interface, which is used to support the standardized conversion of various communication protocols in industrial fields; The cloud-based operation and maintenance management platform layer includes: The microservice management platform has core functions such as decoupling device monitoring, defect analysis, and alarm push. A knowledge graph engine is used to build a correlation model between the characteristics of power equipment defects and handling strategies. The visualization and interaction module is used to map the status of physical devices through a 3D digital twin interface, and to support real-time alarm confirmation and interactive operations through a mobile application.

2. The image recognition system for a converter station intelligent gateway according to claim 1, characterized in that: The intelligent analysis module integrates an improved YOLOv3 object detection algorithm framework, including: The multispectral feature fusion unit is used to add a cross-modal feature alignment layer at the end of the backbone network to achieve pixel-level fusion of visible light and infrared features, improve detection robustness under complex lighting conditions, and adopt an attention-guided feature weighting strategy to dynamically adjust the contribution of dual-channel features. The power-specific anchor frame optimization unit re-clusters anchor frame sizes based on the geometric characteristics of converter station equipment. It designs asymmetric anchor frame ratios for typical equipment components such as large components like transformers, medium components like insulator strings, and small components like bolts, introduces a rotation-sensitive convolutional layer, and enhances the detection capability of tilted equipment. The adaptive computing allocation unit dynamically switches the model depth according to the load status of edge nodes, autonomously switching between lightweight and high-precision modes, and develops a spatial sampling strategy based on device heatmaps to prioritize high-failure-probability areas. The incremental knowledge distillation unit is used to build a cloud-edge collaborative training framework. It periodically distills the knowledge of large cloud models to lightweight edge models and uses adversarial example enhancement technology to improve the generalization ability in small sample scenarios. The multi-scale defect correlation analysis unit is used to establish a rule base for the correlation between local defects and global equipment status, realize progressive judgment from pixel-level anomalies to system-level risks, integrate time-series information to predict defect evolution, and provide early warning of potential failures. The interpretable output architecture unit is used to embed a feature visualization layer into the detection results, intuitively display the key areas of decision-making basis, and generate a standardized diagnostic report template that conforms to power industry standards.

3. The image recognition system for a converter station intelligent gateway according to claim 1, characterized in that: The cloud-based operation and maintenance management platform layer can issue image acquisition strategies to the multispectral image acquisition layer through the edge computing gateway layer via cloud commands. When the converter station intelligent gateway image recognition system determines that an abnormal event exists, the abnormal event triggers the local decision of the edge computing gateway layer and responds in conjunction with the cloud-based operation and maintenance management platform layer.

4. A method for detecting equipment defects, characterized in that: The method is implemented based on the converter station intelligent gateway image recognition system according to any one of claims 1 to 3, and includes the following steps: S1: The multispectral image acquisition layer acquires dual-modal image data of the converter station equipment synchronously and in real time through a visible light camera and an infrared thermal imager, and transmits it to the edge computing gateway layer using the RTSP protocol; S2: Construct preprocessing operations for dual-modal image data to complete the registration of infrared thermal images and visible light images, adaptive histogram equalization, and noise suppression; S3: Embed the SE attention mechanism within the improved YOLOv3 network to extract features from bimodal image data; S4: Combine different algorithms to identify defects such as insulator damage, transformer oil leakage, and mechanical loosening; S5: Generate multi-level risk response warnings by comprehensively considering parameters such as leakage area and temperature gradient. The risk response warnings include a first-level response based on the confidence level of minor defects, a second-level response based on the confidence level of moderate defects in key equipment, and a third-level response based on the confidence level of concurrent severe defects in multiple equipment. S6: Defect diagnosis reports are pushed to the cloud-based operation and maintenance management platform via WebSocket, and the raw data and analysis results are synchronously stored in the time-series database.

5. The equipment defect detection method according to claim 4, characterized in that: In the preprocessing stage, the dual-modal image data uses an adaptive white balance algorithm to dynamically correct color cast, an improved dark channel prior dehazing algorithm to improve the quality of foggy day shooting, and a registration technique based on Harris corner detection to achieve multi-view image alignment. A polarization fusion algorithm is used to suppress specular reflection interference from metal surfaces.

6. The equipment defect detection method according to claim 4 or 5, characterized in that: In the preprocessing stage, efficient feature extraction is performed on the bimodal image data. The specific process is as follows: Step 101: Adaptive gamma correction is applied to the visible light video stream to eliminate the effect of backlight. Simultaneously, non-uniformity correction is performed on the infrared thermal image. The two-point calibration method is used to compensate for the difference in detector response. The gamma value of the adaptive gamma correction is dynamically adjusted and the setting range is 0.8 to 1.

2. Step 102: For dual-spectral image registration, the improved SURF feature detection algorithm is used to extract key points. After RANSAC is used to remove mismatches, the affine transformation matrix is ​​calculated to achieve an alignment accuracy of ±0.5 pixels. Step 103: For typical noise sources in the converter station, a cascaded filtering process is designed. The cascaded filtering process is as follows: first, guided filtering is used to preserve edge details, and then wavelet thresholding is combined to denoise, improving the signal-to-noise ratio by more than 12dB. To enhance defect features, the MSRCRRetinex algorithm is applied to the visible light channel based on a hybrid enhancement strategy, while the infrared channel adopts dynamic histogram projection, dividing it into 8 temperature ranges for mapping. The final output size is normalized to 416×416 tensor data. Bilinear interpolation is used to ensure that the geometric distortion rate is <0.3%, providing standardized input for subsequent improvements to the YOLOv3 network.

7. The equipment defect detection method according to claim 4, characterized in that: The improved YOLOv3 network is pre-optimized and a three-level processing architecture is constructed in the feature extraction stage; The optimization process is as follows: Step 201: Replace standard convolutional layers with depthwise separable convolutions to reduce computational complexity; Step 202: Remove redundant convolutional kernels to compress the model size; Step 203: Perform transfer learning training based on the power equipment defect dataset to improve the model's generalization ability; The three-level processing architecture includes a primary feature layer, a secondary feature layer, and a senior feature layer. The primary feature layer uses a lightweight MobileNetV3 network to extract basic texture features. The secondary feature layer adapts to different camera angles through deformable convolution. The senior feature layer introduces a channel attention mechanism to enhance the feature response in abnormal regions.

8. The equipment defect detection method according to claim 4, characterized in that: For the identification of insulator damage defects, morphological segmentation is used to assist in the judgment. The specific identification process for insulator damage defects is as follows: Step 301: The system first locates the insulator region using the improved YOLOv3 network and then uses adaptive ROI cropping technology to extract high-resolution target images; Step 302: For the linear characteristics of the insulator string, a direction-sensitive morphological processing flow is designed. The direction-sensitive morphological processing flow is as follows: First, a binary image is obtained by dynamic threshold segmentation based on the Otsu algorithm. Then, the shooting tilt error is eliminated by the angle correction module. In the core segmentation stage, a multi-scale morphological operation combination is constructed. A 3×3 circular structural element is used to perform opening operation to eliminate small noise. A 12-pixel linear structural element is used to perform closing operation to connect the broken edges. Step 303: To enhance the defect features, a gradient-enhanced morphological reconstruction algorithm is designed. The enhanced morphological reconstruction algorithm is as follows: first, the edge gradient field is extracted by the Sobel operator, and then the pixel-level fusion is performed with the dilation reconstruction result. Step 304: Design a dual-channel feature extraction in the defect quantitative analysis stage, specifically: calculate the area ratio and aspect ratio of the crack region in the spatial domain, and use wavelet transform in the frequency domain to analyze the energy distribution of texture anomalies. Step 305: Finally, output the fusion report, which includes defect type, location coordinates, morphological feature-based quantitative indicators, and 3D reconstruction suggestions for complex defects that require multi-view observation. The defect types include cracks, damage, dirt, etc. The system integrates an online learning mechanism that automatically triggers a sample collection process to store disputed images when the difference between the morphological segmentation result and the YOLOv3 classification confidence exceeds a threshold.

9. The equipment defect detection method according to claim 4, characterized in that: For the identification of transformer oil leakage defects, motion fuzzy detection is introduced. The specific identification process for transformer oil leakage defects is as follows: Step 401: The system adopts a multimodal fusion detection strategy. First, the temperature field distribution of the transformer is obtained through an infrared thermal imager, and the infrared thermal image is registered with the visible light image at the pixel level. Step 402: Embed a dual-stream feature extraction architecture in the improved YOLOv3 network. The visible light channel uses the Darknet-53 backbone network to extract texture features, and the thermal infrared channel uses the lightweight MobileNetV3 to extract temperature anomaly features. Step 403: To address the unique oil stain diffusion characteristics of leaking oil, a dynamic threshold segmentation algorithm is designed. The dynamic threshold segmentation algorithm is as follows: the expected temperature gradient is calculated based on the heat conduction model. When the measured temperature distribution deviates from the theoretical value by more than 15%, the leaking oil detection process is triggered. In the spatial analysis stage, morphological reconstruction and region growth algorithms are combined. First, thermal noise interference is eliminated through corrosion operation, and then the boundary of the leaking oil area is marked using an improved flood filling algorithm. Step 404: The system integrates a transfer learning mechanism. After pre-training the network using synthetic data, it uses real oil leakage samples for domain adaptation fine-tuning. Fuzzy logic reasoning is introduced in the decision-making process. The system comprehensively evaluates indicators such as the proportion of oil leakage area, the rate of change of temperature gradient, and the speed of oil stain diffusion. It outputs a five-level oil leakage risk warning, overlays and displays abnormal areas on the heat map, arrows indicating the direction of oil leakage diffusion, and suggested disposal measures. The system is then pushed to the cloud operation and maintenance management platform layer in real time via the RTSP protocol. The system automatically performs baseline calibration every 8 hours to eliminate the impact of ambient temperature fluctuations on detection accuracy.

10. The equipment defect detection method according to claim 4, characterized in that: The identification of mechanical loosening defects requires the combination of temperature gradient analysis. The specific process for identifying transformer oil leakage defects is as follows: Step 501: The system achieves mechanical loosening detection by integrating temporal analysis and deep learning. In the video stream processing stage, optical flow is used to calculate the device displacement vector between adjacent frames. When an abnormal vibration mode is detected, a high frame rate acquisition mode is triggered. Step 502: To address motion blur features, a dual-channel evaluation system is designed. The dual-channel evaluation system is as follows: in the frequency domain, an improved Radon transform is used to detect the spectral attenuation characteristics in the motion direction, and in the spatial domain, the LBP-TOP operator is applied to capture the dynamic change patterns of the texture. Step 503: Embed a motion attention module into the improved YOLOv3 network, perform channel-weighted fusion of the blurred feature map and the original feature map, add a motion scoring output head at the end of the network, and regress the device loosening probability through a fully connected layer; Step 504: To improve detection robustness, a multi-scale fuzzy synthetic dataset is constructed. A 6-DOF motion model is used to simulate the fuzzy effect of different degrees of loosening, and an adversarial generative network is used for data augmentation. Step 505: The system combines Kalman filtering to predict motion state. When a continuous low-frequency vibration of 0.5 to 5 Hz is detected and accompanied by directional ambiguity, a three-level loosening warning is generated and a result report is output. The result report includes the positioning box of the loose part, the heat map of the motion trajectory, the ambiguity score, and the suggested maintenance level. Cross-validation is achieved by synchronizing vibration monitoring data with the equipment management system in real time through the OPC UA protocol.

Images