UHV converter station protection system panoramic monitoring image processing and storage method

Abstract

The method belongs to the technical field of panoramic monitoring of ultra-high voltage converter station, and aims to solve the problem that panoramic monitoring image data is directly uploaded to the cloud, occupying a large amount of cloud resources. By adopting multi-scale convolution blocks in the deep multi-scale residual network model to construct low-order and high-order features of images of various scales, the incomplete phenomenon of image detail extraction is avoided, and the residual learning mechanism is adopted to retain low-order rough features, thereby improving the reconstruction ability of the image. Topology optimization constructs the topology structure of the heterogeneous network, and the framework combining deep reinforcement learning and Monte Carlo tree search is used to construct the network according to the pre-defined topology rules. The search result of the Monte Carlo tree strengthens the learning of the deep convolutional neural network, so as to obtain more accurate prediction in the next iteration. After the data is processed in the edge side, it is transmitted to the cloud storage, saving the cloud storage space and transmission bandwidth.

Description

Technical Field

[0001] This invention belongs to the field of panoramic monitoring technology for ultra-high voltage converter stations, and relates to a method for panoramic monitoring image processing and storage of ultra-high voltage converter station protection systems. Background Technology

[0002] With the development of power grids, the scale of grid interconnection is constantly expanding, and electrical connections within the grid are becoming increasingly close. This has led to a growing prominence of the safety and stability issues facing large power grids, significantly increasing the technical difficulty and safety risks associated with operation and management. The safe and reliable operation of ultra-high-voltage (UHV) converter stations plays an undeniably crucial role in the safe and stable operation of the power grid. Therefore, manual inspections are necessary during the daily operation and maintenance of UHV systems to troubleshoot equipment faults and ensure system safety and stability. However, this manual inspection mode is labor-intensive, and its performance is easily affected by the experience and sense of responsibility of the personnel. To improve the efficiency of UHV converter station operation and maintenance management, panoramic monitoring systems are widely deployed within UHV converter stations to monitor the operating status of equipment at all stages.

[0003] The following status signal parameters, applicable to the core links of UHVDC protection, need to be monitored by the protection device of the UHV converter station: A. Monitoring of the status of the outlet pressure plate; B. Temperature measurement of the terminal blocks inside the cabinet; C. Monitoring of the front panel of the secondary equipment inside the cabinet; D. Operating temperature of the secondary equipment inside the cabinet; E. Operating voltage of the secondary equipment inside the cabinet; F. Fiber optic light intensity monitoring; G. Cable insulation detection; H. Outlet circuit detection; I. Auxiliary contact position; J. Cable status detection; K. Detection of environmental parameters, such as temperature and humidity; L. Corrosion status of the terminals.

[0004] However, the operating environment and long-term use of monitoring systems inevitably lead to vibrations and shaking, as well as interference such as dust accumulation and spider webs on the lenses, resulting in blurred video images and inaccurate panoramic monitoring data acquisition. Therefore, there is an urgent need for a super-resolution reconstruction method for panoramic monitoring images of UHV converter station protection systems, so that the reconstructed high-resolution images can meet the panoramic monitoring needs of inspection personnel.

[0005] Traditional image enhancement and reconstruction methods typically enhance image contrast to highlight target objects, including histogram equalization, logarithmic transform, sharpening, wavelet transform, and Retinex at different scales. These methods are computationally inefficient and highly portable, but their enhancement effects are limited as general-purpose algorithms, and the processed images often fail to meet the needs of panoramic surveillance in specific scenarios. Image enhancement and reconstruction is a classic research topic in computer vision, and Single Image Super Resolution (SISR) is a crucial component. SISR utilizes a set of low-quality, low-resolution images to generate a single high-quality, high-resolution image, acquiring a region of interest with higher spatial resolution. This allows for focused analysis of the target object, transforming the image from detection level to recognition level, or even further to fine-resolution level, thereby improving the recognition capability and accuracy of panoramic surveillance images for converter stations.

[0006] Currently, SISR algorithms can be broadly categorized into three types: interpolation-based, reconstruction-based, and deep learning-based. Interpolation algorithms offer low computational cost and high real-time performance, but lack external information features, leading to the loss of high-frequency features after image degradation, resulting in images with noticeable blurring and ringing effects. Compared to interpolation algorithms, reconstruction-based algorithms show more significant improvements, but as the reconstruction magnification increases, high-frequency features become blurred. Deep learning-based methods have become mainstream in recent years, utilizing the mapping relationship between observed low-resolution (LR) images and original high-resolution (HR) images, along with a large number of training samples, to learn more high-frequency details in HR images. However, reconstructed images still suffer from detail distortion and high computational complexity. Convolutional neural networks (CNNs) are widely used in visual analysis due to their powerful image feature learning capabilities. In recent years, SISR algorithms based on CNNs have been proposed and have achieved significant performance improvements. The paper "Image Super-Resolution Using Deep Convolutional Networks" (C. Dong, IEEE Transactions on Pattern Analysis and Machine Intelligence, published in 2016) proposed a CNN model called SRCNN, which replaces dictionary modeling with automatic adjustment of hidden layer parameters, learning the nonlinear mapping relationship from low-resolution input to high-resolution output, improving reconstruction accuracy and reducing computation time. However, SRCNN also has some shortcomings. For example, bicubic interpolation can cause blurred and jagged edges in the image, and with the number of model parameters remaining constant, a larger super-resolution factor indicates a larger input resolution, resulting in higher computational cost. The paper "Accelerating the Super-Resolution Convolutional Neural Network" (Chao D, European Conference on Computer Vision, published in 2016) proposed an improved algorithm, FSRCNN, to address the slow training of SRCNN. It uses deconvolution for upsampling and 1×1 convolutions for dimensionality reduction, reducing the computational cost of the model and accelerating training. The core of ResNet is to add a skip connection between the output of the convolutional layer and the input of the previous convolutional layer to solve the gradient vanishing problem. H(x) represents the underlying mapping fitted by several stacked convolutional layers, where the input of the first convolutional layer is x, and x is connected to the output of the last convolutional layer. The stacked layers only need to learn the mapping F(x) = H(x) - x. If F(x) is zero, the residual unit can fit the identity mapping.

[0007] Because the concurrent processing of various network communications and service data in the panoramic monitoring of UHVDC converter stations is diverse, an unreasonable heterogeneous network topology can lead to dynamic imbalances in data flow access, resulting in poor overall data transmission performance. In severe cases, this can cause data congestion and affect network reliability. Therefore, it is necessary to optimize the heterogeneous network topology to improve its structure and transmission performance.

[0008] In recent years, the topology optimization problem of heterogeneous networks has received widespread attention. The 2015 publication "Logical Topology Design of Multi-Interface Multi-Channel Wireless Mesh Networks" (Bao Xuecai et al., Small and Microcomputer Systems) proposed a logical topology design method under topology reliability constraints. This method uses topology reliability and network path hop count as constraints, with maximum capacity and minimum interference as optimization objectives. It integrates shortest path and minimum spanning tree algorithms into the disjoint path calculation process to obtain an optimized logical topology, but this only improves network robustness. The 2019 publication "Adaptive Protection and Self-Healing Control Method for Distribution Networks Based on Dynamic Topology Analysis" (Zhang Anlong, Power System Protection and Control) proposed a new adaptive distributed topology control algorithm. By adjusting the transmission capacity of nodes under different states, it ensures network connectivity during node failures. Regarding data transmission, tree topology can transmit collected data better than other network topologies and has strong anti-interference capabilities. The 2019 publication, "Improving the Capacity of a Mesh LoRa Network by Spreading-Factor-Based Network Clustering" (Zhu G, Liao CH, Sakdejayont T, et al. IEEE Access), proposes a tree-based algorithm with a set of heuristic rules for constructing tree topologies in multi-hop wireless networks. The advantage of tree topologies lies in their efficient data transmission and aggregation through non-leaf nodes. Throughput in heterogeneous networks is a primary criterion for evaluating the merits of existing network models. Currently, many network topologies have been constructed for good data transmission, such as cluster-based and tree-based topologies. The performance of these topologies in heterogeneous networks demonstrates that topology quality significantly impacts data transmission. Therefore, network topology construction should be based on specific network requirements, accommodating various network types and transmission performance as much as possible.

[0009] The aforementioned network topology optimization methods consider the quality and security of data transmission, but the optimization process is time-consuming when the topology changes, making it difficult to meet the performance requirements of power industry data transmission networks. Finding the optimal reliability topology in the heterogeneous network of an UHVDC transmission system is essentially a combinatorial problem. The 2018 publication "Research on Minimum Spanning Tree Algorithm for Ventilation Networks Based on Weight Matrix" (Tu Peng et al., Journal of Railway Science and Engineering) proposes a minimum spanning tree topology optimization method. This method uses heuristic rules to reduce the number of candidate searches, thus obtaining a suboptimal solution to some extent. However, it still falls short of meeting the real-time and reliability requirements for rapid reconfiguration of the communication network when a UHVDC converter station experiences a network failure. Furthermore, because the search space for all possible topology configurations is extremely large, the complexity of achieving the optimal network configuration through exhaustive search increases exponentially.

[0010] Both the indoor and outdoor terminal boxes are part of the UHVDC converter station protection devices. The most critical protection information is fed back from the indoor and outdoor terminal boxes, therefore, special attention needs to be paid to their protection status. The outdoor terminal boxes and their internal wiring terminals are prone to corrosion due to the humid, dusty, and enclosed environment, threatening the normal operation of the converter station and endangering the safety of the entire power system. Among the core UHVDC protection status signal parameters, the monitoring of the output pressure plate status and the monitoring of the front panel of the secondary equipment inside the cabinet are both located within the cabinet; therefore, image monitoring of the cabinet is necessary.

[0011] In existing technologies, the paper "Corrosion Identification of Fittings Based on Computer Vision" (Zhiren Tian, ​​2019 International Conference on Artificial Intelligence and Advanced Manufacturing (AIAM)) published in 2019 targets the color features of corrosion faults, using HSI space and RGB models for corrosion region identification, segmentation, and detection. The paper "Quantity beats quality for semantic segmentation of corrosion in images" (Will Nash, Research Gate), published in July 2018, segments and extracts corrosion scenes; and the paper "A Corrosion Identification Algorithm for Cable Tunnels Based on Transfer Learning Convolutional Neural Networks" (Zhou Ziqiang et al., China Electric Power), published in April 2019, introduces transfer learning to address the problem of small data samples, improving corrosion detection effectiveness to some extent. All of these target detection algorithms rely on large convolutional neural networks, and the algorithm models suffer from problems such as excessive parameter count and slow detection speed, failing to meet the real-time response requirements of corrosion detection in UHV converter station protection systems.

[0012] Therefore, it is necessary to monitor the corrosion status of the terminal boxes outside the substation in real time through the monitoring system. Once the camera is selected, the host computer continuously collects data on the corrosion of the wiring terminals under the same conditions. As a result, the monitoring data in the substation of the UHV converter station is repetitive, and uploading a large amount of repetitive data is not very meaningful, as it occupies a lot of cloud storage space and transmission bandwidth. Therefore, how to perform lightweight processing on the large amount of repetitive wiring terminal corrosion image data before uploading it to the cloud, so as to avoid a large amount of repetitive data occupying cloud resources, is an urgent problem to be solved by the panoramic monitoring system of the UHV converter station protection system.

[0013] Sometimes, the protection devices in ultra-high voltage converter stations should activate, but they fail to do so. Previously, without monitoring methods, on-site operators were unaware of this situation. Even after the installation of monitoring facilities, subjective factors such as the operator's experience and sense of responsibility could lead to missed detections. Therefore, video analysis technology is now used to assist operators in determining the activation of protection devices.

[0014] The following are the status signal parameters applicable to the core links of UHVDC protection that need to be monitored by the UHVDC converter station protection device: A. Outlet pressure plate status monitoring; B. Terminal block temperature measurement inside the cabinet; C. Front panel monitoring of secondary equipment inside the cabinet; D. Operating temperature of secondary equipment inside the cabinet; E. Operating voltage of secondary equipment inside the cabinet; F. Fiber optic light intensity monitoring; G. Cable insulation detection; H. Outlet circuit detection; I. Auxiliary contact position; J. Cable status detection; K. Detection of environmental parameters, such as temperature and humidity; L. Corrosion status of wiring terminals. Both the cabinet and the outdoor terminal box belong to the UHVDC converter station protection device. The most critical protection information is fed back to the indoor cabinet and the outdoor terminal box. Therefore, special attention needs to be paid to the protection status of the cabinet and the outdoor terminal box. Among the aforementioned UHVDC protection core link status signal parameters, the outlet pressure plate status monitoring and the front panel monitoring of secondary equipment inside the cabinet are both located inside the cabinet, so image monitoring of the cabinet is required.

[0015] Video surveillance is primarily used for image monitoring of power distribution cabinets, and it is widely applied in UHVDC converter stations to monitor the operating status of equipment at various stages. However, due to the limited bandwidth of image data, uploading all video to the cloud places high demands on cloud storage capacity. Once camera and lighting conditions are determined, the host computer continuously collects data under the same conditions, resulting in repetitive monitoring data within the UHVDC converter station's small rooms. Uploading large amounts of repetitive data is not very meaningful and consumes significant cloud storage space and transmission bandwidth. Research on edge-side distributed data processing and fault analysis technology for UHVDC protection systems based on lightweight artificial intelligence aims to move the processing of large amounts of data that would otherwise be processed in the cloud to the edge, thereby achieving lightweight processing. This is a pressing issue that UHVDC converter stations need to address.

[0016] The UHV converter station cabinet is equipped with a glass cover on the outside, and the entire cabinet is placed inside the glass cover. The cabinet is equipped with pressure plates, test data display windows, switches, handles, status indicator lights, etc. The status of the UHV converter station cabinet can be observed by taking pictures of the UHV converter station cabinet through a camera set in the small room of the UHV converter station. Traditional power equipment information processing of monitoring images is mainly done manually, which is not efficient overall, and the accuracy of identifying the equipment operating status varies from person to person. The literature "Zhao Zhenbing, Zhang Wei, Zhai Yongjie, et al. Concept, research status and prospect of power vision technology [J]. Electric Power Science and Engineering, 2020, 36(01):1-8" proposes the concept of power vision technology, which establishes a bridge between power system, computer vision and artificial intelligence. With the development of artificial intelligence and computer vision technology, a large number of research works on panoramic monitoring of UHV DC converter station equipment based on machine vision have emerged. It can efficiently and accurately obtain and identify the operating status characteristics of the equipment, thereby completing effective video inspection work, saving a lot of manpower and improving inspection efficiency.

[0017] However, during video inspections of converter station equipment, certain scenarios exhibit large-area glare in the captured images due to reflected light, affecting image processing efficiency. While some scenarios can be resolved by adjusting camera orientation, the monitoring of secondary equipment cabinets consistently presents the challenge of obscuring certain areas of the captured images due to glass surface reflections. This significantly impacts subsequent machine vision tasks such as target detection and semantic segmentation. In severe cases, the glare can even cover the entire area to be inspected, rendering the task impossible or interrupting the process. This remains an unresolved issue in the research of panoramic monitoring of UHVDC converter station equipment. Summary of the Invention

[0018] The purpose of this invention is to design a method for processing and storing panoramic monitoring images of UHV converter station protection systems, in order to solve the problem that currently, panoramic monitoring image data of UHV converter station protection systems are directly uploaded to the cloud, which consumes a large amount of cloud resources.

[0019] The present invention solves the above-mentioned technical problems through the following technical solutions:

[0020] A method for processing and storing panoramic monitoring images of an ultra-high voltage converter station protection system includes the following steps:

[0021] S1. Perform super-resolution reconstruction on the panoramic surveillance image. The reconstruction method is as follows:

[0022] S11. Establish a deep multi-scale residual network model on the edge side;

[0023] S12. Input sample dataset and train the deep multi-scale residual network model;

[0024] S13. Test the peak signal-to-noise ratio and structural similarity index of the trained deep multi-scale residual network model using a standard dataset.

[0025] S14. Input the panoramic monitoring image of the UHV converter station into the trained deep multi-scale residual network model to complete the super-resolution reconstruction.

[0026] S2. Optimize the ubiquitous heterogeneous network transmission topology for panoramic monitoring of UHV converter stations. The optimization method is as follows:

[0027] S21. Model the heterogeneous network of the UHV converter station as a tree structure, wherein the tree structure has a master station v0 and N-1 data transmission nodes {v1, v2, ..., v...} N-1 Each data transmission node has a unique path to the master station v0;

[0028] S22. Using the main station v0 as the root node of the tree structure, and recursively searching the Monte Carlo tree for each state with the root node as the initial state to obtain the training dataset.

[0029] S23. Input the training dataset obtained from the search into the deep convolutional neural network for training to obtain the value function and policy function, which are used to guide the Monte Carlo tree to recursively search for states with expected rewards and in turn update the training dataset collected by the deep convolutional neural network.

[0030] S24. After training is complete, starting from the initial state s0 = 0, select a sequentially from the strategies predicted by the deep convolutional neural network. t ~π(s) t The action at point ) and the update of state s t+1 =T(s) t ,a t This process continues until a complete tree is reached, thus obtaining the heterogeneous network topology.

[0031] S3. By optimizing the heterogeneous network transmission topology, the panoramic monitoring data is transmitted to the edge side. After being lightweighted at the edge side, it is then transmitted to the cloud for storage.

[0032] Furthermore, the deep multi-scale residual network model includes: an input convolutional layer, an output convolutional layer, and k multi-scale convolutional blocks; the input convolutional layer acts as an encoder to extract the original low-level features of the low-resolution image; the output convolutional layer is used to fuse multi-scale detail features to reconstruct a high-resolution image; skip connections are established between the input and output convolutional layers to establish an identity mapping from the low-resolution image to the high-resolution image for global residual learning; the k multi-scale convolutional blocks are stacked sequentially to obtain the network model depth; the original low-level features are connected to the k multi-scale convolutional blocks through k corresponding paths, and local residual learning enhances the network model's ability to learn complex features;

[0033] Furthermore, both the input and output convolutional layers use convolutional kernels with a stride of 1, and the input convolutional layer uses ReLU activation. The multi-scale convolutional block extracts multi-level detailed features from the input image using convolutional kernels of four scales: 3×3, 3×2, 2×3, and 2×2. Then, the feature maps of the four scales are concatenated pairwise along a specified dimension through a cross-connection mechanism and fed into a 3×3 convolutional layer for feature mapping, generating a new feature map of the same size as the input and feeding it into the next multi-scale convolutional block.

[0034] Furthermore, the local residual learning is defined as follows: H k =G k (H k-1)+F; where G k H is the feature map learned by the k-th multi-scale convolutional block. k H is the output of the k-th multi-scale convolutional block. k-1 is the output of the (k-1)th multi-scale convolutional block, and F is the original low-order feature extracted by the input convolutional layer;

[0035] The mapping of k multi-scale convolutional blocks learned from global and local residuals is represented as follows: Where F0() is the mapping that the input convolutional layer needs to learn, F -1 () represents the mapping that the output convolutional layer needs to learn, where I HR I LR G represents high-resolution and low-resolution images, respectively. k-1 G is the feature map learned by the (k-1)th multi-scale convolutional block. k R is the feature map learned from the first multi-scale convolutional block, and R() is the mapping operation.

[0036] Furthermore, the loss function of the aforementioned deep multi-scale residual network model is: Where represents the parameters of the deep multi-scale residual network, and the loss function is minimized using the Adam optimizer; X (i) For sample dataset The i-th sub-image in Y (i) For the corresponding label, N is a positive integer.

[0037] Furthermore, the panoramic monitoring images include images of secondary equipment, hard pressure plates, and terminal corrosion; the standard datasets include three basic datasets: Set5, Set14, and Urban100.

[0038] Furthermore, the formula for calculating the peak signal-to-noise ratio is as follows: Where MSE is the mean square error between the original image and the processed image, and MAX is the mean square error between the original image and the processed image. I The maximum value of the image color is represented by the following formula: SSIM(X,Y)=L(X,Y)*C(X,Y)*S(X,Y); where u X u Y σ X and σ Y Let σ represent the mean and standard deviation of images X and Y, respectively. XY This represents the covariance of images X and Y. C1, C2, and C3 are constants, typically taken as C1 = (K1 * L). 2 C2 = (K2 * L) 2, C3 = C2 / 2, K1 = 0.01, K2 = 0.03, and L is the range of pixel values.

[0039] Further, the method of modeling the UHV converter station heterogeneous network as a tree structure is as follows: In each round of data collection, node v i Forward the bit data to its parent node, where i ∈ {1, 2,..., N - 1}; is the data generated by v i itself, and the data set is from the child nodes of v i , and the a() function is an aggregation function; the transmission model is used to transmit traffic, and the node transmission traffic related to the topology in the transmission model consists of two parts: data processing and transmission time consumption; and are the time consumption for processing each bit of data and the time consumption for transmitting each bit of data at node v i respectively. The time consumption for transmitting each bit of data depends on the distance to the parent node, and its calculation formula is as follows: where, is the Euclidean distance between node v i and its parent node v j , and ρ is the power amplification constant considering the impact of shadow fading in the link budget.

[0040] Further, the method of Monte Carlo tree recursive search is as follows: Each node on the Monte Carlo tree represents a 5-tuple data (s, a, M(s, a), π(s), Q π (s, a)); At each search step t < N, select the action that maximizes the upper confidence bound. When the search reaches the termination state t = N, obtain the reward and propagate it back along the search path to the root state of all visited states and the actions taken. The Q π value on the path is updated accordingly by the average value on the node; where s is the state of the heterogeneous network; a is the action in this state; M(s, a) is the total number of times (s, a) is visited on the search tree; π(s) is the prior probability of the effective action predicted by the deep convolutional neural network; Q π (s, a) is the state-action value, representing the expected reward starting from state s and taking action a; The calculation formula for the action that maximizes the upper confidence bound is as follows: where, is the visit count of state s, and the action is not considered. c is a hyperparameter that controls the search level.

[0041] Furthermore, the deep convolutional neural network comprises a deep Vgg16 module, a fully connected layer with softmax for the policy, and a fully connected layer with ReLU activation for the value function; the deep Vgg16 module consists of two convolutional layers with 64 convolutional filters, two convolutional layers with 128 convolutional filters, three convolutional layers with 256 convolutional filters, and six convolutional layers with 512 convolutional filters, each convolutional filter having a 3×3 kernel and a max pooling layer.

[0042] Furthermore, the value function satisfies the Bellman equation, indicating that the value of the current state is the reward of that state plus the expected reward of the next state. The formula for the value function is: The formula for the strategy function is as follows:

[0043] The advantages of this invention are:

[0044] The panoramic monitoring image processing and storage method for ultra-high voltage converter station protection systems of this invention avoids incomplete image detail extraction by constructing low-order and high-order features of images at multiple scales using multi-scale convolutional blocks in a deep multi-scale residual network model. A residual learning mechanism is employed in the network model to preserve low-order coarse features, reducing training difficulty, promoting feature reuse, and thus improving image reconstruction capabilities. The reconstructed images exhibit better structural similarity and peak signal-to-noise ratio performance. Experiments on image super-resolution reconstruction and target recognition were conducted using a standard dataset and an ultra-high voltage converter station panoramic monitoring image dataset. The results show that the high-resolution images reconstructed by the method of this invention can meet the panoramic monitoring needs of inspection personnel. A topology control algorithm based on deep reinforcement learning sequentially constructs the topology of a heterogeneous network. A framework combining deep reinforcement learning and Monte Carlo tree search is used to build the network according to predefined topology rules. A deep convolutional neural network is trained to predict the transmission traffic of a partially constructed topology and guides the Monte Carlo tree to search more promising regions in the search space. The search results from the Monte Carlo tree enhance the learning of the deep convolutional neural network, enabling more accurate predictions in the next iteration. Data is lightweighted at the edge before being sent to cloud storage, saving cloud storage space and transmission bandwidth. Attached Figure Description

[0045] Figure 1 This is an architecture diagram of the deep multi-scale residual network model according to an embodiment of the present invention;

[0046] Figure 2 This is a structural diagram of a multi-scale convolutional block according to an embodiment of the present invention;

[0047] Figure 3 These are PSNR performance curves for different network model depths according to embodiments of the present invention.

[0048] Figure 4 , Figure 5 , Figure 6 These are comparison images of the reconstruction effects of the method of this invention and other algorithms on secondary equipment monitoring images, hard pressure plate images, and terminal corrosion images.

[0049] Figure 7 This is a heterogeneous network model according to an embodiment of the present invention;

[0050] Figure 8 This is a heterogeneous network tree structure according to an embodiment of the present invention;

[0051] Figure 9 This is a description of two steps in a finite-time Markov decision process according to an embodiment of the present invention;

[0052] Figure 10 This describes the Monte Carlo tree search process according to an embodiment of the present invention;

[0053] Figure 11 This is the structure of a deep convolutional neural network according to an embodiment of the present invention;

[0054] Figure 12 This describes the convergence and performance of the DRL-TC algorithm proposed in this embodiment of the invention.

[0055] Figure 13 This is an evolution of the training process in the embodiments of the present invention.

[0056] Figure 14 This is a flowchart of a method for lightweight detection of corrosion edge side of wiring terminals of UHV converter station protection device according to an embodiment of the present invention;

[0057] Figure 15 This is a network structure diagram of the lightweight corrosion detection model based on dual-attention MobileNet according to an embodiment of the present invention;

[0058] Figure 16 This is a structural comparison diagram of standard convolution and the depthwise separable convolution of the lightweight rust detection model based on dual attention MobileNet in this embodiment of the invention;

[0059] Figure 17 This is a flowchart of the cascaded attention model of the lightweight corrosion detection model based on dual attention MobileNet according to an embodiment of the present invention.

[0060] Figure 18 This is a diagram showing the detection results of a lightweight detection method for the corrosion edge side of the wiring terminals of the UHV converter station protection device according to an embodiment of the present invention.

[0061] Figure 19 This is a schematic diagram of the structure of a network for monitoring and de-reflection interference removal of the pressure plate of an ultra-high voltage converter station panel, provided in an embodiment of the present invention.

[0062] Figure 20 This is a schematic diagram of the residual block in a network for monitoring and de-reflection interference in a UHV converter station panel pressure plate, provided in an embodiment of the present invention.

[0063] Figure 21 The results of multi-level connection ablation experiments on the publicly available datasets (PSNR1, SSIM1) and the state image datasets (PSNR2, SSIM2) of the UHV converter station panel pressure plate monitoring dereflection interference network provided in the embodiments of the present invention.

[0064] Figure 22 This is the result of de-reflection visual processing of real natural landscape images in a network for monitoring and de-reflection interference of UHV converter station panel pressure plates, as provided in an embodiment of the present invention.

[0065] Figure 23 The image provided in this embodiment of the invention is a de-reflection visual processing result of the pressure plate image of the UHV converter station panel in a de-reflection interference monitoring network. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0068] Example 1

[0069] like Figure 1 As shown, the method for super-resolution reconstruction of panoramic monitoring images of the UHV converter station protection system includes the following steps:

[0070] I. Super-resolution reconstruction of panoramic surveillance images

[0071] 1.1 Establish a deep multi-scale residual network model on the edge side.

[0072] 1.1.1 Deep Multi-scale Residual Network (DMRN)

[0073] Figure 1 This network employs a deep multi-scale residual network architecture, consisting of convolutional layers, k multi-scale convolutional blocks (MC blocks), and skip connections. Stacking the k MC blocks achieves greater depth, while the convolutional operations are improved by employing small kernels of different scales to extract and fuse detailed features at different scales in the image. This enhances the network's ability to reconstruct the microscopic texture and macroscopic geometric features of the input panoramic surveillance image, resulting in more realistic HR images. Residual structures are incorporated during network training to enable feature reuse, reduce network redundancy, accelerate convergence, and address the vanishing gradient problem.

[0074] 1.1.2 Multi-scale convolutional blocks

[0075] DMRN uses a multi-scale convolutional block architecture to perform super-resolution tasks. Convolutional layers with different scales form a multi-scale convolutional block, which can generate and combine detailed features at different levels.

[0076] Figure 2 This is a structural diagram of a single multi-scale convolutional block, where x represents the input of the multi-scale convolutional block and y is the output of the convolutional block. Convolutional blocks of different scales can extract details at different frequencies. In each multi-scale convolutional block, the input image is processed using convolutional kernels of four scales: 3x3, 3x2, 2x3, and 2x2 to extract multi-level detail features. Then, the feature maps of the four scales are concatenated pairwise along a specified dimension using a cross-connection mechanism, and then fed into a 3x3 convolutional layer for feature mapping, generating a new feature map of the same size as the input, which is then fed into the next multi-scale convolutional block. Multi-scale convolutional blocks better preserve the edge information of the image and increase the detail information of the reconstructed high-resolution image.

[0077] 1.1.3 Residual Learning Mechanism

[0078] The DMRN network architecture introduces global residual learning and local residual learning mechanisms for network training. Due to the similarity between low-resolution and high-resolution images, DMRN establishes an identity mapping from low-resolution to high-resolution images through skip connections between input and output to perform global residual learning.

[0079] There are two reasons for using local residual learning: First, the details needed in high-resolution reconstruction are the sum of high-frequency features and low-order features. Figure 1The first convolutional layer in the algorithm acts as an encoder, extracting the original low-order features of the low-resolution image. Local residual learning can preserve these low-order features. Second, there are multiple paths between low-order features and multi-scale convolutional blocks. Through local residual learning, the network's ability to learn more complex features can be enhanced.

[0080] Local residual learning is defined as follows:

[0081] H k =G k (H k-1 )+F (1)

[0082] Among them, G k H is the feature map learned by the k-th multi-scale convolutional block. k is the output of the k-th multi-scale convolutional block, and F is the original low-order feature extracted by the first convolutional layer.

[0083] Let F0 be the mapping that the first convolutional layer (with ReLU) needs to learn. -1 For the mappings that the last convolutional layer (without ReLU) needs to learn, the mappings of the k multi-scale convolutional blocks learned based on the global and local residuals can be expressed as follows:

[0084] I HR =R(I LR ) = I LR +F -1 (G k (G k-1 (…(G 1 (F)+F)…)+F)+F) (2)

[0085] Where F = F0(I LR ) is a primitive, low-level feature.

[0086] 1.1.4 DMRN Network Details

[0087] Figure 1 The main structure of DMRN differs from ResNet. DMRN removes pooling layers and batch normalization layers. This is because SISR aims to achieve accurate pixel prediction, and removing pooling layers helps preserve more image details. Batch normalization layers, which normalize features, eliminate the network's range flexibility and are detrimental to image reconstruction; therefore, they are also removed. DMRN uses convolutional kernels with a stride of 1 and ReLU activation, thus accepting images of arbitrary size as input. Furthermore, DMRN uses two 5×5 convolutional layers in the first and last layers to extract coarse features and fuse multi-scale detail features to reconstruct the HR image.

[0088] 1.2. Input sample data and train the deep multi-scale residual network model.

[0089] Eighty hundred monitoring images, each with a resolution of 1600*1200, were collected from the panoramic monitoring system of the UHV converter station. First, the high-resolution images were reduced to one-third of their original resolution using a bicubic interpolation algorithm, and then their dimensions were adjusted to the original image size. From the adjusted images, 24,000 sub-images of size 32×32 were selected with a step size of 32 as the dataset. Where N = 24000, X (i) For the i-th sub-image, Y (i) The corresponding labels are used. 80% of the images are randomly selected as the training set, and the remaining 20% ​​as the test set. Mean squared error (MSE) is used as the loss function for the network.

[0090]

[0091] Here, θ represents the parameters of DMRN, and the Adam optimizer is used to minimize the loss function.

[0092] 1.3 The trained deep multi-scale residual network model was tested and analyzed using a standard dataset.

[0093] After the DMRN network was trained, it was first tested using three standard datasets: Set5, Set14, and Urban100. Since human vision is more sensitive to changes in brightness, the images were converted to the YCbCr space, and the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) on the Y channel were used to evaluate the performance of super-resolution reconstruction.

[0094] PSNR is defined as the ratio of the maximum power of a signal to the power of noise, measured in decibels (dB). It is commonly used to evaluate the quality of image compression; a higher value indicates a more realistic image. The formula for calculating PSNR is as follows:

[0095]

[0096] Where MSE is the mean square error between the original image and the processed image, and MAX is the mean square error between the original image and the processed image. I This represents the maximum value of the image color.

[0097] SSIM evaluates the similarity between the original image and the processed image, with values ​​ranging from [0, 1]. A higher value indicates less image distortion. The formula for calculating SSIM is as follows:

[0098]

[0099]

[0100]

[0101] SSIM(X,Y)=L(X,Y)*C(X,Y)*S(X,Y) (8)

[0102] Among them, u X u Y σ X and σ Y Let σ represent the mean and standard deviation of images X and Y, respectively. XY This represents the covariance of images X and Y. C1, C2, and C3 are constants, typically taken as C1 = (K1 * L). 2 C2 = (K2 * L) 2 C3 = C2 / 2, K1 = 0.01, K2 = 0.03, and L is the range of pixel values.

[0103] The number of multi-scale convolutional blocks determines the depth of DMRN. Here, models with different numbers of multi-scale convolutional blocks (k = {8, 10, 12, 14}) are selected, such as... Figure 3 As shown, the average PSNR and SSIM performance of 50 randomly selected images from the Set5, Set14, and Urban100 test datasets are presented. With the increase in the number of multi-scale convolutional blocks, the PSNR performance of DMRN on Set5, Set14, and Urban100 steadily improves, indicating that the method of this invention achieves the expected goal of "the deeper the better." However, excessively deep networks also lead to increased computational complexity. The performance improvement of k=14 compared to k=12 is limited; therefore, the parameter setting of k=12 was used in subsequent experiments.

[0104] The SSIM and PSNR values ​​for testing on the standard datasets Set5, Set14, and Urban100 are shown in Tables 1-2. The tables also compare these values ​​with other methods, including Bicubic interpolation, SRCNN, and FSRCNN.

[0105] Table 1. Structural similarity indices for the Set5, Set14, and Urban100 datasets.

[0106]

[0107] Table 2 Peak Signal-to-Noise Ratio of Set5, Set14, and Urban100 Datasets

[0108]

[0109] Here, DMRN with k=12 is selected as the comparison model. As shown in the table, the average SSIM values ​​of SRCNN, FSRCNN, and DMRN are 0.7784, 0.7827, and 0.8082, respectively, while the structural similarity of the algorithm of this invention increases by 0.0043 and 0.0298, respectively. The average PSNR values ​​of SRCNN, FSRCNN, and DMRN are 27.50 dB, 27.67 dB, and 28.33 dB, respectively, while the algorithm of this invention improves by 0.17 dB and 0.83 dB, respectively. This result indicates that the algorithm of this invention can establish a nonlinear mapping relationship from LR to HR by fusing low-order and high-order features and using a combination of global and local residuals.

[0110] 1.4 Input the panoramic monitoring image of the UHV converter station into the trained deep multi-scale residual network model to complete super-resolution reconstruction.

[0111] Figure 4 , Figure 5 , Figure 6 Super-resolution reconstruction images of the panoramic monitoring images of the UHV converter station are presented, including images of secondary equipment, hard pressure plates, and terminal corrosion. The method of this embodiment is compared with Bicubic, SRCNN, and FSRNN, and Tables 3 and 4 show the quantitative experimental results. The images before and after reconstruction were input into the YOLOv3 recognition model used in the UHV converter station, and the recognition results are shown in Table 5. The experimental results show that compared with other methods, DMRN has better SSIM and PSNR performance, recovers clearer edges and more details, such as the indicator lights and corresponding blurred text information in the first image, the switch status and text display of the hard pressure plate in the second image, and the terminal corrosion status in the third image, which can better help inspection personnel to conduct panoramic monitoring.

[0112] Table 3. Structural Similarity Index of Monitoring Images of UHV Converter Stations

[0113]

[0114] Table 4 Peak Signal-to-Noise Ratio of Monitoring Images from UHV Converter Stations

[0115]

[0116]

[0117] Table 5. Image Recognition Results of UHV Converter Station Monitoring

[0118]

[0119] This invention proposes a deep multi-scale residual network (DMRN) to achieve fast super-resolution reconstruction of panoramic monitoring images of UHV converter station protection systems, meeting the panoramic monitoring needs of inspection personnel. In DMRN, multi-scale convolutional blocks are used to construct low-order and high-order features of the image at multiple scales, solving the problem of incomplete image detail extraction. The network employs residual learning to preserve low-order coarse features, reducing training difficulty, promoting feature reuse, and thus improving image reconstruction capabilities. Experimental results show that compared with other methods, DMRN has better SSIM and PSNR performance, recovering clearer edges and more details from standard datasets and UHV panoramic monitoring image sets, improving the quality of high-resolution image reconstruction, and meeting the panoramic monitoring needs of inspection personnel for UHV converter station protection systems.

[0120] II. Optimization of the ubiquitous heterogeneous network transmission topology for panoramic monitoring of UHV converter stations

[0121] 2.1 Establishing a heterogeneous network model for the UHV converter station

[0122] To ensure the stable operation of ultra-high voltage converter stations, comprehensive monitoring of numerous devices within the station is necessary. However, different devices use different networks for data transmission, resulting in a heterogeneous network. Figure 1 As shown. To address the dynamic imbalance in data flow access caused by unreasonable heterogeneous network topology connections, it is necessary to optimize the heterogeneous network topology to meet network communication performance requirements.

[0123] Table 6. Symbols used in the network model

[0124]

[0125] This embodiment takes a ±1100 kV converter station as an example, targeting... Figure 7 The heterogeneous network model shown depicts the heterogeneous network as a tree structure, such as... Figure 8 As shown, this structure has one master station v0 and N-1 data transmission nodes {v1, v2, ..., v...} N-1}, where each node has a unique path to the main station v0. V = {v0, v1, ..., v N-1} is the set of all vertices, and E is the set of directed edges.

[0126] In each round of data collection, node v i Need to The bit data is forwarded to its parent node. Calculated using formula (9):

[0127]

[0128] in, It is made by v i Self-generated data, data set It comes from v i The child nodes, the a() function is an aggregate function, i∈{1,2,...,N-1}.

[0129] This embodiment uses the transmission model shown in formula (10) to transmit traffic, where the topology-related node transmission traffic mainly consists of two parts: data processing (including data reception) and transmission time. The model is shown in formula (10):

[0130]

[0131] in, and They are at node v i The transmission time per bit for data processing and transmission is calculated. The transmission time per bit depends on the distance to the parent node and is further modeled as shown in Equation (11):

[0132] in, It is node v i Its parent node (or main site) v i The Euclidean distance between them, ρ is the power amplification constant in the link budget that takes into account the effects of shadow fading.

[0133] To apply reinforcement learning in this embodiment, an effective tree structure rooted at the master station v0 is first constructed. In each step, a node that is not yet connected is selected and connected to a node in the tree or the master station, until all nodes are connected. Figure 9 As shown, this process can be described by a finite-domain Markov decision process (MDP) with fully observable 4-tuples {S, A, T, R}. At each step t ∈ [0, N], the state s of the system is... t ∈S is the current adjacency matrix of the network. In a t The action at point A is to choose which node to connect to in the tree, or equivalently... Where node v i To connect to node v in the tree j (or the main station). Then, the system evolves to the next state s. t+1 In this case, there is a deterministic transition matrix T(s,a). Upon reaching the terminal state s... N Before all nodes are connected to the tree, the reward at step t is uncertain. Then the lifetime of the heterogeneous network is determined. Return as a reward for each action along the state trajectory.

[0134] The energy efficiency topology optimization framework proposed in this invention follows the following general settings:

[0135] (1) Node v i generated The data size is a random number extracted from a specific distribution in the DRL-TC algorithm;

[0136] (2) The aggregation function a() can be any deterministic function. In this invention, the summation method is used.

[0137] (3) The designed topology control algorithm should be applicable to other network objectives, such as minimizing the overall network time consumption or maximizing the network throughput.

[0138] Will be able to node v i The total traffic transmitted at that location is expressed as Since it is assumed that the main site v0 has no restrictions, therefore This invention defines the lifetime of a heterogeneous network as the minimum transmission traffic of all nodes based on the total number of transmission rounds. This maximization of the lifetime of a heterogeneous network can be expressed as:

[0139]

[0140]

[0141] Where δ(S) is the set of edges. If v i It is v j A subset of v, then v i =1, otherwise 0. Constraint (12b) ensures all nodes are connected, and constraint (12c) ensures each node can only transmit to one parent node at a time. To approximate the complexity of the problem, if the topology is considered as an undirected spanning tree, then according to Cayley's formula, the number of all possible spanning trees in the network is N. N-2 While heuristic rules can reduce the number of search candidates, enumerating all possible solutions remains infeasible for reasonable N values. This invention proposes a real-time DRL-TC algorithm that focuses on more promising regions in the search space with limited computational resources and approaches the optimal solution with increased computational power.

[0142] 2.2 Topology Optimization Algorithm Based on Deep Reinforcement Learning

[0143] 2.2.1 Reinforcement Learning

[0144] Reinforcement learning teaches the agent to take actions in a dynamic environment to maximize reward signals. In step t, the agent performs actions in the environment and receives an immediate reward r. tReceive observations of the environmental state. The action to be taken is determined by a strategy. The strategy can be dependent on s t It can be a set of deterministic actions, or it can be a random strategy using a set of action probabilities.

[0145] Reward r t The agent is told how much the current environmental state necessitates the objective; this is given by a reward function, which may depend on s. t a t and s t+1 When the goal is achieved, it produces a high value; otherwise, it produces a low value. A series of states and actions is called the trajectory motion τ, and the discounted sum of all reward values ​​collected along a trajectory is called the reward, as shown in formula (13):

[0146]

[0147] Here, γ is a discount factor that reduces the value of future rewards. When γ < 1, the reward obtained now is more valuable than the reward obtained later. The return can be a finite-level return collected over the maximum number of time steps, and γ = 1 can be used if needed. Alternatively, the reward can be an unlimited, infinite-level return, in which case γ < 1 is required.

[0148] The value function satisfies the Bellman equation, which states that the value of the current state is the reward of that state plus the expected reward of the next state. The value function and policy function are shown in equations (14) and (15):

[0149]

[0150] The main problem in reinforcement learning is finding a strategy that maximizes this expected reward, and its algorithms typically use approximation functions.

[0151] 2.2.2 Monte Carlo Tree Search

[0152] This embodiment uses a deep convolutional neural network (DCNN) to approximate the policy and value functions. The DCNN requires a training dataset of states, policies, and values ​​to fit the DCNN as a function approximator. One approach is to enumerate and collect all states and their values ​​as the training dataset. However, this approach becomes infeasible as the state space becomes large, as it overfits the DCNN. Instead of using heuristics to reduce the number of search candidates, this embodiment uses MCTS to efficiently collect the training dataset in more promising regions of the search space. Each node in the search tree represents a 5-tuple of data (s, a, M(s, a), π(s), Q). π(s, a)), where s is the state of the heterogeneous network, a is the action in that state, M(s, a) is the total number of times (s, a) is visited on the search tree, π(s) is the prior probability of the valid action predicted by the deep convolutional neural network, and Q π (s, a) is the state-action value, which is defined as the expected reward starting from state s and taking action a, and is calculated using Equation (14). At each search step t < N, the action that maximizes the upper confidence bound (UCB) is selected, as shown in Equation (16):

[0153] Intuitively, this selection strategy initially favors actions with a higher prior probability π, but asymptotically favors actions with a higher state-action value Q π of. As Figure 10 shown, when the search reaches the termination state (i.e., t = N), the reward is obtained and propagated back along the search path to the root state of all visited states and the actions taken, and the Q π values on the path are correspondingly updated by the average value at the nodes. The details of MCTS are described in Algorithm 1, as shown in Table 7.

[0154] Table 7 MCTS Subroutine of the Proposed DRL-TC Algorithm

[0155]

[0156] Each search starts from a specific state and recursively searches for the next state until a new leaf state or a terminal state is reached. By performing multiple MCTS at each state, the posterior visit count M(s) is collected as part of the training dataset used to update the deep convolutional neural network in the next iteration.

[0157] 2.2.3 Deep Convolutional Neural Network

[0158] The stochastic policy π(s) defines the distribution of valid actions in a state, under which the system starts from state s t until the terminal state s N generates the state and action trajectory h(s t [[ID=३३]]) = {s t , a[[ID=३६]] t ,..., s N-1 , a[[ID=४०]] N-1 , s N}. The value function V π (s) is defined as the expected reward of all possible trajectories starting from state s and is calculated using Equation (7).

[0159] In this embodiment, a deep convolutional neural network is used to approximate the policy function and the value function f Θ (s) (parameterized by Θ) approximates the optimal value function V* (s)=max π V π (s) and the optimal strategy π * (s). For example... Figure 11 As shown, the input to the deep convolutional neural network is the training dataset {(s,π(s),V}. π To maintain the feasibility of training multi-layer neural networks while significantly improving the representational power of deep convolutional neural networks, this embodiment employs a deep Vgg16 module. This module consists of two convolutional layers with 64 convolutional filters, two convolutional layers with 128 convolutional filters, three convolutional layers with 256 convolutional filters, and six convolutional layers with 512 convolutional filters. Each convolutional filter has a 3×3 kernel, followed by a max-pooling layer. Then, the deep convolutional neural network is divided into two branches of convolutional layers, followed by fully connected layers with softmax and ReLU activations for the policy and value functions, respectively. The deep convolutional neural network (π(s), V π (s))=f Θ (s) The policy and value of each predicted state contain prior information that guides MCTS to search for states with high rewards and, in turn, collects the training dataset for the deep convolutional neural network.

[0160] Once the deep convolutional neural network (π(s)) is trained, V π (s))=f Θ (s) In order to obtain the tree-like topology of the heterogeneous network, this embodiment starts from the root state s0 = 0, and then sequentially selects a from the strategies predicted by the deep convolutional neural network. t ~π(s) t The action at point ) and the update of state s t+1 =T(s) t ,a t This process continues until a complete tree is reached. This embodiment notes that this topology construction is a stochastic process, and once the deep convolutional neural network has been trained for a sufficient number of iterations, it will converge to a solution.

[0161] 2.2.4 Self-configured DRL-TC algorithm

[0162] The self-configuration and self-optimization characteristics are known as SON (Self-Organizing Network), which better adapts to the flattening and flexibility of network structures, and therefore have attracted widespread attention. In short, the DRL-TC proposed in this embodiment alternates between training a deep convolutional neural network (DCNN) and MCTS (Multi-Channel Search Theory), where the DCNN provides a prior policy to guide the MCTS, and then the MCTS returns posterior visit counts and state values ​​to update the DCNN. In this way, with limited computational resources, the proposed DRL-TC algorithm will focus more on promising search regions and converge to a solution with higher rewards.

[0163] The proposed DRL-TC algorithm can also adapt to dynamic changes in the environment. For example, when nodes are suddenly added or removed, topology rules may make some actions effective or ineffective. In a new run of MCTS, the policy π returned by the deep convolutional neural network for this state will be renormalized for all effective actions. Therefore, the new prior policy π(s) reflects the changes in the network but is still related to historical data. MCTS will collect a new training dataset and use it to update the deep convolutional neural network. Assuming that network changes are slower than training time (depending on available computational resources), the proposed DRL-TC algorithm can track the dynamic changes in the network and reconfigure the network topology accordingly. Algorithm 2 describes the complete algorithm of the proposed DRL-TC, as shown in Table 8.

[0164] Table 8 proposes the DRL-TC algorithm.

[0165]

[0166] 2.3 Simulation Results and Analysis

[0167] 2.3.1 Simulation Settings

[0168] To evaluate the performance of the DRL-TC algorithm, simulation tests were conducted on a heterogeneous network of a ±1100 kV converter station. This heterogeneous network consists of a master node and 12 nodes distributed within a circular area with a radius of 1000 m, uniformly generating 500 to 1000 bits of sensing data in each transmission round. This embodiment assumes that all nodes have sufficient time to transmit data in each round. The data transmission throughput of each unit of all nodes is set to... The power amplification constant is set to ρ = 1.

[0169] In each iteration of the algorithm, from each state N m =Collect N in MCTS with 100 searches e =10 training samples. Batch size B=16, learning rate α=10 -6This embodiment uses the ADAM optimizer to train a deep convolutional neural network. After each training iteration, 100 network topologies are constructed using the deep convolutional neural network, and the results are averaged to evaluate the performance of the algorithm.

[0170] 2.3.2 Results Analysis

[0171] First, this embodiment demonstrates the convergence and performance of the proposed DRL-TC algorithm. Figure 12 The solid lines in the table represent the network latency of the deep convolutional neural network after each training iteration, with the algorithm converging after approximately 50 iterations. Table 9 compares the performance of the proposed DRL-TC algorithm with three heuristics: star topology (all nodes are connected to the master station), random topology (each node randomly selects a node to connect to), and minimum spanning tree (MST) topology, where MST is weighted by the Euclidean distance between nodes. The star topology exhibits the longest network latency due to higher transmission traffic at edge nodes far from the master station. The random topology shows a shorter average network latency, but the differences are significant. The MST topology further reduces network latency by shortening the overall transmission distance. The proposed DRL-TC algorithm in this embodiment significantly outperforms these heuristics, and the algorithm has a short convergence time.

[0172] Table 9 Performance Comparison of DRL-TC Algorithm and Three Heuristic Methods

[0173]

[0174] Figure 13 The proposed DRL-TC demonstrates its ability to adapt to sudden changes in heterogeneous networks, showing the average network latency after each training iteration. Figure 13 Points A to D on the curve show the 100 topological superpositions given by the DRL-TC algorithm after the 1st, 62nd, 63rd, and 100th iterations. Point A indicates that in the first iteration, DRL-TC randomly explores the search space. Because the deep convolutional neural network has no prior information about the state values, the network latency is relatively high. Point B indicates that the network gradually converges after multiple iterations. Point C indicates that it can quickly adapt when the heterogeneous network structure changes. Point D indicates that the algorithm converges to the optimal solution after 100 iterations.

[0175] This invention proposes a novel and unified heterogeneous network topology optimization algorithm based on deep reinforcement learning. The proposed DRL-TC algorithm can adapt to environmental changes and exhibits significantly better data transmission performance than other heuristic algorithms, demonstrating excellent adaptability to network topology changes and enhancing network reliability. The DRL-MCTS framework has great potential in heterogeneous networks, enabling online training without intervening in network services. Furthermore, with the continuous improvement of computing power, this invention anticipates that in the 5G era, DRL-MCTS will see other promising topology control applications in self-organizing and fully automated IoT networks.

[0176] Third, panoramic monitoring data is transmitted to the edge by optimizing the heterogeneous network transmission topology, and then subjected to lightweight processing at the edge.

[0177] 3.1, such as Figure 14 As shown, the method for lightly inspecting the rusted edge of a terminal block includes the following steps:

[0178] 3.1.1 Collect corrosion sample data of the wiring terminals and preprocess the collected corrosion sample data.

[0179] (1) The camera captures images of corroded terminal samples from the UHV converter station and ensures that the terminal samples with corrosion defects cover as many types of power equipment as possible;

[0180] (2) The collected rust sample data is standardized to obtain the terminal rust sample set X. The sample set X is divided into a training dataset and a test dataset in a ratio of 7:3. The training dataset and the test dataset are independent of each other.

[0181] (3) Normalize the standardized sample data. The normalization formula is as follows:

[0182]

[0183] Where a and b are two constants, and a = 0.1 and b = 0.8 are chosen as the maximum and minimum values ​​of each group of factor variables, respectively; x i , x' i These are the values ​​before and after normalization, respectively; x max x min These are the maximum and minimum values ​​in the sample data, respectively.

[0184] 3.1.2 Constructing a lightweight corrosion detection model based on dual-attention MobileNet

[0185] like Figure 15As shown, this invention uses Dual-Attention MobileNet as the base network and selects SSD (Single Shot MultiBox Detector) as the basic network framework. It replaces the SSD feature extraction network VGG16 with an improved Dual-Attention MobileNet as the feature extraction network. This can improve the running speed while ensuring accuracy, and at the same time greatly reduce the amount of computation and parameters. Compared with VGG16, the accuracy is reduced by 0.9%, but the model parameters are only 1 / 32 of VGG.

[0186] The Single Shot MultiBox Detector (SSD) object detection algorithm is a one-stage deep learning object detection algorithm proposed by Liu W et al. in 2016 (SSD: Single shot multibox detector. European Conference on Computer Vision. Amsterdam, The Netherlands. 2016. 21-37.). It incorporates multi-scale detection to improve object detection capabilities at different scales. The SSD object recognition algorithm uses VGG-16 as the feature extraction network, removing the two fully connected layers at the end and replacing them with three convolutional layers to further extract features, while simultaneously reducing the size of the feature maps. To improve its generalization ability for targets with large scale variations, SSD uses feature maps at six different scales for detection. In terms of prior box generation, SSD borrows the anchor strategy from Faster R-CNN, generating 4 to 6 anchor boxes of different sizes and aspect ratios on feature maps at different scales as prior boxes for bounding box regression. This effectively adapts to objects with different aspect ratios and significantly improves detection performance.

[0187] The lightweight corrosion detection model based on dual-attention MobileNet employs feature maps of six scales (38*38, 19*19, 10*10, 5*5, 3*3, and 1*1) for bounding box prediction and object classification. The shallower feature maps, being larger, are used for detecting small objects, while the deeper feature maps, being smaller, are used for detecting salient objects. The 38*38, 19*19, 10*10, and 5*5 feature maps each use six different sizes and aspect ratios of pre-selected boxes, while the 3*3 and 1*1 feature maps each use four different sizes and aspect ratios of pre-selected boxes, resulting in a total of 11,620 pre-selected boxes. The model performs object classification and bounding box regression on the extracted feature maps of the six scales. The classification network outputs the probability value of each class, and the regression network obtains the coordinate values ​​of each predicted box. Non-maximum suppression is used when correcting the candidate box positions.

[0188] like Figure 16 As shown, MobileNet employs depthwise separable convolution. This depthwise separable convolution first uses a 1×1 kernel to convolve each channel, then uses a 3×3 kernel to exchange information between channels. By decomposing the multiplication in standard convolution into addition, it effectively reduces a large number of parameters without sacrificing accuracy. Simultaneously, it replaces the ReLU activation function with the more performant h-swish function. The basic unit of MobileNet is depthwise separable convolution, a structure previously used in the Inception model. Depthwise separable convolution is actually a factorized convolution operation, which can be broken down into two smaller operations: depthwise convolution and pointwise convolution.

[0189] Let the size of the MobileNet input feature map be D. F The size of the convolution kernel is D K If the number of channels in the input feature matrix is ​​M and the number of channels in the output matrix is ​​N, then the size of the standard convolution is D. F ×D K If the computational complexity is ×M, then the ratio of depthwise separable convolution to standard convolution is:

[0190]

[0191] The computational cost of standard convolution is D. K ×D K ×M×N×D F ×D F The computational cost of depthwise convolution is D.K ×D K ×M×D F ×D F The computational complexity of point convolution is 1 × 1 × M × N × D. F ×D F Since N is generally a large value, the ratio in the above formula mainly depends on D. K Since the present invention uses a 3*3 kernel size, the computational cost of depth-separable convolution is only one-ninth that of standard convolution.

[0192] like Figure 17 As shown, the Dual-Att MobileNet of this invention employs a cascaded dual attention model that constructs an attention feature for each position within each channel through simultaneous spatial and channel domain calibration. Then, a cascaded spatial attention and channel attention module is used to enhance the detection performance. The cascaded dual attention model consists of a spatial attention module and a channel attention module. The spatial attention module first flattens the original feature map of size C×H×W into C×N in channel units, then transposes it to N×C. Matrix multiplication of these two feature matrices yields an N×N feature calibration matrix. Each position in this matrix represents the relationship between each pixel in the original feature and other pixels. Normalizing this feature calibration matrix using a two-dimensional softmax function yields a weight mask matrix (FFM). A matrix (FMM) is used where the value at each position in the FMM represents the proportion of information contained in each pixel of the original feature map. Multiplying the weight mask matrix with the expanded original feature map (C×N) allows for the relabeling of the features in the original feature map. Finally, the original feature information is added back to the same residual structure. Its main expression is as follows:

[0193]

[0194] In the above formula, E c For the calibrated feature map, D i A is the feature map before transformation. j The original feature map added to the residual structure. S ij Here is the weight value for the (i,j)th position, which is obtained using the softmax function:

[0195]

[0196] In the above formula, B i For the unfolded N×C feature map, C j This is the expanded C×N feature map.

[0197] Subsequently, a channel attention Squeeze operation is performed. This Squeeze operation reduces the feature map to a 1×1 feature vector after a global average pooling operation. Essentially, it integrates all the information from the feature map onto a single pixel, thus serving as a primary basis for determining feature importance. The formula for the Squeeze operation is as follows:

[0198]

[0199] Where E is the feature map, E(i,j) is the number of pixels in the feature map, and H and W are the sizes of the feature map:

[0200] The excitation operation then proceeds by establishing a fully connected layer densely connected to the preceding feature vectors. This creates a small, learnable, and trainable network for judging the importance of feature vectors and providing a backpropagation path. A sigmoid function then normalizes the information content of all channels to between 0 and 1, explicitly representing the information content of each channel and forming a mask vector. The formula for the excitation operation is as follows:

[0201] S = F ex (z,w)=σ(w2δ(w1z))

[0202]

[0203] Where W is an adjustable parameter and δ is the activation function;

[0204] Finally, a reweighting operation is performed. This operation uses the mask of each channel as a weight and multiplies it by each pixel of the feature map. This weights the proportion of information from each channel onto each feature map, completing the channel-level feature recalibration. The resulting feature map is the one that has undergone spatial and channel dual-attention calibration. The formula for the reweighting operation is as follows:

[0205] x c =F re (E c ,S c ) = S c ·E c

[0206] 3.1.3 Input the training dataset and its labels into the lightweight corrosion detection model based on dual-attention MobileNet for training. After training, input the test dataset to obtain the detection results.

[0207] (1) Input the preprocessed sample data into the base model, and train the dataset in batches according to the batch size. Use stochastic gradient descent (SDG) to backpropagate and update the parameters and save the weights. Stop training after the number of training iterations reaches the set number, and obtain the trained model. The training parameters in this embodiment are set as follows: training batch size is 20, and the number of iterations is 1000.

[0208] (2) Input the test dataset into the trained model to detect the images or photo streams to be detected;

[0209] (3) Determine whether the marked detection boxes in the detection results intersect. After merging all intersecting detection boxes, obtain the minimum bounding matrix of all intersecting detection boxes. Merge the minimum bounding matrix and use it as the final detection result.

[0210] like Figure 18 The image shows the detection results of a lightweight detection method for the corrosion edge of the wiring terminals of the UHV converter station protection device. To further verify the advantages of the proposed algorithm in terms of model size, detection speed, and detection accuracy, this paper compares the standard SSD model with VGG-16 and ResNet-50 as backbone networks with the lightweight SSD model based on the attention upsampling strategy proposed in this paper. The discrimination criteria in this paper mainly consist of precision, recall, weight size, and detection time. The calculation of precision and recall is shown below:

[0211]

[0212]

[0213] In the formula, TP represents the number of positive samples that were correctly identified, FP represents the number of positive samples that were incorrectly identified, and FN represents the number of negative samples that were incorrectly identified.

[0214] Table 10 shows a comparison of the detection performance of different algorithm models on the power equipment corrosion dataset in this paper.

[0215] Table 10 Comparison of detection performance of different network models

[0216] Algorithm Model Recall (%) Precision (%) Weight (MB) Detection time (s) SSD (VGGbase) 78.04 86.49 90.58 1.84 SSD (ResNetbase) 75.61 93.94 97.02 1.24 SSD (MobileNetbase) 63.41 83.87 15.34 0.50 Method of the present invention 78.05 95.89 42.36 1.08

[0217] As shown in Table 1, if only the lightweight MobileNet structure is used to lightweight the SSD model, its detection performance will deteriorate due to parameter loss. However, the method of this invention, by adding upsampling and feature fusion modules, can effectively improve the detection performance, even surpassing the original standard SSD algorithm. In summary, compared with the SSD model that only uses the lightweight MobileNet, the method of this invention expands the network structure on the upsampling network, increasing the number of parameters by 63.7%. However, compared with the standard SSD model with a large number of parameters and VGG-16 as the backbone network, it achieves a 9.4% improvement in accuracy while reducing the number of parameters by 53.23% and increasing the speed by 41.3%. Compared with the standard SSD model with ResNet-50 as the backbone network, it can also achieve a 1.95% improvement in accuracy while reducing the number of parameters by 56.34%.

[0218] 3.2 Monitoring and Reflection Interference Removal of Panel Pressure Plates

[0219] The reflected light source for monitoring the cabinets in ultra-high voltage direct current converter stations is mainly fluorescent lamp light, characterized by being indoors, not natural light, with dispersed light spots and high light intensity. For example... Figure 19 When inputting images, the scattered light spots will likely cover the display screen, pressure plate, and other areas to be monitored. In addition, the installation location of small indoor surveillance cameras is limited by environmental factors such as waterproofing measures and cable routing. They can only be installed on the top of the cabinet for monitoring from a top-down viewing angle. This results in excessively high light intensity, which makes it impossible to correctly read the device status information, thus affecting the image information processing effect and causing great interference to subsequent machine vision tasks such as target detection and semantic segmentation.

[0220] Existing deep neural networks for de-reflection are mostly designed for real-world natural scenarios, typically outdoors under natural light. In these environments, the intensity of reflected light is low and the light spot distribution is uniform, which differs significantly from the lighting characteristics of UHVDC converter station cabinets. Therefore, when constructing a deep neural network for de-reflection of UHVDC converter station cabinets, it is essential to fully consider the operating environment and lighting characteristics of the equipment within the station to efficiently and accurately remove reflection interference and obtain characteristic information about the equipment's operating status.

[0221] To address the aforementioned need for dereflection of monitored images, this invention proposes the following... Figure 19The diagram illustrates a reflection interference removal network for monitoring the pressure plates of ultra-high voltage converter station cabinets, hereinafter referred to as the MA-Net network. The MA-Net network consists of an encoder and a decoder; the first three stages constitute the encoder part, and the remaining four stages constitute the decoder part. Levels are defined based on the size of the feature maps, and a block is defined as a stage. Through connections between multiple levels, MA-Net can connect all the encoder outputs to all the decoder inputs, allowing features of different sizes to be used simultaneously during image reconstruction. (Continue reading...) Figure 19 The MA-Net network comprises three sequentially cascaded encoders and four sequentially cascaded decoders. Each encoder includes two sequentially connected residual blocks and a WRNL block. The output of the current encoder's WRNL block is downsampled and input into the residual block of the next encoder stage. Each decoder includes sequentially connected convolutional layers, two residual blocks, and a WRNL block. The convolutional layers are connected to an SE block. The current decoder input is fused with the previous stage's output and the outputs of each encoder, and then input into the current decoder's SE block. The SE block adjusts the number of channels through a convolutional layer and then inputs the result into the current decoder's residual block. The image with reflection interference is input into the first-stage encoder, and the final decoder outputs an image with reflection interference removed. The following sections describe the multi-stage connection mechanism, WRNL blocks, network training process, and the loss function involved in training. Finally, simulations verify the effectiveness of the invention.

[0222] 3.2.1 Multi-level connection mechanism

[0223] In a network structure resembling U-Net, the feature connections between the same level can alleviate the limitation of low-level features in the decoder in utilizing multi-scale information. However, image dereflection is a low-level visual task that requires more scale-rich features to recover details in the image. This invention constructs a multi-level connection structure, where each level consists of two densely connected residual (DCR) blocks (e.g., ...). Figure 20It consists of WRNL blocks as shown. The residual block includes the first to third residual structures numbered sequentially. The first to third residual structures are cascaded in sequence. Each residual structure includes a 3×3 convolutional layer and a PReLU layer. The input end of the second residual structure receives the input data of the first residual structure and its output result. The input end of the third residual structure receives the input data of the second residual structure and its output result, as well as the input data of the first residual structure. The output data of the third residual structure fuses the input data of the first residual structure as the output of the entire residual block. In the upsampling part of the network, feature information from all scales in the downsampling can be aggregated through multi-level connections. Since features at different levels have different scales, in order to adaptively adjust the channel characteristics after multi-level connections, a squeeze-and-excitation (SE) block is added at each decoder stage, and the number of channels after the excitation block is adjusted through a 1×1 convolutional layer. Among them, the SE block uses the squeeze-and-excitation block of the existing technology. For specific references, please refer to the relevant description of the squeeze-and-excitation block in the literature "Detailed Explanation of SE Module" published on the CSDN blog.

[0224] Let be the output feature of level l (l = 1, 2, 3) in the encoder, then the input feature of each level l (l = 4, 5, 6, 7) in the decoder is:

[0225] Among them,

[0226] Among them, represents the concatenation operation, H up (·) represents the upsampling operation, represents the decoder output feature of level l, W 1×1 represents the 1×1 convolutional layer, f SE (·) represents the SE block, represents the sampling operation from level i to l, that is, the l - i times of downsampling and i - l times of upsampling operations when l > i, l = i, and l < i.

[0227] Multi-level connections can use high-level features when processing low-level features, which helps the network to utilize multiple scale representations when recovering large-scale objects, and vice versa. The present invention uses discrete wavelet transform for upsampling and downsampling operations to find the mapping relationship between feature shapes at different scales. In addition, considering the information loss problem, the present invention selects two-dimensional Haar wavelet for sampling operations.

[0228] 3.2.2, WRNL block (Wide Region Non-Local block)

[0229] The lighting characteristics in the small room of the converter station make the conventional blocks inapplicable. Therefore, the WRNL block is first defined, and then the effectiveness of the WRNL block is analyzed using statistical knowledge.

[0230] The input feature X of WRNL is divided into a feature map of a×b {X k}, (k = 1,..., ab), where k is the number of feature maps, and the output feature is generated through the formula where,

[0231]

[0232]

[0233]

[0234]

[0235] represents the output feature of the k-th feature map at position i, represents the i-th row feature point in the k-th feature map, represents the j-th column feature point in the k-th feature map, () T represents the transpose matrix of the matrix, W θ 、 and W g are all weight matrices with dimensions C×L, C×L, and C×C respectively, and L = C / 2; represents and the correlation between each i in a set of regional positions S ; γ(·) represents the relationship function and γ(·) = 1 / ((·)+1). If a > b, the grid is wider than when a = b. Therefore, when a > b, a = b, and a < b, they are called wide-region rectangular blocks, square blocks, and high-region rectangular blocks respectively.

[0236] Assuming that nonlocal patches recover specific pixels based on information from other pixels within a patch, each patch needs sufficient background information. However, due to the uneven distribution of the reflective layer, regional nonlocal patches struggle to fully utilize background information; wide rectangular patches possess richer background information than square and tall rectangular patches. The image was divided into 16*4, 8*8, and 4*16 grids to obtain wide rectangular, square, and tall rectangular patches. If the difference between a pixel in the input reflective image and the corresponding dereflective image exceeds a certain threshold, the pixel is considered to belong to the reflective layer. The results of ablation experiments for different region types are shown in Table 11. Compared to square and tall rectangular patches, wide rectangular patches exhibit better peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), indicating a more uniform distribution of the reflective layer across all patches, leading to better image recovery.

[0237] Table 11 Ablation experiments of region types in nonlocal blocks

[0238]

[0239] 3.2.3 Network Training Process

[0240] A dataset is constructed; the anti-reflection interference network is trained using the dataset. When the convergence condition is met, the trained anti-reflection interference network is obtained. Real-time acquired images with reflection interference are input into the trained anti-reflection interference network, and the images with reflection interference removed are output. The dataset includes a public dataset and a dataset of images of the pressure plate status of the cabinet in the protection room of a certain UHV converter station. The dataset is randomly divided into training and test sets in a 7:3 ratio.

[0241] The convergence condition is that the loss function reaches its minimum value, and the loss function includes:

[0242] L1 = ||x gt -f(x input )||1+||x gt -f(x input )||2

[0243] Where, x input Represents the input reflection image, x gt Let f represent the corresponding dereflected image, and f represent the output of the dereflection interference network. F This indicates that the F-norm is calculated and F takes values ​​of 1 or 2.

[0244] Furthermore, to better separate the reflective and transmissive layers, a repulsion loss based on the gradient domain is defined. Analysis of the relationship between the edges of the two layers reveals that the transmissive and reflective layers essentially do not overlap at the edges. The edges in image I should be caused by either the transmissive or reflective layer, rather than both. Therefore, the correlation between the predicted transmissive and reflective layers in the gradient domain is minimized, and the repulsion loss is expressed as the product of the normalized gradient fields of the two layers at multiple spatial resolutions. The loss function is constructed as follows:

[0245]

[0246] in,

[0247] θ represents the network weights. D Let I represent the dataset, and n represent the input image, where n is the image downsampling factor. This indicates that the transport layer pass-through factor is 2. n Downsampling of bilinear interpolation of -1 This indicates that the reflective layer has a passivity factor of 2. n Downsampling with bilinear interpolation of -1, where T represents the transport layer of image I, R represents the reflection layer of image I, and λ T and λ R All are normalization factors. ⊙ indicates multiplication in unit order. This represents the gradient map of the transport layer of image I. This represents the gradient map of the reflection layer in image I. express The model, express If the modulus of N is 3, then the total loss function is L = L1 + L excl (θ).

[0248] 3.2.4 Simulation Verification

[0249] (1) Selection of dataset

[0250] To verify the feasibility and effectiveness of the method of the present invention, the SIR dataset published in the literature "Renjie Wan, Boxin Shi, Lingyu Duan, et al. Benchmarking single-image reflection removal algorithms. IEEE International Conference on Computer Vision, 3922-3930, 2017." was used. 2The method of this invention was verified on a dataset of images showing the status of the pressure plate of the cabinet in the protection compartment of a certain UHV converter station. This invention used a total of 4260 images from the Pressure-plate (1400), Object (1500), Postcard (560), and Zhang et al. (800) datasets for training, and the remaining 1820 images from the Pressure-plate (600), Object (640), Postcard (240), and Zhang et al. (340) datasets for quantitative evaluation. The four datasets were randomly divided into training and test sets in a 7:3 ratio.

[0251] The Pressure-plate dataset is an image dataset taken with a Canon EOS 750D camera in an indoor environment, specifically for images of a cabinet under pressure. It contains 220 real image pairs, namely, images with reflections and corresponding reference transmission layers. To simulate different imaging conditions, the following factors were considered when capturing the images: 1) Environment: Indoor; 2) Lighting conditions: Incandescent lamp; 3) Glass thickness: 3mm and 8mm; 4) Distance between glass and camera: 3-15cm; 5) Camera angle: Frontal and oblique views; 6) Camera exposure value: 8.0-16.0; 7) Camera aperture (affecting reflection blur): f / 4.0-f / 16.

[0252] (2) Experimental Results and Analysis

[0253] Multi-level connections can aggregate feature information from all scales in the downsampling part of the network during the upsampling part. However, if the number of levels is too deep, the weight of key information will be reduced, and if the number of levels is too small, the feature information extraction effect will be insignificant. Therefore, it is very important to choose an appropriate number of multi-level connections. Figure 21 Experimental results of multi-level connectivity ablation on publicly available datasets (PSNR1, SSIM1) and cabinet pressure plate state image datasets (PSNR2, SSIM2) are presented. Figure 21 It can be seen that as the number of multi-level connections increases, the PSNR and SSIM indices gradually increase, reaching their maximum values ​​at level 4. However, as the number of levels continues to increase, PSNR and SSIM gradually decrease, indicating that the ability of the constructed deep neural network to aggregate information at various scales gradually decreases. Therefore, the number of subsequent multi-level connections is chosen to be 4 layers.

[0254] Under the condition of selecting 4 connection levels, the proposed method MA-Net was compared with other methods, including CEILNet proposed in the literature "Qingnan Fan, Jiaolong Yang, Gang Hua, et al. A generic deep architecture for single image reflection removal and image smoothing. IEEE International Conference on Computer Vision, 3238-3247, 2017.", BDN proposed in the literature "Jie Yang, Dong Gong, Lingqiao Liu, et al. Seeing deeply and bidirectionally: A deep learning approach for single image reflection removal. European Conference on Computer Vision, 654-669, 2018.", and ERRNet proposed in the literature "Kaixuan Wei, Jiaolong Yang, Ying Fu, et al. Single image reflection removal exploiting misaligned training data and network enhancements. IEEE Conference on Computer Vision and Pattern Recognition, 8178–8187, 2019.". To ensure a fair comparison, the same public dataset and the same dataset of images showing the state of the cabinet pressure plate were used for training. The parameters of each model were fine-tuned, and the best result of the fine-tuned version (indicated by the suffix '-F') is presented.

[0255] like Figure 22 and Figure 23As shown, this invention demonstrates the de-reflection visual processing results of real natural landscape images and converter station panel pressure plate images, where the input images are (column 1), CEILNET (column 2), BDN (column 3), ERRNet (column 4), and the method of this invention (column 5). It can be observed that compared to other methods, the method of this invention achieves more accurate visual results, removing most unwanted reflections. It also shows significant advantages in processing indoor, non-natural light, and light spots with high illumination intensity, while other methods generally suffer from insignificant reflection removal and increased noise. Table 12 summarizes the experimental results of different methods on four real datasets: Pressure-plate, Object, Postcard, and Zhang et al. The number of test images in each dataset is shown after the name. PSNR and SSIM metrics are used; higher PSNR and SSIM values ​​indicate better performance.

[0256] Table 12 Quantitative comparison of different methods on four real datasets

[0257]

[0258] As shown in Table 2, apart from the Zhang et al. dataset published in the paper "Xuaner Zhang, Ren Ng, Qifeng Chen. Single image reflection separation with perceptual losses. IEEE Conference on Computer Vision and Pattern Recognition, 4786-4794, 2018.", the MA-Net of this invention achieved the best performance on all datasets. This is because ERRNet is built upon the Zhang et al. model, and its network model has better generalization ability to this dataset, thus the algorithm performs better on the Zhang et al. dataset. On the average performance across all test datasets, MA-Net outperforms other methods.

[0259] Table 13 compares and analyzes the impact of de-reflection networks on the pressure plate status recognition results of existing pressure plate status recognition methods (clustering matching method, improved BOF method, OpenVINO method, transfer learning method, and improved SSD method), using the pressure plate status dataset Pressure-plate as the object.

[0260] Table 13. Impact of dereflection network on pressure plate status recognition

[0261]

[0262] As shown in Table 3, under conditions of reflection, the recognition accuracies of the five platen status identification methods are 78.22%, 83.55%, 92.90%, 89.63%, and 84.55%, respectively. After removing reflection interference through a de-reflection network, the recognition accuracy of all five methods is improved to varying degrees. Among them, the clustering matching method and the improved BOF method improve the recognition accuracy by 6.28% and 3.66%, respectively, which is significantly higher than the OpenVINO method, transfer learning method, and improved SSD method (0.45%, 1.57%, and 1.87%, respectively). This is because traditional image recognition methods rely more on the information of the original image and have relatively poor anti-interference ability, so the de-reflection network produces better results. This invention studies the reflection problem in platen status monitoring of UHVDC converter stations and proposes a deep de-reflection neural network based on multi-level connections and adaptive region attention to remove reflection interference in images. The MA-Net network can adaptively aggregate features through multi-level connections and compressed excitation blocks, and fully utilize rich remote non-reflection background information based on wide-regional non-local blocks. Experiments show that MA-Net can not only recover the details of the input image, but also almost completely eliminate reflection interference on the real image dataset and the cabinet pressure plate state image dataset, which can effectively improve the detection effect of the pressure plate state.

Claims

1. A method for processing and storing panoramic monitoring images of an ultra-high voltage converter station protection system, characterized in that, Includes the following steps: S1. Perform super-resolution reconstruction on the panoramic surveillance image. The reconstruction method is as follows: S11. Establish a deep multi-scale residual network model on the edge side; S12. Input sample dataset and train the deep multi-scale residual network model; S13. Test the peak signal-to-noise ratio and structural similarity index of the trained deep multi-scale residual network model using a standard dataset. S14. Input the panoramic monitoring image of the UHV converter station into the trained deep multi-scale residual network model to complete the super-resolution reconstruction. S2. Optimize the ubiquitous heterogeneous network transmission topology for panoramic monitoring of UHV converter stations. The optimization method is as follows: S21. Model the heterogeneous network of the UHV converter station as a tree structure, wherein the tree structure has a master station. and Data transmission nodes Each data transmission node has a connection to the master station. The only path; S22, Main Station As the root node of the tree structure, the training dataset is obtained by recursively searching the Monte Carlo tree for each state, with the root node as the initial state. S23. Input the training dataset obtained from the search into the deep convolutional neural network for training to obtain the value function and policy function, which are used to guide the Monte Carlo tree to recursively search for states with expected rewards and in turn update the training dataset collected by the deep convolutional neural network. S24. After training is complete, from the initial state Initially, by sequentially selecting from the strategies predicted by the deep convolutional neural network... The action is performed and the status is updated. This continues until a complete tree is reached, thus obtaining the heterogeneous network topology; S3. By optimizing the heterogeneous network transmission topology, the panoramic monitoring data is transmitted to the edge side. After being lightweighted at the edge side, it is then transmitted to the cloud for storage.

2. The panoramic monitoring image processing and storage method for the UHV converter station protection system according to claim 1, characterized in that, The deep multi-scale residual network model includes: an input convolutional layer, an output convolutional layer, and k The input convolutional layer acts as an encoder to extract the original low-level features of the low-resolution image; the output convolutional layer is used to fuse multi-scale detail features to reconstruct the high-resolution image; the input and output convolutional layers have skip connections to establish an identity mapping from the low-resolution image to the high-resolution image for global residual learning. k The aforementioned multi-scale convolutional blocks are stacked and connected sequentially to obtain the network model depth; the original low-order features and k The multi-scale convolutional blocks are interconnected via... k Each path corresponds to a connection, and local residual learning enhances the network model's ability to learn complex features.

3. The panoramic monitoring image processing and storage method for the UHV converter station protection system according to claim 2, characterized in that, Both the input and output convolutional layers use convolutional kernels with a stride of 1, and the input convolutional layer uses ReLU activation. The multi-scale convolutional block extracts multi-level detailed features from the input image using convolutional kernels of four scales: 3×3, 3×2, 2×3, and 2×2. Then, the feature maps of the four scales are concatenated pairwise along a specified dimension through a cross-connection mechanism and fed into a 3×3 convolutional layer for feature mapping, generating a new feature map of the same size as the input and feeding it into the next multi-scale convolutional block.

4. The panoramic monitoring image processing and storage method for the UHV converter station protection system according to claim 3, characterized in that, The local residual learning is defined as follows: ;in, For the first k The feature maps learned by multi-scale convolutional blocks For the first k The output of a multi-scale convolutional block -1 For the first k-1 The output of a multi-scale convolutional block F The original low-order features extracted by the input convolutional layer; the global residual and local residual learned. k The mapping of a multi-scale convolutional block is represented as follows: ;in, () represents the mapping that the input convolutional layer needs to learn. F -1 () represents the mapping that the output convolutional layer needs to learn, where, , These represent high-resolution and low-resolution images, respectively. -1 For the first k-1 The feature maps learned by multi-scale convolutional blocks The feature map learned from the first multi-scale convolutional block. R () represents a mapping operation.

5. The panoramic monitoring image processing and storage method for the UHV converter station protection system according to claim 4, characterized in that, The loss function of the deep multi-scale residual network model is: ;in, The parameters of the deep multi-scale residual network are defined, and the loss function is minimized using the Adam optimizer. For sample dataset The first in Sub-images, For the corresponding label, N is a positive integer.

6. The panoramic monitoring image processing and storage method for the UHV converter station protection system according to claim 1, characterized in that, The panoramic monitoring images include images of secondary equipment, hard pressure plates, and terminal corrosion; the standard datasets include three basic datasets: Set5, Set14, and Urban100.

7. The panoramic monitoring image processing and storage method for the UHV converter station protection system according to claim 5, characterized in that, The formula for calculating the peak signal-to-noise ratio in the test is as follows: ;in, The mean square error between the original image and the processed image is denoted as . The maximum value of the image color is represented by the following formula: ; ; ; ;in, , , and Representing images respectively and The mean and standard deviation, Representing an image and covariance, , and It is a constant, usually taken as , , , , , This represents the range of pixel values.

8. The panoramic monitoring image processing and storage method for the UHV converter station protection system according to claim 1, characterized in that, The method for modeling the heterogeneous network of the UHV converter station as a tree structure is as follows: In each round of data acquisition, the nodes... Will The bit data is forwarded to its parent node, where, ; It is by Self-generated data, data set It comes from child nodes, The function is an aggregate function; a transfer model is used. Transmission traffic, in the transmission model, the transmission traffic of nodes related to the topology consists of two parts: data processing and transmission time. and They are at the nodes The time consumption for processing each bit of data and the time consumption for transmitting each bit of data are specified. The time consumption for transmitting each bit of data depends on the distance to the parent node, and its calculation formula is as follows: ,in, It is a node with its parent node The Euclidean distance between them It is the power amplification constant in the link budget that takes into account the effects of shadow fading.

9. The panoramic monitoring image processing and storage method for the UHV converter station protection system according to claim 8, characterized in that, The Monte Carlo tree recursive search method is as follows: Each node in the Monte Carlo tree represents a 5-tuple of data. ; in each search step At this point, choose the action that maximizes the upper confidence limit; when the search reaches the termination state... When a reward is obtained, the search path is propagated back to the root state of all visited states and the actions taken, along the path. The value is updated accordingly by the average value on the node; where It is the state of a heterogeneous network; This refers to the action performed in that state. Accessing on the search tree Total number of times; It is the prior probability of an effective action predicted by a deep convolutional neural network; It is a state action value, representing the state. Start and take action The expected reward; the formula for calculating the action that maximizes the confidence upper limit is as follows: ;in, It is a state The access count, without considering actions. It is a hyperparameter that controls the search level.

10. The panoramic monitoring image processing and storage method for the UHV converter station protection system according to claim 9, characterized in that, The deep convolutional neural network comprises a deep Vgg16 module, a fully connected layer with softmax for policy, and a fully connected layer with ReLU activation for value functions. The deep Vgg16 module consists of two convolutional layers with 64 convolutional filters, two convolutional layers with 128 convolutional filters, three convolutional layers with 256 convolutional filters, and six convolutional layers with 512 convolutional filters. Each convolutional filter has a 3×3 kernel and a max pooling layer.

11. The panoramic monitoring image processing and storage method for the UHV converter station protection system according to claim 10, characterized in that, The value function satisfies the Bellman equation, meaning that the value of the current state is the reward of that state plus the expected reward of the next state. The formula for the value function is: The strategy function formula is as follows: .

Images