A method and system for identifying defects in substations

Abstract

This invention discloses a substation defect identification method, comprising: collecting and establishing a defect identification sample library to form a pair of identification samples; data preprocessing, performing image registration and alignment preprocessing on the collected identification sample pairs, and marking the locations of regions with differences in the aligned sample pairs; constructing a Siamese network structure, extracting four different scale feature maps from the neural network for each image pair as image feature representations; feature engineering, extracting feature vectors suitable for target detection and metric learning; designing target detection and similarity measurement branches to achieve the optimization goals of improving prediction box accuracy and improving feature space; and model inference and logical post-processing. This invention also discloses a substation defect identification system based on a Siamese detection model, including modules for the above methods. This invention is applicable to complex scenarios where defect samples are difficult to obtain and conventional identification methods are difficult to distinguish, and has the advantages of wide applicability and high accuracy.

Description

Technical Field

[0001] This invention belongs to the field of substation inspection technology, and in particular relates to a method and system for identifying substation defects. Background Technology

[0002] Currently, with the continuous advancement of power system automation technology and the deepening reform of the power system, unmanned substations have been widely promoted. However, substations are characterized by their large number, wide geographical distribution, and many being located in sparsely populated areas, which brings many inconveniences to the management of higher-level authorities. The contradiction between insufficient production personnel and increased inspection workload is becoming increasingly prominent, and the traditional method of relying on "manual inspection and manual recording" can no longer meet the needs of the modern power industry.

[0003] In the existing technology, the Chinese invention patent with application number "2022108577416" entitled "A Substation Monitoring Method, System, and Electronic Equipment Based on Deep Learning" describes a substation intelligent inspection strategy that utilizes high-definition cameras and inspection robots to acquire equipment images and perform identification tasks such as meter readings and disconnector switch position determination based on single-frame images. Although it can identify common defect types such as oil leaks and failure to wear safety helmets, the types that can be identified are currently fixed and limited. For defect types that are difficult to collect training samples, difficult to perceive by conventional defect identification methods, and have complex and highly variable conditions such as image background lighting, there are problems such as low identification accuracy and lack of identification methods.

[0004] Secondly, while traditional graphics algorithms such as pixel detection and speckle difference detection can detect regions with differences in images, these methods are sensitive to factors such as positional shifts and changes in lighting, resulting in a large number of false detections. Furthermore, deep learning methods using a single model, which preprocess the image before inputting it into the neural network, suffer from lost differential features in areas with subtle changes due to the depth of the network layers, leading to missed detections. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a substation defect identification method based on path spanning tree with wide application scenarios and high detection sensitivity; another purpose of this invention is to provide a substation defect identification system.

[0006] Technical solution: The substation defect identification method based on path spanning tree described in this invention includes the following steps:

[0007] Normal baseline data and abnormal defect data at the same location are obtained from key equipment, meters, entrances and exits and important scenes in the substation to form a pair of discrimination samples;

[0008] The twin network structure is constructed based on the twin network model, and feature maps of different scales are extracted from the neural network for each pair of images as image feature representations;

[0009] Feature engineering is performed on image feature representations to extract feature vectors suitable for object detection and metric learning;

[0010] Different types of features are input into the object detection and similarity measurement branches respectively. By designing loss functions for the two branches, the optimization goals of improving the accuracy of the predicted bounding box and improving the feature space are achieved.

[0011] For model reasoning and logical post-processing, formulate strategies for organizing, merging, and deleting prediction boxes, explore the relationship between prediction boxes and substation equipment, fill and expand prediction boxes, and output the final results.

[0012] In the defect discrimination sample library collection and establishment, a pair of discrimination samples consists of normal baseline data and abnormal defect data. The data should preferably be collected by taking photos of cameras and inspection robots deployed in the substation at fixed preset positions. The collection scenarios include key equipment, important entrances and exits, and the internal environment of the station. The target object should be photographed in the center, and key positions and pointers should be clearly visible. The baseline data and defect data should be photographed using the same collection equipment and the same preset points. The defect discrimination sample library should preferably include at least 10,000 pairs of valid samples.

[0013] The Siamese network structure requires image preprocessing before construction. Image preprocessing includes image registration and alignment of the acquired discrimination sample pairs, marking the locations of differences in the aligned sample pairs, generating XML tag files, and dividing the training and validation sets into a 4:1 ratio. Specifically, this includes the following steps:

[0014] Convert normal baseline image samples and abnormal defect image samples from RGB to grayscale images;

[0015] Feature descriptors were extracted from the two grayscale images using the ORB algorithm.

[0016] The similarity between feature descriptors is calculated using Hamming distance, and feature pairs that meet the matching threshold requirements are extracted.

[0017] Based on the feature pairs extracted by the RANSAC algorithm, the affine transformation matrix is ​​calculated.

[0018] By applying affine transformations to the original image, abnormal images can be registered and aligned with normal images.

[0019] The step of marking out the locations of differences in the aligned sample pairs includes, for scenarios involving abnormal equipment and personnel entering and exiting, using the LabelImg tool to mark out the locations of differences between the two images after registration and alignment. The marked areas are rectangular boxes, and the marked files are saved in the xml format specified by the PASCALVOC dataset.

[0020] The Siamese network consists of two identical first-order object detection models, YOLOv5m, and extracts image feature representations through the following steps:

[0021] Based on the defect identification scenario, customized data augmentation and expansion strategies are implemented; two images use the exact same data augmentation strategies, including image size scaling, blur enhancement, median filtering, HSV color space enhancement, vertical flipping, and horizontal flipping.

[0022] The enhanced image is input into the YOLOV5m model, and the feature maps of the four sub-modules P3-P6 are extracted as image feature representations. The scaling ratios of these four feature maps relative to the original image are 8, 16, 32, and 64, respectively.

[0023] The feature engineering process includes transforming and processing the feature representations of two images to generate feature vectors suitable for metric learning and object detection, specifically including the following steps:

[0024] Based on the XML annotation file and feature map scaling ratio in image preprocessing, a mask matrix of the same size is generated for the feature map of the P3 module of the YOLOV5m model. The matrix element with the marked distinguishable regions has a value of 1, and the other positions have a value of 0.

[0025] The mask matrix is ​​multiplied with the feature map pixel by pixel, and then input into a 3-layer 1*1 convolutional structure to fine-tune the feature vector space, finally generating a metric learning feature vector.

[0026] The system employs a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN) structure. FPN uses operations such as convolution and upsampling to pass strong semantic features from high-level layers down, achieving multi-scale feature fusion. PAN is another bottom-up pyramid structure that passes localization features from lower layers layer by layer. The two-layer pyramid simultaneously possesses semantic and localization information.

[0027] Stack the feature vectors of modules P3-P6 of the two images according to the channel dimension;

[0028] Batch standardization is performed on the stacked feature vectors;

[0029] Two convolutional layers with 1*1 kernels were used to fine-tune the feature vector space, and finally, the target detection feature vector was generated according to the image scaling ratio.

[0030] The calculation of the model loss function in the target detection and similarity measurement branch design includes the following steps:

[0031] By calculating the binary loss function through metric learning feature vectors, the metric space is optimized so that sample pairs with no difference cluster together in the metric space, while sample pairs with difference are moved away from each other in the metric space.

[0032] Generate anchor boxes for target bounding box detection from the target detection feature vectors;

[0033] The positional relationship between the anchor box and the predicted box is optimized by using the cross-union ratio (CUI) through the positional loss function.

[0034] The confidence loss function is used to assess the probability that a target exists at this grid location.

[0035] The model inference includes the following steps:

[0036] According to image preprocessing, images taken at the same location or the same preset position during the inspection process are registered and aligned with the stored normal images to form test sample pairs;

[0037] Load the defect discrimination model based on twin structure;

[0038] Defects are identified in the sample pairs to generate initial inference results.

[0039] The logical post-processing includes the following steps:

[0040] Based on the set intersection-union ratio and confidence threshold, the initial inference results are filtered out;

[0041] Overlapping prediction boxes are filtered out using nonmaximum suppression techniques.

[0042] Use a substation equipment identification algorithm model to identify common equipment in substations;

[0043] Based on the predicted device location, and combined with the coordinate fusion method, each image will ultimately output only one predicted bounding box.

[0044] A substation defect detection system includes:

[0045] The sample acquisition module is used to acquire normal baseline data and abnormal defect data at the same location of key equipment, meters, entrances and exits and important scenes in the substation, forming a pair of discrimination samples;

[0046] The Siamese network structure construction module is used for constructing Siamese network structures. Based on the Siamese network structure model, feature maps of different scales are extracted from the neural network for each pair of images as image feature representations.

[0047] The feature vector extraction module is used to perform feature engineering on image feature representations and extract feature vectors suitable for object detection and metric learning.

[0048] The loss optimization module is used to input different types of features into the object detection and similarity measurement branches respectively. By designing the loss functions of the two branches, the optimization goals of improving the accuracy of the prediction box and improving the feature space are achieved.

[0049] The post-processing module is used for model reasoning and logical post-processing, formulating strategies for organizing, merging, and deleting prediction boxes, exploring the relationship between prediction boxes and substation equipment, filling and expanding prediction boxes, and outputting the final results.

[0050] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0051] (1) The method of the present invention compares the normal baseline data of key equipment, meters, entrances and exits in the substation with the actual inspection images of the same monitoring scene, object and target to determine whether there are defects during the inspection process. It solves the problem of low recognition accuracy in defect scenarios where training samples are difficult to collect, conventional defect recognition methods are difficult to perceive, and image background lighting and other conditions are complex and changeable. It expands the application scenarios of the inspection system.

[0052] (2) When establishing the discrimination training sample library, the images are captured by using in-station robots, drones and cameras to retrieve preset positions, ensuring that the images to be discriminated have no obvious positional offset from the reference images and ensuring the uniformity of the samples in the image domain. Driven by sample quality, the robustness of the algorithm model is improved. According to the task characteristics, sample pair alignment and data augmentation strategies are designed to ensure both the invariance of the target to be identified in spatial position and the diversity of the identification samples.

[0053] (3) This invention constructs a target detection network based on a Siamese structure, extracts feature representations of image sample pairs, and innovatively proposes feature engineering for the discrimination task. Combining the task nature and the target detection network structure, it extracts feature vectors that can be used for loss function calculation and inference output. The neural network structure optimization target designed in this invention combines target detection and metric learning loss functions. While ensuring the correct identification of target categories and accurate location, it also optimizes the feature vector space, making similar samples cluster together as much as possible and dissimilar samples as far apart as possible, thereby improving the sensitivity and robustness of anomaly discrimination detection. In the model inference and testing stages, this invention proposes a logical post-processing strategy. Through threshold filtering, non-maximum suppression, and main device position restrictions, only one prediction box is retained for each image, further filtering out the influence of noise such as illumination, weather, and shadows on the recognition results. Attached Figure Description

[0054] Figure 1This is a schematic diagram of the overall process of the present invention;

[0055] Figure 2 This is a schematic diagram illustrating the feature matching effect between the reference image and the image to be judged in this invention;

[0056] Figure 3 This is a schematic diagram of the target detection model based on Siamese neural networks of the present invention. Detailed Implementation

[0057] The overall process of this invention is as follows: Figure 1 As shown, a method for identifying defects in a substation includes the following steps:

[0058] Step 1: Defect Identification Sample Database Collection and Establishment: An effective defect identification database should contain a pair of identification samples. Data should ideally be collected through cameras deployed within the substation or inspection robots set to fixed preset positions to capture photos and videos. The specific steps are as follows:

[0059] Step 11: For key inspection points such as equipment, meters, important entrances and exits, and the internal environment of the substation, retrieve the preset positions of cameras and robots under normal conditions to ensure that the target object is as centered as possible, and that key positions and pointers are clearly visible. Capture the image under the current normal conditions and save it to a fixed path on the server as the reference image for that point.

[0060] Step 12: Simulate changes in equipment damage, equipment status, meter readings, ladder switch locks, and foreign object positions at the same inspection point, and take another screenshot as the image to be judged.

[0061] Step 13: If there is a significant change between the reference image and the image to be judged during inspection, and there is no large pixel offset between them, then it can be identified as a valid sample pair; otherwise, it is an invalid sample pair and will not be included in the sample library. The sample library finally constructed by this invention contains a total of 9600 valid discrimination samples.

[0062] Step 2: Data preprocessing, including image registration and alignment of the collected discrimination sample pairs, and marking the locations of regions with differences in the aligned sample pairs.

[0063] Step 21: Preprocessing operations for image registration and alignment. This includes locating the differences between the reference image and the image to be identified in the defect discrimination task. However, due to the different shooting times of the two images and the mechanical errors of the shooting equipment causing slight positional shifts, it is necessary to align and register the image sample pairs. The algorithm flow is as follows: Figure 2 As shown, the specific steps are as follows:

[0064] Step 211: Grayscale Conversion of the Color Image. Sample pair alignment and registration primarily consider the device position in the image. Color information is redundant in this step, and considering color channels would increase computational burden. Therefore, this invention requires grayscale conversion of the RGB image acquired by the device. In a grayscale image, each pixel consists of only one value representing grayscale, ranging from 0 to 255. A grayscale value of 255 represents the brightest, and a grayscale value of 0 represents the darkest. This invention uses the mean method to achieve image grayscale conversion, that is, taking the mean of the RGB channel components of the color image to form the grayscale value of the pixel.

[0065] Step 212: Image size adjustment. In this invention, the reference image and the image to be matched are uniformly scaled to a size of 640*640.

[0066] Step 213: Grayscale Image Feature Extraction. This invention uses the ORB (Oriented FAST and Rotated BRIEF) algorithm for feature extraction. The steps include two main steps: finding key points in the image using the FAST corner extraction algorithm, and generating feature descriptors for the key points using the BRIEF algorithm.

[0067] The specific steps of the FAST corner extraction algorithm include: constructing an image pyramid. An image pyramid is a multi-scale representation of a single image, composed of a series of original images at different resolutions. Each level of the pyramid is composed of downsampled versions of the upper-level image. This invention uses a 1 / 2 downsampling ratio, resulting in pyramids of four sizes: 640*640, 320*320, 160*160, and 80*80. Extracting FAST corners: This method mainly targets the degree of local pixel grayscale change in images of each size. The core idea is that if a pixel has a large difference from its neighboring pixels, it may be a corner point. The specific steps are as follows:

[0068] Select pixel p in the image, with its grayscale value set as lp, and set a threshold tfast. Using pixel p as the center, select 16 pixels on a circle with a radius of 3. If N consecutive points on the circle have a grayscale value greater than lp+tfast or less than lp-tfast, then that pixel is considered a potential corner point. Repeat the above steps for each pixel of each size to obtain all possible corner points. Perform non-maximum suppression on the corner points, retaining only the corner points with the largest grayscale changes within a certain area to avoid the problem of excessive corner point concentration.

[0069] The specific steps for extracting FAST corner point orientation information are as follows: 1. Calculate the intensity centroid. Extract the average pixel intensity within the bounding box centered on the key corner point. The pixel with the smallest difference from the average pixel intensity is the intensity centroid. 2. Calculate the intensity vector. The vector connecting the key corner point to the intensity centroid is the intensity vector. 3. Calculate the orientation information. The angle between the intensity vector and the positive x-axis represents the orientation information of the corner point, characterizing the grayscale changes around the corner point.

[0070] Step 214: Extract the BRIEF feature vector. The method generates a set of binary feature descriptors based on a key corner point, meaning the feature vector contains only 1s and 0s. The specific steps are as follows: To reduce the influence of high-frequency noise points in the image on the generated feature descriptors, Gaussian filtering is applied to the neighborhood around the key corner point for smoothing. In this invention, the Gaussian window is set to 3*3 with a variance of 2. The neighborhood value of the key corner point is set, and BRIEF feature extraction is performed within this neighborhood. In this invention, the neighborhood range is set to S = 31. A pixel grayscale value p1 is extracted from the Gaussian distribution centered on the selected key corner point. Another pixel grayscale value p2 is extracted from the Gaussian distribution centered on p1. p1 and p2 are compared. If p1 is brighter than p2, the corresponding bit of the BRIEF descriptor is assigned a value of 1; otherwise, a value of 0 is assigned. The feature calculation is as shown in Equation 1:

[0071]

[0072] In addition, the present invention sets the BRIEF feature dimension to 500, that is, reselects p1 and p2, compares gray levels and assigns feature values, repeats this process 500 times, and forms the BRIEF feature vector f.

[0073] Step 215, Image Feature Matching, the matching effect described in this invention is as follows: Figure 2 As shown. For the reference image and the image to be judged, the multiple key corner points and the 500-dimensional binary codes generated in step 214 need to be registered. The feature vector extracted from the key corner points of the reference image is set as... Where mi and f i1 -f i500 These represent the number of corner points in the baseline image and the 500-dimensional binary feature encoding of the BRIEF algorithm, respectively; the feature vector extracted from the key corner points of the image to be judged is... Among them, mj and f i1 -f i500 These represent the number of key corner points in the image to be judged and the binary feature encoding of the BRIEF algorithm, respectively. and The matching strategy is as follows: If the number of identical elements in corresponding bits of two sets of feature codes is less than 64, then the feature sets must not match; for a pair of features with more than 64 identical elements, this invention measures the similarity between features by calculating the Hamming distance. For each feature pair, the closest one is taken as the matching point. The Hamming distance is calculated as shown in Equation 2:

[0074]

[0075] By sorting the matching corner point pairs from largest to smallest, this invention ultimately selects the top 15% of all matching point pairs as valid matches.

[0076] Step 216: Image registration, i.e., calculating the transformation relationship based on the positions of the matching corner points. This invention requires calculating the mapping relationship between the reference image and the image to be judged. The calculation process can be analogous to a projection mapping from one two-dimensional plane to another; that is, calculating the transformation relationship can be transformed into solving the homography matrix. Let the position vector of the corresponding corner point in the reference image be P1 = {xi; yi; 1}, where xi and yi represent position coordinates, and 1 is a fixed vector; the position vector corresponding to the matching feature in the image to be judged is P2 = {xj; yj; 1}. This invention uses the RANSAC algorithm to calculate the mapping matrix H, as shown in Equation 3:

[0077] P1 = H × P2

[0078]

[0079] The specific registration steps include: randomly selecting 4 pairs of matching corner points and calculating the transformation matrix H1; projecting all matching corner point pairs using H1 to calculate the error between the projected vector and the original vector. When the error is less than a certain threshold, the pair of samples is considered an interior point, and the interior point count variable l is incremented by 1; otherwise, it is an exterior point. The threshold set in this invention is 3. This invention sets the initial optimal number of interior points lbest = 0, and the transformation matrix H is a 3*3 zero matrix. If l > lbest obtained in this iteration, then lbest is updated using l, H1 is updated, and the iteration number k is updated. The above process is repeated until the number of iterations exceeds k. The H updated in the iteration is the final transformation matrix.

[0080] Step 217, Image Perspective Transformation, is essentially projecting the image to be judged onto the view plane of the reference image. The specific steps are as follows: Traverse the rows and columns of the image to be judged to obtain the coordinates (xj; yj; 1) of each pixel; use the transformation matrix H to calculate the corresponding position (xi; yi; 1) of this coordinate in the reference image; assign the pixel value of (xj; yj) of the image to be judged to the position (xi; yi); ignore pixels that exceed the range of the reference image after transformation, and fill in the missing pixels with black.

[0081] Step 218: Swap the reference image and the image to be discriminated, and repeat the above steps to complete the alignment and registration of the reference image relative to the image to be discriminated. In this way, a pair of images can generate two sets of training samples. Therefore, the present invention ultimately provides a total of 19,200 pairs of effective training samples for training the discriminative model.

[0082] Step 22: Image annotation. LabelImg software is used to annotate the 19200 pairs of samples. The annotation principle is to manually compare and align the registered image with the original image. In the registered image, the main equipment located in areas of difference is marked. For example, in the image, the meter shows a large range of pointer changes; the entire meter needs to be labeled, not just the area with the changing pointer. This invention uses rectangular bounding boxes for annotation, with the annotation category being diff. The annotation file is an XML file in PASCALVOC dataset format, with one XML file corresponding to each image.

[0083] The specific relationship between the registration image and the original image mentioned in this invention is as follows: if the registration image is the reference image, the original image is the image to be judged; if the original image is the image to be judged, the registration image is the reference image.

[0084] Step 3: Construction of a discriminative model based on Siamese neural networks

[0085] The twin network structure described in this invention is an improved implementation based on the YOLOv5m target detection model. The network structure is as follows: Figure 3 As shown, the model consists of an input part, a main network, a neck network, a feature extraction network, and a head structure. A twin structure model refers to a main network and a neck network with identical structures and shared parameters. The input part uses the exact same data preprocessing method. This implementation uses an input image size of 640*640*3 as an example. The specific steps include:

[0086] Step 31: Image preprocessing of the input portion mainly includes data augmentation and adaptive anchor box generation, specifically including:

[0087] Step 311: The discrimination task requires processing two images simultaneously and is highly sensitive to positional information in the images. This invention uses the same data augmentation method and parameters for the sample pairs, mainly employing pixel-level data augmentation methods to keep the image positions from changing randomly as much as possible, including translation, scaling, inversion, HSV color gamut transformation, image median filtering, grayscale transformation, image compression, etc.

[0088] Step 312: YOLOv5m is an anchor-based detection model. During network training, predicted boxes need to be identified based on initial anchor boxes, and the distance between these predicted boxes and the ground truth (GT) is calculated. The model parameters are then updated through backpropagation. Object detection models typically use anchor boxes of uniform size, but this method cannot handle the problem of varying sizes of power equipment and different sizes of ground truth boxes. Therefore, this invention employs an adaptive anchor box size generation method, specifically including: loading the training set annotation file to obtain the width and height of all GTs in the training set;

[0089] The maximum width and height of each image are scaled to 640, and the corresponding scaling ratio is obtained. The smaller sides are resized according to this scaling ratio. The ground truth (GT) bounding boxes are scaled accordingly. GT boxes with a length or width less than 3 pixels are filtered out. From the remaining GT samples, k samples are randomly selected as center points. In this invention, k is set to 12 according to the YOLOv5m model requirement, representing the selection of three different sizes of anchor boxes on four different feature maps. The center points are represented by (C1, C2, ..., C12). The Euclidean distance between each sample and each center point is calculated, and the sample is assigned to the nearest center point. All samples assigned to the same center point form a category or cluster. For each category i, according to... Recalculate the cluster centers, where X is a vector sample belonging to the i-th class, consisting of the length and width of the corresponding ground truth box, and n represents the total number of samples contained in the i-th class; repeat the above steps until the cluster centers no longer change.

[0090] Step 32: The main network of the YOLOV5m model described in this invention is still the CSPDarknet structure, consisting of 6 sub-modules composed of convolutional structures, denoted as P1-P6. The main improvements of the network compared to the existing YOLO series networks include the Focus module, CSP network structure, SiLU activation function, and Fast Spatial Pyramid (SPPF) feature unification structure. The specific steps are as follows: The Focus module is a method for image pixel-level processing. It divides each channel matrix of the image into 2*2 blocks. Pixels at the same position in each block are spliced ​​together to obtain 4 independent feature layers. These 4 feature layers are stacked, which expands the input channels of the neural network by 4 times. The width and height information of the image can be converted into channel dimensions. For example, if the original image size is 640*640*3, after passing through the Focus network structure, the output feature map size is 320*320*12.

[0091] Step 321: The CSP network structure divides the input tensor into two branches according to the channels. One branch performs residual or multi-layer convolution calculations, while the other branch only performs a 1*1 Bottleneck operation. Finally, the two branches are stacked according to the channels to keep the number of input and output channels consistent.

[0092] Step 322: The SiLU activation function is a smoothed improvement based on the Sigmoid and ReLU functions. Its calculation formula is shown in Equation 4. Compared to the traditional ReLU activation function, it has continuous smoothness and non-monotonicity. Furthermore, when x ≤ 0, its derivative is not zero, thus not interrupting the backpropagation process of the neural network.

[0093]

[0094] Step 323: SPPF Feature Unification Structure. This invention applies SPPF to the feature space pyramid using pooling kernels of different sizes, increasing the network's receptive field. SPPF is applied to the feature map of the last layer of the main network. Assuming the input image size is 640*640*3, and the output feature map size of the P6 layer is 10*10*1024, the specific SPPF structure is as follows: The convolution kernel is a 1*1 BottleNeck structure (input size: 10*10*1024, output size: 10*10*512); the pooling kernel is a 5*5 three-layer max-pooling concatenated structure, with the outputs of each pooling layer stacked by channel (input size: 10*10*512, output size: 10*10*2048); the convolution kernel is a 1*1 BottleNeck structure to achieve feature fusion (input size: 10*10*2048, output size: 10*10*1024).

[0095] Step 324: In this invention, the outputs of layers P3-P6 of the main network are used as image features for subsequent analysis. The scaling ratios of these four layers relative to the input image are 8, 16, 32, and 64, respectively. Taking an input image size of 640*640*3 as an example, the feature map sizes of layers P3-P6 are 80*80*256, 40*40*512, 20*20*768, and 10*10*1024, respectively.

[0096] Step 4: Feature engineering, analyze and transform the feature maps of layers P3-P6, and design the neck structure of the neural network;

[0097] Step 411 involves extracting feature vectors suitable for object detection and metric learning. The specific steps are as follows:

[0098] Step 411: Extract metric learning feature vectors. The core of the image discrimination task is to locate the differences between two images. This invention designs a metric learning neck structure for the discriminant image and the P3 layer feature map of the reference image extracted in step D, extracts metric learning features, and provides them for the calculation and backpropagation of the head network loss function. The specific steps are as follows:

[0099] Step 412: Scale the annotation area. Based on the annotation file of the image to be judged described in step C, the target box is scaled to obtain its position on the P3 layer feature map. The scaling coordinates of the upper left corner of the target box are rounded down, and the scaling coordinates of the lower right corner are rounded up.

[0100] Step 413: Generate a binary target region mask matrix. Image discrimination tasks suffer from problems such as small target regions and significant influence from weather and environmental factors. Using global image features for metric feature extraction would inevitably affect application performance. Therefore, based on the scaled target bounding box, a binary target region mask matrix for layer P3 is generated, focusing only on regions with different annotations. The matrix has a value of 1 within the scaled target bounding box and 0 in other regions. The mask matrix is ​​a binary representation of the image annotations, and the matrix size should be exactly the same as the feature map size.

[0101] Step 414: Generate the masked image matrix. Perform element-wise product between the mask matrix and the feature matrix of each channel;

[0102] Step 415: Design a 3-layer convolutional neural network to form a metric network and extract metric learning feature vectors. The specific network structure designed in this invention is as follows: a convolutional layer consisting of 128 1*1 convolutional kernels (input size: 80*80*256, output size: 80*80*128); a convolutional layer consisting of 64 1*1 convolutional kernels (input size: 80*80*128, output size: 80*80*64); and a convolutional layer consisting of 32 1*1 convolutional kernels (input size: 80*80*64, output size: 80*80*32).

[0103] Step 416: Extract metric learning feature vectors from the reference image and the image to be matched, respectively, following the steps described above.

[0104] Step 42: Extract target detection feature vectors. The P3-P6 features extracted in step D are input into the Feature Pyramid (FPN) and Path Aggregation Network (PAN) structures. This achieves multi-scale feature fusion while preserving localization information. The fused features from the baseline image and the image to be matched are stacked and fine-tuned through two 1*1 BottleNeck layers. Finally, target detection feature vectors of different sizes are generated according to the scaling ratio. The specific steps are as follows:

[0105] Step 421: Construct a top-down Feature Pyramid (FPN) structure to convey semantic information and achieve feature fusion and detection at different scales. The FPN structure used in this invention has four FPN modules. The specific implementation steps of the modules are as follows: Perform a 1*1 convolution operation on the feature map obtained in step D to keep the number of channels of the feature map consistent before fusion; perform a 2x upsampling on the high-level feature map output by the previous FPN module; add and fuse the feature maps obtained in the previous two steps according to channels; fine-tune and re-fuse the fused feature map through a 3*3 convolution.

[0106] Step 422: Construct a bottom-up path aggregation network (PAN) structure. The semantic information of the low-level features of the object detection network structure is helpful for object recognition, but because the gradient backpropagation path of FPN is relatively long, it increases the difficulty of obtaining accurate localization. PAN complements FPN by propagating the localization features of the low-level feature maps upwards again, ensuring that the designed feature pyramid structure can simultaneously combine localization and semantic information. This invention has three PAN modules, and the specific implementation steps include: copying the lowest-level feature map of FPN to form the first layer of the PAN structure, denoted as P3_PAN; downsampling the first layer features by a factor of 2, denoted as P3_PAN_DS; fine-tuning the second-to-last layer features of FPN using a 3*3 convolution, denoted as P4_FPN; adding P3_PAN_DS and P4_FPN element by element to form P4_PAN_TMP; performing a 3*3 convolution operation on P4_PAN_TMP to form the output feature P4_PAN of the PAN structure; repeating the above operations for each PAN module to obtain four layers of feature maps output by the PAN structure, denoted as P3_PAN, P4_PAN, P5_PAN, and P6_PAN, respectively. The feature map sizes are the same as in step D.2.5.

[0107] Step 423: The present invention adopts a twin detection network structure, which can obtain feature maps of the same size for both the reference image and the image to be judged, and stack them according to the channels. The stacked feature map sizes are 80*80*512, 40*40*1024, 20*20*1536 and 10*10*2048 respectively.

[0108] Step 424: Perform batch normalization on the stacked feature maps;

[0109] Step 425: Design two feature fine-tuning modules for each of the four feature layers. Module 1 includes a 1*1 BottleNeck convolution, batch normalization, and Leaky ReLU activation function. The number of convolution kernels is the same as the number of channels in the stacked feature map. Module 2 has half the number of convolution kernels as the stacked feature map, which restores the feature map to the size of a single image feature map. The specific number of convolution kernels in the fine-tuning modules for each feature map layer is as follows:

[0110] P3 layer: Module 1: 512 convolutional kernels, Module 2: 256 convolutional kernels;

[0111] P4 layer: Module 1: 1024 convolutional kernels, Module 2: 512 convolutional kernels;

[0112] P5 layer: Module 1: 1536 convolutional kernels, Module 2: 768 convolutional kernels;

[0113] P6 layer: Module 1: 2048 convolutional kernels, Module 2: 1024 convolutional kernels;

[0114] The output of each layer module 2 is the target detection feature vector, which is used for target detection loss function optimization and parameter backpropagation.

[0115] Step 5, Design the loss optimization objective

[0116] The loss optimization objective is designed based on the feature vectors extracted in step 4, forming the network's head structure for parameter backpropagation and model updates. The head structure can be divided into a metric learning head and an object detection head. The metric learning head uses a binary loss function based on feature distance, enabling the model to distinguish whether there are differences in images. The object detection head combines feature vectors with anchor boxes, enabling the model to detect locations where differences exist. The specific steps are as follows:

[0117] Step 51: Compared to computer vision tasks such as image classification and object detection, the goal of metric learning is to learn a mapping relationship. This relationship allows sample pairs of the same category but far apart in the high-dimensional feature space to cluster together after being mapped to the low-dimensional space; sample pairs of different categories but close together are dispersed in the low-dimensional space after mapping. This invention uses the contrastive loss optimization function as the metric learning head. This loss function can handle the relationship between a pair of features in a Siamese network, and its expression is shown in Equation 5:

[0118]

[0119] d=||f1-f2||2 (5)

[0120] Where d represents the Euclidean distance between the features f1 and f2 of the image to be judged and the reference image, y is the label of whether the sample pair matches. If they are similar or match, y is 1, otherwise y is 0. margin is a hyperparameter that represents the set distance threshold.

[0121] Step 52: The object detection head combines the feature vectors from layers P3-P6 of the network with the anchor boxes generated in step D.1, and optimizes them using the position loss function and the confidence loss function to enable the model to locate the differences between two images. The specific steps are as follows:

[0122] Step 521: The target detection feature vector extracted in Step 4 is transformed into a unified dimension through a 1*1 convolutional layer with nf convolutional kernels. This unified dimension is used for the calculation of anchor boxes, where nf is the product of the number of anchor boxes na in the feature layer and the dimension of the recognition output vector (5+nc), i.e., nf = na*(5+nc). In this invention, na and nc are 3 and 1 respectively. The additional 5-dimensional vector fixed in the output vector consists of a 4-dimensional representation of position information and a 1-dimensional indicator of the presence or absence of a target object. The tensors after unification in layers P3-P6 are transformed, and the dimensions of the final output feature vectors of each layer are as follows:

[0123] P3 layer: Input dimension: 80*80*256, Output dimension: 3*80*80*6;

[0124] P4 layer: Input dimension: 40*40*512, Output dimension: 3*40*40*6;

[0125] P5 layer: Input dimension: 20*20*768, Output dimension: 3*20*20*6;

[0126] P6 layer: Input dimension: 10*10*1024, Output dimension: 3*10*10*6;

[0127] Step 53: Perform Sigmoid normalization on the predicted output;

[0128] Step 54: Combining the normalized coordinates and anchor box information, deduce the position of the prediction box on the receptive field of the original image.

[0129] Step 55: The loss function used in this invention includes two parts: location loss and confidence loss. The loss function optimizes the distance between the predicted information and the expected information. The closer the predicted information is to the expected information, the smaller the loss function value. Then, through gradient backpropagation, the overall network parameters are updated. The specific form of the loss function is as follows: Generation of the prediction box mask matrix. At each point in the feature map, the network corresponds to three prediction boxes based on the anchor box size. For example, the P3 feature layer corresponds to 3*80*80 prediction boxes. However, not all prediction boxes need to have their loss function values ​​calculated. Therefore, this invention uses a mask matrix to control the calculation of the loss function value. The specific generation steps include: parsing the center coordinates and width / height values ​​of the annotation boxes from the annotation file and converting the annotation values ​​into target boxes suitable for a 640*640 image; calculating the coordinates (xg, yg) of the target box in the feature layer by scaling, and rounding down to obtain integer grid coordinates (x0, y0); based on the position (xg, yg), from (x0, ... Take one neighboring grid cell to the left, right, top, and bottom of position (x0, y0). For example, if (x0, y0) is in the top left corner of a grid cell, take the two grid cells to the left (x0-1, y0) and top (x0, y0-1). This means the target box is considered to be near the selected three points. Compare the feature layer anchor boxes and target box aspect ratios obtained from clustering in step D.1, and remove anchor boxes that do not meet the requirements. Assign values ​​to the mask matrices corresponding to the nine anchor boxes at positions (x0, y0), (x0-1, y0), and (x0, y0-1) according to the above filtering conditions. Repeat the above steps to judge all target boxes in an image and generate the predicted box mask matrix.

[0130] Step 551, Position Loss Function: This invention uses CIOU Loss to measure the position loss of the rectangular box. This function simultaneously considers the overlapping area, center point distance, and aspect ratio. For the predicted box A and the target box B, the calculation is shown in Equation 6:

[0131]

[0132]

[0133]

[0134] In the formula, ρ is the distance between the center points of boxes A and B, c is the diagonal length of the minimum enclosing rectangle of the two boxes, v is the aspect ratio similarity, and α is the influence factor of v. If the overlap area between the two boxes is large, the IOU is large, then α is larger, and thus the influence of v is larger; conversely, if the IOU is small, then α is smaller, and thus the influence of v is smaller. To achieve the goal of higher rectangular box overlap and a loss function closer to 0, the loss function in the optimization is as follows:

[0135] loss CIOU =1-CIOU (7)

[0136] Step 552, Confidence Loss: The model obtains a confidence value for each predicted box in the feature map, representing the credibility of the predicted box. This invention uses a binary cross-entropy loss function to optimize the confidence. When the predicted box mask matrix considers the predicted box to be usable, the CIOU value in Equation 6 is used as the cross-entropy label value; when it is unusable, the label value is 0. The confidence loss function is calculated, and the function expression is shown in Equation 8:

[0137]

[0138]

[0139] loss obj =α obj *I obj +(1-α obj )*I noobj (8)

[0140] In the formula, αobj is the confidence loss weight when the predicted box mask matrix is ​​True, and it is usually between 0.5 and 1, so that the network pays more attention to this situation during training. In this invention, αobj is 0.65.

[0141] The image discrimination task involves a single-class classification algorithm, which only needs to identify and locate the differences between two images. Therefore, classification loss is not used in this invention. For each feature layer from P3 to P6, lossCIOU and lossobj can be calculated, and the final loss function is shown in Equation 9.

[0142] loss CIOU =α1*loss CIOU80 +α2*loss CIOU40 +α3*loss CIOU20 +α4*loss CIOU10

[0143] loss obj =α1*loss obj80 +α2*loss obj40 +α3*loss obj20 +α4*loss obj10

[0144] loss = β CIOU *loss CIOU +β obj *loss obj +β metric *loss metric (9)

[0145] In the formula, α1 to α4 represent the weights of the P3-P6 feature layers in the loss function calculation, respectively. In this invention, they are taken as 0.3, 0.3, 0.2 and 0.2, respectively. βCIOU, βobj and βmetric represent the influence of position, confidence and metric learning on the overall loss function, respectively. In this invention, they are taken as 0.4, 0.4 and 0.2, respectively.

[0146] Step 56: The training parameters of the YOLOV5m algorithm described in this invention are as follows: Images are uniformly scaled to 640×640. Due to the differences between the task and the traditional object detection task, and the model size is moderate, this invention does not use a pre-trained model. The optimizer is SGD, the initial learning rate is 0.01, the momentum term is 0.937, the weight decay coefficient is 5×10-4, the batch training size is 16, and the training is carried out for a total of 100 rounds. The first 5 rounds are used for slow start, with a slow start coefficient of 0.1 and a momentum coefficient of 0.8. After the slow start ends, the learning rate decreases linearly from the 6th round until it decreases to 1 / 10 of the original learning rate in the 100th round.

[0147] Step 6: Application of the discriminant algorithm in reasoning

[0148] The discrimination algorithm model trained in step 5 is applied to a remote intelligent inspection scenario. It compares the differences between images captured during the inspection and reference images stored at the same preset location. If differences are found, the locations of the differences are highlighted and displayed on the front-end interface of the inspection system. The specific steps are as follows:

[0149] Step 61: Set preset positions for points that need image discrimination and save the normal reference image of the device to a fixed path on the server; when performing daily inspection tasks or calling preset position recognition, the system will take pictures of the current point; according to step 2, preprocess and align the captured image with the reference image.

[0150] Step 62: Load the model trained in Step 5;

[0151] Step 63: Use the loaded image to perform inference on the aligned and registered image to generate initial inference results;

[0152] Step 64: Filter out results with a confidence level less than a threshold. In this invention, the threshold is set to 0.3.

[0153] Step 65: Apply non-maximum suppression (NMS) to filter out overlapping prediction boxes and obtain the prediction result output_1_nms;

[0154] Step 66: Apply the existing substation equipment identification algorithm model to identify common equipment in the captured screenshot and obtain the identification result output_1_device of the captured image relative to the reference image. The equipment identification model architecture described in this invention is YOLOV5s, which can identify 15 types of common substation equipment such as main transformers, switchgear, breathers, oil tankers, and meters.

[0155] Step 67: Preprocess and align the reference image relative to the captured screenshot;

[0156] Step 68: Repeat steps 64-68 to obtain the recognition results output_2_nms and output_2_device of the reference image relative to the captured image;

[0157] Step 69: Post-process the four recognition outputs from the two images to generate a prediction bounding box from each captured image. The post-processing strategy specifically includes:

[0158] Step 691: Iterate through each prediction result o_nmsi output by the discriminant model output_1_nms;

[0159] Step 692: Calculate the overlap between o_nmsi and each prediction result o_devicei of the device identification model. If the overlap is greater than 0.6% of o_nmsi and greater than 0.4% of o_devicei, the identification result o_nmsi is considered to be clearly visible on the device body. Then, the device body is taken as the final identification result, that is, o_nmsi is replaced by the position of o_devicei.

[0160] Step 693: Repeat the above process for each record in output_1_nms and output_2_nms to generate the final prediction results output_1 and output_2;

[0161] Step 694: Integrate the model inference results of the images captured during the inspection and the reference image, and finally display only one prediction box in the inspection system result interface. The specific processing steps include: if neither image has output, it is considered that the captured image and the reference image are no different; if only one image has output, for the image with prediction results, integrate the output into a single prediction box that can contain all the results; if both images have output, merge the output results. The specific processing steps are as follows: map the results in output_2 to the reference image through inverse affine transformation to form the updated output result output_2; merge output_1 and output_2, and apply NMS technology to filter out duplicate prediction boxes; merge the remaining prediction boxes using the above method to form the final result; draw the prediction box onto the original image, and push the inference results to the inspection host through the interface agreed upon by the State Grid Corporation's technical specifications, and display it on the inspection system interface.

[0162] A substation defect discrimination system includes: a sample acquisition module for acquiring normal baseline data and abnormal defect data at the same location of key equipment, meters, entrances and exits, and important scenes within the substation, forming a pair of discrimination samples; a preprocessing module for performing image registration and alignment preprocessing on the acquired discrimination sample pairs, marking the locations of differences in the aligned sample pairs, generating XML tag files, and dividing the training set and validation set at a 4:1 ratio; a Siamese network structure construction module for extracting four different scale feature maps from the neural network for each pair of images as image feature representations based on a Siamese network model; a feature vector extraction module for performing feature engineering on the feature representations to extract feature vectors suitable for object detection and metric learning; a loss optimization module for designing branches for object detection and similarity measurement, inputting different types of features into the object detection and similarity measurement branches respectively, and achieving the optimization goals of improving the accuracy of prediction boxes and improving the feature space by designing loss functions for the two branches; and a post-processing module for model inference and logical post-processing, including formulating strategies for organizing, merging, and deleting prediction boxes, mining the relationship between prediction boxes and substation equipment, filling and expanding prediction boxes, and outputting the final results.

[0163] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0164] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0165] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0166] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0167] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0168] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for identifying defects in a substation, characterized in that: Includes the following steps: Normal baseline data and abnormal defect data at the same location are obtained from key equipment, meters, entrances and exits and important scenes in the substation to form a pair of discrimination samples; The twin network structure is constructed based on the twin network model, and feature maps of different scales are extracted from the neural network for each pair of images as image feature representations; Feature engineering is performed on the image feature representations to extract feature vectors suitable for object detection and metric learning. Feature engineering includes transforming and processing the feature representations of two images to generate feature vectors suitable for metric learning and object detection, specifically including the following steps: Based on the XML annotation file and feature map scaling ratio in image preprocessing, a mask matrix of the same size is generated for the feature map of the P3 module of the YOLOV5m model. The matrix element with the marked distinguishable regions has a value of 1, and the other positions have a value of 0. The mask matrix is ​​multiplied with the feature map pixel by pixel, and then input into a 3-layer 1*1 convolutional structure to fine-tune the feature vector space, finally generating a metric learning feature vector. The system employs a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN) structure. The FPN uses convolution and upsampling operations to pass strong semantic features from high-level layers down, achieving multi-scale feature fusion. The PAN is another bottom-up pyramid structure that passes localization features from lower layers layer by layer. The two-layer pyramid simultaneously possesses semantic and localization information. Stack the feature vectors of modules P3-P6 of the two images according to the channel dimension; Batch standardization is performed on the stacked feature vectors; Two convolutional layers with 1*1 kernels were used to fine-tune the feature vector space, and finally, the target detection feature vector was generated according to the image scaling ratio. Different types of features are input into the object detection and similarity measurement branches respectively. By designing loss functions for the two branches, the optimization goals of improving the accuracy of the predicted bounding box and improving the feature space are achieved. For model reasoning and logical post-processing, formulate strategies for organizing, merging, and deleting prediction boxes, explore the relationship between prediction boxes and substation equipment, fill and expand prediction boxes, and output the final results.

2. The substation defect identification method according to claim 1, characterized in that, The defect discrimination sample library is constructed by collecting and establishing a pair of discrimination samples consisting of normal baseline data and abnormal defect data. The data should be collected by taking photos of cameras and inspection robots deployed in the substation at fixed preset positions. The collection scenarios include key equipment, important entrances and exits, and the internal environment of the station. The target object should be photographed in the center, and key positions and pointers should be clearly visible. The baseline data and defect data should be photographed using the same collection equipment and the same preset points. The defect discrimination sample library should include at least 10,000 pairs of valid samples.

3. The substation defect identification method according to claim 2, characterized in that, Before constructing the twin network structure, image preprocessing is required. Image preprocessing includes image registration and alignment of the acquired discrimination sample pairs, marking the locations of differences in the aligned sample pairs, generating XML label files, and dividing the training and validation sets into a 4:1 ratio. Specifically, the preprocessing includes the following steps: Convert normal baseline image samples and abnormal defect image samples from RGB to grayscale images; Feature descriptors were extracted from the two grayscale images using the ORB algorithm. The similarity between feature descriptors is calculated using Hamming distance, and feature pairs that meet the matching threshold requirements are extracted. Based on the feature pairs extracted by the RANSAC algorithm, the affine transformation matrix is ​​calculated. By applying affine transformations to the original image, abnormal images can be registered and aligned with normal images.

4. The substation defect identification method according to claim 1, characterized in that, The annotation of the locations of differences in the aligned sample pairs includes, for scenarios involving abnormal equipment and personnel access, using the LabelImg tool to annotate the locations of differences between the two images after registration and alignment. The annotation area is a rectangle, and the annotation file is saved in the xml format specified by the PASCAL VOC dataset.

5. The substation defect identification method according to claim 1, characterized in that, The Siamese network consists of two identical first-order object detection models, YOLOv5m, and extracts image feature representations through the following steps: Based on the defect identification scenario, customized data augmentation and expansion strategies are implemented; two images use the exact same data augmentation strategies, including image size scaling, blur enhancement, median filtering, HSV color space enhancement, vertical flipping, and horizontal flipping. The enhanced image is input into the YOLOV5m model, and the feature maps of the four sub-modules P3-P6 are extracted as image feature representations. The scaling ratios of these four feature maps relative to the original image are 8, 16, 32, and 64, respectively.

6. The substation defect identification method according to claim 1, characterized in that, The calculation of the model loss function in the target detection and similarity measurement branch design includes the following steps: By calculating the binary loss function through metric learning feature vectors, the metric space is optimized so that sample pairs with no difference cluster together in the metric space, while sample pairs with difference are moved away from each other in the metric space. Generate anchor boxes for target bounding box detection from the target detection feature vectors; The positional relationship between the anchor box and the predicted box is optimized by using the cross-union ratio (CUI) through the positional loss function. The confidence loss function is used to assess the probability that a target exists at this grid location.

7. The substation defect identification method according to claim 1, characterized in that, The model inference includes the following steps: According to image preprocessing, images taken at the same location or the same preset position during the inspection process are registered and aligned with the stored normal images to form test sample pairs; Load the defect discrimination model based on twin structure; Defects are identified in the sample pairs to generate initial inference results.

8. The substation defect identification method according to claim 1, characterized in that, The logical post-processing includes the following steps: Based on the set intersection-union ratio and confidence threshold, the initial inference results are filtered out; Overlapping prediction boxes are filtered out using nonmaximum suppression techniques. Use a substation equipment identification algorithm model to identify common equipment in substations; Based on the predicted device location, and combined with the coordinate fusion method, each image will ultimately output only one predicted bounding box.

9. A substation defect identification system based on a twin detection model, characterized in that, include: The sample acquisition module is used to acquire normal baseline data and abnormal defect data at the same location of key equipment, meters, entrances and exits and important scenes in the substation, forming a pair of discrimination samples; The Siamese network structure construction module is used for constructing Siamese network structures. Based on the Siamese network structure model, feature maps of different scales are extracted from the neural network for each pair of images as image feature representations. The feature vector extraction module is used to perform feature engineering on the image feature representation, extracting feature vectors suitable for object detection and metric learning. Feature engineering includes transforming and processing the feature representations of two images to generate feature vectors suitable for metric learning and object detection, specifically including the following steps: Based on the XML annotation file and feature map scaling ratio in image preprocessing, a mask matrix of the same size is generated for the feature map of the P3 module of the YOLOV5m model. The matrix element with the marked distinguishable regions has a value of 1, and the other positions have a value of 0. The mask matrix is ​​multiplied with the feature map pixel by pixel, and then input into a 3-layer 1*1 convolutional structure to fine-tune the feature vector space, finally generating a metric learning feature vector. The system employs a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN) structure. The FPN uses convolution and upsampling operations to pass strong semantic features from high-level layers down, achieving multi-scale feature fusion. The PAN is another bottom-up pyramid structure that passes localization features from lower layers layer by layer. The two-layer pyramid simultaneously possesses semantic and localization information. Stack the feature vectors of modules P3-P6 of the two images according to the channel dimension; Batch standardization is performed on the stacked feature vectors; Two convolutional layers with 1*1 kernels were used to fine-tune the feature vector space, and finally, the target detection feature vector was generated according to the image scaling ratio. The loss optimization module is used to input different types of features into the object detection and similarity measurement branches respectively. By designing the loss functions of the two branches, the optimization goals of improving the accuracy of the prediction box and improving the feature space are achieved. The post-processing module is used for model reasoning and logical post-processing, formulating strategies for organizing, merging, and deleting prediction boxes, exploring the relationship between prediction boxes and substation equipment, filling and expanding prediction boxes, and outputting the final results.

Images