Power transformation equipment image data enhancement method and system and remote intelligent patrol system
By using the image data enhancement method of equipment component-defect master-slave relationship and significance detection function in the inspection of substation equipment, the problem of monotonous sample background and high construction cost in the prior art is solved, and a real scene is generated Defect samples improve the recognition effect of the defect recognition model.
Patent Information
- Application Number
- CN202410985944.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-12
AI Technical Summary
In the inspection of substation equipment, the manual construction of defect sample data has problems such as monotonous sample background, single defect form, and high construction cost, which makes it difficult for the training data set to accurately reflect the real scene, limiting the performance improvement of the defect recognition algorithm.
Using image data enhancement method based on device component-defect master-slave relationship and significance detection function, the significance detection function is constructed through gradients, defect area segmentation and semantic space construction are carried out, and partition adaptive affine transformation and pixel space conversion technology are combined. , generate defect sample images that match the real scene, and establish a semantic enhanced image sample library for substation equipment.
It realizes the diversity of image data, while retaining the semantic information of the substation equipment, effectively expanding the sample library, improving the recognition effect of the defect recognition model, and is suitable for scenarios where low-frequency defects and sample collection is difficult.
Smart Images

Figure CN118942088A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an image data enhancement method, in particular to a method and system for enhancing image data of substation equipment and a remote intelligent patrol system. Background Art
[0002] The safety of substation equipment is a key link in the operation of substation systems. In order to detect equipment failures in a timely manner, substations need to invest a lot of manpower and time to conduct manual inspections. However, in some high-risk areas that are difficult for manpower to reach, the defects of substation equipment that are difficult to find also pose great safety hazards, which brings huge challenges to the substation equipment protection system that relies on manual inspections.
[0003] With the development and application of artificial intelligence technology, especially the progress of deep learning in image processing and target detection, the main body of safety inspection of substation equipment is gradually shifting from manual to automated defect recognition algorithms relying on air-ground-ground intelligent inspection systems. These algorithms can quickly and accurately identify equipment defects and provide real-time feedback on possible problems through the inspection system, greatly improving the efficiency and accuracy of inspections and providing key support for the sustainable development of the substation industry.
[0004] However, the performance of defect recognition algorithms depends largely on the size and quality of training data. In order to increase the diversity of samples, artificial construction methods are usually used to amplify scarce category data. Although this method effectively increases the amount of defect sample data in these categories, it also has problems such as monotonous sample background, single defect form, and high construction cost. These limitations make it difficult for training data sets to accurately reflect real scenarios, thus greatly restricting the improvement of defect recognition algorithm performance. Summary of the invention
[0005] Purpose of the invention: The purpose of the present invention is to provide an image data enhancement method and system that can quickly and accurately generate a large number of defect samples that conform to real scenes in the substation field; at the same time, a remote intelligent inspection system for substations is provided.
[0006] Technical solution: The method for enhancing image data of substation equipment described in the present invention comprises the following steps:
[0007] Collect defective substation equipment images from the substation, and establish a defect image sample library for substation equipment based on the correspondence between substation equipment and defect categories;
[0008] Establish a defect recognition model for substation equipment, construct a saliency detection function based on the gradient of pixel points, use it to perceive image regions with strong correlation between defect categories, and construct a category semantic space;
[0009] According to the correspondence between the substation equipment and the defect and the category semantic space, the negative sample image in the substation equipment defect image sample library is segmented into defect areas to obtain a defect segmentation image;
[0010] A defect-free equipment operation image of a substation is obtained as a background, and the defect segmentation image is fused with the background to obtain an enhanced image that retains the semantic information of the substation equipment; and a substation equipment semantic enhanced image sample library is established.
[0011] Furthermore, before establishing the substation equipment defect image sample library, the method further includes screening the collected substation equipment images; the screening process includes:
[0012] By extracting the brightness channel of the substation equipment image in the HSV color space, the overexposed or underexposed substation equipment image is deleted;
[0013] By calculating the slope of the image power spectrum under discrete Fourier transform, the blurred substation equipment image is deleted;
[0014] The similarity of substation equipment images in units of days is calculated by using a structural similarity algorithm, and substation equipment images with similarity higher than a threshold are deleted;
[0015] Delete the substation equipment images that do not contain typical electrical equipment.
[0016] Furthermore, the step of establishing a defect image sample library of substation equipment according to the correspondence between the substation equipment and the defects includes:
[0017] Use annotation boxes to annotate substation equipment and defect categories, and construct a category co-occurrence matrix; select defect categories with correlation scores greater than 0.5 in the co-occurrence matrix, and establish a master-slave relationship between substation equipment components and defect categories, where each slave defect category corresponds to at least one master substation equipment component;
[0018] The method for constructing the co-occurrence matrix comprises the following steps:
[0019] (1) Initialize an m×m co-occurrence matrix M. For each value M in the matrix ab It represents the correlation of defect category a to defect category b, and the calculation formula is:
[0020]
[0021] Where Count a&b Indicates the statistical number of the correlation between the annotation boxes of defect categories a and b, Count b Indicates the number of times the annotation box of defect category b appears;
[0022] (2) For each substation equipment image I i∈D, traverse all box pairs (b ij ,b ik ) and update the values in the co-occurrence matrix:
[0023]
[0024] (3) After traversing all substation equipment images and their annotated boxes, the category co-occurrence matrix is constructed.
[0025] Furthermore, the YOLOv5 model is used to train the substation equipment defect image sample library to obtain the substation equipment defect recognition model.
[0026] Furthermore, in the substation defect recognition model, the gradient information of the image of the feature representation layer is extracted to construct a saliency detection function The saliency detection function score J(S,X,l) is the cumulative sum of the gradients on the region S, g ij (X, l) is the gradient value of the defect recognition model for substation equipment at pixel (i, j) for defect category l in substation equipment image X;
[0027] According to the saliency detection function score, the category semantic space is constructed, which specifically includes the following steps:
[0028] If there are defect categories and their corresponding image areas S in the substation equipment image l , then random sampling is performed in the substation equipment image, and the obtained sampling area S satisfies the condition
[0029] The saliency detection function score of the sampling area is calculated. If the score is greater than the threshold, the sampling area is considered to have a strong correlation with the defect category and is used as the category semantic space of the defect category. Otherwise, random sampling is performed again in the substation equipment image until the saliency detection function score is greater than the threshold or the sampling number reaches the upper limit.
[0030] Furthermore, the defect area segmentation is performed on the negative sample image in the defect image sample library of the substation equipment according to the correspondence between the substation equipment and the defect and the category semantic space to obtain the defect segmentation image, which includes:
[0031] According to the correspondence between the substation equipment and the defect category, the main substation equipment component region containing defects and having the largest area in the substation equipment image is selected as the segmentation region of the current substation equipment image;
[0032] If the current substation image does not include the primary substation corresponding to the defect category, the area corresponding to the category semantic space of the defect category is used as the segmentation area of the current substation image.
[0033] Furthermore, the acquiring of a defect-free equipment operation image of the substation as a background, fusing the defect segmented image with the background, and obtaining an enhanced image retaining semantic information of the substation equipment includes:
[0034] Randomly select three of Gaussian blur, color transformation, hue transformation, brightness transformation, Poisson noise, and image sharpening to perform mixed pixel space transformation on the device running image as the background to obtain the background image;
[0035] Performing affine transformation on the defect segmentation image to obtain a segmentation image;
[0036] The segmented image is fused with the background image to obtain an enhanced image.
[0037] The substation equipment image data enhancement system of the present invention comprises:
[0038] A defect image sample library establishment unit is used to collect images of substation equipment containing defects in the substation, and establish a substation equipment defect image sample library according to the correspondence between the substation equipment and the defect category;
[0039] The semantic space construction unit is used to establish a defect recognition model for substation equipment, construct a saliency detection function based on the gradient of pixel points, and is used to perceive image regions with strong correlation between defect categories and construct a category semantic space.
[0040] A segmented image extraction unit, configured to perform defect area segmentation on the negative sample image in the defect image sample library of the substation equipment according to the correspondence between the substation equipment and the defect and the category semantic space, so as to obtain a defect segmentation image;
[0041] The enhanced image sample library establishment unit is used to obtain a defect-free equipment operation image of the substation as a background, fuse the defect segmentation image with the background, obtain an enhanced image that retains the semantic information of the substation equipment, and establish a semantic enhanced image sample library of the substation equipment.
[0042] The remote intelligent patrol system described in the present invention is characterized in that a substation defect recognition enhancement model is obtained by training the substation equipment semantically enhanced image sample library, and the substation defect recognition enhancement model is deployed on a service host of the substation to obtain a remote intelligent patrol system.
[0043] Furthermore, the substation equipment defect identification enhancement model is a DETR model.
[0044] Beneficial effect: Compared with the prior art, the advantages of the present invention are: the present invention automatically enhances data based on the master-slave relationship of equipment components and defects and the significance detection function, and completely retains the semantic information of substation equipment. It innovatively constructs a significance detection function based on the semantic space of substation equipment categories, realizes the automatic extraction of image areas with strong correlation of defect categories, and realizes the automatic enhancement method of image data based on partition adaptive affine transformation technology and pixel space conversion technology, which increases the diversity of images while completely retaining the semantic information of substation equipment in the enhanced image. It is suitable for scenes with low defect occurrence frequency and difficult image sample collection, and can effectively expand the sample library and improve the model recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 The figure is a flow chart of the method for enhancing image data of substation equipment according to the present invention.
[0046] Figure 2 This is a structural diagram of a substation defect identification model according to an embodiment of the present invention.
[0047] Figure 3 Schematic diagram of pixel space conversion according to an embodiment of the present invention.
[0048] Figure 4 FIG. 4 is a schematic diagram of an enhanced image according to an embodiment of the present invention.
[0049] Figure 5 This is a structural diagram of the transformer substation defect identification enhancement model of the present invention. DETAILED DESCRIPTION
[0050] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0051] like Figure 1 As shown, the substation equipment image data enhancement method includes the following steps.
[0052] Step 1: Collect substation equipment images, including positive samples without defects and negative samples with defects, define the master-slave relationship between equipment components and defects, and establish a substation equipment defect image sample library.
[0053] Step 1.1: Use the intelligent acquisition equipment deployed in the substation to collect image data with the normal image of the substation equipment and its appearance defects as the main target, including images of the substation equipment taken by inspection drones, inspection robots, and fixed-point monitoring cameras, and name the images according to the format of "camera number_time_number of frames.jpg"; in order to improve the quality of the substation equipment defect image sample library, the collected image data is screened and processed, which specifically includes the following steps:
[0054] ① Convert each image from RGB color space to HSV color space, extract the V channel of the image, i.e. the brightness channel, and use the sliding window detection method to determine whether the image is overexposed or underexposed, and delete those images that are at risk of losing device detail features due to improper exposure. The formula is as follows:
[0055]
[0056] Where μ represents the average brightness of the image in a window of size s×s starting from the (i, j)th pixel of the image. When its value is greater than the overexposure threshold 233 or less than the underexposure threshold 53, it is the overbright slider N of the image. o Statistics and dark slider statistics N d Add one, if the two are equal to the total number of sliding blocks N t The ratio r o and r d If it is greater than 0.125, the image is considered to be overexposed or underexposed and will be screened out;
[0057] ② Calculate the slope of the image power spectrum under discrete Fourier transform to filter out low-quality blurred images caused by out-of-focus or jitter. The calculation process formula is expressed as follows:
[0058]
[0059] Where I(u,v) represents an image of size M×N after Fourier transformation to the frequency domain, where u=fcosθ and v=fsinθ represent the two-dimensional frequency of the image in polar coordinates. The following approximate representation is obtained by summing the power spectrum S in each θ direction:
[0060]
[0061] Where A is the amplitude proportional factor in each direction. By converting the above formula, the frequency domain component, that is, the power spectrum slope α, can be calculated. If it is greater than the threshold value 10, it is considered that the current image is not blurred. The formula is expressed as follows:
[0062]
[0063] ③Use the Structural Similarity (SSIM) matching algorithm to calculate the similarity of image samples acquired by the same camera on a daily basis, and filter image samples with similarity higher than a threshold of 0.8. The formula is as follows:
[0064] SSIM(x,y)=[l(x,y)] α [c(x,y)] β ·[s(x,y)] γ (5)
[0065] Specifically, SSIM includes three parts: brightness comparison [l(x,y)], contrast comparison [c(x,y)], and structure comparison [s(x,y)]. α, β, and γ represent the weights of the three parts in the calculation, which are all set to 1 in this embodiment. The calculation formulas of the three are:
[0066]
[0067] where μ x and μ y Represent the average brightness of images X and Y respectively. C1 is a constant used to stabilize the calculation and prevent the denominator from being zero.
[0068]
[0069] where σ x and σ y They represent the standard deviation of image X and Y, respectively, reflecting the change of contrast. C2 is also a constant used to stabilize the calculation.
[0070]
[0071] where σ xy Represents the covariance between two images, reflecting the similarity of image structure. The constant C3 is calculated as
[0072] ④Through manual screening, filter out image samples that do not contain typical electrical equipment.
[0073] Step 1.2: Use the minimum bounding rectangle to mark the substation equipment and equipment defect categories, and generate an XML format annotation file with the same prefix name as the image file in PasalVOC format.
[0074] Step 1.3: Use the program to automatically analyze the annotation files, count the annotation rectangles with high overlap, build a category co-occurrence matrix based on the intersection-over-union score, and build a substation equipment component-defect master-slave relationship table for the categories with correlation scores greater than 0.5 in the matrix combined with the expert experience of substation operation and maintenance personnel. The construction process is as follows:
[0075] ① Initialize an m×m co-occurrence matrix M. For each value M in the matrix ab It represents the correlation of defect category a to defect category b, and the calculation formula is:
[0076]
[0077] Where Count a&bIndicates the statistical number of the correlation between the annotation boxes of defect categories a and b, Count b Indicates the number of times the annotation box of defect category b appears;
[0078] ②For each image I i ∈D, traverse all box pairs (b ij ,b ik ) and update the values in the co-occurrence matrix:
[0079]
[0080] ③ After traversing all images and their annotated boxes, the category co-occurrence matrix is constructed. In this embodiment, 10 sets of master-slave relationships are constructed, and each slave defect can have multiple master devices. Some of the master-slave relationships are shown in Table 1:
[0081] Table 1 Substation equipment parts-defect master-slave level relationship table
[0082] Slave equipment defect category Main equipment components Abnormal meter readings Oil storage cabinet, riser, gas relay, cooler Oil stains on the surface of components Oil storage cabinet, riser, gas relay, cooler, bushing The meter dial is blurred and the casing is damaged Oil temperature gauge, oil level gauge, SF6 pressure gauge, leakage current meter Abnormal oil level observation window reading Transformer bushings, current transformers, expanders Abnormal oil level gauge reading Oil storage cabinet
[0083] Step 1.4: Divide the labeled image samples into training sets and test sets in a ratio of 4:1 to construct a sample library of defective images of substation equipment. During the construction process, the structural similarity matching algorithm described in step 1.3 is used to perform similarity verification on the randomly sampled test set images, and the samples with similarity scores higher than the threshold of 0.5 are put back into the training set to ensure the diversity of image scenes in the test set. The sample library of defective images of substation equipment constructed in this embodiment contains 18 types of substation equipment and 15 types of defects, totaling 20,000 image samples.
[0084] Step 2: Train the substation defect recognition model based on YOLOv5, and further build a saliency detection function based on the gradient of the model at the pixel point to realize the perception of the image area with strong correlation between defect categories, and then build the category semantic space. The specific implementation steps are as follows:
[0085] Step 2.1: Use the YOLOv5 model to train the substation defect image sample library described in step 1.4 to obtain a substation defect recognition model. In this embodiment, YOLOv5x is used, the initial learning rate is set to 5e-5, the optimizer is AdamW, and the optimizer momentum decay rate is set to (0.9, 0.99) for 100 rounds. The model structure is as follows Figure 2 As shown in the figure, it includes the backbone network module, feature fusion module, and detection head module, which play the role of feature extraction, feature fusion, and result prediction in the YOLOv5 model architecture. The specific details of the model are as follows:
[0086] 1) The backbone network module of the YOLOv5 model described in the present invention is used to extract high-level features of the input image. The backbone network uses the CSPDarknet53 (Cross Stage Partial Network) architecture, which can effectively reduce the amount of calculation and improve the feature expression ability of the model. First, the P1 layer is constructed by extracting features from the image through the convolution layer, and then the P2-P6 layers are constructed by extracting features layer by layer using convolution. In order to adapt to the equipment management relationship in the substation application scenario and enable the model to obtain better global feature extraction effects, the P6 layer is added on the basis of the original P1-P5 layers.
[0087] 2) The feature fusion module is used to fuse feature maps of different scales to facilitate the detection of targets of different sizes. The original target detection algorithm performs detection on the feature map output by the last convolution. In the case of a deep network, the underlying semantic information will be lost, resulting in a low recognition accuracy of small target objects. The present invention uses a path aggregation network (PAN) module to upsample the high-level features of P3-P6 layer by layer and fuse them with the low-level features through top-down and lateral connections, thereby generating a series of multi-scale feature maps with strong representation capabilities.
[0088] 3) The detection head module consists of multiple classification heads and regression heads, which perform object classification and bounding box regression based on the feature map output by PAN. The output of each detection head includes bounding box coordinates, confidence scores, and category probabilities.
[0089] 4) During model training, the non-maximum suppression (NMS) algorithm is used to post-process the prediction results to eliminate redundant detection boxes. The loss function combines classification loss, bounding box regression loss, and confidence loss.
[0090] Step 2.2: Based on the defect recognition model trained in step 2.1, Figure 2 As shown in the figure, the P3 layer of the feature fusion after the extraction result output module is used as the feature representation layer, and its gradient information is obtained to construct a saliency detection function to characterize the correlation between the image area and the defect category. The specific calculation formula is as follows:
[0091]
[0092] The saliency detection function score J(S,X,l) is the cumulative sum of the model gradients on region S, g ij (X, l) is the specific gradient value of the model at pixel (i, j) for defect category l in the substation image X.
[0093] Step 2.3: Construct the category semantic space based on the saliency detection function score, which specifically includes the following steps:
[0094] ① If there is a defect category l and its corresponding image area S in the substation equipment image X l , then random sampling is performed in the image, and the obtained sampling area S satisfies the condition
[0095] ② Calculate the saliency detection function score J(S,X,l) of the sampling area. If the score is greater than the threshold τ, it is considered that the sampling area S has a strong correlation with the defect category l and is used as the category semantic space of the defect category l.
[0096] ③ If the score is less than the threshold τ, repeat steps ① and ② until J(S,X,l)>τ is satisfied, or the number of sampling times reaches the upper limit. In this embodiment, the upper limit of the number of random sampling times is set to 10 times, and the threshold τ is 0.7.
[0097] Step 3: Segment the defect category-related areas in the image based on the image annotation information and the category semantic space constructed in step 2. The specific implementation steps are as follows:
[0098] Step 3.1: For each image in the initial sample library, according to the annotation file and the equipment component-defect master-slave level relationship table shown in Table 1, the main-level substation equipment component area containing defect l and the largest area in the current image is obtained as the segmentation area of the current image.
[0099] Step 3.2: If there is no main substation equipment corresponding to the defect category in the current image, use the method described in step 2.3 to construct an image area in the current image that has a strong correlation with category l, use it as the category semantic space and perform segmentation.
[0100] Step 3.3: Save the category semantic space image area identified in step 3.1 and step 3.2, and construct a category semantic space image library. The naming rule is "defect category name_xmin_ymin_xmax_ymax_statistics.jpg", where xmin, ymin, xmax, ymax are the coordinates of the adjacent rectangular box of the current defect annotation box in the category semantic space image, calculated from the original image annotation file. After manual screening and filtering out low-quality images, the category semantic space image library constructed in this embodiment contains 8,000 images of 15 categories of defects.
[0101] Step 4: Select a defect-free equipment operation image as the background image, use affine transformation technology and pixel space conversion technology to enhance and fuse the segmented area and background respectively, and retain the semantic information of the substation equipment in the enhanced image.
[0102] Step 4.1: Use the inspection drones, patrol robots, and fixed-point surveillance cameras deployed in the plant to capture a large number of images of substation equipment in normal operation to obtain a background image library of substation equipment.
[0103] Step 4.2: Filter out low-quality images using the data screening method described in step 1.2, and use the defect recognition model obtained in step 2.1 to infer the background images, filter out images in which the model inference has defects, and build a high-quality background image library, which in this embodiment contains 10,000 background image samples.
[0104] Step 4.3: Randomly select three of Gaussian blur, color transformation, hue transformation, brightness transformation, Poisson noise, and image sharpening for mixed pixel space transformation of the background image in the high-quality background image library. The process is as follows: Figure 3 shown.
[0105] Step 4.4: Perform random affine transformations such as random flipping and resizing on the segmented images. In this embodiment, the probability of horizontal flipping of each segmented image is set to 50%, and the image resizing range is set to 50% to 200% of the original image, with a maximum size not exceeding the background image size. Through these affine transformations, the segmented images become more diverse, which helps to improve the model robustness and generalization ability.
[0106] Step 4.5: For each segmented image obtained by affine transformation, fuse it with the randomly sampled background image, such as Figure 4 As shown, an enhanced image that retains the semantic information of the substation equipment is obtained.
[0107] Step 5: For the data in the substation equipment image sample library, repeat the automatic enhancement method from step 2 to step 4 to achieve sample expansion and establish a substation equipment semantic enhanced image sample library.
[0108] Step 5.1: Select the defect category to be enhanced from the substation equipment image sample library, and extract the original image and annotation data containing the category.
[0109] Step 5.2: Apply the defect image data automatic enhancement method of steps 2 to 4 to each image once to achieve sample expansion, enhance sample diversity, and establish a semantic enhanced image sample library for substation equipment. This embodiment includes 18 types of substation equipment and 15 types of defects, totaling 35,000 image samples.
[0110] The substation equipment image data enhancement system of the present invention comprises:
[0111] A defect image sample library establishment unit is used to collect images of substation equipment containing defects in the substation, and establish a substation equipment defect image sample library according to the correspondence between the substation equipment and the defect category;
[0112] The semantic space construction unit is used to establish a defect recognition model for substation equipment, construct a saliency detection function based on the gradient of pixel points, and is used to perceive image regions with strong correlation with defect categories and construct a category semantic space.
[0113] A segmented image extraction unit, configured to perform defect area segmentation on the negative sample image in the defect image sample library of the substation equipment according to the correspondence between the substation equipment and the defect and the category semantic space, so as to obtain a defect segmentation image;
[0114] The enhanced image sample library establishment unit is used to obtain a defect-free equipment operation image of the substation as a background, fuse the defect segmentation image with the background, obtain an enhanced image that retains the semantic information of the substation equipment, and establish a semantic enhanced image sample library of the substation equipment.
[0115] The remote intelligent patrol system described in the present invention uses the substation semantically enhanced image sample library to train a substation defect recognition enhancement model, and deploys the substation defect recognition enhancement model on a service host of a substation to obtain a remote intelligent patrol system.
[0116] The DETR framework is used to train the enhanced model for defect identification of substation equipment. In this embodiment, the initial learning rate is set to 1e-3, the optimizer is AdamW, and the training is performed for 36 rounds under the hyperparameter setting of the optimizer momentum decay rate (0.9, 0.99). The model structure is as follows Figure 5 As shown in the figure, it mainly includes the backbone network module, feature encoder module, feature decoder, and prediction head module, which play the role of feature extraction, feature enhancement, feature query, and result prediction in the DETR model architecture. The specific details of the model are as follows:
[0117] The backbone network module is used to extract high-level features of the input image. The present invention uses the ViT (Vision Transformer) network to extract image features. The network consists of multiple Transformer encoder layers, each of which contains a self-attention mechanism and a feedforward neural network. ViT effectively captures the global information of the image by dividing the input image into fixed-size image blocks, then flattening each image block and embedding it into a high-dimensional feature space for processing.
[0118] The feature encoder module is mainly composed of a series of multi-head self-attention layers and feedforward neural network layers. These layers are stacked together to further process the feature map output by the backbone network to extract higher-level features. Each self-attention layer can capture the relationship between different positions in the feature map, and the feedforward neural network performs nonlinear transformations on these relationships to enhance the feature representation capability.
[0119] The feature decoder module contains a set of learnable query vectors, which are processed layer by layer through the decoder's multi-head self-attention mechanism on the feature maps output by the encoder. Each decoder layer also contains a cross-attention mechanism, which allows the query vector to focus on the important parts of the feature map. Through this process, the decoder generates a set of feature representations, the number of which is consistent with the number of query vectors.
[0120] The prediction head module consists of two parallel feed-forward neural networks, one for predicting the category of the object (classification head) and the other for predicting the bounding box of the object (regression head).
[0121] During the model training process, the Hungarian matching algorithm was used to match the target and the prediction. At the same time, the equipment-component-defect training strategy was adopted, and semantic information was introduced to enhance the model's understanding ability, so that the model can more accurately identify and classify defects on various equipment and their components, improve the detection rate while reducing the false alarm rate, thereby showing higher reliability and accuracy in practical applications.
[0122] The trained model file is converted into TensorRT format and deployed in the substation remote intelligent patrol system. In this embodiment, the input image size of the converted TensorRT format model is 640x640, the inference batch size is 1, and int8 precision quantization inference is used. The substation remote intelligent patrol system analysis host configuration is shown in Table 2:
[0123] Table 2 Configuration table of analysis host of remote intelligent patrol system of substation
[0124] CPU No less than 2GHz main frequency, no less than 8 cores Memory No less than 64G Inference graphics card Configure an independent GPU accelerator card, AI computing power is not less than 64TOPSINT8 power supply Dual hot-swappable redundant power supplies storage System data and operation log data storage time ≥ 1 year
Claims
1. A method for enhancing image data of substation equipment. It is characterized in that The steps include: Collect defective substation equipment images from the substation, and establish a defect image sample library for substation equipment based on the correspondence between substation equipment and defect categories; Establish a defect recognition model for substation equipment, construct a saliency detection function based on the gradient of pixel points, use it to perceive image regions with strong correlation between defect categories, and construct a category semantic space; According to the correspondence between the substation equipment and the defect and the category semantic space, the negative sample image in the substation equipment defect image sample library is segmented to obtain a defect segmentation image; A defect-free equipment operation image of a substation is obtained as a background, and the defect segmentation image is fused with the background to obtain an enhanced image retaining the semantic information of the substation equipment; and a substation equipment semantic enhanced image sample library is established.
2. The method for enhancing image data of substation equipment according to claim 1, It is characterized in that Before establishing the substation equipment defect image sample library, the method further includes screening the collected substation equipment images; the screening process includes: By extracting the brightness channel of the substation equipment image in the HSV color space, the overexposed or underexposed substation equipment image is deleted; By calculating the slope of the image power spectrum under discrete Fourier transform, the blurred substation equipment image is deleted; The similarity of substation equipment images in units of days is calculated by using a structural similarity algorithm, and substation equipment images with similarity higher than a threshold are deleted; Delete the substation equipment images that do not contain typical electrical equipment.
3. The method for enhancing image data of substation equipment according to claim 1, It is characterized in that The step of establishing a defect image sample library of substation equipment according to the correspondence between the substation equipment and the defects includes: Use annotation boxes to annotate substation equipment and defect categories, and construct a category co-occurrence matrix; select defect categories with correlation scores greater than 0.5 in the co-occurrence matrix, and establish a master-slave relationship between substation equipment components and defect categories, where each slave defect category corresponds to at least one master substation equipment component; The method for constructing the co-occurrence matrix comprises the following steps: (1) Initialize an m×m co-occurrence matrix M. For each value M in the matrix ab It represents the correlation of defect category a to defect category b, and the calculation formula is: Where Count a&b Indicates the statistical number of the correlation between the annotation boxes of defect categories a and b, Count b Indicates the number of times the annotation box of defect category b appears; (2) For each substation equipment image I i ∈D, traverse all box pairs (b ij ,b ik ) and update the values in the co-occurrence matrix: (3) After traversing all substation equipment images and their annotated boxes, the category co-occurrence matrix is constructed.
4. The method for enhancing image data of substation equipment according to claim 1, It is characterized in that The YOLOv5 model is used to train the substation equipment defect image sample library to obtain the substation equipment defect recognition model.
5. The method for enhancing image data of substation equipment according to claim 4, It is characterized in that In the substation defect recognition model, the gradient information of the image of the feature representation layer is extracted to construct a saliency detection function. The saliency detection function score J(S,X,l) is the cumulative sum of the gradients on the region S, g ij (X, l) is the gradient value of the defect recognition model for substation equipment at pixel (i, j) for defect category l in substation equipment image X; According to the saliency detection function score, the category semantic space is constructed, which specifically includes the following steps: If there are defect categories and their corresponding image areas S in the substation equipment image l , then random sampling is performed in the substation equipment image, and the obtained sampling area S satisfies the condition The saliency detection function score of the sampling area is calculated. If the score is greater than the threshold, the sampling area is considered to have a strong correlation with the defect category and is used as the category semantic space of the defect category. Otherwise, random sampling is performed again in the substation equipment image until the saliency detection function score is greater than the threshold or the sampling number reaches the upper limit.
6. The method for enhancing image data of substation equipment according to claim 3, It is characterized in that According to the correspondence between the substation equipment and the defect and the category semantic space, the defect area segmentation is performed on the negative sample image in the substation equipment defect image sample library to obtain the defect segmentation image, which includes: According to the correspondence between the substation equipment and the defect category, the main substation equipment component region containing defects and having the largest area in the substation equipment image is selected as the segmentation region of the current substation equipment image; If the current substation image does not include the primary substation corresponding to the defect category, the area corresponding to the category semantic space of the defect category is used as the segmentation area of the current substation image.
7. The method for enhancing image data of substation equipment according to claim 1, It is characterized in that The step of obtaining a defect-free equipment operation image of the substation as a background, fusing the defect segmented image with the background, and obtaining an enhanced image retaining the semantic information of the substation equipment comprises: The device running image as the background is randomly selected to perform mixed pixel space transformation among three of Gaussian blur, color transformation, hue transformation, brightness transformation, Poisson noise, and image sharpening to obtain a background image; Performing affine transformation on the defect segmentation image to obtain a segmentation image; The segmented image is fused with the background image to obtain an enhanced image.
8. A substation equipment image data enhancement system, It is characterized in that include: A defect image sample library establishment unit is used to collect images of substation equipment containing defects in the substation, and establish a substation equipment defect image sample library according to the correspondence between the substation equipment and the defect category; The semantic space construction unit is used to establish a defect recognition model for substation equipment, construct a saliency detection function based on the gradient of pixel points, and is used to perceive image regions with strong correlation with defect categories and construct a category semantic space. A segmented image extraction unit, configured to perform defect area segmentation on the negative sample image in the defect image sample library of the substation equipment according to the correspondence between the substation equipment and the defect and the category semantic space, so as to obtain a defect segmentation image; The enhanced image sample library establishment unit is used to obtain a defect-free equipment operation image of the substation as a background, fuse the defect segmentation image with the background, obtain an enhanced image that retains the semantic information of the substation equipment, and establish a semantic enhanced image sample library of the substation equipment.
9. A remote intelligent patrol system based on the method according to any one of claims 1 to 7, It is characterized in that The substation equipment semantic enhanced image sample library is used to train a substation equipment defect recognition enhancement model, and the substation equipment defect recognition enhancement model is deployed on a service host of a substation to obtain a remote intelligent patrol system.
10. The remote intelligent patrol system according to claim 9, It is characterized in that The substation equipment defect identification enhancement model is a DETR model.
Citation Information
Patent Citations
Material surface defect detection method, device and equipment and storage medium
CN110473197A
Power distribution equipment infrared image fault identification method and system
CN113688947A
Glass panel surface defect detection method based on small sample learning
CN114092389A
Substation complex scene defect identification method and system based on multistage labeling
CN116363053A
Small sample image classification optimization method based on supervised comparative learning and multi-task setting
CN116524242A
Cited By
Method and device for generating power transformation defect image
CN120070304A
