Method for detecting deformation and damage of grading ring based on semantic segmentation

Abstract

The invention provides a semantic segmentation-based grading ring deformation and damage detection method, which adopts a method of combining manual annotation and semantic segmentation, and accurately and efficiently identifies whether various grading ring devices in a transformer substation are deformed and damaged or not by acquiring camera images and automatically detecting. The accurate state of the grading ring is fed back to a worker, so that the operation reliability of a transformer substation and the safety of the worker are improved; the method comprises a training stage and an application stage. The training stage is realized through data acquisition, marking of grading ring detection data, marking of semantic segmentation data, construction of a semantic segmentation data set, construction of a semantic neural network and training, and construction of a binary classification neural network and training; in the application stage, a to-be-recognized grading ring image is collected through an installed camera, the to-be-recognized grading ring image is input into the trained semantic segmentation network and the binary classification neural network to recognize the operation state of the grading ring, whether the grading ring is damaged or not is detected, and safe operation of a power grid is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of semantic segmentation detection, and in particular to a method for detecting deformation and damage of an equalizing ring based on semantic segmentation. Background Art

[0002] The voltage grading ring is a ring-shaped metal device that can improve the voltage distribution of high-voltage power equipment. Its main function is to make the voltage more evenly distributed, ensure that there is no potential difference between different parts of the ring structure, and avoid breakdown and damage to equipment due to local high voltage. Therefore, the voltage grading ring plays an important role in ensuring the smooth operation of the power system. After the voltage grading ring is put into actual operation, it is affected by various environmental factors, which will cause damage and lose its function, and it needs to be replaced in time. At present, there is a lack of effective detection methods for voltage grading rings on site, and manpower is needed to detect whether the voltage grading ring is damaged or deformed. It is inevitable that there will be negligence and omissions, and the accuracy is difficult to guarantee.

[0003] Most of the existing methods based on image analysis use target detection algorithms to directly detect damaged and deformed equalizing rings by training target detection neural networks. Since the deformation and damage of equalizing rings are serious accidents, there are few real equalizing ring deformation samples available for training, and the environment in which the equalizing rings are located is complex. It is difficult to directly train the neural network for target detection, and the false alarm rate is very high. For example, the patent "Equalizing Ring Defect Detection Method, Device, Computer Equipment and Storage Medium" patent number CN117611538A, its main steps are: 1. Acquire the image of the area to be detected; 2. Use the target detection algorithm to detect the end insulator of the image; 3. If the insulator is detected, use the target detection algorithm for the area near the insulator Detect defective grading rings; its disadvantages include first detecting insulators and then detecting defective grading rings near insulators, in order to reduce false detections, assuming that grading rings only appear near insulators; but this approach wastes the ability of advanced neural network algorithms to weight global information. By only training the target detection network for grading rings, the network can learn the connection between grading rings and insulators, and the first step is not necessary; on the other hand, first detecting insulators as a necessary condition for grading rings reduces the flexibility of the algorithm, and is not applicable to grading rings on other devices except insulators. Even if the recognition function is to be added, it is also necessary to add detection of other devices in the first step, which increases the complexity of the algorithm;

[0004] Another example is the patent "Method, device, medium and equipment for detecting defects in equalizing rings based on aerial images"; patent number CN111563893B, the steps are: 1. Collect and annotate image samples of equalizing ring defects to construct a training data set; 2. Use random noise enhancement algorithm, filter enhancement algorithm and color enhancement algorithm to expand the training data set; 3. Train a faster Detection algorithm of equalizing ring defects using rcnn neural network; 4. Apply the trained algorithm to the image to be detected to detect the target frame of the equalizing ring defect; its disadvantages are: using the target detection neural network to directly detect the deformed and damaged equalizing ring, the detection target is a rectangular frame. Due to the large number of power system equipment, the rectangular frame often contains other equipment components. As a coarse-grained target, the rectangular frame is easily disturbed by these equipment components, causing the neural network to learn wrong features and reduce the detection effect; in particular, the number of samples of equalizing ring defects itself is usually small, and the training effect is even worse; the scheme recognizes that the number of samples of equalizing ring defects is small, so a data enhancement method is proposed, which uses a mixture of random noise enhancement algorithm, filtering enhancement algorithm and color enhancement algorithm for training data; however, these are all overall fine-tuning of the image, and do not increase the diversity of the equalizing ring defects themselves, and the effect is limited. For this reason, the applicant proposes a detection method for deformed and damaged equalizing rings based on semantic segmentation according to the difficulty of equalizing ring detection. By collecting camera images, deformed and damaged equalizing rings in substations can be accurately and efficiently identified. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a method for detecting deformation and damage of equalizing rings based on semantic segmentation. By adopting a method combining manual labeling and semantic segmentation, through collecting camera images and automatic detection, it is possible to accurately and efficiently identify whether various types of equalizing ring equipment in the substation are deformed or damaged, and feed back the accurate equalizing ring status to the staff, thereby improving the reliability of substation operation and the safety of the staff.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for detecting deformation and damage of a grading ring based on semantic segmentation, characterized in that: the method for detecting deformation and damage of a grading ring based on semantic segmentation adopts a method combining manual annotation and semantic segmentation, collects camera images, automatically detects, accurately and efficiently identifies whether various grading ring devices in a substation are deformed or damaged, and feeds back accurate grading ring status to staff; it includes two stages, training and application, specifically:

[0008] The steps of the training phase are:

[0009] Step 1: Data collection: Collect pictures of normal pressure equalizing rings and deformed and damaged pressure equalizing rings;

[0010] Step 2: Label the pressure equalizing ring detection data; manually label the data points in the collected normal and damaged photos of the pressure equalizing ring; manually label the normal pressure equalizing ring and the deformed and damaged pressure equalizing ring in the picture, and the labeling format is the coordinates of the circumscribed rectangular frame of the entire pressure equalizing ring and the binary classification label of the pressure equalizing ring (normal / deformed);

[0011] Step 3: Label the semantic segmentation data; manually label the semantic segmentation labels of the manually labeled pressure equalizing ring photos; the semantic segmentation label refers to the mask of the pressure equalizing ring, that is, a binary image with the same size as the original image, in which each pixel has a value of 0 indicating that the pixel is not on the pressure equalizing ring, and a value of 1 indicating that the pixel is on the pressure equalizing ring, and the mask is denoted by M;

[0012] Step 4: Multiply the image with the semantic segmentation label and the mask M by the matrix elements to obtain the semantic segmentation sub-image of the pressure equalization ring, and assign the binary classification label of each pressure equalization ring to the corresponding semantic segmentation sub-image to form a semantic segmentation dataset;

[0013] Step 5: Construct and train a semantic segmentation neural network. Construct a semantic segmentation neural network and train the semantic segmentation neural network using the stochastic gradient descent method on the images labeled with the pressure equalizing ring detection data and the images labeled with the semantic segmentation data.

[0014] Step 6: Build a binary classification neural network and train it; Build a binary classification neural network and use the stochastic gradient descent method to train the binary classification neural network with the data in the semantic segmentation dataset;

[0015] The application phase steps are:

[0016] Step 1: Use a camera to capture an image of the pressure equalizing ring to be identified;

[0017] Step 2: scaling the collected image of the pressure equalizing ring to be identified and inputting it into the trained semantic segmentation neural network for identification, and obtaining the output result of the semantic segmentation neural network;

[0018] Step 3: Use NMS and threshold filtering to calculate the bounding box of the pressure equalizing ring to be identified based on the image output result processed by the semantic segmentation neural network;

[0019] Step 4: Calculate and obtain the semantic segmentation image of the pressure equalizing ring according to the semantic segmentation mask matrix output by the image processed by the semantic segmentation neural network;

[0020] Step 5: Based on the semantic segmentation image of the pressure equalizing ring, the sub-image where the pressure equalizing ring bounding box in step 3 is located is extracted;

[0021] Step 6: Input the sub-image in step 5 into the trained binary classification neural network, calculate and analyze to determine whether the pressure equalizing ring to be identified is damaged or deformed, and send out corresponding information.

[0022] Furthermore, the characteristics of the semantic segmentation neural network in the training phase of the pressure equalizing ring deformation and damage detection method based on semantic segmentation are:

[0023] 1) It is a multi-layer neural network including convolutional layers, pooling layers, and fully connected layers;

[0024] 2) The network input is n color images, n is greater than or equal to 1, and the size of the n images is the same, all H*W;

[0025] 3) The network contains an intermediate output, which is a A four-dimensional matrix, m is a power of 2; the matrix then passes through multiple convolutional layers, pooling layers, and fully connected layers, and finally outputs the aforementioned matrices The detection result of the pressure ring in the corresponding position of the input image in proportion to the dimension, including the confidence and bounding box coordinates, and the size is The voltage-equalizing ring segmentation mask.

[0026] Furthermore, the stochastic gradient descent method in the training phase of the semantic segmentation-based pressure equalizing ring deformation and damage detection method trains the semantic segmentation neural network by using the stochastic gradient descent method to optimize the Euclidean loss function:

[0027]

[0028] Where: y is the size of The neural network output matrix;

[0029] is to scale the annotated mask to The matrix of .

[0030] Furthermore, the characteristics of the binary classification neural network in the training phase of the semantic segmentation-based pressure equalizing ring deformation and damage detection method are:

[0031] 1) It is a multi-layer neural network including convolutional layers, pooling layers, and fully connected layers;

[0032] 2) The network input is n color images, n is greater than or equal to 1, and the n images have the same size;

[0033] 3) The output of the network is an n*2 matrix, which indicates the probability that the n images are normal pressure equalizing rings and pressure equalizing rings that are deformed and damaged.

[0034] According to the method for detecting deformation and damage of a pressure equalizing ring based on semantic segmentation according to claim 1, it is characterized in that: the method for detecting deformation and damage of a pressure equalizing ring based on semantic segmentation adopts a stochastic gradient descent method to train a binary segmentation neural network in the training phase, and uses a stochastic gradient descent method to optimize a cross entropy loss function.

[0035]

[0036] The benefits of this application are:

[0037] 1. The method for detecting deformation and damage of equalizing rings based on semantic segmentation automatically identifies deformed and damaged equalizing rings by analyzing the images collected by the camera, thus reducing the cost of manual inspection;

[0038] 2. The detection method of deformation and damage of the equalizing ring based on semantic segmentation first obtains the semantic segmentation image of the equalizing ring, and then judges the semantic segmentation image to identify whether it is deformed. It eliminates the interference of other objects in the image and only retains the image information of the equalizing ring itself, which can obtain a higher recognition accuracy rate;

[0039] 3. The detection method of deformation and damage of equalizing rings based on semantic segmentation solves the problem that during the training stage of the model, there are usually few training samples of deformation and damage of equalizing rings. Segmenting the equalizing rings first can make it more convenient to produce artificially synthesized training samples, and can synthesize samples more finely, which can further improve the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the flow chart of the training phase of the present invention;

[0041] Figure 2 It is a schematic diagram of the flow chart of the application stage of the present invention;

[0042] Figure 3 It is a schematic diagram for demonstrating the algorithm of the present invention;

[0043] Figure 4 Schematic diagram of the semantic segmentation neural network principle of the present invention. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0045] like Figure 1-3 As shown, a method for detecting deformation and damage of grading rings based on semantic segmentation is shown. The method for detecting deformation and damage of grading rings based on semantic segmentation adopts a method combining manual annotation and semantic segmentation. By collecting camera images, automatic detection, accurate and efficient identification of whether various grading ring equipment in the substation is deformed or damaged is performed, and accurate grading ring status is fed back to the staff; it includes two stages: training and application, specifically:

[0046] like Figure 1 As shown, the training phase steps are:

[0047] Step 1: Data collection: collect pictures of normal pressure equalizing rings and deformed and damaged pressure equalizing rings; ensure that the number of collected images is sufficient, and the number of normal pressure equalizing rings and deformed and damaged pressure equalizing rings is relatively balanced;

[0048] Step 2: Label the pressure equalizing ring detection data; manually label the data points in the collected normal and damaged photos of the pressure equalizing ring; manually label the normal pressure equalizing ring and the deformed and damaged pressure equalizing ring in the picture, and the labeling format is the coordinates of the circumscribed rectangular frame of the entire pressure equalizing ring and the binary classification label of the pressure equalizing ring (normal / deformed);

[0049] Step 3: Label the semantic segmentation data; manually label the semantic segmentation labels of the manually labeled pressure equalizing ring photos; the semantic segmentation label refers to the mask of the pressure equalizing ring, that is, a binary image with the same size as the original image, in which each pixel has a value of 0 indicating that the pixel is not on the pressure equalizing ring, and a value of 1 indicating that the pixel is on the pressure equalizing ring, and the mask is denoted by M;

[0050] Step 4: Multiply the image with the semantic segmentation label and the mask M by the matrix elements to obtain the semantic segmentation sub-image of the pressure equalization ring, and assign the binary classification label of each pressure equalization ring to the corresponding semantic segmentation sub-image to form a semantic segmentation dataset;

[0051] Step 5: Construct and train a semantic segmentation neural network. Construct a semantic segmentation neural network and train the semantic segmentation neural network using the stochastic gradient descent method on the images labeled with the pressure equalizing ring detection data and the images labeled with the semantic segmentation data.

[0052] like Figure 4 As shown, the schematic diagram of the principle of semantic segmentation neural network is shown, and its characteristics are:

[0053] 1) It is a multi-layer neural network including convolutional layers, pooling layers, and fully connected layers;

[0054] 2) The network input is n color images, n is greater than or equal to 1, and the size of the n images is the same, all H*W;

[0055] 3) The network contains an intermediate output, which is a A four-dimensional matrix, m is a power of 2; the matrix then passes through multiple convolutional layers, pooling layers, and fully connected layers, and finally outputs the aforementioned matrices The detection result of the pressure ring in the corresponding position of the input image in proportion to the dimension, including the confidence and bounding box coordinates, and the size is The pressure equalizing ring segmentation mask;

[0056] The stochastic gradient descent method shown in the figure trains the semantic segmentation neural network by optimizing the Euclidean loss function using the stochastic gradient descent method:

[0057]

[0058] Where: y is the size of The neural network output matrix;

[0059] is to scale the annotated mask to Matrix of

[0060] Step 6: Build a binary classification neural network and train it; Build a binary classification neural network and use the stochastic gradient descent method to train the binary classification neural network with the data in the semantic segmentation dataset;

[0061] The properties of the binary classification neural network shown are:

[0062] 1) It is a multi-layer neural network including convolutional layers, pooling layers, and fully connected layers;

[0063] 2) The network input is n color images, n is greater than or equal to 1, and the n images have the same size;

[0064] 3) The output of the network is an n*2 matrix, which indicates that the n images are the probabilities of a normal pressure equalizing ring and a deformed and damaged pressure equalizing ring;

[0065] The stochastic gradient descent method shown in the figure trains the binary segmentation neural network to optimize the cross entropy loss function using the stochastic gradient descent method.

[0066]

[0067] like Figure 2 As shown, the application phase steps are:

[0068] Step 1: Use a camera to capture the image of the pressure equalizing ring to be identified, and scale the image to a size of H*W;

[0069] Step 2: The collected pressure equalizing ring image to be identified is scaled and input into the trained semantic segmentation neural network for identification, and the output result of the semantic segmentation neural network is obtained; the output is divided into two parts: 1) the detection result of the pressure equalizing ring, the format is [x min ,y min , x max ,y max , score], which represent the four coordinates and confidence of the pressure equalization ring bounding box respectively. 2) The semantic segmentation mask matrix of the pressure equalization ring, with a value range of [0, 1];

[0070] Step 3: For the output result of the image processed by the semantic segmentation neural network, use NMS and threshold filtering calculations to obtain the bounding box of the grading ring to be recognized; when performing binarization operations according to a certain threshold, take the threshold T, where 0 < T < 1. For each value v in the mask matrix, let ), to obtain a grading ring mask with clear edges; and perform an element-wise product of this mask and the input image to obtain the semantic segmentation image of the grading ring;

[0071] Step 4: Calculate and obtain the semantic segmentation image of the grading ring according to the semantic segmentation mask matrix output by the image processed by the semantic segmentation neural network;

[0072] Step 5: According to the semantic segmentation image of the grading ring, extract the sub-image where the bounding box of the grading ring in Step 3 is located;

[0073] Step 6: Input the sub-image in Step 5 into the trained binary classification neural network, calculate and analyze to determine whether the grading ring to be recognized is damaged or deformed. When judging, based on whether the probability of its deformation and damage is greater than a certain threshold, determine whether the grading ring to be recognized belongs to deformation and damage, and send corresponding information.

[0074] As Figure 3 shown, the figure shows the algorithm demonstration process, which is fast and accurate in recognition, automatically identifies the grading rings with deformation and damage, and reduces the cost of manual inspection; first obtain the semantic segmentation picture of the grading ring, and then judge the semantic segmentation picture to identify whether there is deformation, excluding the interference of other objects in the picture, and only retaining the image information of the grading ring itself, which can obtain a higher recognition accuracy.

[0075] The above is only a preferred embodiment of the present invention, and it is not a limitation of the present invention in any other form. Any modification or equivalent change made according to the technical essence of the present invention still belongs to the scope protected by the present invention.

Claims

1. A method for detecting deformation and damage of a pressure equalizing ring based on semantic segmentation, characterized in that: The method for detecting deformation and damage of grading rings based on semantic segmentation adopts a method combining manual annotation and semantic segmentation. By collecting camera images, automatic detection, accurate and efficient identification of whether various grading ring devices in the substation are deformed or damaged is performed, and accurate grading ring status is fed back to the staff. The method includes two stages: training and application, specifically: The steps of the training phase are: Step 1: Data collection: Collect pictures of normal pressure equalizing rings and deformed and damaged pressure equalizing rings; Step 2: Label the pressure equalizing ring test data; manually label the data points in the collected normal and damaged photos of the pressure equalizing ring; Manually annotate the normal pressure equalizing ring and the deformed and damaged pressure equalizing ring in the image. The annotation format is the coordinates of the circumscribed rectangular frame of the entire pressure equalizing ring and the binary classification label of the pressure equalizing ring (normal / deformed); Step 3: Label the semantic segmentation data; manually label the semantic segmentation labels of the manually labeled pressure equalizing ring photos; the semantic segmentation label refers to the mask of the pressure equalizing ring, that is, a binary image with the same size as the original image, in which each pixel has a value of 0 indicating that the pixel is not on the pressure equalizing ring, and a value of 1 indicating that the pixel is on the pressure equalizing ring, and the mask is denoted by M; Step 4: Multiply the image with the semantic segmentation label and the mask M by the matrix elements to obtain the semantic segmentation sub-image of the pressure equalization ring, and assign the binary classification label of each pressure equalization ring to the corresponding semantic segmentation sub-image to form a semantic segmentation dataset; Step 5: Construct and train a semantic segmentation neural network. Construct a semantic segmentation neural network and train the semantic segmentation neural network using the stochastic gradient descent method on the images labeled with the pressure equalizing ring detection data and the images labeled with the semantic segmentation data. Step 6: Build a binary classification neural network and train it; Build a binary classification neural network and use the stochastic gradient descent method to train the binary classification neural network with the data in the semantic segmentation dataset; The application phase steps are: Step 1: Use a camera to capture an image of the pressure equalizing ring to be identified; Step 2: scaling the collected image of the pressure equalizing ring to be identified and inputting it into the trained semantic segmentation neural network for identification, and obtaining the output result of the semantic segmentation neural network; Step 3: Use NMS and threshold filtering to calculate the bounding box of the pressure equalizing ring to be identified based on the image output result processed by the semantic segmentation neural network; Step 4: Calculate and obtain the semantic segmentation image of the pressure equalizing ring according to the semantic segmentation mask matrix output by the image processed by the semantic segmentation neural network; Step 5: Based on the semantic segmentation image of the pressure equalizing ring, the sub-image where the pressure equalizing ring bounding box in step 3 is located is extracted; Step 6: Input the sub-image in step 5 into the trained binary classification neural network, calculate and analyze to determine whether the pressure equalizing ring to be identified is damaged or deformed, and send out corresponding information.

2. According to the method for detecting deformation and damage of a pressure equalizing ring based on semantic segmentation according to claim 1, it is characterized in that: The characteristics of the semantic segmentation neural network in the training phase of the semantic segmentation-based pressure equalizing ring deformation and damage detection method are: 1) It is a multi-layer neural network including convolutional layers, pooling layers, and fully connected layers; 2) The network input is n color images, n is greater than or equal to 1, and the size of the n images is the same, all H*W; 3) The network contains an intermediate output, which is a A four-dimensional matrix, m is a power of 2; the matrix then passes through multiple convolutional layers, pooling layers, and fully connected layers, and finally outputs the aforementioned matrices The detection result of the pressure ring in the corresponding position of the input image in proportion to the dimension, including the confidence and bounding box coordinates, and the size is The voltage-equalizing ring segmentation mask.

3. According to the method for detecting deformation and damage of a pressure equalizing ring based on semantic segmentation according to claim 1, it is characterized in that: The stochastic gradient descent method in the training phase of the semantic segmentation-based pressure equalizing ring deformation and damage detection method trains the semantic segmentation neural network by using the stochastic gradient descent method to optimize the Euclidean loss function: Where: y is the size of The neural network output matrix; is to scale the annotated mask to The matrix of .

4. According to claim 1, a method for detecting deformation and damage of a pressure equalizing ring based on semantic segmentation is characterized in that: The characteristics of the binary classification neural network in the training phase of the semantic segmentation-based pressure equalizing ring deformation and damage detection method are: 1) It is a multi-layer neural network including convolutional layers, pooling layers, and fully connected layers; 2) The network input is n color images, n is greater than or equal to 1, and the n images have the same size; 3) The output of the network is an n*2 matrix, which indicates the probability that the n images are normal pressure equalizing rings and pressure equalizing rings that are deformed and damaged.

5. According to claim 1, a method for detecting deformation and damage of a pressure equalizing ring based on semantic segmentation is characterized in that: The method for detecting deformation and damage of the equalizing ring based on semantic segmentation adopts the stochastic gradient descent method to train the binary segmentation neural network in the training phase, and uses the stochastic gradient descent method to optimize the cross entropy loss function.

Images