Substation equipment defect detection method and system based on visual identification

Abstract

The invention belongs to the technical field of defect detection, and discloses a substation equipment defect detection method and system based on visual identification. The method comprises the steps of performing path planning on the inspection trolley; acquiring image information of transformer substation equipment on the planned path, wherein the image information of the transformer substation equipment is acquired by a camera carried by an inspection trolley; based on the substation equipment image information, inputting the substation equipment image information into the trained substation equipment surface pollution identification model to obtain surface pollution categories, the surface pollution categories including a dust pollution category, a mould pollution category, an oil pollution category and a bird droppings pollution category; calling a cleaning device carried by the inspection trolley to carry out corresponding cleaning measures according to the surface pollution types; according to the method, surface pollution identification and surface pollution cleaning are firstly carried out, and then arc burn detection is carried out, so that the accuracy of the arc burn degree and the arc burn shape is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and more specifically, to a method and system for substation equipment defect detection based on visual recognition. Background Art

[0002] The existing Chinese patent with publication number CN111008587A discloses a deep learning-based intelligent visual recognition system for robots, including a defect data set, which includes a characteristic of a substation primary equipment object, a state caused by the attachment of a substation primary equipment object, and a large amount of rust on the substation primary equipment; a target detection model framework CSG Net, which includes a feature extraction part, an FPN network, an RPN network, and a fine-grained network; the feature extraction part uses Res-neXt; the fine-grained network further extracts features for the defective part after the object is selected by the upper target detection frame, which is achieved by adding an APN network; the APN network is trained based on the extracted features to obtain attention area information; the attention area is then cropped out and enlarged, and then used as the input of the next-level scale network; and finally the defect area is obtained. The invention can accurately identify defects in substations.

[0003] The existing technology still has the following problems:

[0004] 1. Due to the high dust content in the coal mine power consumption area and the serious surface pollution of substation equipment, the negative impact of surface pollution category on subsequent arc burn detection results was not fully considered during the defect detection of substation equipment, resulting in inaccurate recognition of arc burn degree and shape.

[0005] 2. The substation equipment defect detection system based on visual recognition is not combined with the inspection vehicle, so effective detection and corresponding maintenance measures cannot be carried out on the bottom of the substation equipment which is inconvenient for drones or staff to observe.

[0006] In view of this, the present invention proposes a substation equipment defect detection method and system based on visual recognition to solve the above problems. Summary of the invention

[0007] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for detecting defects of substation equipment based on visual recognition, comprising:

[0008] Plan the path for the inspection vehicle;

[0009] Collecting substation equipment image information on the planned path, the substation equipment image information is collected by a camera carried by the inspection vehicle; the substation equipment image information includes substation equipment photos, ambient brightness, shooting distance and shooting angle;

[0010] Based on the substation equipment image information, the trained substation equipment surface pollution recognition model is input to obtain the surface pollution category, which includes dust pollution, mold pollution, oil pollution and bird droppings pollution;

[0011] Call the cleaning device on the inspection vehicle to take corresponding cleaning measures according to the surface contamination category;

[0012] Based on the substation equipment image information collected again after the inspection vehicle cleans the surface pollution, the image information is input into the trained arc burn detection model to obtain the defect status of the substation equipment, which includes the degree of arc burn and the shape of arc burn; the degree of arc burn is divided into mild, moderate and severe; the shape of arc burn is divided into circular, linear and irregular shapes;

[0013] Take appropriate maintenance measures according to the defect status of substation equipment.

[0014] Furthermore, the shooting distance is the distance between the camera and the surface of the substation equipment to be inspected when the inspection vehicle takes photos during the inspection. The distance acquisition method includes:

[0015] The camera of the inspection vehicle sends a short pulse of light to the substation equipment when taking photos of the substation equipment, and the camera records the light pulse emission time T1;

[0016] The optical signal reflected from the surface of the substation equipment is received by the camera, and the time when the optical signal is received T2 is recorded by the camera;

[0017] By calculating the time difference between the emission and reception of the short pulse of light, the time it takes for the light signal to travel from the camera to the substation equipment and then return is determined and recorded as the flight time FXT, FXT = |T2-T1|;

[0018] By formula Calculate the distance between the substation equipment and the camera;

[0019] Where DB is the distance between the substation equipment and the camera, in meters; GS is the speed of light;

[0020] The substation equipment refers to the area to be inspected in the substation equipment.

[0021] Furthermore, the method for acquiring the shooting angle includes:

[0022] Use the camera's gyro sensor to obtain the shooting angle when taking photos of substation equipment. The acquisition method includes:

[0023] Read the data from the gyro sensor, which includes the rotation speed ω(t) of the camera on the inspection vehicle;

[0024] By integrating the formula Get the cumulative rotation angle of the camera;

[0025] Where θ(t2) is the cumulative rotation angle of the camera in the time period [t1, t2]; θ(t1) is the initial angle of the camera at time t1; ω(t) is the rotation speed of the camera in the time period [t1, t2], t1 is the moment when the camera starts shooting, and t2 is the moment when the camera finishes shooting.

[0026] Furthermore, the training method of the substation equipment surface pollution identification model includes:

[0027] A historical surface pollution data set is collected, where the historical surface pollution data set includes r groups of historical substation equipment image information and historical surface pollution categories corresponding to the historical substation equipment image information, where r is an integer greater than 1; the historical surface pollution data set is divided into a training set and a test set, a classifier is constructed, the historical substation equipment image information in the training set is used as input data, and the historical surface pollution categories in the training set are used as output data, the classifier is trained to obtain an initial classifier, the initial classifier is tested using the test set, and a classifier that meets a preset accuracy is output as a substation equipment surface pollution recognition model; the substation equipment surface pollution recognition model is a convolutional neural network (CNN).

[0028] Furthermore, the cleaning device includes a trolley body, a high-pressure nozzle, a solenoid valve and four water tanks. The four water tanks are all arranged in the trolley body. The four water tanks are filled with water, weak alkaline detergent, strong alkaline detergent and chlorine bleach in sequence. The input end of the solenoid valve is connected to the four water tanks one by one through a pipeline, and the output end of the solenoid valve is connected to the high-pressure nozzle through a pipeline. A water pump is integrated in the high-pressure nozzle.

[0029] When the output of the substation equipment surface pollution identification model is dust pollution, the solenoid valve is connected to the water tank filled with water, and the high-pressure nozzle is started to flush the surface contaminated area;

[0030] When the output of the substation equipment surface contamination identification model is mold contamination, the solenoid valve is connected to the water tank containing chlorine bleach, and the high-pressure nozzle is started to flush the surface contaminated area;

[0031] When the output of the substation equipment surface pollution identification model is oil pollution, the solenoid valve is connected to the water tank containing strong alkaline detergent, and the high-pressure nozzle is started to flush the surface contaminated area;

[0032] When the output of the substation equipment surface pollution identification model is bird droppings pollution, the solenoid valve is connected to the water tank filled with weak alkaline detergent, and the high-pressure nozzle is started to flush the surface contaminated area.

[0033] Furthermore, the training method of the arc burn detection model includes:

[0034] A historical arc burn data set is collected, where the historical arc burn data set includes e groups of historical substation equipment image information and historical substation equipment defect states corresponding to the historical substation equipment image information, where e is an integer greater than 1; the historical arc burn data set is divided into a training set and a test set, a classifier is constructed, the historical substation equipment image information in the training set is used as input data, and the historical substation equipment defect states in the training set are used as output data, the classifier is trained to obtain an initial classifier, the initial classifier is tested using the test set, and a classifier that meets a preset accuracy is output as an arc burn detection model; the arc burn detection model is a network CSPDarknet53 model.

[0035] Furthermore, the arc burn detection model is optimized using the self-attention mechanism. The optimization method is as follows:

[0036] The self-attention mechanism is used to optimize the historical substation equipment image information; the self-attention mechanism construction method includes:

[0037] Define the input dimension n of the self-attention mechanism input and output dimension n output , where n input and n output is an integer greater than or equal to 1;

[0038] For example, in this embodiment, n input =n output =4;

[0039] Define the input feature tensor X = (A, B, C, D), where A is the feature vector of the substation equipment photo; B is the feature vector of the shooting distance; C is the feature vector of the ambient brightness; D is the feature vector of the shooting angle;

[0040] Define the weight matrix W(n) for linear transformation input ,n output ), and initialize the weight matrix with He, and initialize the weight of the weight matrix to be uniformly distributed. The weight range of the weight matrix is

[0041] Construct query vector and key vector: Linearly transform and combine the output data of the input layer through the fully connected layer to obtain the query tensor Q, key tensor K and value tensor V. The output data includes the arc burn degree and arc burn shape.

[0042] Among them, the linear transformation combinations are: Q = X·W q +b,K=X·W k +b,V=X·W v +b, where X represents the input feature tensor, W q , W k , W v represents the respective weight matrices, and b represents the bias vector;

[0043] The attention score matrix S(Q,K) is calculated by the dot product between the query tensor Q and the key tensor K. Where d k is the dimension of the key tensor K, and c is the shared bias vector in the entire attention score matrix;

[0044] The attention scores are normalized by applying the softmax function to the attention score matrix to obtain the attention weights. The attention score normalization steps include:

[0045]

[0046] Where S(Q,K) i is the i-th score in the attention score vector for each row in the attention matrix, S(Q,K) j is the jth score in the attention score vector of each row in the attention matrix; L is the length of the attention score vector of each row in the attention matrix, and exp(·) represents an exponential function with a natural constant as the base; i=1,2,...,L,j=1,2,...,L;

[0047] The value tensor V is weighted and summed by the attention weight to obtain the weighted value tensor VW. The elements in the weighted value tensor VW are summed, and the elements include A, B, C, and D to obtain the final output tensor OP. The calculation steps of the weighted value tensor VW include:

[0048] VW = softmax(S(Q,K)) i ×V i ;

[0049] Where V i is the i-th element of the value tensor;

[0050] The calculation steps of the output tensor OP include:

[0051]

[0052] Where N is the length of the weighted value tensor VW; (VW) k is the kth element of the weighted value tensor VW;

[0053] The output tensor OP is used as the output of the self-attention mechanism and returned to the shallow network for use as the historical arc burn data for the next set of training.

[0054] Furthermore, when the arc burn is mild, the cleaning device carried by the inspection vehicle is called to clean the target area;

[0055] When the arc burn is moderate, take appropriate measures according to the different shapes of the arc burn;

[0056] When the arc burn shape is circular, the substation equipment is marked for position and protective paint is applied to the arc burn area;

[0057] When the arc burn shape is linear, the thermal imaging camera carried by the inspection vehicle is called to detect the temperature distribution on the surface of the substation equipment, and the insulation performance is inspected and repaired at the temperature abnormality point. The temperature abnormality point is the point in the temperature distribution diagram that exceeds the preset temperature threshold;

[0058] When the arc burn shape is irregular, the substation equipment is marked for position, and the arc burn area is repaired by welding;

[0059] When the arc burn degree is severe, the substation equipment is marked for location and accident analysis is performed.

[0060] Furthermore, it also includes:

[0061] Determine the relative position of the surface contamination area and the inspection vehicle in the visualization area map; obtain the position coordinates (x1, y1) of the camera carried by the vehicle and the position coordinates (x1-d1, y1-d2) of the high-pressure nozzle carried by the vehicle in the visualization area map. The height and pitch angle of the high-pressure nozzle carried by the inspection vehicle are consistent with the camera. When the camera is facing the center point of the surface contamination area, obtain the coordinate point of the surface contamination area (x1, y1+DB), where d1 is the distance between the high-pressure nozzle and the camera on the x-axis of the coordinate system, and d2 is the distance between the high-pressure nozzle and the camera on the y-axis of the coordinate system.

[0062] Calculate the rotation angle δ of the high-pressure nozzle toward the camera side, drive the high-pressure nozzle according to the rotation angle δ, aim the high-pressure nozzle at the surface contamination area, and clean the surface contamination;

[0063] The rotation angle δ of the high-pressure nozzle toward the camera is obtained as follows:

[0064]

[0065] A substation equipment defect detection system based on visual recognition, implementing the substation equipment defect detection method based on visual recognition, comprising:

[0066] Path planning module, used to plan the path of the inspection vehicle;

[0067] An image information acquisition module is used to acquire image information of substation equipment on the planned path. The substation equipment image information is acquired by a camera carried by the inspection vehicle. The substation equipment image information includes photos of the substation equipment, ambient brightness, shooting distance, and shooting angle.

[0068] The surface pollution identification module inputs the trained surface pollution identification model of substation equipment based on the substation equipment image information to obtain the surface pollution category, which includes dust pollution, mold pollution, oil pollution and bird droppings pollution;

[0069] The automated cleaning module is used to call the cleaning device on the inspection vehicle to perform corresponding cleaning measures according to the surface contamination category;

[0070] The arc burn detection module is based on the substation equipment image information collected again after the inspection vehicle cleans the surface pollution, and inputs it into the trained arc burn detection model to obtain the substation equipment defect status, which includes the arc burn degree and arc burn shape; the arc burn degree is divided into mild, moderate and severe; the arc burn shape is divided into circular, linear and irregular shapes;

[0071] The arc burn processing module is used to take corresponding maintenance measures according to the defect status of substation equipment.

[0072] The technical effects and advantages of the visual recognition-based substation equipment defect detection method and system of the present invention are as follows:

[0073] In the substation equipment defect detection system based on visual recognition, surface contamination is first identified and cleaned before arc burn detection, which greatly improves the accuracy of arc burn degree and shape.

[0074] At the same time, it is also combined with the inspection vehicle, which can not only reach the bottom of the substation equipment that is inconvenient for drones or staff to observe, but also effectively deal with the identified surface contamination of substation equipment and the degree of arc burns, thereby improving the safety of staff and the maintenance efficiency of substation equipment;

[0075] Finally, when detecting arc burns on substation equipment, the shooting distance was taken into consideration, and the time-of-flight technology was combined with arc burn detection for the first time, effectively eliminating the impact of shooting the target area at different positions during the trolley inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 This is a schematic diagram of a substation equipment defect detection system based on visual recognition according to Embodiment 1 of the present invention;

[0077] Figure 2 This is a flow chart of a method for detecting substation equipment defects based on visual recognition according to Embodiment 3 of the present invention;

[0078] Figure 3 It is a structural schematic diagram of the inspection vehicle of the present invention;

[0079] Figure 4 This is a schematic diagram of the connection structure between the solenoid valve and the water tank of the present invention;

[0080] Figure 5 This is a schematic diagram of a substation equipment defect detection system based on visual recognition according to Embodiment 2 of the present invention;

[0081] Figure 6 This is a schematic diagram of an electronic device according to Embodiment 4 of the present invention;

[0082] Figure 7 This is a schematic diagram of a storage medium according to Embodiment 5 of the present invention;

[0083] Figure numerals: 1. trolley body, 2. angle adjustment motor, 3. rotating seat, 4. high-pressure nozzle, 5. water tank, 6. solenoid valve. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] Example 1

[0086] See also Figure 1 As shown, the substation equipment defect detection system based on visual recognition described in this embodiment includes a path planning module, an image information acquisition module, a surface contamination identification module, an automatic cleaning module, an arc burn detection module and an arc burn treatment module, and each module is connected by wire and / or wirelessly.

[0087] The path planning module is used to plan the path of the inspection vehicle to obtain image information of substation equipment.

[0088] The image information acquisition module is used to collect substation equipment image information, and the substation equipment image information includes substation equipment photos, ambient brightness, shooting distance and shooting angle; the substation equipment photos provide the substation equipment images themselves that need to be analyzed, and the ambient brightness affects the contrast and clarity of the substation equipment images, thereby affecting the accuracy of detecting defects in the substation equipment; the shooting distance affects the size and proportion of the substation equipment images, thereby affecting the subsequent accurate judgment of the surface contamination category and arc burn degree of the substation equipment; the shooting angle affects the accurate judgment of the arc burn shape.

[0089] Among them, the substation equipment photos are photos of the substation equipment taken by the inspection vehicle during inspection, and the shooting interval is preset to u seconds, where u can be 1, 2 or other.

[0090] The ambient brightness is obtained by an ambient light sensor equipped with a camera device carried by the inspection vehicle;

[0091] The shooting distance is the distance between the camera and the surface of the substation equipment to be inspected when the inspection vehicle takes photos during the inspection. The distance acquisition steps are as follows:

[0092] The camera of the inspection vehicle sends a short pulse of light to the substation equipment when taking photos of the substation equipment, and the camera records the light pulse emission time T1;

[0093] The optical signal reflected from the surface of the substation equipment is received by the camera, and the time when the optical signal is received T2 is recorded by the camera;

[0094] By calculating the time difference between the emission and reception of the short pulse of light, the time it takes for the light signal to travel from the camera to the substation equipment and then return is determined and recorded as the flight time FXT, FXT = |T2-T1|;

[0095] By formula Calculate the distance between the substation equipment and the camera;

[0096] Where DB is the distance between the substation equipment and the camera; GS is the speed of light;

[0097] The substation equipment refers to the area to be inspected in the substation equipment.

[0098] The method for acquiring the shooting angle includes:

[0099] Use the camera's gyro sensor to obtain the shooting angle when taking photos of substation equipment. The steps are as follows:

[0100] Read the data from the gyro sensor, which includes the rotation speed ω(t) of the camera on the inspection vehicle.

[0101] By integrating the formula Get the cumulative rotation angle of the camera.

[0102] Where θ(t2) is the cumulative rotation angle of the camera in the time period [t1, t2]; θ(t1) is the initial angle of the camera at time t1; ω(t) is the rotation speed of the camera in the time period [t1, t2], t1 is the moment when the camera starts shooting, and t2 is the moment when the camera finishes shooting.

[0103] The surface pollution identification module, based on the substation equipment image information, inputs the trained substation equipment surface pollution identification model to obtain the surface pollution category, which includes dust pollution, mold pollution, oil pollution and bird droppings pollution.

[0104] Since dust contamination, mold contamination, oil pollution and bird droppings pollution will affect the detection of arc burn marks, dust contamination will make the surface of substation equipment blurred, mold pollution and bird droppings pollution will form irregular spots on the surface of substation equipment, and oil pollution will form oil film stains on substation equipment, which will cause unclear arc burn marks and further affect the accurate identification of arc burn degree and arc burn shape. Therefore, it is necessary to obtain different results according to the substation equipment surface pollution identification model, and then use the cleaning device carried by the inspection car to take corresponding cleaning measures according to the surface pollution category before detecting the arc burn marks.

[0105] The training method of the substation equipment surface contamination identification model includes:

[0106] A historical surface pollution data set is collected, where the historical surface pollution data set includes r groups of historical substation equipment image information and historical surface pollution categories corresponding to the historical substation equipment image information, where r is an integer greater than 1; the historical surface pollution data set is divided into a training set and a test set, a classifier is constructed, the historical substation equipment image information in the training set is used as input data, and the historical surface pollution categories in the training set are used as output data, the classifier is trained to obtain an initial classifier, the initial classifier is tested using the test set, and a classifier that meets a preset accuracy is output as a substation equipment surface pollution recognition model; the substation equipment surface pollution recognition model is a convolutional neural network CNN or other suitable models.

[0107] The automatic cleaning module is used to call the cleaning device on the inspection vehicle to perform corresponding cleaning measures according to the surface pollution category output by the surface pollution identification model of the substation equipment; see Figure 3 and Figure 4As shown, the cleaning device includes a trolley body 1, a high-pressure nozzle 4, a solenoid valve 6 and four water tanks 5. The four water tanks are all arranged in the trolley body 1. The four water tanks 5 are filled with water, weak alkaline detergent, strong alkaline detergent and chlorine bleach in sequence. The input end of the solenoid valve 6 is connected to the four water tanks 5 one by one through a pipeline, and the output end of the solenoid valve 6 is connected to the high-pressure nozzle 4 through a pipeline. A water pump is integrated in the high-pressure nozzle 4.

[0108] Among them, weak alkaline cleaners can effectively decompose the protein and organic matter in bird droppings to remove bird droppings stains; strong alkaline cleaners can effectively decompose grease to remove oil pollution; chlorine bleach can effectively kill mold to remove mold stains.

[0109] When the output of the substation equipment surface pollution identification model is dust pollution, the electromagnetic valve 6 is connected to the water tank 5 filled with water, and the high-pressure nozzle 4 is started to flush the surface contaminated area.

[0110] When the output of the substation equipment surface pollution identification model is mold contamination, the electromagnetic valve 6 is connected to the water tank 5 filled with chlorine bleach, and the high-pressure nozzle 4 is started to flush the surface contaminated area.

[0111] When the output of the substation equipment surface pollution identification model is oil pollution, the electromagnetic valve 6 is connected to the water tank 5 filled with strong alkaline detergent, and the high-pressure nozzle 4 is started to flush the surface contaminated area.

[0112] When the output of the substation equipment surface pollution identification model is bird droppings pollution, the electromagnetic valve 6 is connected to the water tank 5 filled with weak alkaline detergent, and the high-pressure nozzle 4 is started to flush the surface contaminated area.

[0113] The arc burn detection module is based on the substation equipment image information collected again after the inspection vehicle cleans the surface pollution, and inputs it into the trained arc burn detection model to obtain the defect status of the substation equipment. The substation equipment defect status includes the degree of arc burn and the shape of arc burn; the degree of arc burn is divided into mild, moderate and severe; the shape of arc burn is divided into circular, linear and irregular shapes; by knowing the degree of arc burn and the shape of arc burn, it is helpful to know the cause of arc burn, so as to take targeted maintenance measures.

[0114] The training method of the arc burn detection model includes:

[0115] A historical arc burn data set is collected, where the historical arc burn data set includes e groups of historical substation equipment image information and historical substation equipment defect states corresponding to the historical substation equipment image information, where e is an integer greater than 1; the historical arc burn data set is divided into a training set and a test set, a classifier is constructed, the historical substation equipment image information in the training set is used as input data, and the historical substation equipment defect states in the training set are used as output data, the classifier is trained to obtain an initial classifier, the initial classifier is tested using the test set, and a classifier that meets a preset accuracy is output as an arc burn detection model; the arc burn detection model is a network CSPDarknet53 or other suitable models.

[0116] The collection method of historical substation equipment image information includes:

[0117] Collect photos of substation equipment taken by the inspection car at different shooting distances, ambient brightness and shooting angles.

[0118] By randomly adjusting the color channels of substation equipment photos at different shooting distances and shooting angles, changing the brightness and contrast, and simulating the shooting effects under different lighting conditions, photos of substation equipment at different shooting distances, shooting angles, and ambient brightness are obtained; compared with the prior art of collecting substation equipment image information for each situation, here, the inventors can effectively reduce the difficulty of collecting historical substation equipment image information by randomly adjusting the color channels of substation equipment photos.

[0119] By horizontally flipping photos of substation equipment at different shooting distances, shooting angles, and ambient brightness, the robustness of the arc burn detection model can be trained to improve. Random noise is added to photos of substation equipment to simulate noise interference in the actual shooting process and optimize the application of the model in the actual environment.

[0120] Furthermore, the arc burn detection model is optimized using the self-attention mechanism. The specific optimization method is as follows:

[0121] The self-attention mechanism is used to optimize the historical substation equipment image information; the arc burn detection model is the CSPDarknet53 model; the self-attention module is inserted at the shallow position of the CSPDarknet53 model to improve the CSPDarknet53 model's attention to the details of the input substation equipment image information, so that the CSPDarknet53 model pays more attention to the areas related to the defect status of the substation equipment in the substation equipment photos, thereby improving the CSPDarknet53 model's recognition accuracy of the substation equipment image information taken by the inspection vehicle.

[0122] The methods for building the self-attention mechanism include:

[0123] Define the input dimension n of the self-attention mechanism input and output dimension n output , where n input and n output is an integer greater than or equal to 1;

[0124] For example, in this embodiment, n input =n output =4.

[0125] Define the input feature tensor X = (A, B, C, D), where A is the feature vector of the substation equipment photo; B is the feature vector of the shooting distance; C is the feature vector of the ambient brightness; D is the feature vector of the shooting angle;

[0126] Define the weight matrix W(n) for linear transformation input ,n output ), and initialize the weight matrix with He, and initialize the weight of the weight matrix to be uniformly distributed. The weight range of the weight matrix is

[0127] Construct query vector and key vector: Linearly transform and combine the output data of the input layer through the fully connected layer to obtain the query tensor Q, key tensor K and value tensor V. The output data includes the arc burn degree and arc burn shape.

[0128] Among them, the linear transformation combinations are: Q = X·W q +b,K=X·W k +b,V=X·W v +b, where X represents the input feature tensor, W q , W k , W v represents the respective weight matrices, and b represents the bias vector;

[0129] The attention score matrix S(Q,K) is calculated by the dot product between the query tensor Q and the key tensor K. Where d k is the dimension of the key tensor, c is the shared bias vector in the entire attention score matrix;

[0130] The attention scores are normalized by applying the softmax function to the attention score matrix to obtain the attention weights. The attention score normalization steps include:

[0131]

[0132] Where S(Q,K) i is the i-th score in the attention score vector for each row in the attention matrix, S(Q,K) jis the jth score in the attention score vector of each row in the attention matrix; L is the length of the attention score vector of each row in the attention matrix, and exp(·) represents an exponential function with a natural constant as the base; i=1,2,...,L,j=1,2,...,L;

[0133] The value tensor V is weighted and summed by the attention weight to obtain the weighted value tensor VW. By summing all elements in the weighted value tensor VW, the elements are A, B, C, and D, and the final output tensor OP is obtained; this output tensor reflects the degree of attention of the self-attention mechanism to the value tensor given the query tensor and the key tensor.

[0134] The calculation steps of the weighted value tensor VW include:

[0135] VW = softmax(S(Q,K)) i ×V i ;

[0136] Where V i The i-th element of the value tensor.

[0137] The calculation steps of the output tensor OP include:

[0138]

[0139] Where N is the length of the weighted value tensor VW; (VW) k is the kth element of the weighted value tensor VW.

[0140] The output tensor OP is used as the output of the self-attention mechanism and returned to the shallow network for use as the historical arc burn data for the next set of training.

[0141] By introducing the feature extraction network CSPDarknet53 model with self-attention mechanism and adopting the supervised learning method, the input of the CSPDarknet53 model is the historical substation equipment image information, and the output is the predicted value of the defect status of the substation equipment.

[0142] Supervised learning and self-supervised learning methods are adopted, and labeled image data and unlabeled image data are used for comprehensive training to further optimize model performance.

[0143] The model parameters are optimized by back propagation algorithm, so that the CSPDarknet53 model can accurately predict the defect status of substation equipment.

[0144] The performance of the model was evaluated using the validation set. Different videos containing defects in substation equipment were selected for real-time detection and judgment, and the model was tuned based on the evaluation results.

[0145] Arc burn processing module, used to take corresponding maintenance measures according to the defect status of substation equipment;

[0146] When the arc burn is mild, the cleaning device carried by the inspection vehicle is used to clean the target area to prevent the arc burn marks from continuing to expand.

[0147] When the arc burn is moderate, take appropriate measures according to the different shapes of the arc burn;

[0148] When the arc burn shape is circular, the substation equipment is marked for position, and workers are dispatched to apply protective paint to the arc burn area to prevent oxygen and moisture from directly contacting the metal surface of the substation equipment, thereby slowing down the corrosion rate of the substation equipment and extending its service life.

[0149] When the shape of the arc burn is linear, the thermal imaging camera carried by the inspection vehicle is called to detect the temperature distribution on the surface of the substation equipment, and staff are dispatched to perform insulation performance inspection on the temperature abnormality points. The temperature abnormality points are points in the temperature distribution map that exceed the preset temperature threshold. The temperature threshold is set according to the historical records of the substation equipment.

[0150] When the shape of the arc burn is irregular, the substation equipment is marked, and staff are dispatched to perform repair welding on the arc burn area to repair the damaged parts of the substation equipment and restore its original structural strength and stability to ensure the normal operation of the substation equipment.

[0151] When the arc burn is severe, the substation equipment is marked for location, and staff are dispatched to analyze the accident and record the cause of the accident to prevent the recurrence of severe arc burns.

[0152] In the substation equipment defect detection system based on visual recognition, surface contamination is first identified and cleaned before arc burn detection, which greatly improves the accuracy of arc burn degree and shape.

[0153] At the same time, the system is combined with the inspection vehicle, which can not only reach the bottom of the substation equipment and the connection points of multiple devices that are inconvenient for drones or staff to observe, but also promptly deal with the surface contamination of the substation equipment and the surface of the substation equipment when the degree of arc burns is mild, thereby improving the safety of staff and the maintenance efficiency of substation equipment.

[0154] Finally, when detecting arc burns on substation equipment, the shooting distance was taken into consideration, and the time-of-flight technology was combined with arc burn detection for the first time, effectively eliminating the impact of shooting the target area at different positions during the trolley inspection.

[0155] Example 2

[0156] See also Figure 5 As shown, this embodiment further improves the design on the basis of embodiment 1. To ensure the cleaning efficiency and cleaning effect of the cleaning device, this embodiment provides a substation equipment defect detection system based on visual recognition, which also includes a coordinate positioning module and an execution module.

[0157] The coordinate positioning module is used to determine the relative position of the surface contamination area and the inspection vehicle in the visualization area map; obtain the position coordinates (x1, y1) of the camera carried by the vehicle in the visualization area map and the position coordinates (x1-d1, y1-d2) of the high-pressure nozzle 4 carried by the vehicle. The height and pitch angle of the high-pressure nozzle 4 carried by the inspection vehicle are consistent with the camera; when the camera is facing the center point of the surface contamination area, the coordinate point of the surface contamination area is obtained (x1, y1+DB), where d1 is the distance between the high-pressure nozzle 4 and the camera on the x-axis of the coordinate system, and d2 is the distance between the high-pressure nozzle 4 and the camera on the y-axis of the coordinate system, in meters.

[0158] The execution module calculates the rotation angle δ of the high-pressure nozzle 4 toward the camera side, drives the high-pressure nozzle 4 according to the rotation angle δ, and makes the high-pressure nozzle 4 aim at the surface contaminated area to clean the surface contamination and improve the cleaning efficiency.

[0159] The rotation angle δ of the high-pressure nozzle 4 toward the camera is obtained as follows:

[0160]

[0161] The unit of the rotation angle δ is degree.

[0162] The cleaning device also includes an angle adjustment motor 2 and a rotating seat 3, a high-pressure nozzle 4 is fixed to the rotating seat 3, a driving shaft of the angle adjustment motor 2 is fixed to the bottom of the rotating seat 3, and the angle adjustment motor 2 is fixed in the trolley body 1; the angle adjustment motor 2 rotates to a corresponding stroke according to the rotation angle δ, and the rotating seat 3 drives the high-pressure nozzle 4 to align with the surface contaminated area through a preset rotation angle and stroke.

[0163] By calculating the rotation angle of the high-pressure nozzle 4 carried by the trolley, compared with directly controlling the high-pressure nozzle 4 to clean the surface contaminated area after surface contamination identification, the cleaning efficiency is improved and the utilization rate of different solutions in the water tank is increased.

[0164] Example 3

[0165] See also Figure 2 As shown, this embodiment provides a substation equipment defect detection method based on visual recognition, including:

[0166] Plan the path for the inspection vehicle;

[0167] Collecting substation equipment image information on the planned path, the substation equipment image information is collected by a camera carried by the inspection vehicle; the substation equipment image information includes substation equipment photos, ambient brightness, shooting distance and shooting angle;

[0168] Based on the substation equipment image information, the trained substation equipment surface pollution recognition model is input to obtain the surface pollution category, which includes dust pollution, mold pollution, oil pollution and bird droppings pollution;

[0169] Call the cleaning device on the inspection vehicle to take corresponding cleaning measures according to the surface contamination category;

[0170] Based on the substation equipment image information collected again after the inspection vehicle cleans the surface pollution, the image information is input into the trained arc burn detection model to obtain the defect status of the substation equipment, which includes the degree of arc burn and the shape of arc burn; the degree of arc burn is divided into mild, moderate and severe; the shape of arc burn is divided into circular, linear and irregular shapes;

[0171] Take appropriate maintenance measures according to the defect status of substation equipment.

[0172] Example 4

[0173] See also Figure 6 As shown, according to another aspect of the present application, an electronic device 500 is also provided. The electronic device 500 may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are executed by one or more processors, a substation equipment defect detection method based on visual recognition as described above may be executed.

[0174] The method or system according to the embodiment of the present application can also be used by Figure 6 The electronic device architecture shown in FIG. Figure 6 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, ROM 503, RAM 504, a communication port 505 connected to a network, an input / output 506, a hard disk 507, etc. The storage device in the electronic device 500, such as ROM 503 or hard disk 507, may store a substation equipment defect detection method based on visual recognition provided by the present application. Furthermore, the electronic device 500 may also include a user interface 508. Of course, Figure 6 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 6 One or more components of an electronic device are shown.

[0175] Example 5

[0176] See also Figure 7 As shown, a computer-readable storage medium 600 according to an embodiment of the present application is shown. Computer-readable instructions are stored on the computer-readable storage medium 600. When the computer-readable instructions are executed by the processor, a substation equipment defect detection method based on visual recognition according to an embodiment of the present application described with reference to the above figures can be executed. The storage medium 600 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0177] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0178] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting substation equipment defects based on visual recognition, characterized in that: include: Plan the path for the inspection vehicle; Collecting image information of substation equipment on the planned path, where the image information of substation equipment is collected by a camera carried by the inspection vehicle; The substation equipment image information includes a photo of the substation equipment, ambient brightness, shooting distance and shooting angle; Based on the substation equipment image information, the trained substation equipment surface pollution recognition model is input to obtain the surface pollution category, which includes dust pollution, mold pollution, oil pollution and bird droppings pollution; Call the cleaning device on the inspection vehicle to take corresponding cleaning measures according to the surface contamination category; Based on the substation equipment image information collected again after the inspection vehicle cleans the surface pollution, the image information is input into the trained arc burn detection model to obtain the defect status of the substation equipment, which includes the degree of arc burn and the shape of arc burn; the degree of arc burn is divided into mild, moderate and severe; the shape of arc burn is divided into circular, linear and irregular shapes; Take appropriate maintenance measures according to the defect status of substation equipment.

2. A method for detecting substation equipment defects based on visual recognition according to claim 1, characterized in that: The shooting distance is the distance between the camera and the surface of the substation equipment to be inspected when the inspection vehicle takes photos during the inspection. The distance acquisition method includes: The camera of the inspection vehicle sends a short pulse of light to the substation equipment when taking photos of the substation equipment, and the camera records the light pulse emission time T1; The optical signal reflected from the surface of the substation equipment is received by the camera, and the time when the optical signal is received T2 is recorded by the camera; By calculating the time difference between the emission and reception of the short pulse of light, the time it takes for the light signal to travel from the camera to the substation equipment and then return is determined and recorded as the flight time FXT, FXT = |T2-T1|; By formula Calculate the distance between the substation equipment and the camera; Wherein, DB is the distance between the substation equipment and the camera, in meters; GS is the speed of light.

3. A method for detecting substation equipment defects based on visual recognition according to claim 2, characterized in that: The method for acquiring the shooting angle includes: Use the camera's gyro sensor to obtain the shooting angle when taking photos of substation equipment. The acquisition method includes: Read the data from the gyro sensor, which includes the rotation speed ω(t) of the camera on the inspection vehicle; By integrating the formula Get the cumulative rotation angle of the camera; Where θ(t2) is the cumulative rotation angle of the camera in the time period [t1, t2]; θ(t1) is the initial angle of the camera at time t1; ω(t) is the rotation angle of the camera at time [ The rotation speed in the time period [t1, t2], t1 is the moment when the camera starts shooting, and t2 is the moment when the camera finishes shooting.

4. A method for detecting substation equipment defects based on visual recognition according to claim 3, characterized in that: The training method of the substation equipment surface pollution identification model includes: A historical surface pollution data set is collected, where the historical surface pollution data set includes r groups of historical substation equipment image information and historical surface pollution categories corresponding to the historical substation equipment image information, where r is an integer greater than 1; the historical surface pollution data set is divided into a training set and a test set, a classifier is constructed, the historical substation equipment image information in the training set is used as input data, and the historical surface pollution categories in the training set are used as output data, the classifier is trained to obtain an initial classifier, the initial classifier is tested using the test set, and a classifier that meets a preset accuracy is output as a substation equipment surface pollution recognition model; the substation equipment surface pollution recognition model is a convolutional neural network (CNN).

5. A method for detecting substation equipment defects based on visual recognition according to claim 4, characterized in that: The cleaning device comprises a trolley body (1), a high-pressure nozzle (4), a solenoid valve (6) and four water tanks (5), wherein the four water tanks (5) are all arranged in the trolley body (1), and the four water tanks (5) are filled with water, weak alkaline detergent, strong alkaline detergent and chlorine bleach in sequence, the input end of the solenoid valve (6) is connected to the four water tanks (5) one by one through a pipeline, and the output end of the solenoid valve (6) is connected to the high-pressure nozzle (4) through a pipeline, and a water pump is integrated in the high-pressure nozzle (4); When the output of the substation equipment surface pollution identification model is dust pollution, the electromagnetic valve (6) is connected to the water tank (5) filled with water, and the high-pressure nozzle (4) is started to flush the surface contaminated area; When the output of the substation equipment surface pollution identification model is mold contamination, the electromagnetic valve (6) is connected to the water tank (5) filled with chlorine bleach, and the high-pressure nozzle (4) is started to flush the surface contaminated area; When the output of the substation equipment surface pollution identification model is oil pollution, the electromagnetic valve (6) is connected to the water tank (5) filled with a strong alkaline detergent, and the high-pressure nozzle (4) is started to flush the surface contaminated area; When the output of the substation equipment surface pollution identification model is bird droppings pollution, the electromagnetic valve (6) is connected to the water tank (5) filled with weak alkaline detergent, and the high-pressure nozzle (4) is started to flush the surface contaminated area.

6. A method for detecting substation equipment defects based on visual recognition according to claim 5, characterized in that: The training method of the arc burn detection model includes: A historical arc burn data set is collected, where the historical arc burn data set includes e groups of historical substation equipment image information and historical substation equipment defect states corresponding to the historical substation equipment image information, where e is an integer greater than 1; the historical arc burn data set is divided into a training set and a test set, a classifier is constructed, the historical substation equipment image information in the training set is used as input data, and the historical substation equipment defect states in the training set are used as output data, the classifier is trained to obtain an initial classifier, the initial classifier is tested using the test set, and a classifier that meets a preset accuracy is output as an arc burn detection model; the arc burn detection model is a network CSPDarknet53 model.

7. A method for detecting substation equipment defects based on visual recognition according to claim 6, characterized in that: The arc burn detection model is optimized using the self-attention mechanism. The optimization method is as follows: The self-attention mechanism is used to optimize the historical substation equipment image information; the self-attention mechanism construction method includes: Define the input dimension n of the self-attention mechanism input and output dimension n output , where n input and n output is an integer greater than or equal to 1; Define the input feature tensor X = (A, B, C, D), where A is the feature vector of the substation equipment photo; B is the feature vector of the shooting distance; C is the feature vector of the ambient brightness; D is the feature vector of the shooting angle; Define the weight matrix W(n) for linear transformation input ,n output ), and initialize the weight matrix with He, and initialize the weight of the weight matrix to be uniformly distributed. The weight range of the weight matrix is Construct query vector and key vector: Linearly transform and combine the output data of the input layer through the fully connected layer to obtain the query tensor Q, key tensor K and value tensor V. The output data includes the arc burn degree and arc burn shape. Among them, the linear transformation combinations are: Q = X·W q +b,K=X·W k +b,V=X·W v +b, where X represents the input feature tensor, W q , W k , W v represents the respective weight matrices, and b represents the bias vector; The attention score matrix S(Q,K) is calculated by the dot product between the query tensor Q and the key tensor K. Where d k is the dimension of the key tensor K, and c is the shared bias vector in the entire attention score matrix; The attention scores are normalized by applying the softmax function to the attention score matrix to obtain the attention weights. The attention score normalization steps include: Where S(Q,K) i is the i-th score in the attention score vector for each row in the attention matrix, S(Q,K) j is the jth score in the attention score vector of each row in the attention matrix; L is the length of the attention score vector of each row in the attention matrix, and exp(·) represents an exponential function with a natural constant as the base; i=1,2,...,L,j=1,2,...,L; The value tensor V is weighted and summed by the attention weight to obtain the weighted value tensor VW. The elements in the weighted value tensor VW are summed, and the elements include A, B, C, and D to obtain the final output tensor OP. The calculation steps of the weighted value tensor VW include: VW=softmax(S(Q,K)) i ×V i ; Where V i is the i-th element of the value tensor; The calculation steps of the output tensor OP include: Where N is the length of the weighted value tensor VW; (VW) k is the kth element of the weighted value tensor VW; The output tensor OP is used as the output of the self-attention mechanism and returned to the shallow network for use as the historical arc burn data for the next set of training.

8. A method for detecting substation equipment defects based on visual recognition according to claim 6, characterized in that: When the arc burn is mild, the cleaning device carried by the inspection vehicle is used to clean the target area; When the arc burn is moderate, take appropriate measures according to the different shapes of the arc burn; When the arc burn shape is circular, the substation equipment is marked for position and protective paint is applied to the arc burn area; When the arc burn shape is linear, the thermal imaging camera carried by the inspection vehicle is called to detect the temperature distribution on the surface of the substation equipment, and the insulation performance is inspected and repaired at the temperature abnormality point. The temperature abnormality point is the point in the temperature distribution diagram that exceeds the preset temperature threshold; When the arc burn shape is irregular, the substation equipment is marked for position, and the arc burn area is repaired by welding; When the arc burn degree is severe, the substation equipment is marked for location and accident analysis is performed.

9. A method for detecting substation equipment defects based on visual recognition according to claim 5, characterized in that: Also includes: Determine the relative position of the surface contamination area and the inspection vehicle in the visual area map; Obtain the position coordinates (x1, y1) of the camera carried by the vehicle in the visualization area map and the position coordinates (x1-d1, y1-d2) of the high-pressure nozzle (4) carried by the vehicle. The height and pitch angle of the high-pressure nozzle (4) carried by the inspection vehicle are consistent with those of the camera. When the camera faces the center point of the surface contamination area, obtain the coordinate point (x1, y1+DB) of the surface contamination area, where d1 is the distance between the high-pressure nozzle (4) and the camera on the x-axis of the coordinate system, and d2 is the distance between the high-pressure nozzle (4) and the camera on the y-axis of the coordinate system. Calculating the rotation angle δ of the high-pressure nozzle (4) toward the camera side, and driving the high-pressure nozzle (4) according to the rotation angle δ, so that the high-pressure nozzle (4) is aimed at the surface contamination area to clean the surface contamination; The rotation angle δ of the high-pressure nozzle (4) toward the camera is obtained as follows:

10. A substation equipment defect detection system based on visual recognition, implementing a substation equipment defect detection method based on visual recognition as claimed in any one of claims 1 to 9, characterized in that: include: Path planning module, used to plan the path of the inspection vehicle; An image information acquisition module is used to acquire image information of substation equipment on the planned path. The substation equipment image information is acquired by a camera carried by the inspection vehicle. The substation equipment image information includes photos of the substation equipment, ambient brightness, shooting distance, and shooting angle. The surface pollution identification module inputs the trained surface pollution identification model of substation equipment based on the substation equipment image information to obtain the surface pollution category, which includes dust pollution, mold pollution, oil pollution and bird droppings pollution; The automated cleaning module is used to call the cleaning device on the inspection vehicle to perform corresponding cleaning measures according to the surface contamination category; The arc burn detection module is based on the substation equipment image information collected again after the inspection vehicle cleans the surface pollution, and inputs it into the trained arc burn detection model to obtain the substation equipment defect status, which includes the arc burn degree and arc burn shape; the arc burn degree is divided into mild, moderate and severe; the arc burn shape is divided into circular, linear and irregular shapes; The arc burn processing module is used to take corresponding maintenance measures according to the defect status of substation equipment.

Images