Small sample power transmission line pin defect detection method based on graph convolutional network

Abstract

The invention belongs to the technical field of electric power engineering monitoring, and discloses a small sample power transmission line pin defect detection method based on a graph convolutional network, and the method comprises the steps: collecting a power transmission line pin image through image collection equipment; marking pins in the image based on event logic guidance to obtain a transmission line pin data set with logic association; introducing a cross-domain interaction enhanced pyramid network into the graph convolutional network to obtain an improved graph convolutional network model, and training to obtain an original target detection model; a knowledge graph relation reasoning module is introduced into the original target detection model, and a power transmission line pin defect detection model is obtained; and detecting the preprocessed power transmission line pin image by using a pin defect detection model, so as to generate a pin image with marked pin position classification and confidence. The method can effectively solve the problem that the sample size is small, and improves the recognition accuracy of the pin and the defect of the pin, thereby facilitating the improvement of the efficiency of detecting the defect of the pin of the power transmission line by inspection personnel, and guaranteeing the stable operation of a power system.

Description

Technical Field

[0001] The present invention belongs to the technical field of power engineering monitoring, and relates to defect detection of accessories during a transmission line inspection process, and in particular to a small sample transmission line pin defect detection method based on a graph convolutional network and event logic-oriented annotation. Background Art

[0002] With the continuous construction and expansion of transmission line networks across the country, the inspection of transmission lines is of great significance to the safe and stable transmission of electricity. During the inspection process, among the types of accessories, pins are the smallest and most numerous connection accessories, so defect detection of pins is a very important task. As a common form of fixing and connecting elements in power equipment, the quality and safety of pins are directly related to the normal operation of transmission lines and the stability of power systems. In order to ensure the safe operation of transmission lines, it is a necessary link for inspection staff to use some defect detection technologies to realize defect detection of pins during the inspection process. However, the pins in the inspection process of transmission lines are small in size and easy to confuse, and are often difficult to detect accurately and quickly.

[0003] The most commonly used technique is visual inspection, where inspectors carefully observe the appearance of the pins to check whether there are obvious cracks, wear, deformation, etc. on the surface. This method is simple and intuitive, and can quickly detect obvious defects, but it is not sensitive enough for hidden defects. Ultrasonic testing technology has high sensitivity and strong deep detection capabilities, which can effectively detect hidden defects. In addition, eddy current testing and X-ray testing in non-destructive testing technologies can be used to detect defects in pins during the inspection of transmission lines.

[0004] However, the detection efficiency of the above technologies is not ideal. It is difficult to complete the inspection work in a short time when the transmission line network is so large today, which in turn affects the safe operation of the power grid and the power supply guarantee of the power users. Nowadays, convolutional neural network technology has gradually developed and grown, and image target detection technology has been continuously improved. There is now a pin defect detection method based on the YOLOv5 network, but the deep learning network model established based on this network has the problem of too little defect data, which easily leads to overfitting of the model. Summary of the invention

[0005] The purpose of the present invention is to provide a small sample transmission line pin defect detection method based on graph convolutional network, so as to realize efficient identification of pins on transmission lines and perform defect detection, thereby solving the technical problem that pin defect detection in the prior art generally adopts on-site inspection by patrol personnel, and manual inspection has low efficiency and high cost.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions to achieve it.

[0007] The present invention provides a small sample transmission line pin defect detection method based on graph convolutional network, which comprises the following steps:

[0008] S1 uses an image acquisition device to acquire multiple images, each image includes a pin on a transmission line and an installation accessory, and obtains a transmission line pin image;

[0009] S2 annotates the pins in the image based on event logic guidance to obtain a data set of power transmission line pins with logical associations;

[0010] S3 introduces a cross-domain interactive enhanced pyramid network into the graph convolutional network to obtain an improved graph convolutional network model, and uses a data set of transmission line pins with logical associations for training to obtain an original target detection model;

[0011] S4 introduces the knowledge graph relationship reasoning module into the original target detection model, refines the original detection results, and obtains the power transmission line pin defect detection model;

[0012] S5 pre-processes the transmission line pin image captured by the image acquisition device;

[0013] S6 inputs the preprocessed image into the power transmission line pin defect detection model for detection, and generates a pin image with marked pin position classification and confidence.

[0014] In the above step S1, the image acquisition device is a drone.

[0015] The pins in the image are labeled based on event logic guidance in the above step S2 to obtain a data set of power transmission line pins with logical association, including the following sub-steps:

[0016] S21 uses the transmission line connection standard and the inherent connection relationship between components as the event logic guide, and takes the pin as the final target, so the large target containing the pin is first marked; the large target is the adjustment plate and the yoke plate;

[0017] S22 Mark the pins on the marked yoke plate. Because there is an inherent connection between the adjustment plate, yoke plate and pins, it can serve as a logical guide to further small goals;

[0018] S23 marks the categories of the marked pins. Because there is an inherent connection between the pins and the pin clips, and the number of pins completely missing defects contained in the data set is very small, the categories are divided into pin clip missing and normal;

[0019] S24 uses the labeled labels with logical association as a transmission line pin dataset.

[0020] In the above step S21, there is an inherent connection relationship between the components, such as the insulator connection structure must be composed of a Z-shaped clamp, a yoke plate, a shackle and a drag plate, and the graded shielding ring structure is composed of a groove, a graded shielding ring, a yoke plate, an adjustment plate, a bolt, and an extension ring. Then there is an inherent connection relationship between them, which can be marked as a logical guide.

[0021] The above step S3 introduces a cross-domain interactive enhanced pyramid network into the graph convolutional network to obtain an improved graph convolutional network model, which is trained using a logically associated power line pin dataset to obtain an original target detection model, specifically: first, the self-attention mechanism is applied to the feature layer fusion process, and the target features located on different feature layers are enhanced by calculating the similarity of adjacent layer features. The occurrence of different object types in a high-probability scenario is described by co-occurrence relationships, and the strength of the co-occurrence relationship is quantified by calculating the frequency of co-occurrence. Common co-occurrence relationships include adjustment plates and pins, graded shielding rings and grooves with pins and pin clamps. These co-occurrence relationships can be used to assist object detection at the semantic level, thereby improving detection accuracy. The specific steps are as follows:

[0022] S31 performs feature extraction with self-attention, and objects with similar scales can interact globally in the same feature layer to obtain features with global context information;

[0023] S32 enhances contextual information through cross-domain interactions between objects of different scales to retain as much global contextual information as possible; the Cross-domain Interaction Feature Pyramid Network (CDI-FPN) enhances the original features through interactions between adjacent feature layers, thereby improving the feature pyramid; for small-scale objects, as the network goes deeper, the feature loss of small-scale objects will affect the extraction of contextual information during the interaction process; similarly, for large-size objects, the detailed features of the shallow feature layer are sufficient, but the small receptive field will lead to a lack of contextual information, so feature enhancement is needed to reduce the loss of features and enhance the contextual information of the object.

[0024] S33 divides the data set of logically associated power line pins into a training set and a validation set in a ratio of five to one, sets relevant hyperparameters and performs iterative training. After the model converges, that is, the model loss value remains approximately unchanged, the model parameters are saved as the original target detection model.

[0025] The above step S4 introduces a knowledge graph relationship reasoning module into the original target detection model, refines the original detection results, and obtains a pin defect detection model. The specific steps are as follows:

[0026] S41 establishes a knowledge graph relationship between object categories; nodes A and B of the knowledge graph represent the category of each component. In order to fully consider the occurrence frequency of the component, the edge between A and B is used to represent the co-occurrence relationship, which can be expressed as the conditional probability P(C B |C A ) in the form of:

[0027]

[0028] in, is the probability of B appearing when A exists, is the probability of A appearing, N all is the number of all components; therefore, the co-occurrence frequency matrix of pins and pin clips can be converted into a co-occurrence probability matrix to obtain a constructed knowledge graph;

[0029] S42 uses the knowledge graph constructed in the dataset as a priori through the Knowledge Graph Relation Reasoning (KGRR) module, combines it with the graph convolutional neural network, and adaptively adjusts the feature graph in the original target detection model to obtain deep semantic information;

[0030] S43 applies semantic information to the original detection results, corrects the detection results, excludes impossible objects, and adjusts the occurrence probability according to the closeness of the relationship between different objects to obtain a transmission line pin defect detection model.

[0031] The above step S5 pre-processes the transmission line pin image captured by the image acquisition device, specifically: adjusts the image size, standardizes the image so as to transform the pixel value into a certain range, and converts the data type, that is, converts the image pixel value into a tensor.

[0032] In the above step S6, the preprocessed image is input into the pin defect detection model for detection, so as to generate an image in which the pins in the image are marked and the pin classification and category confidence are given; specifically, the pre-trained model is loaded using Pytorch, and the preprocessed image is passed as input to the model; by calling the forward propagation method of the model, the image input is passed to the model for inference; the model detects the image, generates a bounding box, marks the pins in the image, and classifies them; for each detected pin, the model will also generate a corresponding category confidence, which is used to indicate the model's confidence level that the pin belongs to each category; according to the results output by the model, the marked pins can be drawn on the image, as well as the relevant category and confidence information, to intuitively display the pins in the image and their classification results.

[0033] The small sample power transmission line pin defect detection method based on graph convolutional network provided by the present invention has the following beneficial effects:

[0034] (1) The event logic-oriented labeling method used in the present invention helps to establish the association and connection between the components in the transmission line; it helps the model to better understand and analyze the influence and dependency between component labels, and solves the problem that the model is prone to overfitting due to the small sample size of pin images;

[0035] (2) The present invention can effectively improve the efficiency and accuracy of pin defect detection during the inspection process, and has good visualization effect. It can quickly identify the position and defect category of the pin in the image and output the coordinates of the target bounding box. It also enhances the model's logical reasoning ability by introducing a graph convolutional network model that enhances the pyramid network and knowledge graph relationship reasoning module through cross-domain interaction, thereby improving the accuracy of pin defect detection in small sample transmission lines.

[0036] (3) The present invention has a wide range of application scenarios. After obtaining data sets of multiple components, it can be expanded to realize defect detection of components such as insulators, suspension clamps and graded shielding rings that may be encountered during the inspection of transmission lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of the process flow of a small sample power transmission line pin defect detection method based on a graph convolutional network provided by the present invention;

[0038] Figure 2 is a schematic flow chart of step S2;

[0039] Figure 3 is a schematic flow chart of step S3;

[0040] Figure 4 is a schematic flow chart of step S4;

[0041] Figure 5 An example of a transmission line pin image collected in an embodiment of the present invention;

[0042] Figure 6 Transmission line pin images annotated based on event logic;

[0043] Figure 7 A method for constructing a position enhancement layer according to an embodiment of the present invention;

[0044] Figure 8 A method for constructing a semantic enhancement layer according to an embodiment of the present invention;

[0045] Fig. 9 This is an output picture after the transmission line pin image is detected according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The technical solutions of various embodiments of the present invention will be further described and described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] This embodiment provides a small sample transmission line pin defect detection method based on graph convolutional network, such as Figure 1-Figure 4 As shown, it includes the following steps:

[0048] S1 uses an image acquisition device such as a drone to collect multiple images, each of which contains pins on the power transmission line and installation accessories, to obtain a transmission line pin image, such as Figure 5 shown.

[0049] This embodiment targets conventional overhead transmission lines associated with transmission line towers.

[0050] S2 performs event logic-oriented annotation on the pins in the image to obtain a data set of power transmission line pins with logical associations. This step includes the following sub-steps:

[0051] S21 uses the transmission line connection standard and the inherent connection relationship between components as the event logic guide. For example, the insulator connection structure must be composed of a Z-type clamp, a yoke plate, a shackle and a drag plate, and the graded shielding ring structure is composed of a groove, a graded shielding ring, a yoke plate, an adjustment plate, a bolt and an extension ring. There is an inherent connection relationship between them. This connection relationship is marked as a logical guide. If the pin is the final target, the large targets containing the pin, such as the adjustment plate and the yoke plate, are marked first.

[0052] S22 Mark the pins on the marked yoke plate. Because there is an inherent connection between the adjustment plate, yoke plate and pins, it can serve as a logical guide to further small goals.

[0053] S23 marks the categories of the marked pins. Because there is an inherent connection between the pins and the pin clips, and the number of pin complete missing defects contained in the data set is very small, the pin categories are divided into pin clip missing and normal.

[0054] S24 uses the labeled labels with logical associations as the transmission line pin dataset, such as Figure 6 Shown is an image of a transmission line pin with event-based logic-guided labeling.

[0055] S3 introduces a cross-domain interactive enhanced pyramid network into the graph convolutional network to obtain an improved graph convolutional network model, which is trained using a data set of logically associated power line pins to obtain the original target detection model.

[0056] First, the self-attention mechanism is applied to the feature layer fusion process, and the target features located on different feature layers are enhanced by calculating the similarity of features in adjacent layers. The co-occurrence relationship is used to describe the occurrence of different object types in a high-probability scene, and the strength of the co-occurrence relationship is quantified by calculating the frequency of co-occurrence. Common co-occurrence relationships include adjustment plates and pins, graded shielding rings and grooves, pins and pin clamps. These co-occurrence relationships can be used to assist object detection at the semantic level, thereby improving detection accuracy. The specific steps are as follows:

[0057] S31 performs feature extraction with self-attention, and objects with similar scales can interact globally in the same feature layer to obtain features with global context information.

[0058] S32 enhances contextual information through cross-domain interactions between objects of different scales to preserve as much global contextual information as possible.

[0059] The cross-domain interactive feature pyramid network enhances the original features through the interaction of adjacent feature layers, thereby improving the feature pyramid. For small-scale objects, as the network goes deeper, the feature loss of small-scale objects will affect the extraction of contextual information during the interaction process. Similarly, for large-scale objects, the detailed features of the shallow feature layer are sufficient, but the small receptive field will lead to a lack of contextual information. Therefore, feature enhancement is needed to reduce the loss of features and enhance the contextual information of the object.

[0060] Specifically, the main enhancement method is established by constructing the position enhancement layer (LEL) and the semantic enhancement layer (SEL). First, the feature map of layer q (p-1) is downsampled, and the similarity w between layer p and layer q is calculated based on the self-attention idea. pq , and then the similarity w pq Acting on layer p, that is, performing matrix multiplication with p, we get LEL; the original feature map of the pyramid structure is X, and the enhanced feature map is X e Indicates that the calculation formula is: X e =F mul (w i,j ,v j ), where F mul is the weight aggregation function, w i,j is the similarity, v j is the key value in the self-attention mechanism; the method of obtaining SEL is similar to the previous one; then the feature map of layer q(p+1) is upsampled, and the similarity w between layer p and layer q is calculated based on the self-attention ideapq , and then the similarity w pq Acting on layer p, that is, performing matrix multiplication with p, SEL is obtained; the construction methods of the position enhancement layer and the semantic enhancement layer are respectively as follows Figure 7 and Figure 8 shown.

[0061] S33 divides the data set of logically associated power line pins into a training set and a validation set in a ratio of five to one, sets relevant hyperparameters and performs iterative training. After the model converges, that is, the model loss value remains approximately unchanged, the model parameters are saved as the original target detection model.

[0062] S4 introduces the knowledge graph relationship reasoning module into the original target detection model obtained in the previous step, refines the original detection results, and obtains the pin defect detection model. The specific steps are as follows:

[0063] S41 establishes a knowledge graph relationship between object categories. In this embodiment, nodes A and B of the knowledge graph represent a pin assembly and a pin clip assembly. In order to fully consider the occurrence frequency of the components, the edge between A and B is used to represent the co-occurrence relationship, which can be expressed as the conditional probability P(C B |C A ) in the form of:

[0064]

[0065] in, is the probability of B appearing when A exists, is the probability of A appearing, N all is the number of all components. Therefore, the co-occurrence frequency matrix of pins and pin clips can be converted into a co-occurrence probability matrix to obtain a constructed knowledge graph.

[0066] S42 uses the knowledge graph constructed in the dataset as a priori through the Knowledge Graph Relation Reasoning (KGRR) module, combines it with the graph convolutional neural network, and adaptively adjusts the feature graph in the original target detection model to obtain deep semantic information.

[0067] S43 applies semantic information to the original detection results, corrects the detection results of pins, excludes objects that cannot co-occur with pins in certain scenes, and adjusts the occurrence probability according to the closeness of the relationship between the pins and different components to obtain a transmission line pin defect detection model.

[0068] S5 preprocesses the transmission line pin images taken by the drone, specifically: adjusting the image size to 640*640, normalizing the image to transform the pixel value into the range of [0,1], and converting the image pixel value into a tensor for the next step of model detection.

[0069] S6 inputs the preprocessed image into the pin defect detection model for detection, and can generate an image that marks the pins in the image and gives the pin classification and category confidence. Specifically: use Pytorch to load the pre-trained model, pass the preprocessed image as input to the model; call the model's forward propagation method to pass the image input to the model for inference; the model detects the image, generates a bounding box, marks the pins in the image, and classifies them; for each detected pin, the model generates a corresponding category confidence, which is used to indicate the model's confidence level that the pin belongs to each category; based on the results of the model output, the marked pins can be drawn on the image, as well as the relevant category and confidence information, to intuitively display the pins in the image and their classification results, such as Fig. 9 shown.

[0070] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A small sample transmission line pin defect detection method based on graph convolutional network, characterized in that: The steps include: S1 uses an image acquisition device to acquire multiple images, each image includes a pin on a transmission line and an installation accessory, and obtains a transmission line pin image; S2 annotates the pins in the image based on event logic guidance to obtain a data set of power transmission line pins with logical associations; S3 introduces a cross-domain interactive enhanced pyramid network into the graph convolutional network to obtain an improved graph convolutional network model, and uses a data set of transmission line pins with logical associations for training to obtain an original target detection model; S4 introduces the knowledge graph relationship reasoning module into the original target detection model, refines the original detection results, and obtains the power transmission line pin defect detection model; S5 pre-processes the transmission line pin image captured by the image acquisition device; S6 inputs the preprocessed image into the power transmission line pin defect detection model for detection, and generates a pin image with marked pin position classification and confidence.

2. The small sample power transmission line pin defect detection method based on graph convolutional network according to claim 1 is characterized in that: The image acquisition device is a drone.

3. The small sample power transmission line pin defect detection method based on graph convolutional network according to claim 1 is characterized in that: The step S2 of labeling the pins in the image based on event logic guidance to obtain a data set of power transmission line pins with logical association includes the following sub-steps: S21 uses the transmission line connection standard and the inherent connection relationship between components as the event logic guide, and takes the pin as the final target, so the large target containing the pin is first marked; the large target is the adjustment plate and the yoke plate; S22 marks the pins on the marked yoke plate. There is an inherent connection between the adjustment plate, yoke plate and pins, which can serve as a logical guide to further small goals; S23 marks the categories of the marked pins. Because there is an inherent connection between the pins and the pin clips, and the number of pins completely missing defects contained in the data set is very small, the categories are divided into pin clip missing and normal; S24 uses the labeled labels with logical association as a transmission line pin dataset.

4. The small sample power transmission line pin defect detection method based on graph convolutional network according to claim 1 is characterized in that: In step S21, there is an inherent connection relationship between the components, such as the insulator connection structure is composed of a Z-shaped clamp, a yoke plate, a shackle and a drag plate, and the graded shielding ring structure is composed of a groove, a graded shielding ring, a yoke plate, an adjustment plate, a bolt, and an extension ring. Then there is an inherent connection relationship between them, which can be marked as a logical guide.

5. The small sample power transmission line pin defect detection method based on graph convolutional network according to claim 1 is characterized in that: The specific steps in step S3 are as follows: S31 performs feature extraction with self-attention, and objects with similar scales can interact globally in the same feature layer to obtain features with global context information; S32 enhances context information through cross-domain interactions between objects of different scales to preserve as much global context information as possible; Cross-domain Interaction Feature Pyramid Network (CDI-FPN) enhances the original features through the interaction of adjacent feature layers, thereby improving the feature pyramid. S33 divides the data set of logically associated power line pins into a training set and a validation set in a ratio of five to one, sets relevant hyperparameters and performs iterative training. After the model converges, that is, the model loss value remains approximately unchanged, the model parameters are saved as the original target detection model.

6. The small sample power transmission line pin defect detection method based on graph convolutional network according to claim 1 is characterized in that: The specific steps of step S4 are as follows: S41 establishes a knowledge graph relationship between object categories; nodes A and B of the knowledge graph represent the category of each component. In order to fully consider the occurrence frequency of the component, the edge between A and B is used to represent the co-occurrence relationship, which can be expressed as the conditional probability P(C B |C A ) in the form of: in, is the probability of B appearing when A exists, is the probability of A appearing, N all is the number of all components; therefore, the co-occurrence frequency matrix of pins and pin clips can be converted into a co-occurrence probability matrix to obtain a constructed knowledge graph; S42 uses the knowledge graph constructed in the dataset as a priori through the knowledge graph relation reasoning module, referred to as KGRR, and combines it with the graph convolutional neural network to adaptively adjust the feature graph in the original target detection model to obtain deep semantic information; S43 applies semantic information to the original detection results, corrects the detection results, excludes impossible objects, and adjusts the occurrence probability according to the closeness of the relationship between different objects to obtain a transmission line pin defect detection model.

7. The small sample power transmission line pin defect detection method based on graph convolutional network according to claim 1 is characterized in that: The image of the power transmission line pins captured by the image acquisition device is preprocessed in step S5, specifically: adjusting the image size, standardizing the image so as to transform the pixel value into a certain range, and converting the data type, that is, converting the image pixel value into a tensor.

8. The small sample power transmission line pin defect detection method based on graph convolutional network according to claim 1 is characterized in that: In step S6, the preprocessed image is input into the pin defect detection model for detection, so that an image of the pins in the image is marked and the pin classification and category confidence are given; specifically, Pytorch is used to load the pre-trained model, and the preprocessed image is passed as input to the model; by calling the forward propagation method of the model, the image input is passed to the model for inference; the model detects the image, generates a bounding box, marks the pins in the image, and classifies them; for each detected pin, the model will also generate a corresponding category confidence, which is used to indicate the model's confidence level that the pin belongs to each category; according to the results output by the model, the marked pins can be drawn on the image, as well as the relevant category and confidence information, to intuitively display the pins in the image and their classification results.

Images