A deep learning-based electric tower insulating paper detection method

Abstract

The application discloses a kind of based on deep learning's electric tower insulating paper detection method, realizes the detection of power grid insulating paper on rendering synthesis dataset, and can be used in actual life scene;The application first uses three-dimensional reconstruction technology, obtains a real electric tower model, and uses the model, in combination with the current mainstream rendering engine, rendering obtains the dataset that can be used for training target detection network;Subsequently, the dataset is sent into the feature extraction network;For a picture, multiple different dimensional effective picture features can be obtained, and different picture features are fused and enhanced by a feature fusion structure, and finally the enhanced picture features obtained are sent into the detection head of the detection network to obtain the final effective and accurate detection result.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer vision and computer graphics, and particularly relates to a tower insulating paper detection method based on deep learning. BACKGROUND

[0002] Insulating paper is commonly used as an insulating material for various circuit devices such as cables, coils, transformers, etc., and plays a crucial role in the process of power transmission in the power grid. However, the insulating paper used on the high-voltage transformer of the tower often has a service life limit and may be corroded due to natural reasons such as acid rain, and needs to be regularly inspected and replaced to ensure the safety of circuit transmission. Before that, the inspection of insulating paper usually requires manual on-site visual inspection or image shooting by controlling a drone, which is low in efficiency, time-consuming, and risky for maintenance personnel to shuttle in various complex terrains and landforms of the power grid, which is not conducive to the maintenance of the power grid lines.

[0003] In recent years, with the development of artificial intelligence, deep learning has been widely and successfully applied in solving various visual tasks, such as target detection, target recognition and classification. Using deep learning instead of manual work can save a lot of human resource costs. And computer rendering technology is also gradually improving, using computer rendering virtual scenes can get images close to the real camera shooting effect, which just solves the problem that the effect of deep learning method depends largely on the quality of the data set, but the real data set needs to consume more manpower and material resources to collect. The combination of deep learning and rendering technology can greatly save material and time costs.

[0004] Deep learning and rendering technology have been widely applied, but there is still a lack of corresponding technology to combine the two and apply them to power grid insulating paper detection. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of the prior art and provide a tower insulating paper detection method based on deep learning.

[0006] The purpose of the present application is achieved by the following technical solutions:

[0007] A tower insulating paper detection method based on deep learning, comprising the following steps:

[0008] Collecting tower model parameters, real tower images and various ground background parameters,

[0009] to construct the scene of the tower on various complex terrains;

[0010] Physical rendering, the constructed scene of the tower on various complex terrains is physically rendered,

[0011] To obtain a rendered electric tower image and a rendered electric grid insulation paper picture;

[0012] Construct a training model to train the rendered electric tower image with real electric tower images,

[0013] Obtain an electric tower image training model to construct a detection network;

[0014] Construct an insulation paper detection network to obtain detection results by comparing the insulation paper detection network with the detection network.

[0015] Further, the specific steps of rendering on a variety of complex landscapes are as follows: randomly generating a camera perspective around the electric pole, while ensuring that the camera always faces the center part of the electric pole.

[0016] Further, the specific way to ensure that the camera always faces the center part of the electric pole is:

[0017] In the constructed three-dimensional scene, the center position world coordinates of the electric tower object are obtained through the built-in centroid calculation function ;

[0018] When rendering an electric tower image with a random perspective each time, a random offset is added to the center position of the electric tower object, and the world coordinates of the camera imaging focus point in this rendering image are ;

[0019] Determine the camera focus position, set a random value for the distance from the camera to the focus point , and set a random camera orientation three-dimensional vector , and obtain the world coordinates of the camera as .

[0020] Further, the specific steps of constructing the detection network are as follows:

[0021] Image feature extraction, extracting part of the CSPDarknet structure;

[0022] Feature extraction part FPN, the features obtained by splicing and fusing different dimensions of features extracted in CSPDarknet;

[0023] Network detection head construction, including class classification and detection frame regression prediction part,

[0024] Class classification is used to classify features to determine whether it is insulation paper, and the detection frame regression part is used to adjust the position and shape size of the insulation paper detection frame.

[0025] As preferred, the CSPDarknet structure comprises a Focus structure, an SPP structure and a CSPnet, which are used to construct a detection network convolutional layer, and the specific steps are as follows:

[0026] The Focus structure is used to expand the number of image feature channels, and a value is obtained every other pixel in each feature channel of the image, four independent feature layers are obtained, and the four feature layers are stacked to expand the number of feature channels by four times.

[0027] The SPP structure is used to perform maximum pooling on the image features using different size pooling kernels to improve the receptive field of the network.

[0028] The CSPnet structure is a residual structure, and the backbone part is stacked with convolutional blocks, and the other part is spliced with the backbone part.

[0029] Further, the specific steps of rendering the power grid insulation paper picture are as follows:

[0030] The 8 vertex coordinates of the bounding box of the insulation paper in the three-dimensional space are confirmed, and the coordinate values of the 8 vertex coordinates in the two-dimensional imaging plane are obtained through the camera projection matrix.

[0031] Then, the maximum and minimum values of the x-axis and y-axis are taken as the bounding box range in the two-dimensional plane.

[0032] There are multiple insulation papers in a scene, that is, there are multiple bounding boxes on the two-dimensional image plane, and the bounding boxes are screened through the effective range of image pixel values to determine the effective bounding box as the real label of the image target detection.

[0033] As preferred, the method for screening the bounding box further comprises the following specific steps:

[0034] A specific channel value is assigned to the insulation paper object, and other parts such as power poles and ground are assigned other same channel values, and a semantic segmentation image for the insulation paper is obtained through a channel rendering mode.

[0035] When the pixel proportion of the insulation paper in the bounding box region is greater than 30%, it is set as an effective target, and the real value of the synthetic rendering data set is constructed.

[0036] The detection frame is established through the real value of the data set.

[0037] Further, the specific method for constructing the insulation paper detection network is as follows: ​

[0038] First, the rendered power grid insulation paper picture is sent into the CSPDarknet for feature extraction. The extracted features can be referred to as feature layers, which are a set of features of the input picture. In the main part, three feature layers are obtained for the construction of the next network. The three feature layers are effective feature layers. The three feature layers are located at different positions of the main part CSPdarknet, namely the middle layer, the lower middle layer, and the bottom layer. When the input is (640, 640, 3), the shapes of the three feature layers are feat1=(80, 80, 256), feat2=(40, 40, 512), and feat3=(20, 20, 1024), respectively.

[0039] After obtaining the three effective feature layers in the main part, feature fusion is performed by constructing the FPN layer. The specific construction method is as follows: using the feat3 feature layer to perform 1X1 convolution to adjust the channel to obtain P5, performing upsampling on P5 to combine with the feat2 feature layer to obtain P5_upsample3, performing 1X1 convolution on the P5_upsampl feature layer to adjust the channel to obtain P4, performing upsampling on P4 to combine with the feat1 feature layer to obtain P3_out, performing 3x3 convolution on the P3_out feature layer to downsample, and then stacking P4_out to obtain P4_out, performing 3x3 convolution on the P4_out feature layer to downsample, and then stacking P5 to obtain P5_out.

[0040] The enhanced features p3_out, p4_out, and p5_out obtained by constructing the FPN layer are sent to the detection network detection head for prediction of the detection frame to obtain the final detection result.

[0041] Further, the specific method of the insulation paper detection network for obtaining the detection result compared with the detection network is as follows:

[0042] The enhanced features obtained by the FPN layer in the insulation paper detection network are sent to the detection network detection head for prediction of the detection frame to obtain the final detection result.

[0043] It should be noted that the specific steps of the insulation paper detection network compared with the detection network are as follows:

[0044] Each feature input to the network detection head will obtain three prediction results, Reg, Obj, and Cl. Reg contains the center position and size information of the detection frame, Obj represents whether there is an object similar to insulation paper in the detection frame, and Cls represents the confidence of the object in the detection frame being insulation paper, with a value between 0 and 1.

[0045] After obtaining the three prediction results, score screening and non-maximum suppression are performed.

[0046] Score screening is to judge the confidence of the insulation paper in the detection frame, and only when the confidence is greater than a certain value, the insulation paper is determined, and the value is set to 0.5;

[0047] In the detection result, there may be multiple detection frames containing the same insulation paper, at this time, non-maximum suppression is used for screening of the detection frame, first, the frame with the maximum confidence in all overlapping insulation paper detection frames is taken out, then the overlapping degree IOU value of other insulation paper detection frames and it is calculated, when IOU is greater than a certain value, the corresponding detection frame is removed, the value is set to 0.5;

[0048] After score screening and non-maximum suppression, the final detection result can be obtained.

[0049] The beneficial effects of the present application are:

[0050] The present application trains the deep learning network model by using the power grid insulation paper picture obtained by rendering, which can obtain accurate and effective target detection result, and combines deep learning with rendering technology, which greatly saves the cost of materials and time. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 It is a detection network structure diagram of the present application;

[0052] Figure 2 It is an electric tower image under various landforms in the embodiment of the present application;

[0053] Figure 3 It is a detection frame and mask instance diagram in the embodiment of the present application;

[0054] Figure 4 It is an insulation detection result diagram in the embodiment of the present application. IMPLEMENTATION

[0055] The technical solutions of the present application will be described clearly and completely below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0056] Referring to Figures 1-4 , the present application provides a technical solution:

[0057] A power tower insulation paper detection method based on deep learning, comprising the following steps:

[0058] Collecting power tower model parameters, real power tower images and various ground background parameters,

[0059] To construct scenarios of power towers in various complex terrains,

[0060] This embodiment uses the Blender rendering engine as an example, combining the power tower model with ground backgrounds of various materials to construct scenes of the power tower on various complex terrains, such as... Figure 2 As shown, by using the realistic physical rendering engine Blender, very realistic images of power towers can be rendered. The model can be trained with more realistic images of power towers to improve the model's recognition ability in various real natural environments.

[0061] This embodiment constructs multiple scenes with different terrains. During rendering, the camera viewpoint is randomly generated around the utility pole, while ensuring that the camera always faces the center of the utility pole. The specific operation steps are as follows:

[0062] In this embodiment, the world coordinates of the center position of the power tower object are obtained using Blender's built-in centroid calculation function in a 3D scene constructed by Blender.

[0063] Each time an image of a power tower is rendered from a random viewpoint, this embodiment adds a random offset to the center position of the power tower object. , The value of is within a suitable range to ensure that the power tower is always within the camera's imaging range. The world coordinates of the camera's focal point during this image rendering are...

[0064] After determining the camera's focus position, this embodiment sets a random value for the distance from the camera to the focus point. Simultaneously, a random camera orientation 3D vector is set. The final world coordinates of the camera are:

[0065] By rendering the scene using Blender's built-in physical rendering engine, a highly realistic image of the power tower can be obtained, which serves as the dataset sample for object detection in this embodiment.

[0066] Physically rendered graphics are used to render the constructed power towers across various complex terrains.

[0067] To obtain rendered images of the power tower and the power grid insulation paper;

[0068] Meanwhile, during rendering, this embodiment of the invention uses Blender's built-in functions to pre-determine the coordinates of the eight vertices of the insulating paper's bounding box in three-dimensional space, and obtains the coordinates of these eight vertices on the two-dimensional imaging plane through the camera projection matrix, and then takes... The maximum value and the minimum value on the axis are taken as the bounding box range on a two-dimensional plane. There are multiple insulating papers in a scene, that is, there are multiple bounding boxes on a two-dimensional image plane. Since the insulating paper may be outside the image range, the embodiment of the application filters the bounding box according to the effective range of the image pixel value, and determines the effective bounding box as the target detection real label of the image.

[0069] Meanwhile, since part of the insulating paper in the imaging image result may be blocked, the embodiment of the application uses a mask image as a condition for screening the bounding box, and the specific operation is as follows:

[0070] A specific channel value is given to the insulating paper object in Blender, and other parts such as the power pole and the ground are given other same channel values. Through the channel rendering mode built in Blender, a semantic segmentation image for the insulating paper can be obtained, as shown in the intermediate image. Figure 2

[0071] Meanwhile, in order to avoid invalid labels, that is, only a small part of the insulating paper is within the imaging range, at this time the human eye is also difficult to judge where the insulating paper is, so the detection network should not detect this part of the insulating paper. Therefore, the embodiment of the application sets that when the pixel proportion of the insulating paper in the bounding box region is more than 30%, it is counted as an effective target, thereby constructing the real value of the synthetic rendering data set of the embodiment of the application. The bounding box and the detection frame are shown in Figure 3

[0072] A training model is constructed to train the rendered power tower image with the real power tower image,

[0073] The power tower image training model is obtained to construct a detection network.

[0074] The network as a whole can be divided into three main parts, which are an image feature extraction part CSPDarknet, a feature enhancement extraction part FPN, and a network detection head part. As shown in Figure 1 , the image feature extraction part and the feature enhancement extraction part; the specific structure of each part is as follows:

[0075] ​​The image feature extraction part CSPDarknet includes a Focus structure, an SPP structure and a CSPnet two main parts. The Focus part expands the number of image feature channels. Specifically, a value is taken every other pixel in each feature channel of the original image, four independent feature layers are obtained at this time, and then the four feature layers are stacked, at this time the number of image feature channels is quadrupled. The SPP part uses different size pooling kernels for maximum pooling on the image features to improve the receptive field of the network. The CSPnet can be regarded as a large residual structure. The main part is stacked with convolution blocks, and the other part is processed like a residual edge and directly spliced with the main part.

[0076] The feature extraction part FPN is to splice and fuse the features of different dimensions extracted in the CSPDarknet to obtain better feature representation.

[0077] The network detection head part is to predict the detection result, including class classification and detection frame regression prediction part. The class classification is responsible for classifying the features to determine whether it is insulating paper. The detection frame regression part is responsible for adjusting the position and shape size of the insulating paper detection frame.

[0078] The insulating paper detection network is constructed, and the detection results are obtained by comparing the detection network.

[0079] The specific detection steps of the insulating paper detection network are as follows:

[0080] The rendered power grid insulating paper picture is sent into the CSPDarknet for feature extraction. The extracted features can be called feature layers, which are a feature set of the input picture. In the main part, three feature layers are obtained for the next network construction, which are called effective feature layers. The three feature layers are located at different positions of the main part CSPdarknet, namely the middle layer, the middle lower layer and the bottom layer. When the input is (640, 640, 3), the shapes of the three feature layers are feat1=(80, 80, 256), feat2=(40, 40, 512) and feat3=(20, 20, 1024) respectively.

[0081] After obtaining three effective feature layers in the backbone part, feature fusion is performed by constructing FPN layers. The specific construction method is as follows: using the feat3 feature layer to perform 1x1 convolution to adjust the channel to obtain P5, performing upsampling on P5 to combine with the feat2 feature layer to obtain P5_upsample3, performing 1x1 convolution on the P5_upsample3 feature layer to adjust the channel to obtain P4, performing upsampling on P4 to combine with the feat1 feature layer to obtain P3_out, performing 3x3 convolution on the P3_out feature layer to downsample, and stacking P4 to obtain P4_out, performing 3x3 convolution on the P4_out feature layer to downsample, and stacking P5 after down-sampling to obtain P5_out.

[0082] Through the construction of the FPN layer, enhanced features p3_out, p4_out and p5_out are obtained, and the features are sent to the detection network detection head to predict the detection frame and obtain the final detection result.

[0083] Finally, the detection result is confirmed, as shown in Figure 4

[0084] For each feature input to the network detection head, three prediction results are obtained,

[0085] Respectively, Reg, Obj and Cls, wherein Reg contains the center position and size information of the detection frame;

[0086] Obj represents whether there is an object similar to the insulating paper in the detection frame;

[0087] Cls represents the confidence of the object in the detection frame being insulating paper, and its value is between 0 and 1;

[0088] After obtaining the three prediction results, score screening and non-maximum suppression are performed:

[0089] Score screening is to judge the confidence of the insulating paper in the detection frame. Only when the confidence is greater than a certain value is the insulating paper determined, and the value is set to 0.5;

[0090] There may be multiple detection frames containing the same insulating paper in the detection result. At this time, non-maximum suppression is used to screen the detection frame. First, the detection frame with the highest confidence in all overlapping insulating paper detection frames is taken out, and then the IOU value of the overlapping degree of other insulating paper detection frames and it is calculated. When the IOU is greater than a certain value, the corresponding detection frame is removed, and the value is set to 0.5;

[0091] After score screening and non-maximum suppression, the final detection result is obtained. As can be seen from the detection effect diagram, using the rendered power grid insulating paper picture to train the deep learning network model can obtain accurate and effective target detection results.​

[0092] The foregoing is considered as illustrative only of the principles of the application. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the application to the exact construction and practice described. Accordingly, all such variations are intended to be included within the scope of the present application as defined in the following claims, along with full authority to claim such combinations, modifications and equivalents that come within the scope of the concepts disclosed herein.

Claims

1. A deep learning-based tower insulating paper detection method, characterized in that: Comprise the following steps: Collecting electric tower model parameters, real electric tower images and various ground background parameters, To build the scene of the electric tower on various complex terrains; Physical rendering, the constructed electric tower on various complex terrains is rendered through physical rendering, To obtain the rendered electric tower image and the rendered electric grid insulating paper picture; Build a training model to train the rendered electric tower image with real electric tower images, Get the electric tower image training model to build the detection network; Build an insulating paper detection network to compare the detection results of the insulating paper detection network and the detection network; Specifically, the specific steps of constructing the detection network are: Image feature extraction, extract part of CSPDarknet structure; Feature extraction part FPN, the features obtained by splicing and fusing the features of different dimensions extracted in CSPDarknet; Network detection head construction, including class classification and detection frame regression prediction part, Class classification is used to classify features to determine whether it is insulating paper, and the detection frame regression part is used to adjust the position and shape size of the insulating paper detection frame; The CSPDarknet structure includes Focus structure, SPP structure and CSPnet, which are used to build the detection network convolution layer, and the specific steps are as follows: The Focus structure is used to expand the number of image feature channels, and a value is obtained every other pixel in each feature channel of the image, obtaining four independent feature layers, which are stacked to expand the feature channel number by four times; The SPP structure is used to perform maximum pooling on the image features using different size pooling kernels to improve the network's receptive field; The CSPnet structure is used as a residual structure, and the main part is stacked with convolution blocks, and the other part is spliced with the main part; The specific method of constructing the insulating paper detection network is to send the rendered electric grid insulating paper picture into CSPDarknet for feature extraction; After obtaining three effective feature layers in the main part, feature fusion is performed through the construction of FPN layer; Enhanced features are obtained through the construction of FPN layer; The specific method of the insulating paper detection network compared with the detection network to obtain the detection result is: send the enhanced features obtained by the FPN layer in the insulating paper detection network to the detection head of the detection network for detection frame prediction to obtain the final detection result; Specifically, one enhanced feature is input into the network detection head to obtain three prediction results, Reg result, Reg result includes the center position and size information of the detection frame; Obj result, used to represent whether there is an object similar to insulating paper in the detection frame; Cls result, used to represent the confidence of the object in the detection frame being insulating paper; Score screening and non-maximum suppression are performed on Reg result, Obj result and Cls result, and the final detection result is obtained after score screening and non-maximum suppression.

2. The electric tower insulating paper detection method based on deep learning according to claim 1, wherein: The specific steps of rendering on various complex terrains are as follows: The main camera view is randomly generated around the electric pole, and the main camera view is always directed to the center part of the electric pole.

3. The deep learning-based electric tower insulating paper detection method according to claim 2, characterized in that: The implementation method of the main camera view always directed to the center part of the electric pole is as follows: In the constructed three-dimensional scene, the center position world coordinates of the electric tower object are obtained through centroid calculation ; a random offset is added to the center position of the power tower object each time a power tower image is rendered from a random viewpoint the world coordinates of the camera imaging focus point at the time of the rendered image ; determining a camera focus position, setting a random value for the distance of the camera to the focus point while setting a random camera orientation three-dimensional vector obtaining the world coordinates of the camera as .

4. The deep learning-based electric tower insulating paper detection method according to claim 1, characterized in that: The specific steps of rendering the electric network insulating paper picture are included: Confirm the 8 vertex coordinates of the bounding box of the insulating paper in the three-dimensional space, and obtain the coordinate values of the 8 vertex coordinates in the two-dimensional imaging plane through the camera projection matrix; respectively the maximum and minimum values on the axis as the bounding box range on the two-dimensional plane; When there are multiple insulating papers in the same scene, that is, there are multiple bounding boxes on the two-dimensional image plane, the bounding box is screened through the effective range of image pixel values to determine the effective bounding box as the real label of the target detection of the image.

5. The deep learning-based electric tower insulating paper detection method according to claim 4, characterized in that: The method of screening the bounding box is also included, and the specific steps are as follows: A specific channel value is assigned to the insulating paper object; Other parts, including the electric pole and the ground, are assigned other same channel values; The channel is rendered through the specific value to obtain a semantic segmentation image for the insulating paper; When the pixel proportion of the insulating paper in the bounding box area is greater than 30%, it is regarded as an effective target, and the real value of the synthetic rendering data set is constructed; The detection frame is established through the real value of the data set.

Images