Power line defect detection method, device, equipment, medium and program product

Abstract

The invention relates to a power line defect detection method, device and equipment, a medium and a program product. The power line defect detection method comprises the steps of obtaining a line image of a power line; extracting line feature data in the line image based on a feature extraction network; based on a defect position identification network, determining at least one initial identification area in the line image according to the line feature data; and according to the line feature data, adjusting the at least one initial identification area to obtain a corresponding updated identification area, and identifying a target identification area containing the defect of the power line from each updated identification area. By adopting the method, the determined identification area can be closer to the actual power line defect area, and the identification accuracy of the power line defect position can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of defect detection, and particularly to a method, device, equipment, medium and program product for detecting power line defects. Background Art

[0002] With the continuous expansion of the power grid scale, the complexity of the power system has also increased day by day, and it has become particularly important to effectively identify defects in power lines. This is because the defects in power lines not only affect the stable operation of the power system, but may also lead to serious safety accidents.

[0003] Traditional methods for identifying power line defects mainly rely on manual inspection and image processing technology. Among them, although the manual inspection method can discover some obvious defects to a certain extent, it is limited by human factors, the precision of detection equipment, and the complexity of the detection environment. Compared with the manual inspection method, using image processing technology for defect identification can obviously reduce the interference of human factors, but there is still a problem of low accuracy in defect identification based on image processing technology at present. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, equipment, medium and program product for detecting power line defects to solve the above technical problems and improve the accuracy of power line defect identification.

[0005] In a first aspect, the present application provides a method for detecting power line defects, including:

[0006] Obtaining a line image of a power line;

[0007] Extracting line feature data in the line image based on a feature extraction network;

[0008] Based on a defect location recognition network, determining at least one initial recognition area in the line image according to the line feature data; and,

[0009] Adjusting at least one initial recognition area according to the line feature data to obtain a corresponding updated recognition area;

[0010] Identifying a target recognition area containing a power line defect from each updated recognition area.

[0011] In one embodiment, adjusting at least one initial recognition area according to the line feature data to obtain a corresponding updated recognition area includes: determining geometric adjustment data of each initial recognition area according to the line feature data; and adjusting the corresponding initial recognition area according to the geometric adjustment data of the initial recognition area to obtain an updated recognition area of the initial recognition area.

[0012] In one embodiment, the geometric adjustment data of the initial recognition region includes the adjustment data of the center point of the initial recognition region; correspondingly, according to the line feature data, determining the geometric adjustment data of each initial recognition region includes: according to the line feature data, determining the sampling point adjustment data of each preset sampling point in each initial recognition region; according to the sampling point adjustment data of each preset sampling point in the initial recognition region, determining the center point adjustment data of the corresponding initial recognition region.

[0013] In one embodiment, identifying a target recognition region containing a power line defect from each updated recognition region includes: for each updated recognition region, determining the confidence level of the updated recognition region according to the line feature data; the confidence level is used to characterize the credibility of the power line defect contained in the updated recognition region; determining the target recognition region according to the confidence levels corresponding to the updated recognition regions.

[0014] In one embodiment, it further includes: based on the defect type recognition network, determining the defect type of the power line defect in the target recognition region according to the line feature data; in the line image, correspondingly marking the defect type.

[0015] In one embodiment, it further includes: obtaining a sample line image of the power line and the defect region of the power line defect in the sample line image; based on the feature extraction network to be trained and the defect position recognition network to be trained, determining the sample updated recognition region in the sample line image; for each sample updated recognition region, determining the similarity between the sample updated recognition region and the corresponding defect region; using the sample recognition region with a similarity greater than the similarity threshold as a positive sample; using the sample recognition region with a similarity not greater than the similarity threshold as a negative sample; selecting training samples from each positive sample and each negative sample according to a set ratio to train the feature extraction network and the defect position recognition network to adjust the network parameters of the feature extraction network and the defect position recognition network.

[0016] In a second aspect, the present application also provides a power line defect detection device, including:

[0017] A first acquisition module, configured to acquire a line image of a power line;

[0018] An extraction module, configured to extract line feature data in the line image based on a feature extraction network;

[0019] A first determination module, configured to determine at least one initial recognition region in the line image based on a defect position recognition network according to the line feature data;

[0020] An adjustment module, configured to adjust at least one initial recognition region based on a defect position recognition network according to the line feature data to obtain a corresponding updated recognition region;

[0021] An identification module, configured to identify a target identification area containing power line defects from each updated identification area.

[0022] In a third aspect, the present application further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0023] Obtain a line image of a power line;

[0024] Extract line feature data in the line image based on a feature extraction network;

[0025] Based on a defect position identification network, determine at least one initial identification area in the line image according to the line feature data; and,

[0026] Adjust at least one initial identification area according to the line feature data to obtain corresponding updated identification areas;

[0027] Identify a target identification area containing power line defects from each updated identification area.

[0028] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0029] Obtain a line image of a power line;

[0030] Extract line feature data in the line image based on a feature extraction network;

[0031] Based on a defect position identification network, determine at least one initial identification area in the line image according to the line feature data; and,

[0032] Adjust at least one initial identification area according to the line feature data to obtain corresponding updated identification areas;

[0033] Identify a target identification area containing power line defects from each updated identification area.

[0034] In a fifth aspect, the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0035] Obtain a line image of a power line;

[0036] Extract line feature data in the line image based on a feature extraction network;

[0037] Based on a defect position identification network, determine at least one initial identification area in the line image according to the line feature data; and,

[0038] Adjust at least one initial recognition region according to the line feature data to obtain corresponding updated recognition regions;

[0039] Identify target recognition regions containing power line defects from the updated recognition regions.

[0040] The above power line defect detection method, device, equipment, medium and program product obtain the line image of the power line, extract the line feature data in the line image based on the feature extraction network, thereby providing a data basis for the recognition of power line defects. By using the defect position recognition network, at least one initial recognition region in the line image is determined according to the line feature data, so as to locate the region where power line defects may exist. By adjusting at least one initial recognition region according to the line feature data to obtain corresponding updated recognition regions, the adaptive adjustment of the initial recognition region is realized, the problem of mismatch between the initial recognition region and the line feature data is solved, the determined updated recognition region is closer to the actual power line defect region, which is beneficial to improving the recognition accuracy of the power line defect position. By identifying target recognition regions containing power line defects from the updated recognition regions, the detection of power line defects is realized. Description of the Drawings

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for the description of the embodiments of the present application or related technologies. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other related drawings can be obtained according to these drawings without creative efforts.

[0042] Figure 1 It is a schematic flow chart of the power line defect detection method in one embodiment;

[0043] Figure 2 It is a schematic flow chart of the adjustment steps of the initial recognition region in one embodiment;

[0044] Figure 3A It is a schematic structural diagram of the power line defect detection model in one embodiment;

[0045] Figure 3B It is a schematic structural diagram of the detection head network in one embodiment;

[0046] Figure 4 It is a schematic flow chart of the model training steps in one embodiment;

[0047] Figure 5 It is a reference schematic diagram of the target recognition region in one embodiment;

[0048] Figure 6 It is a schematic flowchart of a power line defect detection method in another embodiment;

[0049] Figure 7 It is a structural block diagram of a power line defect detection device in one embodiment;

[0050] Figure 8 It is an internal structure diagram of a computer device in one embodiment. Detailed implementation manners

[0051] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] In one embodiment, as Figure 1 shown, a power line defect detection method is provided. In this embodiment, it is exemplified that this method is applied to a terminal. It can be understood that this method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0053] S110. Obtain a line image of a power line.

[0054] Among them, the power line may include at least one of line types such as a transmission line and a distribution line, and the present application does not make any limitation on the specific type of the power line. The line image can be understood as a visual image of the power line.

[0055] In an optional embodiment, an original line image of the power line may be obtained, and an image containing a power line defect is screened out from the original line image as the line image of the power line. In another optional embodiment, the original line image of the power line may be used as the line image of the power line.

[0056] Among them, the original line image can be collected by an image collection device, and the image collection device may include at least one of a camera and a scanner. The power line defect may include at least one of types such as conductor strand breakage, foreign object hanging, and insulator rupture, and the present application does not make any limitation on the specific type of the power line defect.

[0057] S120. Extract line feature data in the line image based on a feature extraction network.

[0058] In an optional embodiment, the feature extraction network may include a backbone network and a feature pyramid network.

[0059] Exemplarily, the backbone network can be used to extract multi-scale features in the line. The Feature Pyramid Network can be used to fuse the multi-scale features output by the backbone network to obtain line feature data. By fusing the multi-scale features, the high-level features and low-level features in the line image can be combined to generate a richer and more accurate feature representation, that is, the line feature data.

[0060] Exemplarily, the backbone network can adopt the ResNet50 (Residual Network 50) network.

[0061] S130. Based on the defect location recognition network, at least one initial recognition region in the line image is determined according to the line feature data.

[0062] Among them, the initial recognition region can be understood as a candidate region that may have power line defects initially recognized from the line image.

[0063] In an optional embodiment, the initial recognition region can be represented by an initial recognition box, and the initial recognition box is used to characterize the boundary of the initial recognition region. Among them, the initial recognition box is also the prior anchor box. The present application does not make any limitation on the specific shape and size of the initial recognition box. Exemplarily, the initial recognition box can be at least one of a rectangular box and a circular box, etc.

[0064] In an optional embodiment, the defect location recognition network can include a first dilated convolutional network. The first dilated convolutional network is used to determine at least one initial recognition region in the line image according to the line feature data.

[0065] Exemplarily, the feature map corresponding to the line image can be determined according to the line feature data; the initial recognition regions corresponding to different target feature points in the feature map are determined to obtain the initial recognition region set A. Among them, the initial recognition region set A is also the prior anchor box set. ; where represents the feature map size, C is the number of shape information, and P represents the parameter space. It should be noted that for each target feature point, the number of initial recognition regions corresponding to the target feature point can be one or more, and the present application does not make any limitation on the specific number of initial recognition regions corresponding to the target feature point.

[0066] S140. At least one initial recognition region is adjusted according to the line feature data to obtain the corresponding updated recognition region.

[0067] Among them, the updated recognition region can be represented by an updated recognition box, and the updated recognition box is also the dynamic prior anchor box. The updated recognition box is used to characterize the boundary of the updated recognition region. The present application does not make any limitation on the specific shape and size of the updated recognition box.

[0068] In an alternative embodiment, adjusting the initial recognition region may include: adjusting at least one of the regional position, regional shape, and regional size of the initial recognition region, etc.

[0069] In an alternative embodiment, the defect position recognition network may include a deformable convolutional network. The deformable convolutional network is used to adjust at least one initial recognition region according to the line feature data to obtain corresponding updated recognition regions. Among them, the deformable convolutional network is a learnable network, and the line feature data and at least one initial recognition region can be input into the deformable convolutional network to obtain corresponding updated recognition regions.

[0070] Exemplarily, at least one initial recognition region can be adjusted through the following formula to obtain corresponding updated recognition regions:

[0071]

[0072] Among them, A represents the set of initial recognition regions; represents the set of updated recognition regions; represents the deformable convolutional network.

[0073] S150. Identify target recognition regions containing power line defects from each updated recognition region.

[0074] In an alternative embodiment, a non-maximum suppression method can be adopted to screen out duplicate or overlapping target recognition regions from each target recognition region.

[0075] In an alternative embodiment, for each updated recognition region, the confidence of the updated recognition region can be determined according to the line feature data; the confidence is used to characterize the credibility of the power line defect contained in the updated recognition region; the target recognition region is determined according to the confidence corresponding to each updated recognition region.

[0076] Exemplarily, the updated recognition region with the highest confidence can be selected from each updated recognition region as the target recognition region according to the number of targets. Exemplarily, the preset number of regions can be one. Correspondingly, the updated recognition region with the highest confidence can be selected from each updated recognition region as the target recognition region.

[0077] Exemplarily, the updated recognition regions with a confidence greater than the preset confidence can be selected from each updated recognition region as the target recognition regions. The preset confidence can be set by those skilled in the art according to needs or experience, or determined through a large number of experiments, and the present application does not make any limitation thereto.

[0078] In an optional embodiment, the target recognition area of the power line defect can be correspondingly marked in the line image.

[0079] In an optional embodiment, based on the defect type recognition network, according to the line feature data, the defect type of the power line defect in the target recognition area can be determined; in the line image, the defect type is correspondingly marked.

[0080] Optionally, the defect type recognition network may include a second dilated convolutional network, and the second dilated convolutional network is used to determine the defect type of the power line defect in the target recognition area according to the line feature data.

[0081] In an optional embodiment, based on the detection head, and according to the following formula, the detection result of the power line defect can be determined :

[0082]

[0083] Wherein, represents the updated recognition area set; represents the detection head.

[0084] It should be noted that the detection result may include the classification score and the regional position . The classification score is used to determine the defect type of the power line defect, and the regional position is used to determine the target recognition area of the power line defect.

[0085] Among them, the classification score can be expressed as: ; represents the feature map size, and M represents the defect type number.

[0086] Among them, the regional position can be expressed as: ; represents the feature map size, and N represents the number of target recognition area parameters.

[0087] The above power line defect detection method obtains the line image of the power line, extracts the line feature data in the line image based on the feature extraction network, thereby providing a data basis for the identification of power line defects. Based on the defect position recognition network, at least one initial recognition area in the line image is determined according to the line feature data, so as to locate the area where power line defects may exist. By adjusting at least one initial recognition area according to the line feature data, the corresponding updated recognition area is obtained, thereby realizing the adaptive adjustment of the initial recognition area, solving the problem that the initial recognition area does not match the line feature data, making the determined updated recognition area closer to the actual power line defect area, and being beneficial to improving the recognition accuracy of the power line defect position. By identifying the target recognition area containing the power line defect from each updated recognition area, the detection of the power line defect is realized.

[0088] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment, in which the adjustment step of the initial recognition area is refined.

[0089] See Figure 2 The adjustment step of the initial recognition area shown includes:

[0090] S210. Determine the geometric adjustment data of each initial recognition area according to the line feature data.

[0091] In an optional embodiment, the geometric adjustment data of the initial recognition area may include the sampling point adjustment data of the initial recognition area and / or the center point adjustment data of the initial recognition area.

[0092] Optionally, the sampling point adjustment data of each preset sampling point in each initial recognition area may be determined according to the line feature data.

[0093] Optionally, the line feature data and the area data corresponding to the initial recognition area may be input into the convolutional layer of the deformable convolutional network to obtain the sampling point adjustment data of each preset sampling point in each initial recognition area. Exemplarily, taking a convolutional kernel of size 3*3 as an example, the sampling point adjustment data of a certain initial recognition area includes: the adjustment data of the 9 preset sampling points corresponding to the 3*3 convolutional kernel in the horizontal and vertical directions.

[0094] Optionally, the center point adjustment data of the corresponding initial recognition area may be determined according to the sampling point adjustment data of each preset sampling point in the initial recognition area. Exemplarily, the average value of the sampling point adjustment data of each preset sampling point may be taken in the horizontal and vertical directions respectively to obtain the center point adjustment data.

[0095] S220. Adjust the corresponding initial recognition region according to the geometric adjustment data of the initial recognition region to obtain an updated recognition region of the initial recognition region.

[0096] In an optional embodiment, the corresponding initial recognition region may be adjusted according to the sampling point adjustment data of the initial recognition region and / or the center point adjustment data of the initial recognition region to obtain an updated recognition region of the initial recognition region.

[0097] Optionally, based on deformable convolution, the sampling positions of the preset sampling points of the corresponding initial recognition region may be adjusted according to the sampling point adjustment data of the initial recognition region to obtain an updated recognition region of the initial recognition region.

[0098] Optionally, based on deformable convolution, the center point of the corresponding initial recognition region may be adjusted according to the center point adjustment data of the initial recognition region to obtain an updated recognition region of the initial recognition region. Exemplarily, the spatial position of the target feature point in the feature map corresponding to the initial recognition region may be remapped to the line image to obtain the center point of the initial recognition region. Among them, the center point of the initial recognition region is also the prior position.

[0099] In the above steps, by determining the geometric adjustment data of each initial recognition region according to the line feature data, a basis is provided for the position adjustment of the initial recognition region. By adjusting the corresponding initial recognition region according to the geometric adjustment data of the initial recognition region, an updated recognition region is obtained, solving the problem of mismatch between the initial recognition region and the line feature data. In addition, by using a learnable deformable convolution network to determine the geometric adjustment data of each initial recognition region, it is beneficial to improve the determination efficiency and data accuracy of the geometric adjustment data.

[0100] In one embodiment, as Figure 3A shown, a power line defect detection model is provided, including a backbone network 1, a feature pyramid network 2, and a detection head network 3.

[0101] Among them, the backbone network 1 can be used to extract multi-scale features in the line.

[0102] Exemplarily, the backbone network 1 may adopt a ResNet50 (Residual Network 50, 50-layer residual neural network) network. Optionally, the backbone network includes a first network block, a second network block, a third network block, a fourth network block, and a fifth network block. Among them, the first network block is used for preliminary feature extraction of the line image. The second network block, the third network block, the fourth network block, and the fifth network block are used to output feature data of different scales. The feature intermediate layers of the third network block, the fourth network block, and the fifth network block are all connected to the feature pyramid network.

[0103] Among them, the Feature Pyramid Network 2 can be used to fuse the multi-scale features output by the backbone network 1 to obtain line feature data.

[0104] It should be noted that the network structure of the Feature Pyramid Network 2 can perform coarse-to-fine learning on the feature data of different scales output by the backbone network 1. In the network structure of the Feature Pyramid Network 2, based on its top-down connection structure, high-level semantic information can be transmitted downward, and at the same time, it can ensure that more target detail feature information is retained in the shallower layers.

[0105] Reference Figure 3B As shown, the detection head network 3 includes a classification branch 31 and a regression branch 32. The detection head network includes an atrous convolution network on each of the regression branch 31 and the classification branch 32. For the sake of distinction, the atrous convolution network corresponding to the regression branch 31 is regarded as the first atrous convolution network, and the atrous convolution network corresponding to the classification branch 32 is regarded as the second atrous convolution network. In addition, the detection head network 3 also includes a deformable convolution network. Based on this, on the regression branch 31, a defect position recognition network composed of the first atrous convolution network and the deformable convolution network is formed; on the classification branch 32, a defect type recognition network composed of the second atrous convolution network is formed.

[0106] For the sake of easy understanding, the above-mentioned networks will be described in detail below.

[0107] Continue to refer to Figure 3B , for the regression branch 31, the first atrous convolution network is used to determine at least one initial recognition region in the line image according to the line feature data; the deformable convolution network is used to determine the geometric adjustment data, that is, the offset, of each initial recognition region according to the line feature data; according to the geometric adjustment data of the initial recognition region, the corresponding initial recognition region is adjusted, that is, feature alignment is performed, and the updated recognition region of the initial recognition region is obtained.

[0108] For the classification branch 32, the second atrous convolution network is used to determine the defect type of the power line defect in the target recognition region according to the line feature data.

[0109] Based on the technical solutions of the above embodiments, the present application also provides an optional embodiment, in which a model training step is added.

[0110] See Figure 4 As shown, the model training step includes:

[0111] S410. Obtain a sample line image of a power line and a defect region of a power line defect in the sample line image.

[0112] Among them, the sample line image may include line images with different types of power line defects. Exemplarily, it may include 3 types of defect types: broken conductor strands, hanging foreign objects, and damaged insulators. Among them, the sample line image may be a line image obtained by acquisition or a line image after augmentation processing.

[0113] S420. Based on the feature extraction network to be trained and the defect position recognition network to be trained, determine the sample update recognition region in the sample line image.

[0114] Exemplarily, the sample line image may be input into the feature extraction network to be trained to obtain the sample line feature data of the sample line image; the output of the feature extraction network to be trained is used as the input of the defect position recognition network to be trained, and the sample update recognition region output by the defect position recognition network to be trained is obtained.

[0115] S430. For each sample update recognition region, determine the similarity between the sample update recognition region and the corresponding defect region.

[0116] In an optional embodiment, the generalized Jensen-Shannon divergence may be used to determine the distance score between each sample update recognition region and the corresponding defect region. . Among them, the distance score is used to characterize the similarity between the sample update recognition region and the corresponding defect region.

[0117] Among them, both the sample update recognition region and the defect region can be represented by anchor boxes. For the convenience of distinction, the sample update recognition region is represented by the first anchor box P, and the defect region is represented by the second anchor box Q.

[0118] In an optional embodiment, the determination steps of the distance score include:

[0119] a. Determine the two-dimensional Gaussian distributions of the first anchor box P and the second anchor box Q respectively.

[0120] It should be noted that for a certain target anchor box , its two-dimensional Gaussian distribution can be expressed as . Among them, the target anchor box B can be the first anchor box P or the second anchor box Q. (x, y) represents the center point coordinates of the target anchor box; w represents the width of the target anchor box; h represents the length of the target anchor box; represents the mean value of; represents the covariance matrix of.

[0121] Optionally, the mean value of can be determined by the following formula :

[0122]

[0123] Among them, (x, y) represents the center point coordinates of the updated recognition area.

[0124] Optionally, it can be determined by the following formula of the covariance matrix :

[0125]

[0126] Among them, R represents the rotation matrix, represents the detection angle. It should be noted that since a horizontal detection frame is adopted in this embodiment, therefore . The rotation matrix R can be determined according to the following formula:

[0127]

[0128] Among them, .

[0129] It can be understood that the target anchor box B can be used as the first anchor box P, that is, the two-dimensional Gaussian distribution of the first anchor box P is obtained . represents the two-dimensional Gaussian distribution of the mean; represents the two-dimensional Gaussian distribution of the covariance matrix.

[0130] It can be understood that the target anchor box B can be used as the second anchor box Q, that is, the two-dimensional Gaussian distribution of the second anchor box Q is obtained . represents the two-dimensional Gaussian distribution of the mean; represents the two-dimensional Gaussian distribution of the covariance matrix.

[0131] b. Determine the generalized Jensen-Shannon divergence between the two-dimensional Gaussian distribution of the first anchor box P and the two-dimensional Gaussian distribution of the second anchor box Q .

[0132] Optionally, the generalized Jensen-Shannon divergence can be determined by the following formula :

[0133]

[0134] Among them, represents and the Kullback-Leibler divergence between, represents and The average distribution. represents the mean of the average distribution, represents the covariance matrix of the average distribution.

[0135] Optionally, the covariance matrix of the average distribution can be determined by the following formula :

[0136]

[0137] where, represents the covariance matrix of the two-dimensional Gaussian distribution ; represents the covariance matrix of the two-dimensional Gaussian distribution ;

[0138] Optionally, the mean of the average distribution can be determined by the following formula :

[0139]

[0140] where, represents the covariance matrix of the two-dimensional Gaussian distribution ; represents the covariance matrix of the two-dimensional Gaussian distribution ; represents the mean of the two-dimensional Gaussian distribution ; represents the mean of the two-dimensional Gaussian distribution ;

[0141] c. Determine the distance score between the first anchor box P and the second anchor box Q according to the generalized Jensen-Shannon divergence .

[0142] Optionally, the generalized Jensen-Shannon divergence can be mapped to the range (0, 1] to obtain the distance score between the first anchor box P and the second anchor box Q.

[0143] Exemplarily, the generalized Jensen-Shannon divergence can be mapped to the range (0, 1] by the following formula:

[0144]

[0145] where, represents the generalized Jensen-Shannon divergence between the two-dimensional Gaussian distribution of the first anchor box P and the two-dimensional Gaussian distribution

[0146] S440. Take the sample recognition region corresponding to a similarity greater than the similarity threshold as the positive sample; take the sample recognition region corresponding to a similarity not greater than the similarity threshold as the negative sample.

[0147] In an optional embodiment, the similarity threshold may specifically be a distance score threshold. Correspondingly, the sample recognition region with a distance score greater than the distance score threshold can be taken as the positive sample; the sample recognition region with a distance score not greater than the distance score threshold can be taken as the negative sample. Among them, the distance score threshold can be set by those skilled in the art according to needs or experience, or determined through a large number of experiments, and the present application does not make any limitations on this.

[0148] S450. Select training samples from each positive sample and each negative sample according to a set ratio, and train the feature extraction network and the defect location recognition network to adjust the network parameters of the feature extraction network and the defect location recognition network.

[0149] In an optional embodiment, the training samples can be input into a preset loss function to train the feature extraction network and the defect location recognition network to adjust the network parameters of the feature extraction network and the defect location recognition network. Further, based on the training samples, the feature extraction network, the defect location recognition network, and the defect type recognition network can also be trained simultaneously.

[0150] Optionally, the preset loss function may include a classification loss function , and the classification loss function can be specifically expressed as:

[0151]

[0152] Among them, represents the classification score of the power line defect in the i-th first anchor box; represents the number of samples input into the classification loss function ; represents the first weight parameter; represents the second weight parameter.

[0153] Exemplarily, the first weight parameter can be set to 0.25, and the second weight parameter can be set to 2.

[0154] Optionally, the preset loss function may include a localization loss function, and the localization loss function can be specifically expressed as:

[0155]

[0156] Among them, denotes the i-th first anchor box; denotes the second anchor box corresponding to the i-th first anchor box; denotes the minimum bounding rectangle between the i-th first anchor box and the corresponding second anchor box; denotes the number of samples input to the classification loss function in it.

[0157] In the above steps, by taking the sample recognition regions with similarity greater than the similarity threshold as positive samples and the sample recognition regions with similarity not greater than the similarity threshold as negative samples, the determination of positive and negative samples is achieved. By selecting training samples from each positive sample and each negative sample according to a set ratio, the allocation of positive and negative samples is achieved. Training the feature extraction network and the defect location recognition network with positive and negative samples is beneficial to improving the training efficiency and the accuracy of the power line defect detection model.

[0158] Based on the technical solutions of the above embodiments, the present application also provides a verification embodiment. In this embodiment, a test set is selected, and this test set contains three types of defect types: broken conductor strands, foreign object hanging, and insulator damage. Among them, the ratio of the training set to the test set can be 7:3. The detection indicators include the mAP (mean Average precision) indicator. In addition, the detection indicators also include the precision rate indicators measured under different intersection over union ratios, namely the mAP_50 indicator and the mAP_75 indicator. Among them, the mAP_50 indicator refers to the precision rate indicator with an intersection over union of 0.5; the mAP_75 indicator refers to the precision rate indicator with an intersection over union of 0.75.

[0159] After the verification experiment, using the power line defect detection method of the present application, the mAP indicator reached 91.3%, the mAP_50 indicator reached 96.5%, and the mAP_75 indicator reached 93.9. As Figure 5 shown in the reference schematic diagram of the target recognition region, it can be seen that using the power line defect detection provided by the present application, the power line defects can be accurately identified and marked.

[0160] It should be noted that the power line defect detection method provided by the present application can be implemented by programming in the Python language under the Pytorch deep learning framework. The computer CPU (Central Processing Unit) can be configured as I7-7700K@4.20GHz, the graphics card can be configured as 1080ti, and the operating system can be Ubuntu20.04.

[0161] Based on the technical solutions of the above embodiments, the present application also provides an alternative embodiment, in which the method for detecting power line defects is described in detail.

[0162] See Figure 6 The power line defect detection method shown includes:

[0163] S601. Obtain a line image of the power line.

[0164] S602. Based on the feature extraction network, extract the line feature data in the line image.

[0165] S603. Based on the defect location recognition network, determine at least one initial recognition region in the line image according to the line feature data.

[0166] S604. According to the line feature data, determine the sampling point adjustment data of each preset sampling point in each initial recognition region.

[0167] S605. According to the sampling point adjustment data of each preset sampling point in the initial recognition region, determine the center point adjustment data of the corresponding initial recognition region.

[0168] S606. According to the center point adjustment data of the initial recognition region, adjust the corresponding initial recognition region to obtain the updated recognition region of the initial recognition region.

[0169] S607. For each updated recognition region, determine the confidence level of the updated recognition region according to the line feature data; the confidence level is used to represent the credibility of the updated recognition region containing power line defects.

[0170] S608. Determine the target recognition region according to the confidence levels corresponding to the updated recognition regions.

[0171] S609. Based on the defect type recognition network, determine the defect type of the power line defect in the target recognition region according to the line feature data.

[0172] S610. In the line image, correspondingly mark the target recognition region and the defect type.

[0173] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this document, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0174] Based on the same inventive concept, an embodiment of the present application further provides a power line defect detection device for implementing the above-mentioned power line defect detection method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the power line defect detection device provided below can refer to the limitations on the power line defect detection method in the above text, and will not be repeated here.

[0175] In an exemplary embodiment, as Figure 7 shown, a power line defect detection device is provided, including: a first acquisition module 710, an extraction module 720, a first determination module 730, an adjustment module 740, and an identification module 750.

[0176] The first acquisition module 710 is used to acquire the line image of the power line.

[0177] The extraction module 720 is used to extract the line feature data in the line image based on the feature extraction network.

[0178] The first determination module 730 is used to determine at least one initial recognition area in the line image based on the defect position recognition network according to the line feature data.

[0179] The adjustment module 740 is used to adjust at least one initial recognition area based on the defect position recognition network according to the line feature data to obtain the corresponding updated recognition area.

[0180] The identification module 750 is used to identify the target recognition area containing the power line defect from each updated recognition area.

[0181] In one embodiment, the adjustment module 740 includes: a first determination unit configured to determine geometric adjustment data of each initial recognition region according to the line feature data; and an adjustment unit configured to adjust the corresponding initial recognition region according to the geometric adjustment data of the initial recognition region to obtain an updated recognition region of the initial recognition region.

[0182] In one embodiment, the geometric adjustment data of the initial recognition region includes center point adjustment data of the initial recognition region; the first determination unit includes: a first determination subunit configured to determine sampling point adjustment data of each preset sampling point in each initial recognition region according to the line feature data; and a second determination subunit configured to determine the center point adjustment data of the corresponding initial recognition region according to the sampling point adjustment data of each preset sampling point in the initial recognition region.

[0183] In one embodiment, the recognition module 750 includes: a second determination unit configured to determine, for each updated recognition region, a confidence level of the updated recognition region according to the line feature data, where the confidence level is used to characterize the credibility of the presence of a power line defect in the updated recognition region; and a third determination unit configured to determine a target recognition region according to the confidence levels corresponding to the respective updated recognition regions.

[0184] In one embodiment, the power line defect detection device further includes: a second determination module configured to determine a defect type of a power line defect in the target recognition region based on a defect type recognition network according to the line feature data; and a first processing module configured to correspondingly label the defect type in the line image.

[0185] In one embodiment, the power line defect detection device further includes: a second acquisition module configured to acquire a sample line image of a power line and a defect region of a power line defect in the sample line image; a third determination module configured to determine a sample updated recognition region in the sample line image based on a feature extraction network to be trained and a defect position recognition network to be trained; a fourth determination module configured to determine, for each sample updated recognition region, a similarity between the sample updated recognition region and the corresponding defect region; a second processing module configured to use a sample recognition region corresponding to a similarity greater than a similarity threshold as a positive sample, and use a sample recognition region corresponding to a similarity not greater than the similarity threshold as a negative sample; and a training module configured to select training samples from the respective positive samples and the respective negative samples according to a set ratio to train the feature extraction network and the defect position recognition network to adjust network parameters of the feature extraction network and the defect position recognition network.

[0186] Each module in the above power line defect detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0187] In an exemplary embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 8 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a power line defect detection method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0188] Those skilled in the art can understand that Figure 8 the structure shown in

[0189] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component arrangement.

[0190] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0191] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0192] Those of ordinary skill in the art can understand that all or part of the processes of implementing the above method embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0193] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0194] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application shall be subject to the appended claims.

Claims

1. A method for detecting power line defects, characterized in that, The method includes: Obtaining a line image of a power line; Extracting line feature data in the line image based on a feature extraction network; Based on a defect position recognition network, determining at least one initial recognition region in the line image according to the line feature data; and Adjusting at least one of the initial recognition regions according to the line feature data to obtain corresponding updated recognition regions; Identifying a target recognition region containing a power line defect from each of the updated recognition regions.

2. The method according to claim 1, wherein Adjusting at least one of the initial recognition regions according to the line feature data to obtain corresponding updated recognition regions includes: Determining geometric adjustment data of each of the initial recognition regions according to the line feature data; Adjusting the corresponding initial recognition region according to the geometric adjustment data of the initial recognition region to obtain an updated recognition region of the initial recognition region.

3. The method according to claim 2, wherein The geometric adjustment data of the initial recognition region includes center point adjustment data of the initial recognition region; correspondingly, determining geometric adjustment data of each of the initial recognition regions according to the line feature data includes: Determining sampling point adjustment data of each preset sampling point in each of the initial recognition regions according to the line feature data; Determining center point adjustment data of the corresponding initial recognition region according to the sampling point adjustment data of each preset sampling point in the initial recognition region.

4. The method according to claim 1, characterized in that Identifying a target recognition region containing a power line defect from each of the updated recognition regions includes: For each updated recognition region, determining a confidence level of the updated recognition region according to the line feature data; the confidence level is used to represent the credibility of the updated recognition region containing the power line defect; Determining the target recognition region according to the confidence levels corresponding to the updated recognition regions.

5. The method according to claim 1, wherein It further includes: Based on a defect type recognition network, determining a defect type of the power line defect in the target recognition region according to the line feature data; Correspondingly marking the defect type in the line image.

6. The method according to any one of claims 1-5, characterized in that, It further includes: Obtaining a sample line image of a power line and a defect region of a power line defect in the sample line image; Determining a sample updated recognition region in the sample line image based on a feature extraction network to be trained and a defect position recognition network to be trained; Determining a similarity between the sample updated recognition region and the corresponding defect region for each sample updated recognition region; Taking the sample recognition region with a similarity greater than a similarity threshold as a positive sample; taking the sample recognition region with a similarity not greater than the similarity threshold as a negative sample; Selecting training samples from each of the positive samples and each of the negative samples according to a set ratio to train the feature extraction network and the defect position recognition network so as to adjust network parameters of the feature extraction network and the defect position recognition network.

7. A power line defect detection device, characterized in that, The device includes: A first acquisition module for acquiring a line image of a power line; An extraction module for extracting line feature data in the line image based on a feature extraction network; The first determination module is configured to determine at least one initial recognition region in the line image based on the defect position recognition network and according to the line feature data; The adjustment module is configured to adjust at least one of the initial recognition regions based on the defect position recognition network and according to the line feature data to obtain corresponding updated recognition regions; The recognition module is configured to recognize a target recognition region containing a power line defect from each of the updated recognition regions.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images