Power transmission line fault detection method and device based on image recognition

Abstract

The invention relates to the technical field of image recognition, in particular to a power transmission line fault detection method and device based on image recognition, and the method comprises the steps: obtaining an original image of a power transmission line, and carrying out the preprocessing of the original image, and obtaining a restored image of the power transmission line; extracting feature points of the restored image and feature points of the template image, and matching the feature points of the restored image with the feature points of the template image to obtain a mapping relation between the restored image and the template image; and performing registration processing on the restored image according to the mapping relation to obtain a registration image of the restored image, and identifying a fault detection result of the power transmission line according to the image difference between the registration image and the template image. According to the method, the fault existing in the power transmission line can be effectively detected, image processing is relatively fine, the average accuracy and the average reliability of an image recognition result are effectively improved, the application range is relatively wide, and practical application in various scenes is facilitated.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method and device for detecting power transmission line faults based on image recognition. Background Art

[0002] High-voltage transmission lines are the carriers of long-distance power transmission. Their line conductors, poles and towers are exposed to the wild natural environment for a long time. They are affected by factors such as continuous mechanical tension, electrical flashover, material aging, natural disasters, and human destruction, resulting in damages such as broken strands, collapsed towers, wear and corrosion. If not repaired or replaced in time, the originally minor defects may continue to expand and eventually lead to serious accidents.

[0003] In the related technologies, the line inspection method widely used in power systems is manual visual inspection. This method is not only labor-intensive but also difficult to carry out. In particular, it is very difficult to inspect power transmission lines in mountainous areas and across large rivers. With the advancement of technology, power companies have gradually started to use robots for power line inspection. This method uses high-definition cameras mounted on drones to fly along fixed lines set in advance and take images, which are then sent to the background fault diagnosis system. The system uses image recognition technology to extract fault features in the images, and then perform fault identification and diagnosis. This method can improve the operating efficiency of the power system and reduce maintenance costs.

[0004] However, in the related technology, the use of image recognition technology to extract fault features in images to identify transmission line faults involves the processing of massive amounts of image information, which requires high system detection capabilities and is difficult to guarantee accuracy. How to improve image recognition technology and improve the accuracy of transmission line fault identification and detection results is an urgent issue to be addressed. Summary of the invention

[0005] The present application provides a method and device for power transmission line fault detection based on image recognition to solve the problems in related technologies, such as using image recognition technology to extract fault features in images to identify power transmission line faults, which involves processing massive amounts of image information, and requires high system detection capabilities while the accuracy is difficult to guarantee. The problem also lies in how to improve image recognition technology and improve the accuracy of power transmission line fault identification and detection results.

[0006] The first aspect of the present application provides a method for power transmission line fault detection based on image recognition, comprising the following steps: acquiring an original image of a transmission line, and preprocessing the original image to obtain a restored image of the transmission line; extracting feature points of the restored image and feature points of a template image, and matching the feature points of the restored image with feature points of the template image to obtain a mapping relationship between the restored image and the template image; performing registration processing on the restored image according to the mapping relationship to obtain a registered image of the restored image, so as to identify a fault detection result of the transmission line according to the image difference between the registered image and the template image.

[0007] Optionally, in one embodiment of the present application, the preprocessing of the original image to obtain the restored image of the transmission line includes: using a preset peak signal-to-noise ratio as a metric for image restoration, and filtering the initial restored image to obtain the restored image.

[0008] Optionally, in one embodiment of the present application, matching the feature points of the restored image with the feature points of the template image to obtain a mapping relationship between the restored image and the template image includes: representing the feature points of the restored image and the feature points of the template image with feature vectors to obtain the feature vectors of the restored image and the feature vectors of the template image, respectively; calculating the Euclidean distance between the feature vectors of the restored image and the feature vectors of the template image; judging whether the feature points of the restored image and the feature points of the template image are successfully matched based on the Euclidean distance, and if the match is successful, obtaining the mapping relationship between the restored image and the template image based on the successfully matched feature points, otherwise refusing to accept the feature points that failed to match.

[0009] Optionally, in one embodiment of the present application, the restored image is registered according to the mapping relationship to obtain a registered image of the restored image, including: based on the mapping relationship, matching the feature points of the restored image and the feature points of the template image at the same position to obtain a transformation matrix; mapping the image portion of the restored image that overlaps with the template image to the template image coordinate system for registration according to the transformation matrix to obtain a registered image of the restored image.

[0010] Optionally, in one embodiment of the present application, identifying the fault of the transmission line based on the image difference between the registered image and the template image includes: analyzing the image difference between the binarized image of the registered image and the binarized image of the template image to obtain the image difference between the registered image and the template image based on the image difference value and a preset difference threshold; identifying the image category of the registered image based on the image difference between the registered image and the template image; and determining the fault detection result of the transmission line based on the image category.

[0011] The second aspect of the present application provides a transmission line fault detection device based on image recognition, including: a processing module, used to obtain an original image of a transmission line, and pre-process the original image to obtain a restored image of the transmission line; a matching module, used to extract feature points of the restored image and feature points of a template image, and match the feature points of the restored image with the feature points of the template image to obtain a mapping relationship between the restored image and the template image; an identification module, used to perform registration processing on the restored image according to the mapping relationship to obtain a registered image of the restored image, so as to identify the fault detection result of the transmission line according to the image difference between the registered image and the template image.

[0012] Optionally, in one embodiment of the present application, the processing module includes: a processing unit, configured to filter the initial restored image using a preset peak signal-to-noise ratio as a metric for image restoration to obtain the restored image.

[0013] Optionally, in one embodiment of the present application, the matching module includes: a characterization unit, used to characterize the feature points of the restored image and the feature points of the template image with feature vectors to obtain the feature vectors of the restored image and the feature vectors of the template image, respectively; a calculation unit, used to calculate the Euclidean distance between the feature vector of the restored image and the feature vector of the template image; and a matching unit, used to determine whether the feature points of the restored image and the feature points of the template image are successfully matched based on the Euclidean distance, and if the match is successful, obtain the mapping relationship between the restored image and the template image based on the successfully matched feature points, otherwise refuse to accept the feature points that failed to match.

[0014] Optionally, in one embodiment of the present application, the recognition module includes: a corresponding unit, used to correspond the feature points of the restored image and the feature points of the template image at the same position based on the mapping relationship to obtain a transformation matrix; and a registration unit, used to map the image portion of the restored image that overlaps with the template image into the template image coordinate system for registration according to the transformation matrix to obtain a registered image of the restored image.

[0015] Optionally, in one embodiment of the present application, the recognition module includes: an analysis unit, used to analyze the image difference between the binary image of the registration image and the binary image of the template image, so as to obtain the image difference between the registration image and the template image according to the image difference value and a preset difference threshold; a recognition unit, used to identify the image category of the registration image according to the image difference; and a determination unit, used to determine the fault detection result of the transmission line according to the image category.

[0016] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the power transmission line fault detection method based on image recognition as described in the above embodiment.

[0017] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned image recognition-based transmission line fault detection method.

[0018] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned image recognition-based transmission line fault detection method.

[0019] The embodiment of the present application can perform multiple processing on the original image of the transmission line, and identify the fault detection result of the transmission line based on the processed image. This achieves the improvement of image quality through image restoration, so that the image can be further processed and then compared with the template image, which effectively improves the accuracy and reliability of the image recognition result, expands the scope of application of the present application, and helps to implement it in various scenarios. This solves the problem that the use of image recognition technology to extract fault features in images to identify transmission line faults in the related technology involves the processing of massive image information, and the accuracy is difficult to guarantee while requiring high system detection capabilities. How to improve image recognition technology and improve the accuracy of transmission line fault identification and detection results is solved.

[0020] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1This is a structural schematic diagram of a power transmission line fault diagnosis system based on image recognition according to an embodiment of the present application;

[0023] Figure 2 A flowchart of a method for detecting a power transmission line fault based on image recognition according to an embodiment of the present application;

[0024] Figure 3 A flowchart of image restoration according to an embodiment of the present application;

[0025] Figure 4 A schematic diagram of an image restoration effect according to an embodiment of the present application;

[0026] Figure 5 This is a schematic diagram of image matching according to an embodiment of the present application;

[0027] Figure 6 A schematic diagram of an imaging model according to an embodiment of the present application;

[0028] Figure 7 This is a schematic diagram of the registration image effect of an embodiment of the present application;

[0029] Figure 8 A schematic diagram of different template images at the same position according to an embodiment of the present application;

[0030] Fig. 9 A schematic diagram of a registered image of different template images at the same position according to an embodiment of the present application;

[0031] Fig.10 This is a schematic diagram of the change detection result of the image difference method according to one embodiment of the present application;

[0032] Fig.11 A schematic diagram of a fitting curve of the relationship between grayscale threshold and difference information according to an embodiment of the present application;

[0033] Fig.12 A schematic diagram of an aerial image with abnormal conditions and its template image according to an embodiment of the present application;

[0034] Fig.13 A schematic diagram of an aerial image change detection result with abnormal conditions according to an embodiment of the present application;

[0035] Fig.14 Schematic diagram of the structure of a power transmission line fault detection device based on image recognition according to an embodiment of the present application;

[0036] Fig.15 Schematic diagram of the structure of an electronic device according to an embodiment of the present application.

[0037] Reference numerals:

[0038] 10-Power transmission line fault detection device based on image recognition: 100-processing module, 200-matching module and 300-identification module; 1501-memory, 1502-processor and 1503-communication interface. DETAILED DESCRIPTION

[0039] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0040] The following describes the transmission line fault detection method and device based on image recognition of the embodiment of the present application with reference to the accompanying drawings. In view of the related technologies mentioned in the above background technology, the use of image recognition technology to extract fault features in images to identify transmission line faults involves the processing of massive image information, which requires high system detection capabilities and the accuracy is difficult to guarantee. How to improve image recognition technology and improve the accuracy of transmission line fault identification and detection results. The present application provides a transmission line fault detection method based on image recognition, in which the original image of the transmission line can be multi-processed, and the fault detection result of the transmission line can be identified based on the processed image. In this way, the quality of the image is improved by image restoration, so that the image is further processed, and then compared with the template image, which effectively improves the accuracy and reliability of the image recognition result, expands the scope of application of the present application, and is helpful for practical application in various scenarios. This solves the problem in related technologies that using image recognition technology to extract fault features in images to identify transmission line faults involves processing massive amounts of image information, places high demands on system detection capabilities, and the accuracy is difficult to guarantee. It also solves the problem of how to improve image recognition technology and increase the accuracy of transmission line fault identification and detection results.

[0041] Before explaining the power transmission line fault detection method based on image recognition provided in the embodiment of the present application, the structure of the power transmission line fault diagnosis system based on image recognition involved in the embodiment of the present application is first explained.

[0042] Figure 1 FIG. 1 is a schematic diagram of a transmission line fault diagnosis system based on image recognition according to an embodiment of the present application. Figure 1 As shown, the power transmission line fault diagnosis system based on image recognition involved in the embodiments of the present application is mainly composed of image acquisition, image preprocessing, feature extraction and selection, fault identification and diagnosis and other parts.

[0043] Among them, the acquisition of images mainly relies on drones or robots to take aerial photos through high-definition cameras during fixed-route inspections. Due to the influence of factors such as autofocus and aircraft movement during the shooting process, it is inevitable that the image will be blurred and jittery. Therefore, it is necessary to pre-process such images. After pre-processing, image recognition uses a feature point extraction algorithm to extract feature points of the template image and the sample image based on the establishment of a standard aerial image template library; then, a template matching the sample is found to establish a one-to-one correspondence between the sample and the template; then the sample image is further registered, and the difference between the sample image and the template image is obtained using the image change detection method, and the image is determined to be an abnormal image based on the set threshold.

[0044] Specifically, Figure 2 A flowchart of a method for detecting power line faults based on image recognition provided in an embodiment of the present application.

[0045] like Figure 2 As shown, the power transmission line fault detection method based on image recognition includes the following steps:

[0046] In step S201, an original image of the power transmission line is acquired, and the original image is preprocessed to obtain a restored image of the power transmission line.

[0047] It is understandable that when the fault of the power transmission line is detected by image recognition technology, the images used are mainly images taken by robots or drones. During the shooting process, the image taken may be blurred due to various reasons such as weather or the machine itself, which will have a certain impact on the accuracy of the recognition result. Therefore, when the fault of the power transmission line is detected based on image recognition, the embodiment of the present application can first restore the original image of the power transmission line obtained, so as to perform multi-step processing of subsequent images through the restored image.

[0048] Image restoration here can be understood as establishing a corresponding mathematical model based on the cause of degradation and extracting the required information from the distorted image signal. In a broad sense, it can be understood as an inverse problem. Figure 3 FIG. 1 is a flowchart of image restoration according to an embodiment of the present application. Figure 3 As shown, the embodiment of the present application can use the image restoration model to restore the original image of the transmission line.

[0049] Specifically, f(x, y) can be used to describe the distribution of scene radiation energy on the object plane. The image obtained on the image plane through the imaging system H is H[f(x, y)]. Adding noise n(x, y), the actual image g(x, y) can be represented by the following degradation model:

[0050] g(x,y)=H[f(x,y)]+n(x,y) (1)

[0051] Among them, H[·] is a function that integrates all degradation factors.

[0052] The embodiment of the present application can restore the acquired original image of the power transmission line, and use the restored image to identify the fault of the power transmission line, thereby improving the efficiency of original image processing and effectively improving the accuracy of fault detection results.

[0053] Optionally, in one embodiment of the present application, the original image is preprocessed to obtain a restored image of the transmission line, including: using a preset peak signal-to-noise ratio as a metric for image restoration, and filtering the initial restored image to obtain a restored image.

[0054] In some embodiments, when the present application restores the original image of the power transmission line, the image restoration model used is mainly but not limited to an image restoration filter, that is, the image is filtered by the image restoration filter to restore the image.

[0055] For example, the minimum mean square error filter is a filter proposed by Norman Wiener in the 1940s, also called the Wiener filter, which assumes that the input of the linear filter is the sum of the useful signal and the noise, both of which are wide-sense stationary processes and their second-order statistical characteristics are known. According to the minimum mean square error criterion, the optimal linear filter parameters can be obtained. Therefore, the embodiment of the present application can use the Wiener filter to filter the original jittered image to obtain a restored image.

[0056] Furthermore, in the restoration process, a certain peak signal-to-noise ratio of an image can be selected as a metric for image restoration to determine the degree of image restoration. Its definition can be expressed as follows:

[0057]

[0058] Among them, the size of the normal image f(x,y) is M×N, and the image to be compared is

[0059] For example, taking the peak signal-to-noise ratio of an image reaching 40 as a metric, Figure 4 This is a schematic diagram of an image restoration effect of an embodiment of the present application. Figure 4 The image processed by Wiener filtering is obtained by applying Wiener filtering to the original jittered image. The peak signal-to-noise ratio of the image increases from 12.038 to 45.617, which means that the image restoration is successful. It can be seen that Wiener filtering has achieved a good image restoration effect, and the image has become clearer from blurry.

[0060] It should be noted that the preset peak signal-to-noise ratio here can be understood as a preset peak signal-to-noise ratio of the image, which can be specifically set by professionals in the field according to actual conditions. It is only used as an example here for illustration without specific limitation.

[0061] The embodiment of the present application can filter the image and use a certain peak signal-to-noise ratio of the image as a measurement criterion, thereby ensuring that the degree of image restoration can achieve the expected effect while restoring the image, thereby improving the reliability of the image restoration result.

[0062] Step S202 , extracting feature points of the restored image and feature points of the template image, and matching the feature points of the restored image and feature points of the template image to obtain a mapping relationship between the restored image and the template image.

[0063] In other embodiments, after obtaining the restored image of the transmission line image, in order to more intuitively observe whether the restored image in the image is abnormal, the template image can be used as a reference for identification and judgment. Furthermore, in order to ensure the correctness and accuracy of the image recognition result, the embodiment of the present application can use the feature points of the restored image and the template image as the main body.

[0064] The template image here can be understood as an image in a transmission line aerial photography template library composed of historical aerial photography fault-free images of transmission lines.

[0065] When using the feature points of the restored image and the feature points of the template image, the feature points of the restored image and the feature points of the template image may be matched first to obtain a mapping relationship between the two, and then the next step of image processing may be performed according to the mapping relationship.

[0066] For example, after restoring the sample image actually taken, the corresponding template in the template library can be found and matched, and then based on the comparison between the restored image and the template image, it can be determined whether an abnormality occurs here.

[0067] Next, the process of matching the feature points of the restored image with the feature points of the template image to obtain a mapping relationship between the two is further explained.

[0068] Optionally, in one embodiment of the present application, feature points of the restored image are matched with feature points of the template image to obtain a mapping relationship between the restored image and the template image, including: representing the feature points of the restored image and the feature points of the template image with feature vectors to obtain feature vectors of the restored image and feature vectors of the template image, respectively; calculating the Euclidean distance between the feature vector of the restored image and the feature vector of the template image; judging whether the feature points of the restored image and the feature points of the template image are successfully matched based on the Euclidean distance, and if the match is successful, obtaining the mapping relationship between the restored image and the template image based on the successfully matched feature points, otherwise refusing to accept the feature points that failed to match.

[0069] In the actual implementation process, before matching the feature points of the restored image with the feature points of the template image, it is necessary to extract the feature points of the restored image and the template image before matching them. Next, the process of image feature extraction and matching is further explained.

[0070] For example, the feature point extraction process of the image in the embodiment of the present application can be performed using a feature extraction algorithm, such as the SIFT algorithm. The Scale Invariant Feature Transform (SIFT) algorithm is a scale-space-based local feature of an image that remains invariant to image translation, rotation, scaling, and affine transformation, as well as the descriptor of the feature. In the process of matching an image sample with a template, the SIFT algorithm can well extract the inherent feature information of an image.

[0071] Furthermore, in the embodiment of the present application, each extracted key point can be described by a feature vector, and the Euclidean distance of the feature vector can be used as a measure of the similarity between the key points of the two images. That is, assuming that the two feature vectors are a = (a1, a2, a3, ...) and b = (b1, b2, b3, ...), the Euclidean distance calculation formula can be expressed as follows:

[0072]

[0073] In practical applications, in order to reduce the amount of calculation, the embodiment of the present application can, but is not limited to, use formula (4) to calculate the Euclidean distance between two vectors. The formula can be expressed as follows:

[0074] d=(a1-b1) 2 +(a2-b2) 2 +(a3-b3) 2 …(4)

[0075] Figure 5 This is a schematic diagram of image matching according to an embodiment of the present application. Figure 5 As shown, take a key point in the restored image and find the first two key points d1 and d2 with the closest Euclidean distance to the template image. Among these two key points, if the closest distance d1 divided by the second closest distance d2 is less than a certain ratio threshold (such as 0.75, 0.8, etc.), then accept this pair of matching points, otherwise reject this pair of matching points.

[0076] It should be noted that, by lowering the ratio threshold, the number of SIFT matching points will decrease, but it will be more stable. Therefore, professionals in this field can select or set the ratio threshold according to actual conditions and needs. This is only an exemplary description without specific limitation.

[0077] For example, in the embodiment of the present application, 260 and 453 key points were detected in the template image and the restored image respectively, and SIFT feature matching was performed on them. Taking the ratio threshold of 0.75 as an example, 119 pairs of key points were matched. Through manual screening, it was found that there were 104 pairs of correct matches and 15 pairs of incorrect matches, with a matching accuracy of 87.4% (it should be noted that for ease of display, only a part of the matching pairs are drawn in the figure for example only).

[0078] After obtaining the matching results of the template image and the restored image, the embodiment of the present application can obtain the mapping relationship between the restored image and the template image based on the successfully matched feature points, so that the mapping relationship can be used to determine the geometric transformation model of the restored image and the template image for further processing of the restored image.

[0079] Step S203 , registering the restored image according to the mapping relationship to obtain a registered image of the restored image, so as to identify the fault detection result of the power transmission line according to the image difference between the registered image and the template image.

[0080] It is understood by those skilled in the art that, since there are differences in translation, rotation, scaling, etc. of different scales between the actual aerial image and the template image, after completing the feature point matching of the two images, it is necessary to determine their geometric transformation model based on the relationship between the feature point matching pairs. Furthermore, the restored image of the actual aerial sample image can also be registered based on the geometric transformation model.

[0081] As a possible implementation method, the embodiment of the present application can obtain the mapping relationship between the restored image and the template image in terms of the general outline by SIFT feature extraction and aerial image registration. Figure 6 As shown, it is a schematic diagram of an imaging model of shooting the same position at different angles in a three-dimensional scene according to an embodiment of the present application. The points representing the same position in the two figures correspond one to one, that is, the transformation relationship (mapping relationship) between the restored image and the template image is a one-to-one correspondence.

[0082] Next, the embodiment of the present application can use the mapping relationship to obtain a geometric transformation model to align the restored image, and can compare the registered restored image and the template image again to detect the changes in the details of the restored image relative to the template image, that is, to detect the changes in the image, thereby determining whether there is a problem in the restored image, and through image recognition, whether there is a fault in the power transmission line photographed by a robot or drone.

[0083] The embodiments of the present application can determine whether there is a fault in the transmission line based on the difference between the restored image of the transmission line and the template image, identify the fault detection result of the transmission line, and perform registration processing on the restored image through the mapping relationship between the restored image and the template image, thereby improving the image recognition efficiency and effectively grasping the accuracy of the recognition result.

[0084] Optionally, in one embodiment of the present application, the restored image is registered according to the mapping relationship to obtain a registered image of the restored image, including: based on the mapping relationship, matching the feature points of the restored image and the feature points of the template image at the same position to obtain a transformation matrix; mapping the image portion where the restored image overlaps with the template image to the template image coordinate system for registration according to the transformation matrix to obtain a registered image of the restored image.

[0085] Based on the relevant descriptions of other embodiments, it can be understood that the mapping relationship between the restored image and the template image can be obtained by using the successfully matched feature points between the two images, and the restored image can be registered using the mapping relationship to obtain a registered image of the restored image.

[0086] Specifically, taking the one-to-one correspondence between the feature point mapping relationship between the two images as an example, we can assume that the coordinates of the points m1 and m2 representing the same position in the two images are (X1, y1) and (x2, y2), respectively, and use the direct linear transformation algorithm (The Direct Linear Transformation) to estimate the parameters of the transformation matrix H. Writing it in homogeneous coordinate form, we can get: (X1, y1, z1) and (x2′, y2′, z2), where x2′ / z2=x2, y2′ / z2=y2, and the matrix form can be expressed as follows:

[0087]

[0088] Let z1 = 1, we can get:

[0089] x2(H 31 x1+H 32 y1+H 33 )=H 11 x1+H 12 y1+H 13 (6)

[0090] y2(H 31 x1+H 32 y1+H 33 )=H 21 x1+H 22 y1+H 23 (7)

[0091] From equation 6 and equation 7, we can deduce:

[0092] a x T h=0 (8)

[0093] a y T h=0 (9)

[0094] in:

[0095] h=[H 11 , H 12 , H 13 , H 21 , H 22 , H 23 , H 31 , H 32 , H 33 ] (10)

[0096] a x =(-x1,-y1,-1,0,0,0,x2x1,x2y1,x2′) T (11)

[0097] a y =(0, 0, 0, -X1, -y1, -1, y2x1, y2y1, y2′) T (12)

[0098] Given a set of corresponding points of two images, the equation can be constructed as follows:

[0099] Ah=0 (13)

[0100] in:

[0101] A=(a x T 1, a y T 1, a x T 2, a y T 2, …a x T N , a y TN ) T (14)

[0102] Taking the N matching pairs obtained above as an example, a 2N×9 matrix A can be obtained. As can be seen from the above formula, only 4 matching pairs are needed to calculate the transformation matrix, but in order to reduce the error, more matching pairs can be taken. In the embodiment of the present application, the value of N is taken as the number of matching pairs.

[0103] Next, the obtained 2N×9 matrix A is subjected to singular value decomposition (SVD) to obtain: A = UDVT. The singular values ​​on the diagonal of D are arranged in descending order, and the last column of V is h in equation 10. Based on h, the transformation matrix H is constructed, and the parameters of the matrix satisfy the minimum error under the least squares method.

[0104] by Figure 5 Taking the template image and the restored image in as an example, assuming that 119 SIFT feature matching pairs are obtained, the above method is used to calculate the transformation matrix of the 119 matching pairs, and the results can be obtained as follows:

[0105]

[0106] After calculating the transformation matrix, the transformation matrix H is used to map the part of the image to be registered that overlaps with the template image to the template image coordinate system to obtain the registration result image as shown in Figure 7 shown.

[0107] Figure 7 This is a schematic diagram of the registration image effect of an embodiment of the present application, from Figure 7 It can be seen that the registered image retains the same part of the sample as the template, and corrects this part through coordinate transformation, achieving a good registration effect.

[0108] Additionally, during the aerial photography process, the restored image of the same sample image may overlap with several template images at that location. Figure 8 This is a schematic diagram of different template images at the same position in an implementation of the present application. Figure 8 As shown, assuming that both images are Figure 5 The templates R1 and R2 at the location of the sample image in the image are matched, and the sample image can be matched with both images during the matching process. The registration result is as follows: Fig. 9 shown.

[0109] In this case, the template image with the most matching pairs can be used as the matching template of the sample image. Figure 7 The template image in Figure 8 The number of matching pairs between template image R1 and template image R2 and sample image is 119, 86, and 98 respectively. Therefore, in the matching process, Figure 7The template image is used as the matching template of the sample image.

[0110] Optionally, in one embodiment of the present application, a transmission line fault is identified based on an image difference between a registered image and a template image, including: analyzing an image difference value between a binarized image of the registered image and a binarized image of the template image to obtain an image difference between the registered image and the template image based on the image difference value and a preset difference threshold; identifying an image category of the registered image based on the image difference; and determining a transmission line fault detection result based on the image category.

[0111] It can be understood that the preset difference threshold here can be understood as a threshold of a pre-set image difference value. For example, if the grayscale threshold at a certain point in the image is 80, when the actual grayscale threshold at that point in the image is greater than 80, it can be determined that the image is abnormal.

[0112] In other embodiments, after obtaining the registration image of the restored image, the fault condition of the power transmission line can be identified based on the image difference between the registration image and the template image. Since the registration image is still in the basic image style, it is more difficult to identify and the accuracy is not high. In order to make the image difference more obvious and easy to identify, the embodiment of the present application can further process the registration image. For example, the registration image and the template image are partially processed at the same position, and the difference image is binarized, and then the image difference between the registration image and the template image is obtained by the image difference value of the binarized image of the registration image and the binarized image of the template image and the preset difference threshold, so as to realize the change detection of the registration image relative to the template image.

[0113] For the process of detecting changes in the registered image relative to the template image, the embodiments of the present application can be implemented, but are not limited to, using image change detection methods. Image change detection is the main technical means for detecting changes between images taken in the same scene but at different time periods. An appropriate threshold can be selected to compare with the size of the changed part to determine whether the image is abnormal. Image change detection methods mainly include, but are not limited to, image difference method, image ratio method, classification comparison method, principal component analysis method, etc.

[0114] Taking the image difference method as an example, when identifying the fault detection result of the power transmission line by the difference between the registered image and the template image, the present application can, but is not limited to, select two images taken in the same area at different times, perform strict spatial geometric registration on the two images, and then subtract the grayscale values ​​of each pixel of the two images to obtain a difference image. By comparing the subtraction result with the selected threshold, the changed area and the unchanged area can be distinguished.

[0115] If selected Figure 7The restored image after registration is subtracted from the image of the same position in the template image, and the difference image is binarized to obtain the binary image of the restored image and the template image. The two are subtracted, and if the grayscale value of the pixel point of the difference image after subtraction is less than a certain threshold, it is set to 1; otherwise, it is set to 0. Fig.10 This is a schematic diagram of the change detection result of the image difference method according to an embodiment of the present application. The final result is as follows Fig.10 shown.

[0116] It should be noted that since the lighting, angle and other conditions of the two images cannot be exactly the same, their grayscale values ​​are slightly different. The grayscale values ​​of the pixels in the difference image obtained after subtraction cannot be completely 0. The image still retains the general outline of the original image, such as Fig.10 As shown in the difference image in , the grayscale of the image is 0 to 255. When the template and the sample are very different in a certain area, the grayscale value of the area in the difference image is also large. Therefore, it is necessary to select an appropriate grayscale threshold to filter out the area with a small difference and retain the area with a large difference. For the same image, the smaller the threshold setting, the less difference information between the two images obtained in the binary image.

[0117] In order to set the binarization threshold here, the embodiment of the present application can, but is not limited to, take 10 fault images and 10 normal images and their corresponding template images for difference processing. During the processing, the threshold value is taken from 0 to 210 every 30, and the corresponding binary image is obtained. Then the amount of difference information in the binary image is calculated (that is, the number of points with a pixel value of 0 in the image), and finally the average value of the 10 fault images and the 10 normal images is calculated to obtain the fitting curve. Fig.11 The figure is a schematic diagram of a fitting curve of the relationship between the grayscale threshold and the difference information in one embodiment of the present application.

[0118] from Fig.11 It can be seen from the figure that when the grayscale threshold is around 50, the difference in area between the fault image and the normal image is the largest, that is, the distinction between the fault image and the normal image is most obvious at this time (only a small number of test images used in the embodiment of the present application, a large number of images are required for testing in actual applications). Therefore, 50 can be taken as the grayscale threshold for binarization of the difference image in the embodiment of the present application, and finally the following can be obtained: Fig.10 The binarized image shown.

[0119] After obtaining the binary images of the registration image and the template image, and subtracting the two binary images, the pixel grayscale values ​​of the differential image obtained are set to 1 or 0 according to a certain difference threshold, and then the image difference between the registration image and the template image can be obtained according to the number of 1s and 0s.

[0120] In some embodiments, the fault detection result of the transmission line may be no fault, or it may be confirmed that there is a fault, or it may be that the fault cannot be confirmed and is only a suspected fault. Therefore, before determining the fault detection result of the transmission line, the embodiment of the present application can first identify the image category of the registered image based on the actual image difference between the registered image and the template image, and finally determine the fault detection result of the transmission line.

[0121] For example, the amount of difference information in the binary image (i.e., the number of points with a pixel value of 0) can be used to determine whether the image is a suspected fault image. Further, in order to expand the scope of application of the present application, when determining whether an image is a suspected fault image, professionals in the field can set the number of points with a pixel value of 0 that is considered a suspected fault image based on actual conditions or actual needs. This is only an exemplary description without any specific explanation.

[0122] For example, take Fig.11 The middle value between the fault image area and the normal image area corresponding to the point with a medium grayscale threshold of 50 is about 120. Fig.10 The number of pixel values ​​of the binary image that are 0 is 39, which is less than the threshold and can be judged as a normal image.

[0123] When there are abnormalities in the aerial image, it will be significantly different from the template image, and a large amount of difference information will be obtained after performing image difference and binarization. Fig.12 As shown in the figure, they are respectively the fault-free template image of aerial photography and the abnormal image of actual aerial photography. The results after difference processing and binarization processing of the abnormal image of actual aerial photography are as follows Fig.13 As shown, Fig.13 The binary image in the image has 607 points with a pixel value of 0, which is much larger than the set threshold. Fig.13 The binary image in will be judged as a suspected fault image.

[0124] According to the transmission line fault detection method based on image recognition proposed in the embodiment of the present application, the original image of the transmission line can be multi-processed, and the fault detection result of the transmission line can be identified based on the processed image. In this way, the quality of the image is improved by image restoration, and the image is further processed, and then compared with the template image, which effectively improves the accuracy and reliability of the image recognition result, expands the scope of application of the present application, and is helpful for practical application in various scenarios. In this way, the problems in the related art that the use of image recognition technology to extract fault features in images to identify transmission line faults involve the processing of massive image information, and the accuracy is difficult to guarantee while requiring high system detection capabilities, and how to improve image recognition technology and improve the accuracy of transmission line fault identification and detection results are solved.

[0125] Next, a power transmission line fault detection device based on image recognition proposed in accordance with an embodiment of the present application will be described with reference to the accompanying drawings.

[0126] Fig.14 It is a structural schematic diagram of a power transmission line fault detection device based on image recognition according to an embodiment of the present application.

[0127] like Fig.14 As shown, the power transmission line fault detection device 10 based on image recognition includes: a processing module 100 , a matching module 200 and an identification module 300 .

[0128] The processing module is used to obtain the original image of the power transmission line and pre-process the original image to obtain a restored image of the power transmission line.

[0129] The matching module is used to extract feature points of the restored image and feature points of the template image, and match the feature points of the restored image and feature points of the template image to obtain a mapping relationship between the restored image and the template image.

[0130] The recognition module is used to perform registration processing on the restored image according to the mapping relationship to obtain a registered image of the restored image, so as to recognize the fault detection result of the power transmission line according to the image difference between the registered image and the template image.

[0131] Optionally, in one embodiment of the present application, the processing module 100 includes: a processing unit.

[0132] The processing unit is used to filter the initial restored image using a preset peak signal-to-noise ratio as a metric for image restoration to obtain a restored image.

[0133] Optionally, in one embodiment of the present application, the matching module 200 includes: a characterization unit, a calculation unit and a matching unit.

[0134] The characterization unit is used to characterize the feature points of the restored image and the feature points of the template image using feature vectors to obtain the feature vector of the restored image and the feature vector of the template image respectively.

[0135] The calculation unit is used to calculate the Euclidean distance between the feature vector of the restored image and the feature vector of the template image.

[0136] The matching unit is used to determine whether the feature points of the restored image and the feature points of the template image are matched successfully according to the Euclidean distance. If the match is successful, the mapping relationship between the restored image and the template image is obtained according to the successfully matched feature points. Otherwise, the feature points that failed to match are rejected.

[0137] Optionally, in one embodiment of the present application, the recognition module 300 includes: a corresponding unit and a registration unit.

[0138] The corresponding unit is used to correspond the feature points of the restored image and the feature points of the template image at the same position based on the mapping relationship to obtain a transformation matrix.

[0139] The registration unit is used to map the image portion where the restored image overlaps with the template image into the template image coordinate system for registration according to the transformation matrix, so as to obtain a registered image of the restored image.

[0140] Optionally, in one embodiment of the present application, the identification module 300 includes: an analysis unit, an identification unit and a determination unit.

[0141] The analyzing unit is used to analyze the image difference value between the binary image of the registration image and the binary image of the template image, so as to obtain the image difference between the registration image and the template image according to the image difference value and a preset difference threshold.

[0142] The recognition unit is used to recognize the image category of the registration image according to the image difference between the registration image and the template image.

[0143] The determination unit is used to determine the fault detection result of the transmission line according to the image category.

[0144] It should be noted that the aforementioned explanation of the embodiment of the power transmission line fault detection method based on image recognition is also applicable to the power transmission line fault detection device based on image recognition in this embodiment, and will not be repeated here.

[0145] According to the transmission line fault detection device based on image recognition proposed in the embodiment of the present application, the original image of the transmission line can be multi-processed, and the fault detection result of the transmission line can be identified based on the processed image. In this way, the quality of the image is improved by image restoration, and the image is further processed, and then compared with the template image, which effectively improves the accuracy and reliability of the image recognition result, expands the scope of application of the present application, and is helpful for practical application in various scenarios. In this way, the problems in the related art that the use of image recognition technology to extract fault features in images to identify transmission line faults involve the processing of massive image information, and the system detection force is required to be high while the accuracy is difficult to guarantee, how to improve the image recognition technology, and improve the accuracy of transmission line fault identification and detection results, etc. are solved.

[0146] Fig.15 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0147] A memory 1501 , a processor 1502 , and a computer program stored in the memory 1501 and executable on the processor 1502 .

[0148] When the processor 1502 executes the program, the power transmission line fault detection method based on image recognition provided in the above embodiment is implemented.

[0149] Furthermore, the electronic device further comprises:

[0150] The communication interface 1503 is used for communication between the memory 1501 and the processor 1502 .

[0151] The memory 1501 is used to store computer programs that can be executed on the processor 1502 .

[0152] The memory 1501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0153] If the memory 1501, the processor 1502 and the communication interface 1503 are implemented independently, the communication interface 1503, the memory 1501 and the processor 1502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0154] Optionally, in a specific implementation, if the memory 1501, the processor 1502 and the communication interface 1503 are integrated on a chip, the memory 1501, the processor 1502 and the communication interface 1503 can communicate with each other through an internal interface.

[0155] The processor 1502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0156] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for detecting power transmission line faults based on image recognition.

[0157] An embodiment of the present application also provides a computer program product, including a computer program, which can run computer instructions. When the computer instructions are executed by a processor, the image recognition-based transmission line fault detection method provided in the embodiment of the present application is implemented.

[0158] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0159] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0160] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0161] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0162] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0163] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0164] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0165] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for detecting power transmission line faults based on image recognition. It is characterized in that The following steps are involved: Acquire an original image of the power transmission line, and preprocess the original image to obtain a restored image of the power transmission line; Extracting feature points of the restored image and feature points of the template image, and matching the feature points of the restored image with the feature points of the template image to obtain a mapping relationship between the restored image and the template image; The restored image is registered according to the mapping relationship to obtain a registered image of the restored image, so as to identify the fault detection result of the power transmission line according to the image difference between the registered image and the template image.

2. The method according to claim 1, It is characterized in that The preprocessing of the original image to obtain the restored image of the power transmission line includes: The restored image is filtered using a preset peak signal-to-noise ratio as a metric for image restoration to obtain the restored image.

3. The method according to claim 1, It is characterized in that The matching of the feature points of the restored image with the feature points of the template image to obtain a mapping relationship between the restored image and the template image includes: Characterizing the feature points of the restored image and the feature points of the template image with feature vectors to obtain the feature vector of the restored image and the feature vector of the template image respectively; Calculating the Euclidean distance between the feature vector of the restored image and the feature vector of the template image; Whether the feature points of the restored image and the feature points of the template image are matched successfully is determined according to the Euclidean distance. If the match is successful, the mapping relationship between the restored image and the template image is obtained according to the successfully matched feature points; otherwise, the feature points that fail to match are rejected.

4. The method according to claim 1, It is characterized in that The performing registration processing on the restored image according to the mapping relationship to obtain a registered image of the restored image includes: Based on the mapping relationship, the feature points of the restored image and the feature points of the template image at the same position are matched to obtain a transformation matrix; The image portion where the restored image overlaps with the template image is mapped to the template image coordinate system for registration according to the transformation matrix to obtain a registered image of the restored image.

5. The method according to claim 1, It is characterized in that The identifying the fault of the power transmission line according to the image difference between the registered image and the template image comprises: Analyzing an image difference value between a binarized image of the registration image and a binarized image of the template image to obtain an image difference between the registration image and the template image according to the image difference value and a preset difference threshold; identifying an image category of the registered image based on the image difference; A fault detection result of the power transmission line is determined according to the image category.

6. A transmission line fault detection device based on image recognition, It is characterized in that include: A processing module, used for acquiring an original image of the power transmission line, and preprocessing the original image to obtain a restored image of the power transmission line; A matching module, used for extracting feature points of the restored image and feature points of the template image, and matching the feature points of the restored image with the feature points of the template image to obtain a mapping relationship between the restored image and the template image; The recognition module is used to perform registration processing on the restored image according to the mapping relationship to obtain a registered image of the restored image, so as to recognize the fault detection result of the power transmission line according to the image difference between the registered image and the template image.

7. The device according to claim 6, It is characterized in that The processing module comprises: The processing unit is used to filter the restored image using a preset peak signal-to-noise ratio as a metric for image restoration to obtain the restored image.

8. An electronic device, It is characterized in that include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for detecting power transmission line faults based on image recognition as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that The program is executed by a processor to implement the power transmission line fault detection method based on image recognition as described in any one of claims 1 to 5.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed, it is used to implement the power transmission line fault detection method based on image recognition as described in any one of claims 1 to 5.

Images