Intelligent infrared zero measurement method and system for super-long string insulator

Abstract

The application discloses an intelligent infrared zero measurement method and system for an ultra-long string insulator, and the method comprises the following steps: acquiring an insulator key frame image; registering the insulator key frame image, and integrating the insulator key frame image into a mapping of a same coordinate system; adopting a cylindrical space curved surface as a projection surface to project and deform the integrated insulator key frame image, so as to obtain a projected and deformed insulator key frame image; performing fusion processing on the projected and deformed insulator key frame image, so as to obtain a seamless full-string insulator infrared image; extracting infrared temperature curve information of the seamless full-string insulator infrared image, and judging whether the seamless full-string insulator contains a zero-value insulator or not; and the application can realize full-automatic aerial photography of a UAV and real-time infrared diagnosis of an ultra-long string insulator, and has higher automation and better real-time performance compared with the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of insulator diagnosis, and in particular to an intelligent infrared zero detection method and system for an ultra-long string insulator. Background Art

[0002] Insulators are critical insulating and supporting components in power systems. Due to long-term exposure to electromechanical loads and external factors during operation, they are prone to gradual deterioration, resulting in low zero-value insulators. This can lead to serious accidents such as string breakage and line drop, posing a serious threat to the safe and stable operation of power transmission and transformation equipment. Therefore, regular insulator low-zero-value testing is necessary to ensure stable power system operation.

[0003] Infrared thermal imaging is a commonly used live-wire inspection technology for insulator low-zero-value diagnosis, characterized by its fast response and non-contact measurement. However, due to the limited resolution of existing industrial infrared thermal imagers, it is difficult to clearly capture the infrared spectrum of an entire string of insulators. The resulting temperature distribution curve would result in significant errors, making it prone to missed or misjudgment. Currently, the main methods for diagnosing low-zero-value insulators in very long strings rely on temperature curve splicing and infrared image splicing.

[0004] The temperature curve stitching method extracts temperature curves from infrared images and then stitches the temperature curves together. The infrared image stitching method directly processes infrared images, using multiple insulator infrared images as input to stitch together infrared images of a whole string of insulators. The key technology is image registration, which is the prerequisite for image stitching. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides an intelligent infrared zero detection method and system for ultra-long insulator strings. The method uses a drone to imitate the line, that is, the drone flies along the installation direction of the insulator and records the infrared image of the insulator in real time to form high-definition infrared photography. Subsequently, after extracting the key frames in the infrared video, the key frames can be aligned and spliced ​​to obtain an infrared image of the entire insulator string that meets the infrared detection requirements, thereby improving the accuracy and efficiency of infrared detection.

[0006] To achieve the above object, the present invention provides the following technical solution: an intelligent infrared zero detection method for ultra-long string insulators, comprising the following steps:

[0007] Step S1: using a drone to fly along a line to take aerial photos of the entire string of insulators, obtaining an infrared video of the insulators, and extracting key frames of the insulators from the infrared video to obtain key frame images of the insulators;

[0008] Step S2: registering the insulator key frame images using a sparse feature point extraction algorithm, and integrating the insulator key frame images into a mapping of the same coordinate system;

[0009] Step S3: using a cylindrical space surface as a projection surface to perform projection deformation on the integrated insulator key frame image to obtain a projection-deformed insulator key frame image;

[0010] Step S4: performing fusion processing on the projected deformed insulator key frame image to obtain a seamless infrared image of the entire insulator string;

[0011] Step S5: extracting infrared temperature curve information of the infrared image of the seamless full string insulators, and determining whether the seamless full string insulators contain zero-value insulators.

[0012] Furthermore, the specific process of extracting the insulator key frame image from the insulator infrared video is as follows: using the LBP feature as the operator of the Fisher classifier, classifying the insulator infrared video into background images and target images through the Fisher classifier after replacing the operator, and finally selecting the target image in the insulator infrared video as the insulator key frame image.

[0013] Furthermore, the specific steps of classifying the infrared video of the insulator into background images and target images by the Fisher classifier after the replacement operator are as follows:

[0014] Define the detection window of the insulator image in the insulator infrared video and divide the insulator image into 16×16 areas;

[0015] The operator is set as follows: the pixel value of the central pixel of each area of ​​the insulator image is used as the threshold, and compared with the pixels of the adjacent 8 areas. When the pixel value of the adjacent pixel is greater than the pixel value of the central pixel, the adjacent pixel is recorded as 1; when the pixel value of the adjacent pixel is less than the pixel value of the central pixel, the adjacent pixel is recorded as 0;

[0016] Calculating the histogram of each region of the insulator image according to the operator, and performing normalization processing on the histogram of each region;

[0017] The normalized histograms of each region are connected to obtain the LBP feature vector of the insulator image texture. The LBP feature vector is input into the Fisher classifier after the replacement operator for classification to obtain the background image and the target image.

[0018] Furthermore, the specific process of registering the insulator key frame images is as follows: first, a probability model is used to establish an adjacency table between the insulator key frame images based on the correspondence between the insulator key frame images. The correspondence is whether there is an overlapping area or not between the insulator key frame images. The probability model is used to assume that the correctly matched feature points and the incorrectly matched feature points between two adjacent insulator key frame images conform to the Bernoulli distribution;

[0019] The probability that the insulator key frame image pair is an adjacent image is calculated according to the Bayesian rule; when the probability that the insulator key frame image pair is an adjacent image is greater than a preset value, the insulator key frame image pair is determined to be an adjacent image.

[0020] Furthermore, in step S4, the projected deformed insulator key frame images are fused using a weighted linear fusion method, that is, different weight coefficients are assigned to the pixels of the two overlapping insulator key frame images in the overlapping area between the insulator key frame images.

[0021] Furthermore, step S2 also includes: using the approximate nearest neighbor search library algorithm FLANN to further improve the extraction speed of feature points, and at the same time using the KNN ratio screening method to retain the K-neighbor matching results and calculate the ratio of the nearest matching point to the second nearest matching point. When the ratio is greater than a preset value, the matching result is eliminated.

[0022] Furthermore, step S3 also includes: projecting the insulator key frame images onto a projection surface in a unified coordinate system through the mapping relationship of the insulator key frame images, and automatically interpolating the size changes of the insulator key frame images during the mapping process.

[0023] An intelligent infrared zero detection system for ultra-long string insulators, comprising:

[0024] The shooting and extraction module is used to use a drone to fly along the line to take aerial photos of the entire string of insulators, obtain infrared videos of the insulators, and extract key frames of the insulators from the infrared videos to obtain key frame images of the insulators;

[0025] A registration and integration module is used to register the insulator key frame images using a sparse feature point extraction algorithm and integrate the insulator key frame images into a mapping of the same coordinate system;

[0026] A projection deformation module is used to use a cylindrical space surface as a projection surface to perform projection deformation on the integrated insulator key frame image to obtain a projection deformed insulator key frame image;

[0027] A fusion processing module is used to fuse the projected deformed insulator key frame images to obtain a seamless infrared image of the entire insulator string;

[0028] The extraction and judgment module is used to extract the infrared temperature curve information of the infrared image of the seamless full-string insulator and judge whether the seamless full-string insulator contains zero-value insulators.

[0029] Compared with the existing technology, the present invention has the following beneficial effects:

[0030] (1) The unmanned aerial vehicle full-automatic aerial photography and real-time infrared diagnosis of the super-long string insulator can be realized, the automation degree is higher, the real-time performance is better, the problem that the super-long string insulator cannot be clearly photographed is solved, and the accuracy of the infrared diagnosis of the super-long string insulator is improved.

[0031] (2) The LBP feature is used as the operator of the Fisher classifier, the key frame image is extracted through the Fisher classifier after the operator is replaced, the accuracy of insulator identification and the stability of the feature are improved, the data processing efficiency is optimized through the automatic classification process, the robustness of the system is enhanced, and a solid foundation is laid for subsequent image analysis steps.

[0032] (3) The sparse feature point extraction algorithm and the approximate nearest neighbor search library algorithm are adopted to quickly and accurately extract feature points, improve the processing speed, and the KNN ratio screening method is used to further improve the accuracy of feature point matching; the adjacency table is established through the probability model, the insulator key frame image can be effectively registered, the corresponding relationship between images is ensured to be accurate, and a foundation is laid for subsequent projection deformation and fusion processing.

[0033] (4) The cylindrical space curved surface is used as the projection surface for projection deformation, the actual shape of the insulator can be more truly reflected, the image is projected to a unified coordinate system through the mapping relationship, and the size change is automatically interpolated, the continuity and consistency of the image are ensured, and high-quality input is provided for subsequent image fusion. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 It is a method flowchart of the application.

[0035] Figure 2 It is an LBP feature extraction schematic diagram. DETAILED DESCRIPTION

[0036] As shown in Figure 1 , the application provides a technical scheme: a super-long string insulator intelligent infrared zero measurement method, comprising the following steps:

[0037] Step S1: using an unmanned aerial vehicle to fly along a line to photograph a full string of insulators, obtaining an insulator infrared video, extracting insulator key frames in the insulator infrared video, and obtaining insulator key frame images.

[0038] Step S2: using a sparse feature point extraction algorithm to register the insulator key frame images, and integrating the insulator key frame images into a mapping of the same coordinate system.

[0039] Step S3: using a cylindrical space curved surface as a projection surface to perform projection deformation on the integrated insulator key frame image to obtain the insulator key frame image after projection deformation.

[0040] Through the mapping relationship of the insulator key frame images, the insulator key frame images are projected onto a projection surface in a unified coordinate system, and the size changes of the insulator key frame images during the mapping process are automatically interpolated.

[0041] Step S4: performing fusion processing on the projected deformed insulator key frame images to obtain a seamless infrared image of the entire insulator string.

[0042] Step S5: extracting infrared temperature curve information of the infrared image of the seamless full string insulators, and determining whether the seamless full string insulators contain zero-value insulators.

[0043] The specific process of obtaining the infrared video of the insulator in step S1 is: automatically identifying the positions of power transmission line equipment such as wires and insulators through a visual recognition algorithm, and realizing operations such as flight attitude adjustment and tracking shooting of the UAV.

[0044] Among them, the pre-trained YOLO V7 target recognition network is used as the visual recognition algorithm to automatically identify the location of power transmission line equipment such as wires and insulators. The specific steps are: first, use the camera mounted on the drone to take real-time aerial videos of the entire string of insulators, and then use the real-time video of the entire string of insulators as the input of the pre-trained YOLO V7 target recognition network to detect the location of power transmission line equipment such as wires and insulators in real time.

[0045] Among them, the specific process of extracting the insulator key frame image from the insulator infrared video in step S1 is as follows: the insulator infrared video obtained by the drone's line-simulating aerial photography of the entire string of insulators has the characteristics of large changes in background and shooting angle, and the LBP feature with stable grayscale and rotation characteristics is used as the operator of the Fisher classifier. The insulator infrared video is classified into background class images and target class images by the Fisher classifier after replacing the operator, and finally the target class image in the insulator infrared video is selected as the insulator key frame image.

[0046] The specific steps for classifying the infrared video of the insulator into background images and target images by the Fisher classifier after the replacement operator are as follows:

[0047] A. Define the detection window of the insulator image in the insulator infrared video and divide the insulator image into 16×16 areas.

[0048] B. Set the operator as follows: take the pixel value of the center pixel of each area of ​​the insulator image as the threshold and compare it with the pixels of the adjacent 8 areas, such asFigure 2 As shown, if the pixel value of the adjacent pixel point is greater than the pixel value of the central pixel point, then the adjacent pixel point is recorded as 1; if the pixel value of the adjacent pixel point is less than the pixel value of the central pixel point, then the adjacent pixel point is recorded as 0; execute the above steps to obtain Figure 2 The 8-bit binary number shown is the LBP value of the insulator image.

[0049] C. Calculate the histogram of each region of the insulator image according to the operator, and normalize the histogram of each region.

[0050] D. Connect the normalized histograms of each region to obtain the LBP feature vector of the insulator image texture. Input the LBP feature vector into the Fisher classifier after the replacement operator for classification to obtain the background image and the target image.

[0051] The sparse feature point extraction algorithm in step S2 is the SURT algorithm. The SURT algorithm can convert the process of calculating the Hessian matrix into integral graph calculation, thereby accelerating the construction of the scale pyramid.

[0052] Among them, the approximate nearest neighbor search library algorithm FLANN is used to further improve the extraction speed of feature points. At the same time, the KNN ratio screening method is used to retain the K-neighboring matching results and calculate the ratio of the nearest matching point to the second nearest matching point. If the ratio is greater than the preset value, the matching result is eliminated to improve the accuracy of feature point matching.

[0053] The specific process of registering the insulator key frame images in step S2 is as follows: first, an adjacency table between the insulator key frame images is established using a probabilistic model based on the correspondence between the insulator key frame images. The correspondence is whether there is an overlapping area or not between the insulator key frame images. The probabilistic model assumes that the correctly matched feature points (inliers) and incorrectly matched feature points (outliers) between two adjacent insulator key frame images conform to the Bernoulli distribution, and its mathematical expression is:

[0054] ;

[0055] ;

[0056] Where, Indicates the probability of correct matching of feature points when there is a corresponding relationship between the insulator key frame images; It represents the probability of feature point mismatch when there is no corresponding relationship between the insulator key frame images; Indicates the number of matching feature points, Indicates the number of correctly matched feature points; Indicates whether there is a corresponding relationship. Indicates that there is a corresponding relationship between the insulator key frame images, It means that there is no corresponding relationship between the key frame images of the insulator; It represents the probability that a pair of matching points between insulator key frame images are correctly matched feature points when there is a corresponding relationship between the insulator key frame image pairs formed after insulator key frame image registration; It represents the probability that a pair of matching points between insulator key frame images are correctly matched feature points when there is no corresponding relationship between the insulator key frame image pairs formed after insulator key frame image registration; represents the binomial distribution.

[0057] The probability that the insulator key frame image pair is an adjacent image is calculated according to the Bayesian rule:

[0058] ;

[0059] Where, It indicates the posterior probability that the insulator key frame image pair is an adjacent image when the feature points are correctly matched; represents the prior probability that the insulator key frame image pair is an adjacent image; represents the prior probability that the insulator key frame image pair is not an adjacent image; Indicates the probability of correct matching of feature points; when When it is greater than a preset value, the insulator key frame image pair is determined to be adjacent images.

[0060] The confidence determination conditions are:

[0061] ;

[0062] Where, and is the confidence parameter; the insulator key frame images with confidence greater than 1 are selected as adjacent images.

[0063] In step S4, the projected deformed insulator key frame images are fused using a weighted linear fusion method, that is, different weight coefficients are assigned to the pixels of the two overlapping insulator key frame images in the overlapping area between the insulator key frame images. The mathematical expression is as follows:

[0064] ;

[0065] Where, is the pixel value after fusion; Key frame image for the insulator Pixel value of Key frame image for the insulator The pixel value of is the pixel horizontal coordinate, is the pixel ordinate; and Represent the key frame images of insulators and insulator keyframe images The weight coefficient is as follows:

[0066] ;

[0067] Where, Insulator key frame image The horizontal coordinate of the pixel is , the pixel vertical coordinate is The weight coefficient of the location; Insulator key frame image The horizontal coordinate of the pixel is , the pixel vertical coordinate is The weight coefficient of the location, Indicates the The vertical coordinate of a pixel, when the vertical coordinate of the pixel remains unchanged, the horizontal coordinate of the pixel changes; and The weight coefficient can be expressed as the pixel horizontal coordinate the law of change; It is represented as the left boundary of the overlapping area between the insulator key frame images; It is represented as the right boundary of the overlapping area between the insulator key frame images; and With pixel vertical coordinate changes with the changes of .

[0068] An intelligent infrared zero detection system for ultra-long string insulators, comprising:

[0069] The shooting and extraction module is used to use a drone to fly along the line to take aerial photos of the entire string of insulators, obtain infrared videos of the insulators, extract key frames of the insulators in the infrared videos, and obtain key frame images of the insulators.

[0070] The registration and integration module is used to register the insulator key frame images using a sparse feature point extraction algorithm and integrate the insulator key frame images into a mapping of the same coordinate system.

[0071] The projection deformation module is used to use a cylindrical space surface as a projection surface to perform projection deformation on the integrated insulator key frame image to obtain the insulator key frame image after projection deformation.

[0072] The fusion processing module is used to perform fusion processing on the projected deformed insulator key frame images to obtain a seamless full-string insulator infrared image.

[0073] The extraction and judgment module is used to extract the infrared temperature curve information of the infrared image of the seamless full-string insulator and judge whether the seamless full-string insulator contains zero-value insulators.

[0074] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent infrared zero detection method for ultra-long string insulators, characterized in that: The steps include: Step S1: using a drone to fly along a line to take aerial photos of the entire string of insulators, obtaining an infrared video of the insulators, and extracting key frames of the insulators from the infrared video to obtain key frame images of the insulators; Step S2: registering the insulator key frame images using a sparse feature point extraction algorithm, and integrating the insulator key frame images into a mapping of the same coordinate system; Step S3: using a cylindrical space surface as a projection surface to perform projection deformation on the integrated insulator key frame image to obtain a projection-deformed insulator key frame image; Step S4: performing fusion processing on the projected deformed insulator key frame image to obtain a seamless infrared image of the entire insulator string; Step S5: extracting infrared temperature curve information of the infrared image of the seamless full string insulators to determine whether the seamless full string insulators contain zero-value insulators; The specific process of registering the insulator key frame images is as follows: first, a probability model is used to establish an adjacency table between the insulator key frame images based on the correspondence between the insulator key frame images. The correspondence relationship is whether there is an overlapping area or not between the insulator key frame images. The probability model is used to assume that the correctly matched feature points and the incorrectly matched feature points between two adjacent insulator key frame images conform to the Bernoulli distribution, which can be expressed as: ； ； Where, Indicates the probability of correct matching of feature points when there is a corresponding relationship between the insulator key frame images; It represents the probability of feature point mismatch when there is no corresponding relationship between the insulator key frame images; Indicates the number of matching feature points, Indicates the number of correctly matched feature points; Indicates whether there is a corresponding relationship. Indicates that there is a corresponding relationship between the insulator key frame images, It means that there is no corresponding relationship between the key frame images of the insulator; It represents the probability that a pair of matching points between insulator key frame images are correctly matched feature points when there is a corresponding relationship between the insulator key frame image pairs formed after insulator key frame image registration; It represents the probability that a pair of matching points between insulator key frame images are correctly matched feature points when there is no corresponding relationship between the insulator key frame image pairs formed after insulator key frame image registration; represents the binomial distribution; The probability that the insulator key frame image pair is an adjacent image is calculated according to the Bayesian rule: ； Where, It indicates the posterior probability that the insulator key frame image pair is an adjacent image when the feature points are correctly matched; represents the prior probability that the insulator key frame image pair is an adjacent image; represents the prior probability that the insulator key frame image pair is not an adjacent image; Indicates the probability of correct matching of feature points; when When it is greater than a preset value, the insulator key frame image pair is determined to be adjacent images.

2. The intelligent infrared zero detection method for ultra-long string insulators according to claim 1, characterized in that: The specific process of extracting the insulator key frame image from the insulator infrared video is as follows: using the LBP feature as the operator of the Fisher classifier, classifying the insulator infrared video into background images and target images through the Fisher classifier after replacing the operator, and finally selecting the target image in the insulator infrared video as the insulator key frame image.

3. The intelligent infrared zero detection method for ultra-long string insulators according to claim 2, characterized in that: The specific steps of classifying the insulator infrared video into background images and target images through the Fisher classifier after the replacement operator are as follows: Define the detection window of the insulator image in the insulator infrared video and divide the insulator image into 16×16 areas; The operator is set as follows: the pixel value of the central pixel of each area of ​​the insulator image is used as the threshold, and compared with the pixels of the adjacent 8 areas. When the pixel value of the adjacent pixel is greater than the pixel value of the central pixel, the adjacent pixel is recorded as 1; when the pixel value of the adjacent pixel is less than the pixel value of the central pixel, the adjacent pixel is recorded as 0; Calculating the histogram of each region of the insulator image according to the operator, and performing normalization processing on the histogram of each region; The normalized histograms of each region are connected to obtain the LBP feature vector of the insulator image texture. The LBP feature vector is input into the Fisher classifier after the replacement operator for classification to obtain the background image and the target image.

4. The intelligent infrared zero detection method for ultra-long string insulators according to claim 3, characterized in that: In step S4, the projected deformed insulator key frame images are fused using a weighted linear fusion method, that is, different weight coefficients are assigned to the pixels of the two overlapping insulator key frame images in the overlapping area between the insulator key frame images.

5. The intelligent infrared zero detection method for ultra-long string insulators according to claim 4, characterized in that: Step S2 also includes: using the approximate nearest neighbor search library algorithm FLANN to further improve the extraction speed of feature points, and using the KNN ratio screening method to retain the K-nearest matching results and calculate the ratio of the nearest matching point to the next nearest matching point. When the ratio is greater than a preset value, the matching result is eliminated.

6. The intelligent infrared zero detection method for ultra-long string insulators according to claim 5, characterized in that: Step S3 also includes: projecting the insulator key frame images onto a projection surface in a unified coordinate system through the mapping relationship of the insulator key frame images, and automatically interpolating the size changes of the insulator key frame images during the mapping process.

7. An intelligent infrared zero detection system for ultra-long string insulators, applied to an intelligent infrared zero detection method for ultra-long string insulators according to any one of claims 1 to 6, characterized in that: include: The shooting and extraction module is used to use a drone to fly along the line to take aerial photos of the entire string of insulators, obtain infrared videos of the insulators, and extract key frames of the insulators from the infrared videos to obtain key frame images of the insulators; A registration and integration module is used to register the insulator key frame images using a sparse feature point extraction algorithm and integrate the insulator key frame images into a mapping of the same coordinate system; A projection deformation module is used to use a cylindrical space surface as a projection surface to perform projection deformation on the integrated insulator key frame image to obtain a projection deformed insulator key frame image; A fusion processing module is used to fuse the projected deformed insulator key frame images to obtain a seamless infrared image of the entire insulator string; The extraction and judgment module is used to extract the infrared temperature curve information of the infrared image of the seamless full-string insulator and judge whether the seamless full-string insulator contains zero-value insulators.

Images