A power transmission tower swaying estimation method and system based on image fusion matching

Abstract

The application provides a power transmission tower swing estimation method and system based on image fusion matching. Real-time two-dimensional optical images of a power transmission tower and real-time infrared images of the power transmission tower are obtained. Real-time two-dimensional optical images of the power transmission tower after preprocessing and real-time infrared images of the power transmission tower after preprocessing are obtained through preprocessing. A plurality of enhanced corner points of the real-time two-dimensional optical images of the power transmission tower after preprocessing and a plurality of enhanced corner points of the real-time infrared images of the power transmission tower after preprocessing are obtained through Harris corner point detection and corner point intensity threshold judgment and screening. Each group of optical infrared enhanced corner point pairs is constructed. Real-time fused optical infrared images and a plurality of fused corner points of the real-time fused optical infrared images are obtained through image fusion by a remapping method. Each group of stable detection between the fused optical infrared enhanced corner point pairs is constructed. Whether the power transmission tower swings is determined through L2 norm distance calculation. The application improves the swing estimation accuracy of the power transmission tower and can adapt to complex and severe weather environments.

Description

Technical Field

[0001] This invention belongs to the field of high-voltage transmission line safety detection technology, and in particular relates to a method and system for estimating transmission tower sway by image fusion and matching. Background Technology

[0002] Power transmission lines, as carriers of high-load electrical energy, are a widely used lifeline project. Therefore, the need to optimize traditional power tower monitoring technology has emerged, which not only has significant economic and social implications but also reflects a commitment to actively responding to the nation's call for energy conservation.

[0003] Currently, the most popular methods for measuring the sway displacement of large, tall buildings in China include GNSS measurement, laser measurement, and video technology measurement. However, these methods all have significant drawbacks. First, GNSS measurement suffers from signal shielding affecting positioning accuracy, and its accuracy is also limited by satellite position or clock deviation. Second, laser measurement is sensitive to environmental conditions; for example, atmospheric turbulence and dust can affect beam propagation and reception, thus reducing measurement accuracy. Third, video technology measurement is greatly affected by lighting conditions, potentially leading to image blurring. Summary of the Invention

[0004] To overcome the shortcomings of the existing technology, this invention proposes a method and system for estimating transmission tower sway by image fusion matching.

[0005] The technical solution of this invention is an image fusion matching method for estimating transmission tower sway, comprising the following steps:

[0006] Step 1: Acquire real-time two-dimensional optical images and real-time infrared images of the transmission tower;

[0007] Step 2: The real-time two-dimensional optical image and the real-time infrared image of the transmission tower are processed by denoising and image enhancement respectively to obtain the real-time pre-processed two-dimensional optical image and the real-time pre-processed infrared image of the transmission tower.

[0008] Step 3: Using the Harris corner detection method, corner points are extracted and corner point intensity thresholds are determined and filtered to obtain the pixel positions and intensities of multiple corner points in the real-time preprocessed infrared image of the transmission tower, as well as multiple enhanced corner points in the real-time preprocessed infrared image of the transmission tower.

[0009] Step 4: Construct each pair of optical infrared enhanced corner points by extracting neighboring pixels and comparing pixel mean square error thresholds;

[0010] Step 5: Combine the real-time preprocessed 2D optical image of the transmission tower and the real-time preprocessed infrared image of the transmission tower with the affine transformation matrix and perform image fusion using the remapping method to obtain the real-time fused optical-infrared image and multiple fused corner points of the real-time fused optical-infrared image.

[0011] Step 6: Construct fused optical infrared enhanced corner point pairs between each group of stable detections by using multiple fused corner points corresponding to the real-time fused optical infrared images during the stabilization period, and the real-time fused optical infrared images and corresponding multiple fused corner points during the detection period.

[0012] Step 7: Calculate the distance between the fused optical-infrared enhanced corner point pairs between each group of stable detections using the L2 paradigm. If the distance exceeds the threshold, it is determined that the power tower is shaking.

[0013] Preferably, step 3 involves corner extraction using the Harris corner detection method, as detailed below:

[0014] The real-time preprocessed two-dimensional optical image of the transmission tower is used to extract corners using the Harris corner detection method, thereby obtaining the pixel positions and intensities of multiple corners of the real-time preprocessed two-dimensional optical image of the transmission tower.

[0015] The real-time preprocessed infrared image of the transmission tower is used to extract corner points using the Harris corner detection method to obtain the pixel position and intensity of multiple corner points in the real-time preprocessed infrared image of the transmission tower.

[0016] Step 3, the corner strength threshold determination and filtering, is as follows:

[0017] Multiple corner points with intensity greater than the corner point threshold are selected from multiple corner points in the real-time preprocessed two-dimensional optical image of the transmission tower to construct multiple enhanced corner points in the real-time preprocessed two-dimensional optical image of the transmission tower.

[0018] Multiple corner points with intensity greater than the corner point threshold are selected from multiple corner points in the real-time preprocessed infrared image of the transmission tower to construct multiple enhanced corner points in the real-time preprocessed infrared image of the transmission tower.

[0019] As a preferred embodiment, the extraction of neighboring pixels in step 4 is specifically as follows:

[0020] The pixel positions of eight neighboring pixels of each enhanced corner point in the 3×3 image region are extracted from each enhanced corner point of the real-time preprocessed two-dimensional optical image of the transmission tower to construct the description vector of each enhanced corner point of the real-time preprocessed two-dimensional optical image of the transmission tower.

[0021] The pixel positions of eight neighboring pixels of each enhanced corner point in the real-time preprocessed infrared image of the transmission tower are extracted in a 3×3 image area to construct the description vector of each enhanced corner point in the real-time preprocessed infrared image of the transmission tower.

[0022] The pixel mean square error threshold comparison in step 4 is as follows:

[0023] Calculate the pixel mean square error between the description vector of each enhanced corner point of the real-time preprocessed two-dimensional optical image of the transmission tower and the description vector of each enhanced corner point of the real-time preprocessed infrared image of the transmission tower.

[0024] Enhanced corner points of real-time preprocessed two-dimensional optical images of transmission towers and enhanced corner points of real-time preprocessed infrared images of transmission towers with pixel mean square error less than the pixel mean square error threshold are selected to construct each pair of optical and infrared enhanced corner points.

[0025] Step 4 describes the construction of each pair of optical infrared enhancement corner points, defined as follows:

[0026] The enhanced corner points of the two-dimensional optical image of the transmission tower after preprocessing (where the pixel mean square error is less than the pixel mean square error threshold) and the enhanced corner points of the infrared image of the transmission tower after preprocessing (where the pixel mean square error is less than the pixel mean square error threshold) constitute each pair of optical infrared enhanced corner points.

[0027] Preferably, the affine transformation matrix in step 5 is constructed as follows:

[0028] An affine transformation matrix is ​​constructed by combining multiple sets of optical infrared enhanced corner point pairs using the RANSAC method.

[0029] As a preferred embodiment, the calculation process for the multiple fused corner points corresponding to the real-time fused optical-infrared image during the stabilization period in step 6 is as follows:

[0030] During the stabilization period of the transmission tower, real-time fused optical infrared images and corresponding multiple fused corner points are obtained through steps 1-5.

[0031] The calculation process for multiple fused corner points corresponding to the real-time fused optical-infrared image during the detection period described in step 6 is as follows:

[0032] During the detection process, multiple fused corner points corresponding to the real-time fused optical infrared image are obtained through steps 1-5.

[0033] Step 6 describes the construction of fused optical-infrared enhanced corner point pairs between each group of stable detections, as follows:

[0034] Step 4 is used to construct fused optical infrared enhanced corner point pairs between each group of stable detections by combining multiple fused corner points of the real-time fused optical infrared image during the stabilization period and multiple fused corner points of the real-time fused optical infrared image during the detection period.

[0035] The technical solution of this invention is an image fusion and matching transmission tower sway estimation system, comprising:

[0036] The image acquisition module is used to acquire real-time two-dimensional optical images and real-time infrared images of the transmission tower.

[0037] The image preprocessing module is used to obtain real-time preprocessed two-dimensional optical images and real-time preprocessed infrared images of the transmission tower by denoising and image enhancement processing, respectively.

[0038] The enhanced corner extraction module is used to extract corners and filter them by using the Harris corner detection method, so as to obtain the pixel position and intensity of multiple corners of the preprocessed infrared image of the transmission tower in real time, as well as multiple enhanced corners of the preprocessed infrared image of the transmission tower in real time.

[0039] The optical infrared enhanced corner point pair construction module is used to construct each optical infrared enhanced corner point pair by extracting neighboring pixels and comparing pixel mean square error thresholds;

[0040] The fused corner point construction module is used to combine the real-time preprocessed two-dimensional optical image of the transmission tower and the real-time preprocessed infrared image of the transmission tower with the affine transformation matrix and perform image fusion through the remapping method to obtain the real-time fused optical-infrared image and multiple fused corner points of the real-time fused optical-infrared image.

[0041] The stable detection corner point pair construction module is used to construct fused optical infrared enhanced corner point pairs between each stable detection by using multiple fused corner points corresponding to the real-time fused optical infrared image during the stabilization period, and the real-time fused optical infrared image and multiple fused corner points corresponding to the detection period.

[0042] The swaying detection module is used to calculate the distance between the fused optical and infrared enhanced corner point pairs between each group of stable detections using the L2 paradigm. If the distance exceeds the distance threshold, the power tower is determined to be swaying.

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] This invention, through the use of computer vision, achieves relatively accurate results with a small number of templates, reducing algorithm implementation costs while maintaining high accuracy and effectively minimizing manual labor costs. It leverages the advantages of multimodal imaging: optical and infrared images each have their strengths. Optical images provide high-resolution detail within the visible light range, while infrared images capture the thermal radiation information of objects, which is particularly important for monitoring under various environmental conditions. Combining these two modalities improves the robustness and accuracy of the monitoring system. It also enhances the accuracy of feature point detection: by first identifying points of interest in the optical image and then mapping these points to the infrared image, the richer detail information in the optical image can guide the analysis of the infrared image. This reduces the false recognition rate caused by the lower resolution of infrared images. Attached Figure Description

[0045] Figure 1 : Flowchart of the method according to an embodiment of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0048] To verify the effectiveness of the present invention, the specific implementation scenario of the present invention is as follows:

[0049] The infrared and optical images were captured by infrared and optical lenses located at the base of the transmission tower. There are 500 infrared images and 500 optical images, including images taken in clear daytime, nighttime, and inclement weather such as rainy days.

[0050] The following is combined Figure 1 Specific embodiments of the present invention are described below.

[0051] like Figure 1 As shown, the specific embodiment of the method of the present invention is an image fusion matching method for estimating transmission tower sway, and the specific steps are as follows:

[0052] Step 1: Acquire real-time two-dimensional optical images and real-time infrared images of the transmission tower;

[0053] Step 2: The real-time two-dimensional optical image and the real-time infrared image of the transmission tower are processed by denoising and image enhancement respectively to obtain the real-time pre-processed two-dimensional optical image and the real-time pre-processed infrared image of the transmission tower.

[0054] The specific method for noise reduction is the mean filtering method;

[0055] The specific method for image enhancement processing is linear grayscale transformation;

[0056] Step 3: Using the Harris corner detection method, corner points are extracted and corner point intensity thresholds are determined and filtered to obtain the pixel positions and intensities of multiple corner points in the real-time preprocessed infrared image of the transmission tower, as well as multiple enhanced corner points in the real-time preprocessed infrared image of the transmission tower.

[0057] Step 3 describes the corner extraction using the Harris corner detection method, as detailed below:

[0058] The real-time preprocessed two-dimensional optical image of the transmission tower is used to extract corners using the Harris corner detection method, thereby obtaining the pixel positions and intensities of multiple corners of the real-time preprocessed two-dimensional optical image of the transmission tower.

[0059] The real-time preprocessed infrared image of the transmission tower is used to extract corner points using the Harris corner detection method to obtain the pixel position and intensity of multiple corner points in the real-time preprocessed infrared image of the transmission tower.

[0060] Step 3, the corner strength threshold determination and filtering, is as follows:

[0061] Multiple corner points with intensity greater than the corner point threshold are selected from multiple corner points in the real-time preprocessed two-dimensional optical image of the transmission tower to construct multiple enhanced corner points in the real-time preprocessed two-dimensional optical image of the transmission tower.

[0062] Multiple corner points with intensity greater than the corner point threshold are selected from multiple corner points in the real-time preprocessed infrared image of the transmission tower to construct multiple enhanced corner points in the real-time preprocessed infrared image of the transmission tower.

[0063] Step 4: Construct each pair of optical infrared enhanced corner points by extracting neighboring pixels and comparing pixel mean square error thresholds;

[0064] Step 4, the extraction of neighboring pixels, is as follows:

[0065] The pixel positions of eight neighboring pixels of each enhanced corner point in the 3×3 image region are extracted from each enhanced corner point of the real-time preprocessed two-dimensional optical image of the transmission tower to construct the description vector of each enhanced corner point of the real-time preprocessed two-dimensional optical image of the transmission tower.

[0066] The pixel positions of eight neighboring pixels of each enhanced corner point in the real-time preprocessed infrared image of the transmission tower are extracted in a 3×3 image area to construct the description vector of each enhanced corner point in the real-time preprocessed infrared image of the transmission tower.

[0067] The pixel mean square error threshold comparison in step 4 is as follows:

[0068] Calculate the pixel mean square error between the description vector of each enhanced corner point of the real-time preprocessed two-dimensional optical image of the transmission tower and the description vector of each enhanced corner point of the real-time preprocessed infrared image of the transmission tower.

[0069] Enhanced corner points of real-time preprocessed two-dimensional optical images of transmission towers and enhanced corner points of real-time preprocessed infrared images of transmission towers with pixel mean square error less than the pixel mean square error threshold are selected to construct each pair of optical and infrared enhanced corner points.

[0070] Step 4 describes the construction of each pair of optical infrared enhancement corner points, defined as follows:

[0071] The enhanced corner points of the two-dimensional optical image of the transmission tower after preprocessing (where the pixel mean square error is less than the pixel mean square error threshold) and the enhanced corner points of the infrared image of the transmission tower after preprocessing (where the pixel mean square error is less than the pixel mean square error threshold) constitute each pair of optical infrared enhanced corner points.

[0072] Step 5: Combine the real-time preprocessed 2D optical image of the transmission tower and the real-time preprocessed infrared image of the transmission tower with the affine transformation matrix and perform image fusion using the remapping method to obtain the real-time fused optical-infrared image and multiple fused corner points of the real-time fused optical-infrared image.

[0073] The affine transformation matrix mentioned in step 5 is constructed as follows:

[0074] A multi-set optical infrared enhanced corner point pair is combined to construct an affine transformation matrix using the RANSAC method. The matrix size is 3×3.

[0075] Step 6: Construct fused optical infrared enhanced corner point pairs between each group of stable detections by using multiple fused corner points corresponding to the real-time fused optical infrared images during the stabilization period, and the real-time fused optical infrared images and corresponding multiple fused corner points during the detection period.

[0076] The calculation process for the multiple fused corner points corresponding to the real-time fused optical-infrared image during the stabilization period described in step 6 is as follows:

[0077] During the stabilization period of the transmission tower, real-time fused optical infrared images and corresponding multiple fused corner points are obtained through steps 1-5.

[0078] The calculation process for multiple fused corner points corresponding to the real-time fused optical-infrared image during the detection period described in step 6 is as follows:

[0079] During the detection process, multiple fused corner points corresponding to the real-time fused optical infrared image are obtained through steps 1-5.

[0080] Step 6 describes the construction of fused optical-infrared enhanced corner point pairs between each group of stable detections, as follows:

[0081] Step 4 is used to construct fused optical infrared enhanced corner point pairs between each group of stable detections by combining multiple fused corner points of the real-time fused optical infrared image during the stabilization period and multiple fused corner points of the real-time fused optical infrared image during the detection period.

[0082] Step 7: Calculate the distance between the fused optical-infrared enhanced corner point pairs between each group of stable detections using the L2 paradigm. If the distance exceeds the threshold, it is determined that the power tower is shaking.

[0083] Table 1: Tower Displacement Measurement Table

[0084]

[0085] The tower displacement measurement results are shown in Table 1. It can be seen that, regardless of whether it is sunny or rainy, this method can improve the measurement accuracy from centimeter level to millimeter level, and can adapt to harsh weather conditions such as complex rainy days.

[0086] The technical solution of this invention is an image fusion and matching transmission tower sway estimation system, comprising:

[0087] The image acquisition module is used to acquire real-time two-dimensional optical images and real-time infrared images of the transmission tower.

[0088] The image preprocessing module is used to obtain real-time preprocessed two-dimensional optical images and real-time preprocessed infrared images of the transmission tower by denoising and image enhancement processing, respectively.

[0089] The enhanced corner extraction module is used to extract corners and filter them by using the Harris corner detection method, so as to obtain the pixel position and intensity of multiple corners of the preprocessed infrared image of the transmission tower in real time, as well as multiple enhanced corners of the preprocessed infrared image of the transmission tower in real time.

[0090] The optical infrared enhanced corner point pair construction module is used to construct each optical infrared enhanced corner point pair by extracting neighboring pixels and comparing pixel mean square error thresholds;

[0091] The fused corner point construction module is used to combine the real-time preprocessed two-dimensional optical image of the transmission tower and the real-time preprocessed infrared image of the transmission tower with the affine transformation matrix and perform image fusion through the remapping method to obtain the real-time fused optical-infrared image and multiple fused corner points of the real-time fused optical-infrared image.

[0092] The stable detection corner point pair construction module is used to construct fused optical infrared enhanced corner point pairs between each stable detection by using multiple fused corner points corresponding to the real-time fused optical infrared image during the stabilization period, and the real-time fused optical infrared image and multiple fused corner points corresponding to the detection period.

[0093] The swaying detection module is used to calculate the distance between the fused optical and infrared enhanced corner point pairs between each group of stable detections using the L2 paradigm. If the distance exceeds the distance threshold, the power tower is determined to be swaying.

[0094] The image acquisition module, image preprocessing module, enhanced corner point extraction module, optical infrared enhanced corner point pair construction module, fused corner point construction module, stable detection corner point pair construction module, and shaking determination module are all deployed on the server.

[0095] It should be understood that any parts not described in detail in this specification belong to the prior art.

[0096] It should be understood that the above description of the embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art can make substitutions or modifications under the guidance of this invention without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A method for estimating transmission tower sway using image fusion and matching, characterized in that, Includes the following steps: Step 1: Acquire real-time two-dimensional optical images and real-time infrared images of the transmission tower; Step 2: The real-time two-dimensional optical image and the real-time infrared image of the transmission tower are processed by denoising and image enhancement respectively to obtain the real-time pre-processed two-dimensional optical image and the real-time pre-processed infrared image of the transmission tower. Step 3: Corner extraction and corner intensity threshold determination are performed using the Harris corner detection method to obtain multiple enhanced corners of the real-time preprocessed two-dimensional optical image of the transmission tower and multiple enhanced corners of the real-time preprocessed infrared image of the transmission tower. Step 4: Construct each pair of optical infrared enhanced corner points by extracting neighboring pixels and comparing pixel mean square error thresholds; Step 5: Combine the real-time preprocessed two-dimensional optical image of the transmission tower and the real-time preprocessed infrared image of the transmission tower with the affine transformation matrix and perform image fusion using the remapping method to obtain the real-time fused optical infrared image and multiple fused corner points of the real-time fused optical infrared image. Step 6: Construct fused optical infrared enhanced corner point pairs between each group of stable detections by using multiple fused corner points corresponding to the real-time fused optical infrared images during the stabilization period, and the real-time fused optical infrared images and corresponding multiple fused corner points during the detection period. Step 7: Calculate the distance between the fused optical-infrared enhanced corner point pairs between each group of stable detections using the L2 paradigm. If the distance exceeds the threshold, it is determined that the power tower is shaking. Step 4, the extraction of neighboring pixels, is as follows: The pixel positions of eight neighboring pixels of each enhanced corner point in the 3*3 image region are extracted from each enhanced corner point of the real-time preprocessed two-dimensional optical image of the transmission tower to construct the description vector of each enhanced corner point of the real-time preprocessed two-dimensional optical image of the transmission tower. The pixel positions of eight neighboring pixels of each enhanced corner point in the real-time preprocessed infrared image of the transmission tower are extracted in a 3*3 image region to construct the description vector of each enhanced corner point in the real-time preprocessed infrared image of the transmission tower. The pixel mean square error threshold comparison in step 4 is as follows: Calculate the pixel mean square error between the description vector of each enhanced corner point of the real-time preprocessed two-dimensional optical image of the transmission tower and the description vector of each enhanced corner point of the real-time preprocessed infrared image of the transmission tower. Enhanced corner points of real-time preprocessed two-dimensional optical images of transmission towers and enhanced corner points of real-time preprocessed infrared images of transmission towers with pixel mean square error less than the pixel mean square error threshold are selected. Step 4 describes the construction of each pair of optical infrared enhancement corner points, defined as follows: The enhanced corner points of the two-dimensional optical image of the transmission tower after preprocessing (where the pixel mean square error is less than the pixel mean square error threshold) and the enhanced corner points of the infrared image of the transmission tower after preprocessing (where the pixel mean square error is less than the pixel mean square error threshold) constitute each pair of optical infrared enhanced corner points. The calculation process for the multiple fused corner points corresponding to the real-time fused optical-infrared image during the stabilization period described in step 6 is as follows: During the stabilization period of the transmission tower, real-time fused optical infrared images and corresponding multiple fused corner points are obtained through steps 1-5. The calculation process for multiple fused corner points corresponding to the real-time fused optical-infrared image during the detection period described in step 6 is as follows: During the detection process, multiple fused corner points corresponding to the real-time fused optical infrared image are obtained through steps 1-5. Step 6 describes the construction of fused optical-infrared enhanced corner point pairs between each group of stable detections, as follows: Step 4 involves constructing fused optical infrared enhanced corner point pairs between each stable detection group using multiple fused corner points from the real-time fused optical infrared image during the stabilization period and multiple fused corner points from the real-time fused optical infrared image during the detection period.

2. The image fusion matching method for estimating transmission tower sway according to claim 1, characterized in that: Step 3 describes the corner extraction using the Harris corner detection method, as detailed below: The real-time preprocessed two-dimensional optical image of the transmission tower is used to extract corners using the Harris corner detection method, thereby obtaining the pixel positions and intensities of multiple corners of the real-time preprocessed two-dimensional optical image of the transmission tower. The real-time preprocessed infrared image of the transmission tower is used to extract corner points using the Harris corner detection method, thereby obtaining the pixel position and intensity of multiple corner points in the real-time preprocessed infrared image of the transmission tower.

3. The image fusion matching method for estimating transmission tower sway according to claim 2, characterized in that: Step 3, the corner strength threshold determination and filtering, is as follows: Multiple corner points with intensity greater than the corner point threshold are selected from multiple corner points in the real-time preprocessed two-dimensional optical image of the transmission tower to construct multiple enhanced corner points in the real-time preprocessed two-dimensional optical image of the transmission tower. Multiple corner points with intensity greater than a corner point threshold are selected from multiple corner points in the real-time preprocessed infrared image of the transmission tower to construct multiple enhanced corner points in the real-time preprocessed infrared image of the transmission tower.

4. The image fusion matching method for estimating transmission tower sway according to claim 3, characterized in that: The affine transformation matrix mentioned in step 5 is constructed as follows: An affine transformation matrix is ​​constructed by combining multiple sets of optical infrared enhanced corner point pairs using the RANSAC method.

5. A transmission tower sway estimation system using the image fusion matching method for estimating transmission tower sway according to any one of claims 1 to 4, characterized in that, include: The image acquisition module is used to acquire real-time two-dimensional optical images and real-time infrared images of the transmission tower. The image preprocessing module is used to obtain real-time preprocessed two-dimensional optical images and real-time preprocessed infrared images of the transmission tower by denoising and image enhancement processing, respectively. The enhanced corner extraction module is used to extract corners and filter them by using the Harris corner detection method, so as to obtain the pixel position and intensity of multiple corners of the preprocessed infrared image of the transmission tower in real time, as well as multiple enhanced corners of the preprocessed infrared image of the transmission tower in real time. The optical infrared enhanced corner point pair construction module is used to construct each optical infrared enhanced corner point pair by extracting neighboring pixels and comparing pixel mean square error thresholds; The fused corner point construction module is used to combine the real-time preprocessed two-dimensional optical image of the transmission tower and the real-time preprocessed infrared image of the transmission tower with the affine transformation matrix and perform image fusion through the remapping method to obtain the real-time fused optical-infrared image and multiple fused corner points of the real-time fused optical-infrared image. The stable detection corner point pair construction module is used to construct fused optical infrared enhanced corner point pairs between each stable detection by using multiple fused corner points corresponding to the real-time fused optical infrared image during the stabilization period, and the real-time fused optical infrared image and multiple fused corner points corresponding to the detection period. The swaying detection module is used to calculate the distance between the fused optical infrared enhanced corner point pairs between each group of stable detections using the L2 paradigm. If the distance exceeds the threshold, the power tower is determined to be swaying.

Images