Electric power iron tower state monitoring method, device and equipment and storage medium

Abstract

The invention discloses an electric power iron tower state monitoring method, device and equipment and a storage medium, and the method comprises the steps: collecting a current image of an electric power iron tower at a preset position through an unmanned plane, carrying out the preprocessing, carrying out the target detection based on a target detection model, and determining the current position information of each target; determining the current pixel offset of each target based on the current feature point information and the current position information corresponding to each target in the current image and the standard feature point information and the standard position information corresponding to each target in a predetermined standard image; the current pixel offset is converted, and the current actual offset of each target is determined; and determining the current state of the electric iron tower based on the current actual offset of each target. Through the technical scheme of the embodiment of the invention, automatic monitoring of the state of the electric power iron tower is realized, and the monitoring efficiency and accuracy are improved, so that problems are found in time, and the requirement of real-time monitoring of the electric power iron tower is met.

Description

Technical Field

[0001] The present invention relates to the field of electrical engineering technology, and in particular to a method, device, equipment and storage medium for monitoring the status of a power tower. Background Art

[0002] Power towers are important facilities for power supply, and their safety needs to be reliably guaranteed. However, towers are generally installed outdoors, widely distributed, and many are installed in remote areas, which are easily affected and damaged by nature and manpower. Therefore, it is necessary to monitor the status of power towers in real time to facilitate early warning of possible problems and protect the towers.

[0003] In the prior art, the monitoring of the status of power towers usually relies mainly on regular inspections and manual monitoring of the status of power towers. However, this manual monitoring method requires a lot of manpower costs, has low monitoring efficiency and accuracy, is not easy to detect problems in time, and cannot meet the needs of real-time monitoring of power towers. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for monitoring the status of an electric tower, so as to realize automatic monitoring of the status of an electric tower, improve monitoring efficiency and accuracy, thereby timely discovering problems and meeting the demand for real-time monitoring of electric towers.

[0005] In a first aspect, an embodiment of the present invention provides a method for monitoring the state of a power tower, comprising:

[0006] Using a drone to collect a current image of a power tower at a preset position, wherein multiple targets are deployed at multiple key locations of the power tower;

[0007] Preprocess the current image, and perform target detection on the preprocessed current image based on the target detection model to determine the current position information of each target in the current image;

[0008] Extract feature points based on the current position information of each target in the current image, and determine the current feature point information corresponding to each target in the current image;

[0009] Determine the current pixel offset of each target in the horizontal direction and the vertical direction respectively based on the current feature point information and the current position information corresponding to each target in the current image and the standard feature point information and the standard position information corresponding to each target in the predetermined standard image;

[0010] Converting the current pixel offset to determine the current actual offset of each target in the horizontal direction and the vertical direction, respectively, wherein the current actual offset in the horizontal direction is used to characterize the degree of inclination at the target position, and the current actual offset in the vertical direction is used to characterize the degree of settlement at the target position;

[0011] Based on the current actual offset of each target in the horizontal direction and the vertical direction, the current state of the power tower is determined.

[0012] In a second aspect, an embodiment of the present invention further provides a power tower state monitoring device, comprising:

[0013] An image acquisition module, used to acquire a current image of an electric power tower at a preset position by using a drone, wherein a plurality of targets are deployed at a plurality of key positions of the electric power tower;

[0014] A position information determination module is used to preprocess the current image, and perform target detection on the preprocessed current image based on the target detection model to determine the current position information of each target in the current image;

[0015] A feature information determination module is used to extract feature points based on the current position information of each target in the current image, and determine the current feature point information corresponding to each target in the current image;

[0016] A current offset determination module, used to determine the current pixel offset of each target in the horizontal direction and the vertical direction respectively based on the current feature point information and the current position information corresponding to each target in the current image and the standard feature point information and the standard position information corresponding to each target in the predetermined standard image;

[0017] an actual offset determination module, used to convert the current pixel offset to determine the current actual offset of each target in the horizontal direction and the vertical direction, wherein the current actual offset in the horizontal direction is used to characterize the degree of inclination at the target position, and the current actual offset in the vertical direction is used to characterize the degree of settlement at the target position;

[0018] The current state determination module is used to determine the current state of the power tower based on the current actual offset of each target in the horizontal direction and the vertical direction respectively.

[0019] In a third aspect, an embodiment of the present invention further provides an electronic device, characterized in that the electronic device comprises: at least one processor; and

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the power tower condition monitoring method provided in any embodiment of the present invention.

[0022] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the power tower status monitoring method provided by any embodiment of the present invention when executing.

[0023] The technical solution of the embodiment of the present invention is to collect the current image of the power tower at a preset position by a drone, preprocess the current image, and perform target detection on the preprocessed current image based on the target detection model, so as to determine the current position information of each target in the current image. Feature point extraction is performed based on the current position information of each target in the current image, and the current feature point information corresponding to each target in the current image is determined. Based on the current feature point information and current position information corresponding to each target in the current image and the standard feature point information and standard position information corresponding to each target in the predetermined standard image, the current pixel offset of each target in the horizontal direction and the vertical direction is determined. The current pixel offset is converted to determine the current actual offset of each target in the horizontal direction and the vertical direction, that is, to obtain the inclination and settlement degree at each target position, so that the inclination and settlement degree of multiple targets deployed at multiple key parts of the power tower can accurately determine the current state of the power tower, realize automatic monitoring of the state of the power tower, improve monitoring efficiency and accuracy, and then timely discover problems of the power tower, meeting the demand for real-time monitoring of the power tower.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of a method for monitoring the status of a power tower provided according to Embodiment 1 of the present invention;

[0027] Figure 2 is a flow chart of a method for monitoring the status of a power tower provided according to Embodiment 2 of the present invention;

[0028] Figure 3A This is an example diagram of a power tower status monitoring method according to the second embodiment of the present invention;

[0029] Figure 3B is a schematic structural diagram of a reference target and a monitoring target involved in Embodiment 2 of the present invention;

[0030] Figure 4 is a schematic structural diagram of a power tower status monitoring device provided according to Embodiment 3 of the present invention;

[0031] Figure 5 It is a structural schematic diagram of an electronic device for implementing the power tower status monitoring method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "target", "current", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment 1

[0035] Figure 1 A flowchart of a method for monitoring the status of a power tower is provided for the first embodiment of the present invention. This embodiment is applicable to monitoring the status of a power tower. Figure 1As shown, the method can be performed by a power tower status monitoring device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, the method specifically comprises the following steps:

[0036] S110. Collecting a current image of a power tower at a preset position by using a drone, wherein multiple targets are deployed at multiple key locations of the power tower.

[0037] Among them, the preset position may refer to the pre-set position or coordinates of the drone. The power tower may refer to a steel frame structure used to support power transmission lines, usually composed of steel pipes, steel plates and welded parts. The current image may refer to the image of the power tower collected by the drone at the current moment. The key parts may refer to the parts on the power tower that are easily affected and deformed. For example, the key parts may be the connection points, key support points, etc. of the power tower. The target may refer to a marker that is easily observed. For example, the target may be an object with a specific pattern. A target may be deployed on each key part so that the degree of inclination and settlement at the key part can be determined by using the target.

[0038] Specifically, a target can be pre-deployed at each key part of the power tower, and a drone suitable for power tower monitoring can be selected, equipped with high-performance navigation equipment and high-definition cameras to ensure that the drone can fly stably in complex outdoor environments and can accurately obtain visual information of the tower. The drone can be controlled in real time to perform high-definition image acquisition of the power tower where the target is deployed at the preset position, and the current image of the power tower containing the target can be obtained, thereby facilitating accurate positioning and identification of the status of the power tower based on the target.

[0039] S120, preprocessing the current image, and performing target detection on the preprocessed current image based on the target detection model to determine the current position information of each target in the current image.

[0040] The target detection model may refer to a computer vision model for monitoring and identifying a specific target or object in an image. For example, the specific target or object may be a target. The current position information may refer to a specific position of each target detected in the current image in the image. For example, the current position information of each target in the current image may be represented by the identification box position information.

[0041] Specifically, the current image collected by the drone is subjected to image denoising (such as median filtering, mean filtering, Gaussian filtering, etc.), image enhancement (such as histogram equalization, contrast stretching, sharpening, etc.) and image correction (such as camera calibration, perspective transformation, etc.) to complete the preprocessing of the current image, thereby improving image quality and reducing noise interference. The preprocessed current image is input into a pre-trained target detection model for target detection, and based on the output of the target detection model, the current position information of each target in the current image is obtained, so that the target position information can be automatically detected using the target detection model.

[0042] S130 , extracting feature points based on the current position information of each target in the current image, and determining current feature point information corresponding to each target in the current image.

[0043] The current feature point information may refer to specific information of the image feature points existing at each target position, such as the position and direction information of the existing extreme value points.

[0044] Specifically, after the current position information of each target is determined by the target detection model, the current position information is used as the feature point extraction position, the specific information of the target's feature points is extracted from the current image, and the current feature point information corresponding to each target in the current image is determined, so that the position and change of the target in the current image can be determined.

[0045] Exemplarily, S130 may include: based on the current position information of each target in the current image, cropping the current image to determine the current local image corresponding to each target; based on a scale-invariant feature transformation method, extracting feature points of the current local image corresponding to each target to obtain the current feature point information corresponding to each target in the current image.

[0046] The current local image may refer to a local area image where the target in the current image is located. The current local image corresponds to the position and size of the target in the original image. Scale-Invariant Feature Transform (SIFT) may refer to a feature extraction algorithm used in image processing and computer vision.

[0047] Specifically, based on the current position information of each target, the position and size of each target in the current image are determined, and the current image is cropped based on the position and size information corresponding to each target to obtain the current local image corresponding to each target. Based on the scale-invariant feature transformation method, the scale space corresponding to the current local image is constructed, and the current local image is Gaussian blurred at different scales to simulate the effect of human eyes observing objects at different scales. In the scale space, by comparing the response of each pixel point at different scales, local extreme points are found as candidate feature points. By fitting the Gaussian function, the position and scale of each candidate feature point are accurately located, and the erroneous candidate points caused by noise or image blur are eliminated, thereby obtaining the current feature point information corresponding to each target in the current image. By using the scale-invariant feature transformation method, the feature point information can be extracted more accurately, thereby further improving the accuracy of state monitoring.

[0048] S140, determining the current pixel offset of each target in the horizontal direction and the vertical direction respectively based on the current feature point information and the current position information corresponding to each target in the current image and the standard feature point information and the standard position information corresponding to each target in the predetermined standard image.

[0049] Among them, the standard image may refer to an image collected by a drone in advance at a preset position for an electric tower without tilt and subsidence, which is used as a benchmark for comparison and analysis with the current image, so as to accurately determine the degree of tilt and subsidence at different target positions in the current image. Standard feature point information may refer to the specific information of the image feature points existing at each target position in the standard image, such as the position and direction information of the existing extreme points. Standard position information may refer to the specific position of each target monitored in the standard image in the image. The current pixel offset may refer to the offset of the entire target in the current image relative to the entire target in the standard image in the pixel coordinate system.

[0050] Specifically, based on the current feature point information corresponding to each target in the current image and the standard feature point information corresponding to each target in the predetermined standard image, the feature points in the current image are matched with the feature points in the standard image to determine the corresponding relationship between the feature points corresponding to each target in the current image and the feature points corresponding to each target in the predetermined standard image. For each matched feature point, according to the current position information of each feature point in the current image and the standard position information in the standard image, the pixel offset of each target in the horizontal direction and the vertical direction is determined, so that dynamic monitoring and analysis of the target can be achieved.

[0051] Exemplarily, S140 may include: performing feature point matching and sub-pixel fitting on current feature point information corresponding to each target in the current image and standard feature point information corresponding to each target in a predetermined standard image to determine the correspondence between the target pixel points in the current image and the target pixel points in the standard image; based on the correspondence, the current pixel coordinates of each target pixel point in the current image and the standard pixel coordinates of each target pixel point in the standard image, determining the current pixel offset of the same target in the horizontal and vertical directions, respectively.

[0052] Among them, the target pixel point may refer to each pixel point in the local image where the target is located. The current pixel coordinates may refer to the coordinates of the target pixel point in the current image. The current position information corresponding to each target in the current image includes: the current pixel coordinates of each pixel point in the current local image where the target is located. The standard pixel coordinates may refer to the coordinates of the target pixel point in the standard image. The standard position information corresponding to each target in the standard image includes: the standard pixel coordinates of each pixel point in the standard local image where the target is located. The correspondence between the target pixel point in the current image and the target pixel point in the standard image may refer to the target pixel point in the current image and the corresponding target pixel point in the standard image being the pixel point at the same position of the same target.

[0053] Specifically, feature point matching and sub-pixel fitting are performed on the current feature point information corresponding to each target in the current image and the standard feature point information corresponding to each target in the predetermined standard image, and the corresponding relationship between each target pixel in the current image and each target pixel in the standard image is determined based on the results of the feature point matching and sub-pixel fitting. For each target in the current image, based on the corresponding relationship, the current pixel coordinates of each target pixel in the target in the current image and the standard pixel coordinates of each target pixel in the target in the standard image, the standard pixel horizontal and vertical coordinates of the corresponding target pixel in the standard image are respectively subtracted from the current pixel horizontal and vertical coordinates of each target pixel in the current image to obtain the current offset of each target pixel corresponding to the target in the current image in the horizontal direction and the vertical direction, and all the current offsets corresponding to the target can be averaged, for example, the largest current offset and the smallest current offset can be removed and the remaining current offsets can be averaged to obtain the average deviations in the horizontal direction and the vertical direction, and the average offset is determined as the current pixel offset of the target in the horizontal direction and the vertical direction. Similarly, the current pixel offset of each target in the horizontal and vertical directions can be determined, thereby realizing dynamic detection and analysis of the target posture.

[0054] S150, converting the current pixel offset to determine the current actual offset of each target in the horizontal direction and the vertical direction, respectively, wherein the current actual offset in the horizontal direction is used to characterize the degree of inclination at the target position, and the current actual offset in the vertical direction is used to characterize the degree of settlement at the target position.

[0055] The current actual offset may refer to the actual offset distance of the target in the horizontal and vertical directions at the current moment in the spatial coordinate system. The settlement degree may refer to the displacement or sinking degree of the target in the vertical direction.

[0056] Specifically, the current pixel offset of each target in the horizontal direction is converted into the current actual offset in the horizontal direction, and the current pixel offset of each target in the vertical direction is converted into the current actual offset in the vertical direction, so as to more accurately monitor the posture changes of the targets on the power tower.

[0057] S160: Determine the current state of the power tower based on the current actual offset of each target in the horizontal direction and the vertical direction.

[0058] The current state may refer to a dangerous state or a safe state. A dangerous state may refer to a state where the power tower is tilted and sunk to a certain extent and is more prone to collapse. A safe state may refer to a state where the power tower is not prone to collapse.

[0059] Specifically, the current actual offset of each target in the horizontal direction and the vertical direction is compared with the offset threshold of each target in the horizontal direction and the vertical direction when the power tower is in a safe state, and the current state of the power tower is determined based on the comparison result. When the current state is determined to be a dangerous state, it is necessary to generate early warning information corresponding to the power tower, and report the early warning information to the monitoring platform for timely processing, so as to realize real-time monitoring of the state of the power tower, timely discover problems of the power tower, and ensure the safety of the power tower.

[0060] Exemplarily, S160 may include: comparing the current actual offset of each target in the horizontal direction with the corresponding horizontal offset threshold, and comparing the current actual offset of each target in the vertical direction with the corresponding vertical offset threshold; if there is at least one target whose current actual offset in the horizontal direction is greater than or equal to the corresponding horizontal offset threshold, or whose current actual offset in the vertical direction is greater than or equal to the corresponding vertical offset threshold, then determining that the current state of the power tower is a dangerous state; if the current actual offsets of all targets in the horizontal direction are less than the corresponding horizontal offset threshold, and the current actual offsets in the vertical direction are less than the corresponding vertical offset threshold, then determining that the current state of the power tower is a safe state.

[0061] The horizontal offset threshold may refer to the preset minimum offset of the power tower in the horizontal direction when the power tower is in a dangerous state. The vertical offset threshold may refer to the preset minimum offset of the power tower in the vertical direction when the power tower is in a dangerous state. It should be noted that the horizontal offset threshold and the vertical offset threshold corresponding to different targets are different.

[0062] Specifically, the current actual offset of each target in the horizontal direction is compared with the corresponding horizontal offset threshold, and the current actual offset of each target in the vertical direction is compared with the corresponding vertical offset threshold. If there is at least one target whose current actual offset in the horizontal direction is greater than or equal to the corresponding horizontal offset threshold, the current state of the power tower is determined to be a dangerous state; if there is at least one target whose current actual offset in the vertical direction is greater than or equal to the corresponding vertical offset threshold, the current state of the power tower is determined to be a dangerous state; if the current actual offsets of all targets in the horizontal direction are less than the corresponding horizontal offset threshold, and the current actual offsets in the vertical direction are less than the corresponding vertical offset threshold, the current state of the power tower is determined to be a safe state, thereby realizing real-time monitoring of the state of the power tower, timely discovering problems with the power tower, and ensuring the safety of the power tower. It should be noted that in a dangerous state, it is necessary to generate warning information corresponding to the power tower, and report the warning information to the monitoring platform for timely processing.

[0063] The technical solution of the embodiment of the present invention is to collect the current image of the power tower at a preset position by a drone, preprocess the current image, and perform target detection on the preprocessed current image based on the target detection model, so as to determine the current position information of each target in the current image. Feature point extraction is performed based on the current position information of each target in the current image, and the current feature point information corresponding to each target in the current image is determined. Based on the current feature point information and current position information corresponding to each target in the current image and the standard feature point information and standard position information corresponding to each target in the predetermined standard image, the current pixel offset of each target in the horizontal direction and the vertical direction is determined. The current pixel offset is converted to determine the current actual offset of each target in the horizontal direction and the vertical direction, that is, to obtain the inclination and settlement degree at each target position, so that the inclination and settlement degree of multiple targets deployed at multiple key parts of the power tower can accurately determine the current state of the power tower, realize automatic monitoring of the state of the power tower, improve monitoring efficiency and accuracy, and then timely discover problems of the power tower, meeting the demand for real-time monitoring of the power tower.

[0064] Embodiment 2

[0065] Figure 2 This is a flow chart of a method for monitoring the state of a power tower provided in the second embodiment of the present invention. Based on the above embodiments, this embodiment optimizes the step of "converting the current pixel offset to determine the current actual offset of each target in the horizontal direction and the vertical direction". The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0066] See also Figure 2 Another power tower status monitoring method provided in this embodiment specifically includes the following steps:

[0067] S210: Collect the current image of the power tower at a preset position by using a drone.

[0068] S220, preprocessing the current image, and performing target detection on the preprocessed current image based on the target detection model to determine the current position information of each target in the current image.

[0069] S230 , extracting feature points based on the current position information of each target in the current image, and determining current feature point information corresponding to each target in the current image.

[0070] S240, based on the current feature point information and current position information corresponding to each target in the current image and the standard feature point information and standard position information corresponding to each target in the predetermined standard image, determine the current pixel offset of each target in the horizontal direction and the vertical direction respectively.

[0071] S250, performing temperature compensation calibration on the current pixel offset of each target in the horizontal direction and the vertical direction, and performing vibration interference calibration on the current pixel offset after the temperature compensation calibration.

[0072] Specifically, since the temperature change of the image sensor in the camera of the drone for collecting images will cause the error of the pixel offset, it is necessary to correct the pixel offset according to the temperature change of the image sensor to improve the measurement accuracy. The influence of temperature change on the current pixel offset of each target can be determined in advance, so as to perform temperature compensation calibration on the current pixel offset of each target in the horizontal and vertical directions, thereby eliminating the image drift caused by temperature. Since the drone may be affected by mechanical vibration, wind or other external interference when collecting images, vibration interference is generated, resulting in inaccurate pixel offset. Therefore, it is necessary to determine the influence of vibration interference on the current pixel offset of each target, and perform vibration interference calibration on the current pixel offset after temperature compensation calibration, so as to eliminate the influence of vibration interference on the current pixel offset.

[0073] Exemplarily, "performing temperature compensation calibration for the current pixel offset of each target in the horizontal direction and the vertical direction respectively" in S250 may include: obtaining the current image sensor temperature when acquiring the current image; determining the current pixel drift corresponding to the current image sensor temperature based on the current image sensor temperature and the pixel drift at each image sensor temperature determined in advance by simulation; and performing temperature compensation calibration for the current pixel offset of each target in the horizontal direction and the vertical direction respectively based on the current pixel drift.

[0074] The current image sensor temperature may refer to the temperature of the image sensor when the drone is collecting the current image. The image sensor may refer to a device that converts an optical image into an electronic signal. The current pixel drift may refer to the offset of each target in the image relative to the original position at the current image sensor temperature.

[0075] Specifically, the current image sensor temperature during the current image acquisition is obtained in real time through the temperature sensor. In a laboratory or simulation environment, the image acquisition process under different ambient temperatures is simulated, the image sensor temperature during image acquisition under different ambient temperatures is determined, and the pixel offset at each image sensor temperature is recorded. According to the current image sensor temperature and the pixel drift amount at each image sensor temperature determined in advance by simulation, the current pixel drift amount corresponding to the current image sensor temperature is determined. Based on the determined current pixel drift amount, the current pixel offset amount of each target in the horizontal direction and the vertical direction is subtracted from the current pixel drift amount, and the temperature compensation calibration of each target in the horizontal direction and the vertical direction is completed, thereby eliminating the image drift caused by temperature.

[0076] Exemplarily, "performing vibration interference calibration on the current pixel offset after temperature compensation calibration" in S250 may include: based on the current pixel offset of the reference target after temperature compensation calibration in the horizontal direction and the vertical direction, respectively, performing vibration interference calibration on the current pixel offset of each monitoring target after temperature compensation calibration in the horizontal direction and the vertical direction.

[0077] Wherein, a reference target is deployed on the ground around the power tower. The reference target may refer to a target deployed on the ground around the power tower and will not deform or sink. The reference target is within the image acquisition range of the power tower. The monitoring target may refer to a target deployed on a key part of the power tower.

[0078] Specifically, Figure 3AAs shown in the figure, the black origin is the deployment position of the target. The reference target is deployed on the ground around the power tower, and the monitoring target is deployed on the key part of the power tower. Both the reference target and the monitoring target are within the image acquisition range of the drone. The specific structures of the targets corresponding to the reference target position and the monitoring target position are shown in Figure 1. Figure 3B As shown. Based on the reference target after temperature compensation calibration, the current pixel offsets of the reference target after temperature compensation calibration in the horizontal and vertical directions are determined respectively, and the current pixel offsets of the reference target in the horizontal and vertical directions are determined as the pixel offsets of each monitoring target caused by vibration interference in the horizontal and vertical directions respectively. Based on the current pixel offsets of the reference target in the horizontal and vertical directions after temperature compensation calibration, the current pixel offsets of each monitoring target in the horizontal and vertical directions after temperature compensation calibration are subtracted from the pixel offsets of each reference target caused by vibration interference in the horizontal and vertical directions respectively, to complete the vibration interference calibration, thereby eliminating the influence of vibration interference on the current pixel offset. Exemplarily, as Figure 3B As shown, dx is the horizontal displacement of each monitoring target in the horizontal direction after the vibration interference calibration, that is, the current pixel offset of each monitoring target in the horizontal direction after the vibration interference calibration; dy is the vertical displacement of each monitoring target in the vertical direction after the vibration interference calibration, that is, the current pixel offset of each monitoring target in the vertical direction after the vibration interference calibration.

[0079] S260: Convert the current pixel offset after vibration interference calibration to determine the current actual offset of each target in the horizontal direction and the vertical direction.

[0080] Specifically, the current pixel offset in the horizontal direction of each monitoring target after vibration interference calibration is converted into the current actual offset in the horizontal direction, and the current pixel offset in the vertical direction of each monitoring target after vibration interference calibration is converted into the current actual offset in the vertical direction, so as to better understand and analyze the position and posture changes of the target in the image.

[0081] S270: Determine the current state of the power tower based on the current actual offset of each target in the horizontal direction and the vertical direction.

[0082] The technical solution of the embodiment of the present invention can eliminate the image drift caused by temperature by performing temperature compensation calibration on the current pixel offset of each target in the horizontal direction and the vertical direction, and can eliminate the interference of vibration on the pixel by performing vibration interference calibration on the current pixel offset after the temperature compensation calibration. The current pixel offset after the vibration interference calibration is converted to determine the current actual offset of each target in the horizontal direction and the vertical direction, so as to further improve the accuracy of the current actual offset by calibrating the current pixel offset, thereby improving the accuracy of the power tower status monitoring.

[0083] Embodiment 3

[0084] Figure 4 This is a schematic diagram of the structure of a power tower status monitoring device provided by the third embodiment of the present invention. Figure 4 As shown, the device includes: an image acquisition module 310, a position information determination module 320, a feature information determination module 330, a current offset determination module 340, an actual offset determination module 350 and a current state determination module 360.

[0085] The image acquisition module 310 is used to acquire a current image of the power tower at a preset position by using a drone, wherein multiple targets are deployed at multiple key positions of the power tower;

[0086] The position information determination module 320 is used to preprocess the current image, and perform target detection on the preprocessed current image based on the target detection model to determine the current position information of each target in the current image;

[0087] A feature information determination module 330 is used to extract feature points based on the current position information of each target in the current image, and determine the current feature point information corresponding to each target in the current image;

[0088] A current offset determination module 340 is used to determine the current pixel offset of each target in the horizontal direction and the vertical direction respectively based on the current feature point information and the current position information corresponding to each target in the current image and the standard feature point information and the standard position information corresponding to each target in the predetermined standard image;

[0089] An actual offset determination module 350 is used to convert the current pixel offset to determine the current actual offset of each target in the horizontal direction and the vertical direction, wherein the current actual offset in the horizontal direction is used to characterize the degree of inclination at the target position, and the current actual offset in the vertical direction is used to characterize the degree of settlement at the target position;

[0090] The current state determination module 360 ​​is used to determine the current state of the power tower based on the current actual offset of each target in the horizontal direction and the vertical direction respectively.

[0091] The technical solution of this embodiment is to collect the current image of the power tower at a preset position by a drone, preprocess the current image, and perform target detection on the preprocessed current image based on the target detection model, so as to determine the current position information of each target in the current image. Feature point extraction is performed based on the current position information of each target in the current image, and the current feature point information corresponding to each target in the current image is determined. Based on the current feature point information and current position information corresponding to each target in the current image and the standard feature point information and standard position information corresponding to each target in the predetermined standard image, the current pixel offset of each target in the horizontal direction and the vertical direction can be determined, which is helpful to further analyze the motion state of the target. The current pixel offset is converted to determine the current actual offset of each target in the horizontal direction and the vertical direction. Based on the current actual offset of each target in the horizontal direction and the vertical direction, the current state of the power tower is determined, and the automatic monitoring of the state of the power tower is realized, the monitoring efficiency and accuracy are improved, and the problems of the power tower can be discovered in time, meeting the demand for real-time monitoring of the power tower.

[0092] Optionally, the feature information determination module 330 is specifically used to: based on the current position information of each target in the current image, crop the current image to determine the current local image corresponding to each target; based on a scale-invariant feature transformation method, extract feature points of the current local image corresponding to each target to obtain the current feature point information corresponding to each target in the current image.

[0093] Optionally, the current offset determination module 340 is specifically used to: perform feature point matching and sub-pixel fitting on the current feature point information corresponding to each target in the current image and the standard feature point information corresponding to each target in a predetermined standard image, to determine the correspondence between the target pixel points in the current image and the target pixel points in the standard image; based on the correspondence, the current pixel coordinates of each target pixel point in the current image and the standard pixel coordinates of each target pixel point in the standard image, determine the current pixel offsets of the same target in the horizontal and vertical directions, respectively.

[0094] Optionally, the actual offset determination module 350 includes:

[0095] A calibration unit, used to perform temperature compensation calibration on the current pixel offset of each target in the horizontal direction and the vertical direction, and to perform vibration interference calibration on the current pixel offset after the temperature compensation calibration;

[0096] The actual offset determination unit is used to convert the current pixel offset after vibration interference calibration to determine the current actual offset of each target in the horizontal and vertical directions, wherein the current actual offset in the horizontal direction is used to characterize the degree of inclination at the target position, and the current actual offset in the vertical direction is used to characterize the degree of settlement at the target position.

[0097] Optionally, the calibration unit is specifically used to: obtain the current image sensor temperature when acquiring the current image; determine the current pixel drift amount corresponding to the current image sensor temperature based on the current image sensor temperature and the pixel drift amount at each image sensor temperature determined in advance by simulation; and perform temperature compensation calibration processing on the current pixel offset of each target in the horizontal direction and the vertical direction respectively based on the current pixel drift amount.

[0098] Optionally, a reference target is deployed on the ground around the power tower, and the reference target is within the image acquisition range of the power tower; the target deployed at the key position of the power tower is a monitoring target; the calibration unit is specifically used to: based on the current pixel offset of the reference target in the horizontal direction and the vertical direction after the temperature compensation calibration, perform vibration interference calibration on the current pixel offset of each monitoring target in the horizontal direction and the vertical direction after the temperature compensation calibration.

[0099] Optionally, the current state determination module 360 ​​is specifically used to: compare the current actual offset of each target in the horizontal direction with the corresponding horizontal offset threshold, and compare the current actual offset of each target in the vertical direction with the corresponding vertical offset threshold; if there is at least one target whose current actual offset in the horizontal direction is greater than or equal to the corresponding horizontal offset threshold, or whose current actual offset in the vertical direction is greater than or equal to the corresponding vertical offset threshold, then determine that the current state of the power tower is a dangerous state; if the current actual offsets of all targets in the horizontal direction are less than the corresponding horizontal offset threshold, and the current actual offsets in the vertical direction are less than the corresponding vertical offset threshold, then determine that the current state of the power tower is a safe state.

[0100] The power tower state monitoring device provided in the embodiment of the present invention can execute the power tower state monitoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0101] Figure 5A schematic diagram of the structure of an electronic device 12 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as desktop computers, workstations, servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0102] like Figure 5 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0103] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0104] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0105] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0106] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0107] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0108] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing a method for monitoring the state of a power tower provided in the embodiment of the present invention, the method comprising:

[0109] Using a drone to collect a current image of a power tower at a preset position, wherein multiple targets are deployed at multiple key locations of the power tower;

[0110] Preprocess the current image, and perform target detection on the preprocessed current image based on the target detection model to determine the current position information of each target in the current image;

[0111] Extract feature points based on the current position information of each target in the current image, and determine the current feature point information corresponding to each target in the current image;

[0112] Determine the current pixel offset of each target in the horizontal direction and the vertical direction respectively based on the current feature point information and the current position information corresponding to each target in the current image and the standard feature point information and the standard position information corresponding to each target in the predetermined standard image;

[0113] Converting the current pixel offset to determine the current actual offset of each target in the horizontal direction and the vertical direction, respectively, wherein the current actual offset in the horizontal direction is used to characterize the degree of inclination at the target position, and the current actual offset in the vertical direction is used to characterize the degree of settlement at the target position;

[0114] Based on the current actual offset of each target in the horizontal direction and the vertical direction, the current state of the power tower is determined.

[0115] Of course, those skilled in the art can understand that the processor can also implement the technical solution of the power tower status monitoring method provided by any embodiment of the present invention.

[0116] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the power tower state monitoring method provided in any embodiment of the present invention are implemented. The method includes:

[0117] Using a drone to collect a current image of a power tower at a preset position, wherein multiple targets are deployed at multiple key locations of the power tower;

[0118] Preprocess the current image, and perform target detection on the preprocessed current image based on the target detection model to determine the current position information of each target in the current image;

[0119] Extract feature points based on the current position information of each target in the current image, and determine the current feature point information corresponding to each target in the current image;

[0120] Determine the current pixel offset of each target in the horizontal direction and the vertical direction respectively based on the current feature point information and the current position information corresponding to each target in the current image and the standard feature point information and the standard position information corresponding to each target in the predetermined standard image;

[0121] Converting the current pixel offset to determine the current actual offset of each target in the horizontal direction and the vertical direction, respectively, wherein the current actual offset in the horizontal direction is used to characterize the degree of inclination at the target position, and the current actual offset in the vertical direction is used to characterize the degree of settlement at the target position;

[0122] Based on the current actual offset of each target in the horizontal direction and the vertical direction, the current state of the power tower is determined.

[0123] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0124] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0125] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0126] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0127] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0128] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for monitoring the state of a power tower, characterized in that: include: Using a drone to collect a current image of a power tower at a preset position, wherein multiple targets are deployed at multiple key locations of the power tower; Preprocess the current image, and perform target detection on the preprocessed current image based on the target detection model to determine the current position information of each target in the current image; Extract feature points based on the current position information of each target in the current image, and determine the current feature point information corresponding to each target in the current image; Determine the current pixel offset of each target in the horizontal direction and the vertical direction respectively based on the current feature point information and the current position information corresponding to each target in the current image and the standard feature point information and the standard position information corresponding to each target in the predetermined standard image; Converting the current pixel offset to determine the current actual offset of each target in the horizontal direction and the vertical direction, respectively, wherein the current actual offset in the horizontal direction is used to characterize the degree of inclination at the target position, and the current actual offset in the vertical direction is used to characterize the degree of settlement at the target position; Based on the current actual offset of each target in the horizontal direction and the vertical direction, the current state of the power tower is determined.

2. The method according to claim 1, characterized in that Extracting feature points based on current position information of each target in the current image to determine current feature point information corresponding to each target in the current image includes: Based on the current position information of each target in the current image, cropping the current image to determine a current local image corresponding to each target; Based on the scale-invariant feature transformation method, feature points are extracted from the current local image corresponding to each target to obtain current feature point information corresponding to each target in the current image.

3. The method according to claim 1, characterized in that Based on the current feature point information and current position information corresponding to each target in the current image and the standard feature point information and standard position information corresponding to each target in the predetermined standard image, determining the current pixel offset of each target in the horizontal direction and the vertical direction, respectively, including: Performing feature point matching and sub-pixel fitting on current feature point information corresponding to each target in the current image and standard feature point information corresponding to each target in a predetermined standard image to determine a corresponding relationship between target pixel points in the current image and target pixel points in the standard image; Based on the corresponding relationship, the current pixel coordinates of each target pixel in the current image and the standard pixel coordinates of each target pixel in the standard image, the current pixel offsets of the same target in the horizontal direction and the vertical direction are determined.

4. The method according to claim 1, characterized in that: Converting the current pixel offset to determine the current actual offset of each target in the horizontal direction and the vertical direction, respectively, includes: Performing temperature compensation calibration on the current pixel offset of each target in the horizontal direction and the vertical direction, and performing vibration interference calibration on the current pixel offset after the temperature compensation calibration; The current pixel offset after vibration disturbance calibration is converted to determine the current actual offset of each target in the horizontal direction and the vertical direction respectively.

5. The method according to claim 4, characterized in that Perform temperature compensation calibration on the current pixel offset of each target in the horizontal and vertical directions, including: Get the current image sensor temperature when acquiring the current image; Determine a current pixel drift amount corresponding to the current image sensor temperature based on the current image sensor temperature and the pixel drift amount at each image sensor temperature determined in advance by simulation; Based on the current pixel drift amount, temperature compensation calibration processing is performed on the current pixel offset of each target in the horizontal direction and the vertical direction respectively.

6. The method according to claim 4, characterized in that A reference target is deployed on the ground around the power tower, and the reference target is within the image acquisition range of the power tower; The targets deployed on the key parts of the power tower are monitoring targets; Perform vibration interference calibration on the current pixel offset after temperature compensation calibration, including: Based on the current pixel offsets of the reference target after the temperature compensation calibration in the horizontal direction and the vertical direction respectively, the current pixel offsets of each monitoring target after the temperature compensation calibration in the horizontal direction and the vertical direction are calibrated for vibration interference.

7. The method according to claim 1, characterized in that Determining the current state of the power tower based on the current actual offset of each target in the horizontal direction and the vertical direction, including: Comparing the current actual offset of each target in the horizontal direction with the corresponding horizontal offset threshold, and comparing the current actual offset of each target in the vertical direction with the corresponding vertical offset threshold; If there is at least one target whose current actual offset in the horizontal direction is greater than or equal to the corresponding horizontal offset threshold, or whose current actual offset in the vertical direction is greater than or equal to the corresponding vertical offset threshold, then the current state of the power tower is determined to be a dangerous state; If the current actual offsets of all targets in the horizontal direction are smaller than the corresponding horizontal offset threshold, and the current actual offsets in the vertical direction are smaller than the corresponding vertical offset threshold, it is determined that the current state of the power tower is a safe state.

8. A power tower status monitoring device, characterized in that: include: An image acquisition module, used to acquire a current image of an electric power tower at a preset position by using a drone, wherein a plurality of targets are deployed at a plurality of key positions of the electric power tower; A position information determination module is used to preprocess the current image, and perform target detection on the preprocessed current image based on the target detection model to determine the current position information of each target in the current image; A feature information determination module is used to extract feature points based on the current position information of each target in the current image, and determine the current feature point information corresponding to each target in the current image; A current offset determination module, used to determine the current pixel offset of each target in the horizontal direction and the vertical direction respectively based on the current feature point information and the current position information corresponding to each target in the current image and the standard feature point information and the standard position information corresponding to each target in the predetermined standard image; an actual offset determination module, used to convert the current pixel offset to determine the current actual offset of each target in the horizontal direction and the vertical direction, wherein the current actual offset in the horizontal direction is used to characterize the degree of inclination at the target position, and the current actual offset in the vertical direction is used to characterize the degree of settlement at the target position; The current state determination module is used to determine the current state of the power tower based on the current actual offset of each target in the horizontal direction and the vertical direction respectively.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the power tower condition monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the power tower state monitoring method according to any one of claims 1 to 7 when executed.

Images