Electric power tower defect three-dimensional space positioning method and equipment based on de-weighting technology

The method addresses inefficiencies in drone inspection by grouping and deduplicating power tower defect images, enhancing spatial positioning accuracy and efficiency using three-dimensional laser point cloud data.

CN119090803AInactive Publication Date: 2024-12-06STATE GRID SHANGHAI MUNICIPAL ELECTRIC POWER CO
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411011803.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Current drone inspection technologies for power tower defects fail to deduplicate image defects found at different angles and lack efficient three-dimensional spatial positioning, leading to manual review inefficiencies.

Method used

A method for three-dimensional spatial positioning of power tower defects involves grouping defect images by type, establishing relationships between image coordinates and real-world coordinates, calculating the shortest distance between images, and using three-dimensional laser point cloud data to deduplicate defects, thereby improving positioning efficiency.

Benefits of technology

The method accurately deduplicates defects and enhances positioning efficiency by reducing data requirements and manual workload through automated defect identification and spatial positioning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090803A_ABST
    Figure CN119090803A_ABST
Patent Text Reader

Abstract

The invention relates to an electric power tower defect three-dimensional space positioning method and device based on a de-duplication technology. The method comprises the following steps: obtaining an unmanned aerial vehicle patrol tower image shot according to an unmanned aerial vehicle route; defect labeling is carried out on the unmanned aerial vehicle patrol tower image, wherein defect types and defect coordinate information are labeled; the marked unmanned aerial vehicle patrol tower image defects are divided according to defect types, and the same defect is divided into one group; and for each group of defect images, establishing a relationship between image defect coordinates and real coordinates of a world system, calculating a nearest distance between two images in the same group of defects, performing defect image de-weighting by combining the three-dimensional laser point cloud data of the electric power tower, and performing three-dimensional space positioning based on the defect images after de-weighting. Compared with the prior art, the method has the advantages that the image defects are subjected to duplicate removal, and three-dimensional space positioning of the image defects of the electric power tower is rapidly achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of power pole tower defect positioning, and in particular to a three-dimensional space positioning method and device for power pole tower defects based on de-duplication technology. Background Art

[0002] Currently, drone inspections have been widely promoted in the power industry, and drone inspections of transmission and distribution line towers have been applied by power companies in various provinces. Companies also use artificial intelligence algorithms to identify defects in drone inspection images of towers, but there is currently no relevant technology that can remove duplicates of image defects found at different angles and perform three-dimensional spatial positioning, resulting in a large amount of manual review work, resulting in low positioning efficiency, and at the same time, it cannot really reduce the workload of front-line personnel.

[0003] After searching, Chinese invention patent publication number CN112767391B discloses a method for locating defects in power line components by integrating three-dimensional point clouds and two-dimensional images, including step S1: calibrating camera parameters of the camera carried by the drone so that the pixel coordinate system of the camera can be converted to and from the world coordinate system; step S2: extracting shooting point information corresponding to the image from the visible light image F obtained by the drone for aerial inspection of the power line, and using the information to obtain three-dimensional point cloud data of the inspection target in the visible light image F; step S3: using a deep learning model to intelligently identify defective components in the inspection target and obtain the center coordinates of the defective components; step S4: performing a projection transformation on the three-dimensional point cloud data to process its three-dimensional image information and obtain the coordinates with the camera as the origin. The coordinates of the three-dimensional point cloud data in the calibration system are transformed from the world coordinate system to the image coordinate system according to the camera calibration parameters to obtain the two-dimensional image F'; step S5: perform fusion processing. In order to ensure the accurate matching of the defective parts in the inspection target, feature matching and position matching are used for confirmation; first, feature matching is performed on the visible light image F and the two-dimensional image F' as a whole to obtain the position offset of the overall target in the two images; then the center coordinates and offset of the defective parts identified in the image F are calculated to obtain the coordinate position of the defective parts in the image F'; step S6: perform inverse coordinate transformation; transform the coordinate position of the defective parts in the image F' to the position in the three-dimensional world coordinate system, establish the corresponding relationship between the component state and the actual position and record the results. This existing patent has the problem of not deduplicating image defects and slow positioning speed.

[0004] How to quickly achieve three-dimensional spatial positioning of power tower defects based on de-duplication technology has become a technical problem that needs to be solved. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a three-dimensional spatial positioning method and device for power tower defects based on de-duplication technology.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] According to one aspect of the present invention, a method for three-dimensional spatial positioning of power tower defects based on deduplication technology is provided, the method comprising the following steps:

[0008] Step S1, obtaining an image of a tower patrolled by a drone taken along the drone route;

[0009] Step S2, marking defects on the image of the tower inspected by the drone in step S1, including marking defect types and defect coordinate information;

[0010] Step S3, classifying the defects of the tower inspection image of the drone marked in step S2 according to the defect type, and classifying the same type of defects into one group;

[0011] Step S4, for each group of defect images, establish the relationship between the image defect coordinates and the real coordinates of the world system, calculate the shortest distance between two images in the same group of defects, deduplicate the defect images in combination with the three-dimensional laser point cloud data of the power tower, and perform three-dimensional spatial positioning based on the deduplicated defect images.

[0012] Preferably, the step S4 comprises:

[0013] Step S101, randomly select two defect images of the same group, and obtain the relationship between the image defect coordinates and the real coordinates of the world system;

[0014] Step S102, according to the relationship constraint between the image defect coordinates and the real coordinates of the world system, the shortest distance between the real coordinates of the world system corresponding to the two defect images is obtained;

[0015] Step S103, judging whether the closest distance is less than or equal to a set threshold value, if yes, then the next step is to judge whether the two world system real coordinates are within the range of the tower laser point cloud three-dimensional model when the closest distance is reached, if they are within the range of the tower laser point cloud three-dimensional model, then proceed to step S104; if the closest distance is greater than the threshold value or the world system real coordinates when the closest distance is reached are outside the range of the tower laser point cloud three-dimensional model, then proceed to step S105;

[0016] Step S104, determining that the two image defects are the same position defect, and calculating the world system real coordinates of the defect;

[0017] Step S105 , determining that the two image defects are defects at different positions, and calculating the world system real coordinates of the defects.

[0018] More preferably, the relationship between the image defect coordinates and the real coordinates of the world system is specifically:

[0019] Take any two defect images of the same group, the image coordinates of the center point of the defect are [u,v] and [u ’ ,v']; the corresponding real world coordinates are set to [X w ,Y w ,Z w ] and [X ’ w ,Y ’ w ,Z ’ w ]; According to the coordinate information of the UAV image defect and the camera imaging model, the relationship between the image coordinates and the real coordinates of the world system is obtained as follows:

[0020]

[0021] in, is the camera internal parameter, fx and fy are the pixel ratios, u 0 、v 0 are the pixel coordinates of the center of the phase plane; and are the external parameters of the camera taken at two different positions, R and R′ are the rotation matrices between the world coordinates and the camera coordinates, T and T′ are the offsets between the world coordinate origin and the camera coordinate origin, Zc and Z ’ c are the Z-direction coordinate values ​​of two different real points in the camera coordinate system.

[0022] More preferably, the shortest distance between the real coordinates of the world system corresponding to the two defect images is specifically:

[0023] The formula for calculating the closest distance between the real-world coordinates of two defect images is:

[0024]

[0025] Among them, [Xw, Yw, Zw] and [X ’ w ,Y ’ w ,Z ’ w ] are the real coordinates of the world system corresponding to the closest distance between the two defect images.

[0026] More preferably, the real coordinates of the world system corresponding to when the distance between the two defect images is the shortest are obtained by solving the relationship between the image coordinates and the real coordinates of the world system.

[0027] More preferably, if the real coordinates of the world system are within the convex hull of the tower laser 3D point cloud, or the real coordinates of the world system are inside any triangle of the triangular mesh, then it is determined that the real coordinates of the world system are within the range of the tower laser point cloud 3D model.

[0028] More preferably, in step S104, the real coordinates of the defect in the world system are taken as the average value of the real coordinates of the two world systems.

[0029] More preferably, in step S105, the world system real coordinates of each image defect are the intersection values ​​of two world system real coordinates satisfying the relationship between the image coordinates and the world system real coordinates and the convex hull formed by the three-dimensional laser point cloud of the tower.

[0030] More preferably, the defect marking is performed by manual marking or artificial intelligence algorithm to identify defects.

[0031] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1) The present invention groups the drone inspection images of the power transmission and distribution line towers according to defect categories, establishes the relationship between the image defect coordinates and the world system real coordinates for each group of defect images, calculates the shortest distance between two images in the same group of defects, and combines the three-dimensional laser point cloud data of the power towers to perform defect deduplication, thereby reducing the amount of data required for spatial positioning, and then performs three-dimensional spatial positioning to accurately obtain the real point coordinates of each group of defect images, thereby improving the efficiency of three-dimensional spatial positioning of drone inspection of power tower image defects and solving the problem of manual load of existing manual review.

[0034] 2) The present invention utilizes the relationship constraints between the image defect coordinates and the world system real coordinates to obtain the closest distance between the world system real coordinates corresponding to the two defect images, and based on different situations of the closest distance and threshold comparison, combined with the positions of the two world system real coordinates in the laser point cloud three-dimensional model, determines whether the defects are at the same position, deduplicates the defects at the same position, and then calculates the world system real coordinates of the defect based on the deduplicated data. After deduplication, the positioning efficiency is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the overall process of the three-dimensional spatial positioning method of power tower defects based on de-duplication technology in the present invention;

[0036] Figure 2 It is a schematic diagram of the flow of the three-dimensional spatial positioning method for each group of defect images in the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] Example 1

[0039] This embodiment relates to a three-dimensional spatial positioning method for power tower defects based on de-duplication technology, such as Figure 1 , the method comprising:

[0040] Step S1, obtaining the route information of the tower drone inspection, including the latitude and longitude coordinates and altitude of the drone, and the deflection angle of the drone gimbal;

[0041] Step S2, obtaining drone patrol photos taken along the drone route;

[0042] Step S3, identify defects in the drone patrol photos through manual annotation or artificial intelligence algorithm, and mark the specific type and coordinate information of the defects; the artificial intelligence algorithm for defect identification belongs to the existing technology and will not be expanded here.

[0043] Step S4, obtaining three-dimensional laser point cloud data of the power tower;

[0044] Step S5, classify the marked defects in the drone inspection tower image according to the defect type, classify the same type of defects into one group, perform three-dimensional spatial positioning on each group of defect images, and obtain the real point coordinates of the defect image.

[0045] The process of three-dimensional spatial positioning of each set of defect images is as follows: Figure 2 As shown, including:

[0046] Step S101: randomly select two defect images of the same group, whose image coordinates of the center point of the defect are [u,v] and [u ’ ,v']. The corresponding real world coordinates are set to [X w ,Y w, ,Z w ] and [X ’ w ,Y ’ w, ,Z ’ w ]. According to the coordinate information of the UAV image defect and the camera imaging model, the relationship between the image coordinates and the real coordinates of the world system is obtained as shown in the following formula:

[0047]

[0048] in, is the camera internal parameter, fx and fy are the pixel ratios, u 0 、v 0 are the pixel coordinates of the center of the phase plane. and are the external parameters of the cameras taken at two different positions, R and R′ are the rotation matrices between the world coordinates and the camera coordinates, and T and T′ are the offsets between the world coordinate origin and the camera coordinate origin. ’ c is the Z-direction coordinate value of the real point in the camera coordinate system. The camera internal and external parameters can be obtained through camera calibration. Since the coordinate information of the image defect has been obtained before, [u,v] and [u ’ ,v'] is known, and [Xw,Yw,Zw] and [X ’ w ,Y ’ w ,Z ’ w ], Zc and Z ’ c is the unknown quantity to be determined. [Xw,Yw,,Zw] and [X ’ w ,Y ’ w ,Z ’ w ] respectively satisfy the equation of the straight line.

[0049] Step S102: According to the constraints of the above formulas (1) and (2), [Xw, Yw, Zw] and [X ’ w ,Y ’ w ,Z ’ w ] are closest to each other and their closest distance is calculated.

[0050]

[0051] Step S103, determine whether the closest distance is less than or equal to a set threshold. If so, the next step is to determine the real world coordinates [Xw, Yw, Zw] and [X ’ w ,Y ’ w ,Z ’ w] is within the range of the tower laser point cloud 3D model. If so, proceed to step S104. If the closest distance is greater than the threshold or the real world coordinates [Xw, Yw, Zw] and [X ’ w ,Y ’ w ,Z ’ w ] is outside the range of the tower laser point cloud 3D model, then proceed to step S105;

[0052] Step S104: determine that the two image defects are defects at the same position, and the real point coordinates of the defects are [Xw, Yw, Zw] and [X ’ w ,Y ’ w ,Z ’ w ]; whether it is within the range of the three-dimensional model of the tower laser point cloud can be determined by judging whether the real-world coordinates are within the convex hull of the three-dimensional point cloud or whether the real-world coordinates are within any triangle of the triangular mesh. Set the size threshold of each defect, for example, the radius of a bird's nest on a tower is generally not more than 0.5 meters.

[0053] Step S105, the two image defects are the same type of defects at different positions. At this time, the real coordinates of each image defect are [Xw, Yw,, Zw] and [X ’ w ,Y ’ w ,Z ’ w ] and the convex hull of the tower 3D laser point cloud. According to actual verification results, the accuracy of this 3D spatial positioning method is currently within 0.3 meters.

[0054] Example 2

[0055] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0056] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0057] The processing unit performs the various methods and processes described above, such as methods S1 to S5. For example, in some embodiments, methods S1 to S5 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S5 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S5 in any other appropriate manner (e.g., by means of firmware).

[0058] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0059] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0060] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A three-dimensional spatial positioning method for power tower defects based on deduplication technology, characterized in that: The method comprises the following steps: Step S1, obtaining an image of a tower patrolled by a drone taken along the drone route; Step S2, marking defects on the image of the tower inspected by the drone in step S1, including marking defect types and defect coordinate information; Step S3, classifying the defects of the tower inspection image of the drone marked in step S2 according to the defect type, and classifying the same type of defects into one group; Step S4, for each group of defect images, establish the relationship between the image defect coordinates and the real coordinates of the world system, calculate the shortest distance between two images in the same group of defects, deduplicate the defect images in combination with the three-dimensional laser point cloud data of the power tower, and perform three-dimensional spatial positioning based on the deduplicated defect images.

2. According to claim 1, a method for three-dimensional positioning of power tower defects based on deduplication technology is characterized in that: The step S4 comprises: Step S101, randomly select two defect images of the same group, and obtain the relationship between the image defect coordinates and the real coordinates of the world system; Step S102, according to the relationship constraint between the image defect coordinates and the real coordinates of the world system, the shortest distance between the real coordinates of the world system corresponding to the two defect images is obtained; Step S103, judging whether the closest distance is less than or equal to a set threshold value, if yes, then the next step is to judge whether the two world system real coordinates are within the range of the tower laser point cloud three-dimensional model when the closest distance is reached, if they are within the range of the tower laser point cloud three-dimensional model, then proceed to step S104; if the closest distance is greater than the threshold value or the world system real coordinates when the closest distance is reached are outside the range of the tower laser point cloud three-dimensional model, then proceed to step S105; Step S104, determining that the two image defects are the same position defect, and calculating the world system real coordinates of the defect; Step S105 , determining that the two image defects are defects at different positions, and calculating the world system real coordinates of the defects.

3. The method for three-dimensional positioning of power tower defects based on deduplication technology according to claim 2 is characterized in that: The relationship between the image defect coordinates and the real coordinates of the world system is specifically: Take any two defect images of the same group, the image coordinates of the center point of the defect are [u,v] and [u ’ ,v']; the corresponding real world coordinates are set to [X w ,Y w ,Z w ] and [X ’ w ,Y ’ w ,Z ’ w ]; According to the coordinate information of the UAV image defect and the camera imaging model, the relationship between the image coordinates and the real coordinates of the world system is obtained as follows: in, is the camera internal parameter, fx and fy are the pixel ratios, u0 and v0 are the pixel coordinates of the center of the phase plane; and are the external parameters of the camera taken at two different positions, R and R′ are the rotation matrices between the world coordinates and the camera coordinates, T and T′ are the offsets between the world coordinate origin and the camera coordinate origin, Zc and Z ’ c are the Z-direction coordinate values ​​of two different real points in the camera coordinate system.

4. The three-dimensional spatial positioning method for power tower defects based on deduplication technology according to claim 3 is characterized in that: The closest distance between the real coordinates of the world system corresponding to the two defect images is specifically: The formula for calculating the closest distance between the real-world coordinates of two defect images is: Among them, [Xw, Yw, Zw] and [X ’ w ,Y ’ w ,Z ’ w ] are the real coordinates of the world system corresponding to the closest distance between the two defect images.

5. The method for three-dimensional positioning of power tower defects based on deduplication technology according to claim 4 is characterized in that: The real coordinates of the world system corresponding to when the distance between the two defect images is the shortest are obtained by solving the relationship between the image coordinates and the real coordinates of the world system.

6. The method for three-dimensional positioning of power tower defects based on deduplication technology according to claim 2 is characterized in that: If the real coordinates of the world system are within the convex hull of the tower laser three-dimensional point cloud, or the real coordinates of the world system are inside any triangle of the triangular mesh, then it is judged that the real coordinates of the world system are within the range of the tower laser point cloud three-dimensional model.

7. The method for three-dimensional positioning of power tower defects based on deduplication technology according to claim 2 is characterized in that: In step S104, the real coordinates of the defect in the world system are taken as the average value of the real coordinates of the two world systems.

8. The method for three-dimensional positioning of power tower defects based on deduplication technology according to claim 2 is characterized in that: In the step S105, the world system real coordinates of each image defect are the intersection values ​​of two world system real coordinates satisfying the relationship between the image coordinates and the world system real coordinates and the convex hull formed by the three-dimensional laser point cloud of the tower.

9. The method for three-dimensional positioning of power tower defects based on deduplication technology according to claim 2 is characterized in that: The defect marking is performed by manual marking or artificial intelligence algorithm to identify defects.

10. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 9 is implemented.

Citation Information

Patent Citations

  • A method for locating defects in power grid line components by integrating 3D point clouds and 2D images

    CN112767391B

  • Iron tower bolt defect display method and system based on three-dimensional live-action model

    CN111537515A

  • Accurate positioning method for power transmission line unmanned aerial vehicle inspection image detection defects

    CN112013830A

  • Power grid line component defect positioning method fusing three-dimensional point cloud and two-dimensional image

    CN112767391A

  • Inspection method and system for photovoltaic power station

    CN113286129A