Automatic annotation method and system for inspection image of power transmission line and electronic equipment

Abstract

The invention discloses an automatic annotation method and system for a power transmission line inspection image and electronic equipment. The method comprises the following steps: acquiring an infrared image and a visible light image of a target power transmission line; performing dual-light registration based on the infrared image and the visible light image to obtain a mapping relation of pixel coordinates of the infrared image and the visible light image; identifying the power equipment based on the visible light image, and acquiring a first pixel coordinate of the power equipment in the visible light image; calculating a second pixel coordinate of the power equipment in the infrared image based on the mapping relation and the first pixel coordinate; and automatically marking the power equipment in the infrared image based on the second pixel coordinates. According to the invention, efficient marking of the power transmission line unmanned aerial vehicle inspection infrared image can be realized, the efficiency and accuracy of power transmission line unmanned aerial vehicle inspection are improved, and the power transmission line inspection safety is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of overhead transmission and distribution line operation and maintenance, and more particularly to a power transmission line inspection image automatic labeling method and system and electronic equipment. BACKGROUND

[0002] As a core element supporting key infrastructure and social operation, the stable operation of power equipment is of great importance. However, a large number of equipment are in outdoor harsh environments for a long time, and are affected by factors such as aging, overload, poor contact, etc., which can easily cause overheating of electrical connection parts, fittings, insulators, transformers, and even short circuit, open circuit and other fault risks. As a non-contact, fast and efficient online monitoring method, infrared temperature measurement technology can accurately capture the temperature field distribution of the equipment surface, effectively identify abnormal heating points, and provide important technical support for ensuring the safe and reliable operation of the power grid.

[0003] Deep learning has been widely applied in the field of image recognition. It has formed a relatively mature framework system in the fields of power transmission line inspection and insulator defect recognition. For the infrared inspection images taken, the current mainstream artificial intelligence recognition algorithms such as Faster RCNN, YOLO, SSD, etc. are used to extract high-dimensional target features through convolutional neural networks to locate the equipment area from the infrared images. However, the excellent performance of deep learning models is highly dependent on the training support of massive and high-quality labeled data. In the field of power equipment infrared thermal image analysis, obtaining accurate and sufficient labeled data is facing challenges such as high labeling professional requirements, high labeling costs, and difficulty in standardizing labeling.

[0004] In the unmanned aerial vehicle inspection of power transmission lines, visible light and infrared registration is an important technology. It combines the detailed information of visible light images and the temperature information of infrared images through the calibration and data fusion of visible light cameras and infrared thermal imaging cameras, forming more comprehensive inspection data. On this basis, high-resolution visible light images and infrared thermal images of the same device and the same perspective can be collected synchronously, and the pixel-level or target-level accurate correspondence between the two can be established by using image registration and fusion algorithms. There are mainly two technical routes for existing dual-light registration: ① feature matching through image region features. This method is simple and intuitive and easy to implement, but lacks the ability to handle image deviations caused by size differences and perspective differences between visible light images and infrared images. ② generating dual-light fusion images through GAN and other generation methods for registration. This method has good registration and fusion effect for power equipment and strong expandability, but requires a large amount of hardware equipment calculation, which is not conducive to subsequent edge deployment on unmanned aerial vehicles.

[0005] Therefore, there is a need for an accurate, efficient and fast dual-light registration method for power transmission and distribution lines. SUMMARY

[0006] This invention proposes an automatic annotation method, system, and electronic equipment for power transmission line inspection images to solve the problem of how to automatically annotate equipment in infrared images of power transmission lines.

[0007] To address the aforementioned problems, according to one aspect of the present invention, an automatic annotation method for transmission line inspection images is provided, the method comprising:

[0008] Acquire infrared and visible light images of the target transmission line;

[0009] Dual-light registration is performed based on the infrared image and the visible light image to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image;

[0010] Based on the visible light image, identify the power equipment and obtain the first pixel coordinates of the power equipment in the visible light image;

[0011] Based on the mapping relationship and the first pixel coordinates, calculate the second pixel coordinates of the power equipment in the infrared image;

[0012] The power equipment is automatically labeled in the infrared image based on the second pixel coordinates.

[0013] Preferably, the dual-light registration based on the infrared image and the visible light image to obtain the mapping relationship of pixel coordinates between the infrared image and the visible light image includes:

[0014] The phase consistency algorithm was used to obtain edge structure feature maps of power equipment in infrared and visible light images, respectively.

[0015] Based on the edge structure feature map, feature matching algorithm is used to perform feature matching, and feature point data of the edge structure of power equipment in infrared image and visible light image are obtained respectively. Based on the feature point data, feature matching is performed to obtain the mapping relationship of pixel coordinates of infrared image and visible light image.

[0016] Preferably, the feature detection algorithm includes: SIFT feature point detection algorithm and ORB feature point detection algorithm.

[0017] Preferably, the mapping relationship includes:

[0018]

[0019] Among them, u k and v k The position coordinates of the object within the pixel coordinate system of the visible light image; u h and v his the position coordinate of the object in the pixel coordinate system of the infrared image; T is the transformation mapping matrix between the position of the object in the pixel coordinate system of the visible light image and the position of the object in the pixel coordinate system of the infrared image.

[0020] According to another aspect of the present invention, an automatic annotation system for transmission line inspection images is provided, the system comprising:

[0021] The image acquisition unit is used to acquire infrared and visible light images of the target transmission line;

[0022] The registration unit is used to perform dual-light registration based on the infrared image and the visible light image to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image;

[0023] The target recognition unit is used to identify power equipment based on the visible light image and obtain the first pixel coordinates of the power equipment in the visible light image.

[0024] A pixel coordinate determination unit is used to calculate the second pixel coordinates of the power equipment in the infrared image based on the mapping relationship and the first pixel coordinates;

[0025] An automatic annotation unit is used to automatically annotate the power equipment in the infrared image based on the second pixel coordinates.

[0026] Preferably, the registration unit performs dual-light registration based on the infrared image and the visible light image to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image, including:

[0027] The phase consistency algorithm was used to obtain edge structure feature maps of power equipment in infrared and visible light images, respectively.

[0028] Based on the edge structure feature map, feature matching algorithm is used to perform feature matching, and feature point data of the edge structure of power equipment in infrared image and visible light image are obtained respectively. Based on the feature point data, feature matching is performed to obtain the mapping relationship of pixel coordinates of infrared image and visible light image.

[0029] Preferably, the feature detection algorithm includes: SIFT feature point detection algorithm and ORB feature point detection algorithm.

[0030] Preferably, the mapping relationship includes:

[0031]

[0032] Among them, u k and v k The position coordinates of the object within the pixel coordinate system of the visible light image; u h and vh is the position coordinate of the object in the pixel coordinate system of the infrared image; T is the transformation mapping matrix between the position of the object in the pixel coordinate system of the visible light image and the position of the object in the pixel coordinate system of the infrared image.

[0033] According to another aspect of the present invention, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the steps of an automatic annotation method for transmission line inspection images.

[0034] According to another aspect of the present invention, the present invention provides an electronic device, comprising:

[0035] The aforementioned computer-readable storage medium; and

[0036] One or more processors for executing a program in the computer-readable storage medium.

[0037] This invention provides an automatic annotation method, system, and electronic device for power transmission line inspection images, comprising: acquiring infrared and visible light images of a target power transmission line; performing dual-light registration based on the infrared and visible light images to obtain a mapping relationship between the pixel coordinates of the infrared and visible light images; identifying power equipment based on the visible light image to obtain the first pixel coordinates of the power equipment in the visible light image; calculating the second pixel coordinates of the power equipment in the infrared image based on the mapping relationship and the first pixel coordinates; and automatically annotating the power equipment in the infrared image based on the second pixel coordinates. This invention enables efficient annotation of infrared images from UAV inspections of power transmission lines, improving the efficiency and accuracy of UAV inspections of power transmission lines and ensuring the safety of power transmission line inspections. Attached Figure Description

[0038] Exemplary embodiments of the present invention can be more fully understood by referring to the following figures:

[0039] Figure 1 This is a flowchart of automatic annotation 100 for transmission line inspection images according to an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of the automatic annotation process according to an embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of the automatic annotation system 300 for transmission line inspection images according to an embodiment of the present invention. Detailed Implementation

[0042] Exemplary embodiments of the invention will now be described with reference to the accompanying drawings. However, the invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to fully and completely disclose the invention and to fully convey its scope to those skilled in the art. The terminology used in the exemplary embodiments illustrated in the drawings is not intended to limit the invention. In the drawings, the same units / elements are referred to by the same reference numerals.

[0043] Unless otherwise stated, the terms used herein (including technical terms) have their common meaning as understood by one of ordinary skill in the art. Furthermore, it is understood that terms defined in commonly used dictionaries should be understood to have a meaning consistent with the context of their relevant field, and not to be interpreted as having an idealized or overly formal meaning.

[0044] This invention provides a dual-light automatic annotation method for transmission line inspection images based on the registration of edge structure features of power equipment. By extracting stable and similar edge structure features of power equipment in dual-light images, the difference between dual-light images can be improved, and the registration rate can be increased. At the same time, a feature matching-based strategy is adopted to ensure high registration efficiency. Then, a visible light-infrared synchronous annotation framework is designed in the annotation system. The equipment target is initially located through a visible light recognition model, and the mapping relationship of the corresponding pixel coordinates of the dual-light images is calculated. The equipment target is initially located through a visible light recognition model, and efficient annotation of infrared images is achieved.

[0045] Figure 1 This is a flowchart illustrating the automatic annotation 100 of transmission line inspection images according to an embodiment of the present invention. For example... Figure 1 As shown, the automatic annotation method for transmission line inspection images provided by this embodiment of the invention can achieve efficient annotation of infrared images of transmission line UAV inspections, improve the efficiency and accuracy of transmission line UAV inspections, and ensure the safety of transmission line inspections. The automatic annotation method 100 for transmission line inspection images provided by this embodiment of the invention starts at step 101, in which the infrared image and visible light image of the target transmission line are acquired.

[0046] In step 102, dual-light registration is performed based on the infrared image and the visible light image to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image.

[0047] Preferably, the dual-light registration based on the infrared image and the visible light image to obtain the mapping relationship of pixel coordinates between the infrared image and the visible light image includes:

[0048] The phase consistency algorithm was used to obtain edge structure feature maps of power equipment in infrared and visible light images, respectively.

[0049] Based on the edge structure feature map, feature matching algorithm is used to perform feature matching, and feature point data of the edge structure of power equipment in infrared image and visible light image are obtained respectively. Based on the feature point data, feature matching is performed to obtain the mapping relationship of pixel coordinates of infrared image and visible light image.

[0050] Preferably, the feature detection algorithm includes: SIFT feature point detection algorithm and ORB feature point detection algorithm.

[0051] Preferably, the mapping relationship includes:

[0052]

[0053] Among them, u k and v k The position coordinates of the object within the pixel coordinate system of the visible light image; u h and v h is the position coordinate of the object in the pixel coordinate system of the infrared image; T is the transformation mapping matrix between the position of the object in the pixel coordinate system of the visible light image and the position of the object in the pixel coordinate system of the infrared image.

[0054] Combination Figure 2 As shown, in this invention, images of power transmission line UAV inspections are acquired, including infrared images and visible light images. Then, dual-light registration is performed based on the edge structure features of the infrared images and visible light images to obtain the mapping relationship between the two images.

[0055] The inherent differences in the imaging mechanisms of infrared and visible light lead to a significant divergence in their apparent characteristics: visible light images primarily reflect the reflective properties of an object's surface, exhibiting rich texture details and distinct edge gradients; while infrared thermal images characterize the distribution of radiant energy of a target, modulated by factors such as material thermal properties, ambient temperature, and operating load, often displaying characteristics such as uneven grayscale distribution, low contrast, and blurred boundaries. Despite these apparent differences, power equipment exhibits strong consistency in its physical structure, such as the periodic contours of insulator strings and the geometry of clamps, providing a structural-level correlation basis for the collaborative analysis of dual-light images.

[0056] To effectively capture such cross-modal common features, traditional gradient operators (such as Sobel and Canny) are susceptible to imaging differences due to their over-reliance on local gray-level gradients, making robust structure alignment difficult. In response, the Phase Congruency (PC) model provides an illumination- and contrast-invariant edge detection mechanism. Phase congruency quantifies the saliency of image structure by the degree of phase alignment of Fourier components—at edges or texture features, different frequency harmonics are highly consistent in phase, a characteristic independent of absolute gray-level values. Its calculation can be expressed as:

[0057]

[0058] Among them, A n The amplitude of the frequency component. For local phase, W n For frequency band weighting, The average phase.

[0059] Frequency component amplitude A n The amplitude intensity of the nth frequency component of the image at position n is obtained through multi-scale filter bank decomposition. A n This reflects the energy distribution of the image structure within the frequency band. In images of power equipment, high-amplitude regions correspond to abrupt changes in texture or thermal defects such as insulator damage or wire clamp overheating.

[0060] Local phase The instantaneous phase angle of the nth frequency component is represented by the Hilbert transform or orthogonal filter. It describes the periodic alignment of the frequency components in space. When different frequency components are phase aligned, local phase markers such as edges / corners exist.

[0061] Bandwidth weight W n The normalized weighting coefficient assigned to the nth frequency component is typically calculated dynamically based on the band signal-to-noise ratio or energy saliency. The formula is as follows:

[0062] ,

[0063] Where η is a small constant to avoid division by zero.

[0064] Average phase As a reference standard for phase consistency. If and If the difference is close to 0 or π, it indicates a high phase correlation, suggesting a strong structural feature at this point. The calculation formula is

[0065] .

[0066] Phase consistency passed and Alignment degree rather than The absolute intensity of edge detection makes it highly invariant to low contrast in infrared images and shadow / illuminance variations in visible light images, significantly outperforming traditional gradient detectors. This mechanism ensures that phase consistency calculations extract only edge features strongly correlated with the device's physical structure, resulting in stable edge structure feature maps for both infrared and visible light images, providing robust input for dual-light registration.

[0067] Therefore, in this invention, the phase consistency algorithm is used to obtain the edge structure feature maps of the power equipment in the infrared image and the visible light image respectively. After obtaining the edge structure feature maps of the power equipment, the feature matching algorithm is used to perform feature matching in order to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image.

[0068] The feature detection algorithm is either the SIFT feature point detection algorithm or the ORB feature point detection algorithm.

[0069] Taking SIFT as an example, SIFT (Scale-Invariant Feature Transform) is an algorithm for image processing designed to detect and describe local features in images. It performs image registration by finding extreme points in an image and extracting their position, scale, and rotation invariants. SIFT features are very stable local features, exhibiting high tolerance even to changes in lighting, noise, and minute viewpoints. This allows SIFT features to maintain a high detection rate even when partially occluded.

[0070] SIFT feature detection has four main steps:

[0071] ① Scale-space extremum detection: Retrieve images across all scale spaces and detect keypoint locations and scales by constructing a Difference of Gaussian (DoG) pyramid. The DoG calculation formula is:

[0072] L(x,y,σ)=G(x,y,σ)*I(x,y),

[0073] D(x,y,σ)=|G(x,y,kσ)-G(x,y,σ)|*I(x,y)=L(x,y,kσ)-L(x,y,σ),

[0074] Where G(x,y,σ) is the Gaussian kernel function.

[0075] ,

[0076] Here, σ is called the scale space factor, which is the standard deviation of the Gaussian normal distribution and reflects the degree of blurring of the image. The larger the value, the more blurred the image, and the larger the corresponding scale. L(x,y,σ) represents the Gaussian scale space of the image.

[0077] ② Geometric consistency optimization: Remove low-contrast extreme points and unstable edge response points to ensure the robustness and matching accuracy of feature points.

[0078] ③ Determining the principal direction of the feature point: Calculate the argument and magnitude of the gradient for each pixel within a neighborhood centered on the feature point and with a radius of 3 × 1.5σ. Then, use a histogram to statistically analyze the argument of the gradient. The horizontal axis of the histogram represents the direction of the gradient, and the vertical axis represents the cumulative value of the gradient magnitude corresponding to the gradient direction. The direction corresponding to the highest peak in the histogram is the direction of the feature point.

[0079] ④ Feature point description generation. The coordinate axes are rotated to the direction of the feature points. The gradient magnitude and direction of the pixels in a 16×16 window centered on the feature points are used to divide the pixels in the window into 16 blocks. Each block is a histogram of the pixels in 8 directions, which can form a total of 128-dimensional feature vectors.

[0080] The SIFT feature detection algorithm can ultimately generate multiple feature point data. Each feature point data includes a 128-dimensional feature description vector encoding local structural information, providing a highly discriminative and strongly invariant feature representation for two-light image registration.

[0081] After finding the feature points of the edge structure of power equipment in visible light and infrared images based on the SIFT algorithm, a correspondence matrix T between the pixel coordinate systems of the visible light and infrared images is established, thus obtaining the mapping relationship as follows:

[0082]

[0083] Among them, u k v k u represents the position of an object within the pixel coordinate system of a visible light image. h v h This represents the position of the object in the pixel coordinate system of the infrared image, and T is the transformation mapping matrix between the position of the object in the pixel coordinate system of the visible light image and the position of the object in the pixel coordinate system of the infrared image. After obtaining the mapping matrix, the position of the power equipment in the infrared image can be automatically identified by a visible light target recognition algorithm and then calculated from the left side of the transformation matrix.

[0084] In step 103, the electrical equipment is identified based on the visible light image, and the first pixel coordinates of the electrical equipment in the visible light image are obtained.

[0085] In this invention, the YOLOv8 model is used to identify power equipment in a visible light image dataset to obtain the first pixel coordinates of the power equipment in the visible light image. The YOLOv8 model is an upgraded version of the YOLOv5 model, integrating unified support for three major tasks: image classification, object detection, and instance segmentation. Through systematic reconstruction of the backbone network, detection head, and loss function, it significantly improves model performance and deployment flexibility. YOLOv8 includes a scaling factor design, providing detection models with five parameter levels (N / S / M / L / X) and an instance segmentation model based on YOLACT, covering the needs of all scenarios from edge computing to cloud services.

[0086] In this invention, the architecture of YOLOv8 is improved, focusing on the synergistic enhancement of accuracy, speed, and training stability, mainly reflected in three aspects of innovation:

[0087] ① Backbone network and Neck enhancement

[0088] The C2f module, which is richer in gradient flow, is introduced to replace the C3 structure of YOLOv5. Drawing on the ELAN design concept of YOLOv7, the feature representation capability of small targets is improved by retaining the reuse of high-order features.

[0089] ②Detection head decoupling and Anchor-Free paradigm

[0090] A decoupled head is used to separate the classification and regression branches, avoiding task conflicts and improving convergence efficiency. An anchor-free paradigm is adopted to eliminate the matching bias of the preset anchor box for complex targets and enhance the robustness of localization for irregular structures.

[0091] ③ Optimization of training strategy and loss function

[0092] We propose a Task Aligned Assigner positive sample allocation strategy, which dynamically selects high-quality samples through task alignment metrics to alleviate false detection problems in complex backgrounds; we adopt Distribution Focal Loss to optimize the bounding box probability distribution modeling and improve regression accuracy; and we integrate a Mosaic-enhanced progressive shutdown strategy to reduce generalization error caused by data perturbation and significantly improve the model convergence stability.

[0093] In step 104, based on the mapping relationship and the first pixel coordinates, the second pixel coordinates of the power equipment in the infrared image are calculated.

[0094] In step 105, the power equipment is automatically labeled in the infrared image based on the second pixel coordinates.

[0095] In this invention, after obtaining the first pixel coordinates and mapping relationship, the position of the power equipment in the infrared image, i.e., the second pixel coordinates, can be obtained based on the first pixel coordinates and mapping relationship; then, the power equipment can be automatically labeled in the infrared image based on the second pixel coordinates.

[0096] To verify the effectiveness of the semi-automatic image annotation of this invention, the registration accuracy and temperature measurement precision were used as evaluation criteria. The accuracy was calculated based on an IOU (Intersection over Union) of more than 0.6 between the annotated and target images. The calculation method was as follows:

[0097]

[0098] The predicted bounding boxes are the system output boxes, and the ground truth bounding boxes are manually labeled boxes. If the Intersection over Union (IOU) is greater than 0.6, the labeling is considered correct; otherwise, it is considered incorrect.

[0099] The target recognition algorithm and the dual-light registration system were imported into the dual-light annotation system, and automatic annotation was achieved through the algorithm. Using the dual-light image semi-automatic annotation technology, 4000 infrared and visible light images were registered and annotated, including 5254 defects in insulators, hardware, and conductors. The registration results are shown in Table 1.

[0100] Table 1. Dual-light annotation and registration test

[0101] Device type Sample number Correctly labeled number Accuracy Insulator 1778 1635 91.95％ Fitting 2049 1871 91.31％ Ground wire 1427 1254 87.87％ Overall 5254 4760 90.59％

[0102] The dual-light automatic annotation technology for transmission line inspection images based on edge structural feature registration of power equipment achieved an overall equipment annotation accuracy of 90.59% for single-type defects, specifically 91.95% for insulators, 91.31% for hardware, and 87.87% for conductors and ground wires. Its overall accuracy exceeds 90% and can be applied to subsequent large-scale infrared sample annotation.

[0103] This invention provides an automatic dual-light annotation method for transmission line inspection images based on edge structure feature registration of power equipment. It identifies the equipment type and location of the transmission line in the visible light image using a YOLOv8 model, and then achieves visible light and infrared dual-light registration using an edge structure feature dual-light registration algorithm. The mapping relationship of the corresponding pixel coordinates in the dual-light images is calculated, thereby achieving efficient annotation of the infrared image. Experimental results show that the proposed edge structure feature-based dual-light registration algorithm has a significantly smaller error than the direct registration method, and the overall equipment annotation accuracy for infrared image defects is higher than 90%.

[0104] Figure 3 This is a schematic diagram of the structure of an automatic annotation system 300 for transmission line inspection images according to an embodiment of the present invention. Figure 3As shown, the automatic annotation system 300 for transmission line inspection images provided in this embodiment of the invention includes: an image acquisition unit 301, a registration unit 302, a target recognition unit 303, a pixel coordinate determination unit 304, and an automatic annotation unit 305.

[0105] Preferably, the image acquisition unit 301 is used to acquire infrared images and visible light images of the target transmission line.

[0106] Preferably, the registration unit 302 is used to perform dual-light registration based on the infrared image and the visible light image to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image.

[0107] Preferably, the registration unit 302 performs dual-light registration based on the infrared image and the visible light image to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image, including:

[0108] The phase consistency algorithm was used to obtain edge structure feature maps of power equipment in infrared and visible light images, respectively.

[0109] Based on the edge structure feature map, feature matching algorithm is used to perform feature matching, and feature point data of the edge structure of power equipment in infrared image and visible light image are obtained respectively. Based on the feature point data, feature matching is performed to obtain the mapping relationship of pixel coordinates of infrared image and visible light image.

[0110] Preferably, the feature detection algorithm includes: SIFT feature point detection algorithm and ORB feature point detection algorithm.

[0111] Preferably, the mapping relationship includes:

[0112]

[0113] Among them, u k and v k The position coordinates of the object within the pixel coordinate system of the visible light image; u h and v h is the position coordinate of the object in the pixel coordinate system of the infrared image; T is the transformation mapping matrix between the position of the object in the pixel coordinate system of the visible light image and the position of the object in the pixel coordinate system of the infrared image.

[0114] Preferably, the target recognition unit 303 is used to identify power equipment based on the visible light image and obtain the first pixel coordinates of the power equipment in the visible light image.

[0115] Preferably, the pixel coordinate determination unit 304 is used to calculate the second pixel coordinates of the power equipment in the infrared image based on the mapping relationship and the first pixel coordinates.

[0116] Preferably, the automatic labeling unit 305 is used to automatically label the power equipment in the infrared image based on the second pixel coordinates.

[0117] The automatic annotation system 300 for transmission line inspection images in this embodiment of the present invention corresponds to the automatic annotation method 100 for transmission line inspection images in another embodiment of the present invention, and will not be described again here.

[0118] According to another aspect of the present invention, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the steps of an automatic annotation method for transmission line inspection images.

[0119] According to another aspect of the present invention, the present invention provides an electronic device, comprising:

[0120] The aforementioned computer-readable storage medium; and

[0121] One or more processors for executing a program in the computer-readable storage medium.

[0122] The present invention has been described with reference to a few embodiments. However, it will be apparent to those skilled in the art that other embodiments besides those disclosed above fall equivalently within the scope of the present invention.

[0123] Generally, all terms used in this invention are interpreted according to their ordinary meaning in the art, unless otherwise expressly defined herein. All references to “a / the / the [device, component, etc.]” ​​are openly interpreted as at least one instance of said device, component, etc., unless otherwise expressly stated. The steps of any method disclosed herein need not be performed in the exact order disclosed unless explicitly stated otherwise.

[0124] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0126] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention.

Claims

1. An automatic labeling method of a power transmission line inspection image, characterized by, The method includes: Acquire infrared and visible light images of the target transmission line; Dual-light registration is performed based on the infrared image and the visible light image to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image; Based on the visible light image, identify the power equipment and obtain the first pixel coordinates of the power equipment in the visible light image; Based on the mapping relationship and the first pixel coordinates, calculate the second pixel coordinates of the power equipment in the infrared image; The power equipment is automatically labeled in the infrared image based on the second pixel coordinates.

2. The method of claim 1, wherein, Dual-light registration is performed based on the infrared image and the visible light image to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image, including: The phase consistency algorithm was used to obtain edge structure feature maps of power equipment in infrared and visible light images, respectively. Based on the edge structure feature map, feature matching algorithm is used to perform feature matching, and feature point data of the edge structure of power equipment in infrared image and visible light image are obtained respectively. Based on the feature point data, feature matching is performed to obtain the mapping relationship of pixel coordinates of infrared image and visible light image.

3. The method of claim 2, wherein, The feature detection algorithms include: SIFT feature point detection algorithm and ORB feature point detection algorithm.

4. The method of claim 1, wherein, The mapping relationship includes: wherein u k and v k are the position coordinates of the object in the pixel coordinate system of the visible light image; u h and v h are the position coordinates of the object in the pixel coordinate system of the infrared image; and T is the conversion mapping matrix of the position of the object in the pixel coordinate system of the visible light image and the position of the object in the pixel coordinate system of the infrared image.

5. An automatic labeling system for power line inspection images, characterized in that, The system includes: The image acquisition unit is used to acquire infrared and visible light images of the target transmission line; The registration unit is used to perform dual-light registration based on the infrared image and the visible light image to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image; The target recognition unit is used to identify power equipment based on the visible light image and obtain the first pixel coordinates of the power equipment in the visible light image. A pixel coordinate determination unit is used to calculate the second pixel coordinates of the power equipment in the infrared image based on the mapping relationship and the first pixel coordinates; An automatic annotation unit is used to automatically annotate the power equipment in the infrared image based on the second pixel coordinates.

6. The system of claim 5, wherein, The registration unit performs dual-light registration based on the infrared image and the visible light image to obtain the mapping relationship between the pixel coordinates of the infrared image and the visible light image, including: The phase consistency algorithm was used to obtain edge structure feature maps of power equipment in infrared and visible light images, respectively. Based on the edge structure feature map, feature matching algorithm is used to perform feature matching, and feature point data of the edge structure of power equipment in infrared image and visible light image are obtained respectively. Based on the feature point data, feature matching is performed to obtain the mapping relationship of pixel coordinates of infrared image and visible light image.

7. The system of claim 6, wherein, The feature detection algorithms include: SIFT feature point detection algorithm and ORB feature point detection algorithm.

8. The system of claim 5, wherein, The mapping relationship includes: wherein u k and v k are the position coordinates of the object in the pixel coordinate system of the visible light image; u h and v h are the position coordinates of the object in the pixel coordinate system of the infrared image; and T is the conversion mapping matrix of the position of the object in the pixel coordinate system of the visible light image and the position of the object in the pixel coordinate system of the infrared image.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-4.

10. An electronic device, comprising: include: The computer-readable storage medium as described in claim 11; as well as One or more processors for executing a program in the computer-readable storage medium.

Images