A method and device for detecting and positioning external damage of a power transmission tower

Abstract

The application discloses a kind of transmission tower outer damage hidden danger detection positioning method and device, comprising: real-time shooting area to be detected, and by target detection model, the image to be identified obtained by shooting is detected to outer damage hidden danger, to obtain the detection result whether there is outer damage hidden danger pixel point in image to be identified;When the detection result is that there is outer damage hidden danger pixel point in image to be identified, based on the coordinate mapping relationship between pixel coordinate system and space coordinate system, the first coordinate information of outer damage hidden danger pixel point under pixel coordinate system is converted into the first point cloud data under space coordinate system;Wherein, target detection model is the model obtained by training using YOLOv4-tiny algorithm, pixel coordinate system is established based on image to be identified.The application is detected to outer damage hidden danger to the image of real-time shooting by the model based on YOLOv4-tiny algorithm, reduce the occupation of computing resource in detection process.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of external force damage hidden danger detection of power systems, and in particular to a power transmission tower external damage hidden danger detection positioning method and device. BACKGROUND

[0002] Due to the characteristics of large regional span and complex surrounding terrain of the power transmission line in the power system, the environment of the power transmission tower on the power transmission line is uncertain, and the performance of the power transmission tower is greatly affected by the geographical environment and external factors, and is easily damaged by external forces. In the power system, the power transmission tower is an important infrastructure, which undertakes the task of power transmission and transmission, and makes a great contribution to the normal operation of the power system. However, in the long-term operation process, the power transmission tower is often affected by various external forces, which may cause various potential problems and hidden dangers of the power transmission tower, such as bolt loosening, component fracture, and tower body bending, etc. due to the lack of safety distance or the current distance exceeding the safety distance when high machinery operates around the power facility, and the impact of vehicles or wild animals, etc. external factors, which may cause abnormal operation problems of the power system where the power transmission tower is located.

[0003] In order to prevent external damage from threatening the power system, it is necessary to monitor the surrounding area of the power transmission tower and other facilities. The traditional monitoring method mainly adopts manual patrol mode, which is low in efficiency and high in cost, and the result is difficult to achieve comprehensive and accurate effect, and it is easy to cause power accidents when the personnel responsible for the patrol arrive. At present, the external damage hidden danger detection system of the power transmission tower mainly detects the site conditions through infrared detection or ultrasonic detection. However, the detection range of the monitoring method of infrared detection and ultrasonic detection is limited, and it is easy to be affected by non-external hidden danger objects and produce false alarms, and the detection effect is also easily affected by the weather. SUMMARY

[0004] The embodiment of the present application provides a power transmission tower external damage hidden danger detection positioning method and device, which utilizes YOLOv4-tiny algorithm to construct a target detection model, and utilizes the model to detect external damage hidden danger of the image, reduces the occupation of computing resources, and improves the external damage hidden danger detection precision and external damage hidden danger detection efficiency.

[0005] In order to solve the above technical problems, the embodiment of the present application provides a power transmission tower external damage hidden danger detection positioning method, which comprises:

[0006] Real-time shooting of the to-be-detected area to obtain a to-be-recognized image of the to-be-detected area; wherein the to-be-detected area contains the power transmission tower, or the to-be-detected area is adjacent to the power transmission tower;

[0007] The image to be identified is input into the target detection model, so that the target detection model can detect external damage hazards in the image to be identified and output the detection result of whether there are pixels with external damage hazards in the image to be identified; wherein, the target detection model is obtained by training an initial model pre-built using the YOLOv4-tiny algorithm;

[0008] When the detection result indicates that there are pixels with potential external damage in the image to be identified, the first coordinate information of the pixels with potential external damage in the pixel coordinate system is converted into the first point cloud data in the spatial coordinate system based on the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, so as to complete the location of the potential external damage of the transmission tower; wherein, the pixel coordinate system is established based on the image to be identified.

[0009] This invention utilizes the YOLOv4-tiny algorithm, which boasts small scale and high detection accuracy, to construct an initial model. This initial model is then trained to obtain a target detection model for detecting external damage hazards in the image to be identified. The image to be identified, captured in real-time in the area to be detected, is then input into the target detection model. This allows the target detection model to quickly detect external damage hazards in the image with fewer parameters and less computational resources. Furthermore, the detection process avoids the loss of detection data and storage space caused by complex computational procedures, thereby improving the accuracy and efficiency of external damage hazard detection. Additionally, by establishing a pixel coordinate system, the positional information of the detected external damage hazard pixels in the image to be identified can be accurately represented. Based on the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, the first coordinate information in the pixel coordinate system is transformed into first point cloud data in the spatial coordinate system. This enables precise location of external damage hazards in actual space, facilitating targeted repair and maintenance of transmission towers.

[0010] As a preferred embodiment, the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system is obtained as follows:

[0011] The camera is calibrated to obtain its intrinsic parameters;

[0012] The area to be detected is scanned in real time using a laser scanner to obtain the second point cloud data of the area to be detected, and the area to be detected is captured in real time using a camera to obtain the first image;

[0013] Using the PNP algorithm, the extrinsic parameters of the camera are calculated based on the second point cloud data and the first image; wherein, the extrinsic parameters of the camera include the camera's rotation matrix and translation vector;

[0014] The target region image to be identified is extracted from the first image, and then the target region image is subjected to feature extraction to obtain a number of corresponding feature points, and the second coordinate information of each feature point in the pixel coordinate system is obtained.

[0015] Based on the intrinsic parameters, the extrinsic parameters, the second coordinate information corresponding to each feature point, and the third coordinate information of each feature point in the spatial coordinate system, each feature point is matched to obtain the correspondence between the second coordinate information and the third coordinate information corresponding to each feature point. Then, based on the correspondence between the second coordinate information and the third coordinate information corresponding to all feature points, a coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system is established.

[0016] In a preferred embodiment of the present invention, the camera's extrinsic parameters include the camera's rotation matrix and translation vector, which describe the camera's position and attitude in space, provide the correspondence between the camera and the world coordinate system, and provide accurate geometric relationships for subsequent location of potential external damage hazards. Then, based on the camera's intrinsic parameters, extrinsic parameters, the second coordinate information of each feature point in the target area image in the pixel coordinate system, and the third coordinate information of each feature point in the spatial coordinate system, each feature point is matched to determine the correspondence between the second and third coordinate information corresponding to each feature point. Based on the correspondence between the second and third coordinate information corresponding to all feature points, a coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system is established. This allows the pixel coordinates in the image to be identified obtained by the camera to be mapped to three-dimensional coordinates in the real world using the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, thereby locating the location of potential external damage hazards in the real world.

[0017] As a preferred embodiment, the step of extracting the target region image to be identified from the first image, then performing feature extraction on the target region image to obtain a number of corresponding feature points, and obtaining the second coordinate information of each feature point in the pixel coordinate system, specifically involves:

[0018] Object recognition is performed on the first image, and the region in the first image containing the reference target is divided into a target region image;

[0019] The target region image is preprocessed, and then features are extracted from the preprocessed target region image to obtain several feature points of the reference target, and the second coordinate information of each feature point in the pixel coordinate system is obtained; wherein, the preprocessing of the target region image includes image enhancement, edge detection and binarization.

[0020] In a preferred embodiment of the present invention, object recognition is performed on the first image. The region containing the reference target in the first image is divided into a target region image, which helps to define the region of interest, reduce the processing scope, and improve the efficiency and accuracy of establishing coordinate mapping relationships. Furthermore, image enhancement of the target region image improves its visual quality, making it easier to analyze. Edge detection obtains the edge contour information of objects in the target region image, providing more valuable information for subsequent feature extraction. Binarization transforms the target region image into a binary image, enabling analysis of the object and background. This helps reduce background interference in object recognition, thereby highlighting the features of the target object and improving feature extraction accuracy.

[0021] As a preferred embodiment, when the detection result indicates the presence of pixels with potential external damage in the image to be identified, the first coordinate information of the pixels with potential external damage in the pixel coordinate system is converted into first point cloud data in the spatial coordinate system based on the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, thereby completing the location of potential external damage to the transmission tower. Specifically:

[0022] When the detection result indicates that there are pixels with potential external damage in the image to be identified, the first coordinate information of the pixels with potential external damage in the pixel coordinate system is converted into the second coordinate information in the camera coordinate system according to the camera's intrinsic parameters.

[0023] Based on the camera's position information and shooting angle, the second coordinate information is transformed into a third coordinate information in the world coordinate system;

[0024] Using 3D point cloud technology, the third coordinate information is converted into first point cloud data in the spatial coordinate system to locate the external damage hazard of the transmission tower.

[0025] The camera is used to capture images of the area to be detected in real time to obtain the image to be identified, and the camera coordinate system is a coordinate system with the optical center of the camera as the origin.

[0026] The preferred embodiment of the present invention uses camera intrinsic parameters and location information to convert the coordinate information of pixels with external damage hazards into point cloud data in a spatial coordinate system. This helps to realize applications in spatial positioning of hazards, problem analysis, hazard management and maintenance, thereby providing accurate and comprehensive information for locating external damage hazards of transmission towers and helping to improve safety and maintenance efficiency.

[0027] As a preferred embodiment, the step of capturing real-time images of the area to be detected to obtain an image of the area to be identified specifically involves:

[0028] Using a camera installed on the transmission tower, the area to be detected is photographed in real time to obtain an initial image;

[0029] The initial image is preprocessed to obtain the image to be identified for the region to be detected;

[0030] The preprocessing of the initial image includes image denoising, image enhancement, and edge detection.

[0031] In a preferred embodiment of the present invention, during real-time shooting, noise may be generated in the image due to factors such as light and environment. By performing image denoising on the initial image obtained by the camera to remove noise interference in the initial image, the quality and recognizability of the initial image can be improved. Furthermore, image enhancement technology can improve the performance of the image in terms of brightness, contrast, and clarity, which helps to make the area to be detected more distinct and improve the recognizability of the object. Edge detection is also performed on the initial image to obtain the edge contour of the object so that the target object can be identified and its features extracted in the subsequent process.

[0032] To address the same technical problem, embodiments of the present invention also provide a device for detecting and locating potential external damage to power transmission towers, comprising:

[0033] An area imaging system is used to capture images of an area to be detected in real time to obtain an image of the area to be identified; wherein the area to be detected includes the power transmission tower, or the area to be detected is adjacent to the power transmission tower;

[0034] An external damage hazard detection system is used to input the image to be identified into a target detection model, so that the target detection model can detect external damage hazards in the image to be identified and output the detection result of whether there are external damage hazard pixels in the image to be identified; wherein, the target detection model is obtained by training an initial model pre-built using the YOLOv4-tiny algorithm;

[0035] An external damage hazard location system is used to, when the detection result indicates the presence of external damage hazard pixels in the image to be identified, convert the first coordinate information of the external damage hazard pixels in the pixel coordinate system into first point cloud data in the spatial coordinate system based on the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, so as to complete the location of external damage hazards of the transmission tower; wherein, the pixel coordinate system is established based on the image to be identified.

[0036] As a preferred embodiment, the device for detecting and locating potential external damage to transmission towers further includes a data acquisition system; wherein, the data acquisition system specifically includes:

[0037] The calibration module is used to calibrate the camera and obtain its intrinsic parameters.

[0038] The scanning and imaging module is used to scan the area to be detected in real time using a laser scanner to obtain the second point cloud data of the area to be detected, and to capture the area to be detected in real time using the camera to obtain the first image;

[0039] The calculation module is used to calculate the extrinsic parameters of the camera based on the second point cloud data and the first image using the PNP algorithm; wherein the extrinsic parameters of the camera include the camera's rotation matrix and translation vector;

[0040] The extraction module is used to extract the target region image to be identified from the first image, then perform feature extraction on the target region image to obtain a number of corresponding feature points, and obtain the second coordinate information of each feature point in the pixel coordinate system.

[0041] The matching module is used to match each feature point according to the intrinsic parameters, the extrinsic parameters, the second coordinate information corresponding to each feature point, and the third coordinate information of each feature point in the spatial coordinate system, to obtain the correspondence between the second coordinate information and the third coordinate information corresponding to each feature point, and then establish the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system based on the correspondence between the second coordinate information and the third coordinate information corresponding to all feature points.

[0042] As a preferred embodiment, the extraction module specifically includes:

[0043] The segmentation unit is used to perform object recognition on the first image and to segment the region in the first image containing the reference target into a target region image.

[0044] The extraction unit is used to preprocess the target region image, then extract features from the preprocessed target region image to obtain several feature points of the reference target, and obtain the second coordinate information of each feature point in the pixel coordinate system; wherein, the preprocessing of the target region image includes image enhancement, edge detection and binarization.

[0045] As a preferred embodiment, the external damage hazard location system specifically includes:

[0046] The first coordinate transformation module is used to transform the first coordinate information of the pixel with external damage risk in the pixel coordinate system into the second coordinate information in the camera coordinate system according to the camera's intrinsic parameters when the detection result indicates that there is a pixel with external damage risk in the image to be identified. The camera is used to capture the area to be detected in real time to obtain the image to be identified, and the camera coordinate system is a coordinate system with the optical center of the camera as the origin.

[0047] The second coordinate transformation module is used to transform the second coordinate information into third coordinate information in the world coordinate system based on the position information and shooting angle of the camera.

[0048] The third coordinate transformation module is used to convert the third coordinate information into first point cloud data in the spatial coordinate system using three-dimensional point cloud technology, so as to complete the location of external damage hazards of the transmission tower.

[0049] As a preferred embodiment, the area imaging system specifically includes:

[0050] The imaging module is used to capture images of the area to be detected in real time using a camera installed on the power transmission tower, so as to obtain an initial image;

[0051] The preprocessing module is used to preprocess the initial image to obtain the image to be identified of the region to be detected; wherein, the preprocessing of the initial image includes image denoising, image enhancement and edge detection. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a method for detecting and locating potential external damage to power transmission towers according to Embodiment 1 of the present invention.

[0053] Figure 2 : A schematic diagram of the structure of a target detection model provided in Embodiment 1 of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure of a detection and positioning device for external damage hazards of power transmission towers provided in Embodiment 1 of the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Example 1:

[0057] Please refer to Figure 1 This invention provides a method for detecting and locating potential external damage to power transmission towers. The method includes steps S1 to S3, each of which is detailed below:

[0058] Step S1: Take a real-time picture of the area to be detected to obtain the image of the area to be identified.

[0059] The area to be tested includes power transmission towers, or is adjacent to power transmission towers. The area to be tested represents the environment in which the power transmission towers are located.

[0060] As a preferred embodiment, step S1 includes steps S11 to S12, each of which is detailed below:

[0061] Step S11: Use a camera set up at the transmission tower to take real-time pictures of the area to be detected to obtain an initial image.

[0062] It should be noted that using a camera to capture real-time images of the area to be detected can yield video. Subsequently, target recognition is performed on the captured video to determine the target area image in the video.

[0063] Step S12: Preprocess the initial image to obtain the image to be identified for the region to be detected.

[0064] The preprocessing of the initial image includes image denoising, image enhancement, and edge detection.

[0065] Step S2: Input the image to be identified into the target detection model so that the target detection model can detect potential external damage to the image and output the detection result of whether there are pixels with potential external damage in the image.

[0066] The object detection model was obtained by training the initial model using the sample dataset. The initial model was pre-built using the YOLOv4-tiny algorithm and consists of four parts: an input layer, a backbone feature extraction network layer, a neck network layer, and an output layer.

[0067] Specifically, please refer to Figure 2In the initial model, CSPDarkNet53_tiny was used to replace the CSPDarkNet53 module as the backbone feature extraction network layer to reduce the computational cost of the convolutional neural network. The backbone feature extraction network layer consists of a first Darknet convolution, a second Darknet convolution, a first residual block, a second residual block, and a third Darknet convolution. The LeakyReLU activation function is used to reduce the number of parameters in the model and improve the computational speed of the convolutional neural network. The neck network layer uses a feature pyramid network (FPN) structure. The FPN upsamples the last effective feature layer after convolution and then stacks and convolves it with the previous effective feature layer.

[0068] It should be noted that the accuracy of the target detection model in detecting and identifying pixels with potential external damage can be improved by increasing the amount of point cloud data under different environmental conditions in the sample dataset. These pixels are those that are at risk of being damaged by external forces. Furthermore, by improving the performance of the camera's monocular camera, the impact of environmental factors on image quality can be mitigated to some extent.

[0069] Step S3: When the detection result indicates that there are pixels with potential external damage in the image to be identified, the first coordinate information of the pixels with potential external damage in the pixel coordinate system is converted into the first point cloud data in the spatial coordinate system based on the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, so as to complete the location of potential external damage to the transmission tower.

[0070] The pixel coordinate system is established based on the image to be recognized. The pixel coordinate system is a two-dimensional coordinate system uv with the upper left corner of the image to be recognized as the origin.

[0071] As a preferred embodiment, step S3 includes steps S31 to S33, and the specific details of each step are as follows:

[0072] Step S31: When the detection result indicates that there are pixels with potential external damage in the image to be identified, the first coordinate information of the pixels with potential external damage in the pixel coordinate system is converted into the second coordinate information in the camera coordinate system according to the camera's intrinsic parameters.

[0073] The camera is used to capture images of the area to be detected in real time to obtain the image to be identified. The camera coordinate system is a three-dimensional coordinate system Oc-XcYcZc with the optical center of the camera as the origin.

[0074] Step S32: Based on the camera's position information and shooting angle, the second coordinate information is converted into third coordinate information in the world coordinate system.

[0075] The world coordinate system is a three-dimensional coordinate system Ow-XwYwZw used to describe the camera's position.

[0076] Specifically, step S32 includes steps S321 to S322, each of which is as follows:

[0077] Step S321: Convert the second coordinate information of the external damage hazard pixel in the camera coordinate system into homogeneous coordinate form, i.e. [x,y,z,1].

[0078] Step S322: Project the points in homogeneous coordinate form through the camera intrinsic parameter matrix to obtain two-dimensional coordinates [u,v,1] on the image plane. The camera intrinsic parameter matrix is ​​a matrix composed of the camera's intrinsic parameters.

[0079] Step S323: Based on the camera's extrinsic parameter matrix, the two-dimensional coordinates [u,v,1] are transformed into three-dimensional coordinates [xw,yw,zw] in the world coordinate system, which is the third coordinate information. The camera's extrinsic parameter matrix is ​​a matrix composed of the camera's extrinsic parameters.

[0080] It is important to note that when performing coordinate calculations, the influence of camera distortion parameters on the coordinates needs to be considered, which can be addressed through distortion correction methods.

[0081] Step S33: Using 3D point cloud technology, the third coordinate information is converted into first point cloud data in a spatial coordinate system to locate the external damage hazards of the transmission tower.

[0082] As a preferred solution, the process for obtaining the "coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system" mentioned in step S3 includes steps S01 to S05, and the specific steps are as follows:

[0083] Step S01: Calibrate the camera to obtain its intrinsic parameters.

[0084] In this embodiment, the camera's intrinsic parameters include the camera's focal length, principal point coordinates, and distortion parameters, and the distortion coefficients include radial distortion coefficients and tangential distortion coefficients.

[0085] Step S02: The area to be detected is scanned in real time using a laser scanner to obtain the second point cloud data of the area to be detected, and the area to be detected is captured in real time using a camera to obtain the first image.

[0086] Step S03: Using the PNP (Perspective-N-Points) algorithm, the camera's extrinsic parameters are calculated based on the second point cloud data and the first image.

[0087] The camera's extrinsic parameters include the camera's rotation matrix R and translation vector T. The rotation matrix R contains rotation parameters for three axes (ω, δ, θ), and the translation vector T contains translation parameters for three axes (Tx, Ty, Tz). The rotation matrix R and translation vector T together form a 3*4 matrix, used to achieve coordinate transformations between different coordinate systems.

[0088] Step S04: Extract the target region image to be identified from the first image, then perform feature extraction on the target region image to obtain several corresponding feature points, and obtain the second coordinate information of each feature point in the pixel coordinate system.

[0089] As a preferred embodiment, step S04 includes steps S041 to S042, and the specific steps are as follows:

[0090] Step S041: Perform object recognition on the first image and divide the region in the first image containing the reference target into a target region image.

[0091] It should be noted that the target area image corresponds to the target area in actual space.

[0092] Step S042: Preprocess the target region image, and then use feature extraction algorithms such as SIFT and SURF to extract features from the preprocessed target region image to obtain several feature points of the reference target, and obtain the second coordinate information of each feature point in the pixel coordinate system.

[0093] The reference target refers to the object within the target area. Preprocessing of the target area image includes image enhancement, edge detection, and binarization.

[0094] It's important to note that SIFT (Scale-Invariant Feature Transform) is a computer vision feature extraction algorithm used to detect and describe local features in images. SURF (Speeded Up Robust Features) is a robust image recognition and description algorithm. It's an efficient variant of SIFT, also extracting scale-invariant features, but using a different method, making it more efficient than SIFT.

[0095] Step S05: Based on the intrinsic parameters, extrinsic parameters, the second coordinate information corresponding to each feature point, and the third coordinate information of each feature point in the spatial coordinate system, the feature points are matched to obtain the correspondence between the second and third coordinate information of each feature point. Then, based on the correspondence between the second and third coordinate information of all feature points, a coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system is established. The third coordinate information of all feature points in the spatial coordinate system is the three-dimensional coordinate information of the reference target in actual space.

[0096] In this embodiment, point cloud data refers to a set of three-dimensional vectors generated by photographing various objects in the target area in various ways. The feature points of different objects have different three-dimensional coordinates in the point cloud data.

[0097] Please refer to Figure 3 This is a schematic diagram of a device for detecting and locating external damage hazards on power transmission towers, provided in an embodiment of the present invention. The device includes a regional imaging system M1, an external damage hazard detection system M2, and an external damage hazard location system M3. The specific components of each system are as follows:

[0098] The area imaging system M1 is used to capture images of the area to be detected in real time to obtain an image of the area to be identified; wherein the area to be detected contains power transmission towers, or the area to be detected is adjacent to power transmission towers;

[0099] The external damage hazard detection system M2 is used to input the image to be identified into the target detection model, so that the target detection model can detect external damage hazards in the image to be identified and output the detection result of whether there are pixels with external damage hazards in the image to be identified; wherein, the target detection model is obtained by training an initial model pre-built using the YOLOv4-tiny algorithm;

[0100] The M3 external damage hazard location system is used to locate the external damage hazard of the transmission tower when the detection result indicates that there are pixels with external damage hazards in the image to be identified. Based on the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, it converts the first coordinate information of the external damage hazard pixel in the pixel coordinate system into the first point cloud data in the spatial coordinate system, so as to complete the location of the external damage hazard of the transmission tower. The pixel coordinate system is established based on the image to be identified.

[0101] For the preferred option, please refer to Figure 3 The present invention provides a device for detecting and locating potential external damage to power transmission towers, which further includes a data acquisition system M4. The data acquisition system M4 specifically includes a calibration module 41, a scanning and imaging module 42, a calculation module 43, an extraction module 44, and a matching module 45, with each module described in detail below:

[0102] The calibration module 41 is used to calibrate the camera and obtain the camera's intrinsic parameters.

[0103] The scanning and imaging module 42 is used to scan the area to be detected in real time using a laser scanner to obtain the second point cloud data of the area to be detected, and to capture the area to be detected in real time using a camera to obtain the first image.

[0104] The calculation module 43 is used to calculate the camera's extrinsic parameters based on the second point cloud data and the first image using the PNP algorithm; wherein the camera's extrinsic parameters include the camera's rotation matrix and translation vector;

[0105] The extraction module 44 is used to extract the target region image to be identified from the first image, then perform feature extraction on the target region image to obtain several corresponding feature points, and obtain the second coordinate information of each feature point in the pixel coordinate system.

[0106] The matching module 45 is used to match each feature point according to the intrinsic parameters, extrinsic parameters, the second coordinate information corresponding to each feature point, and the third coordinate information of each feature point in the spatial coordinate system, to obtain the correspondence between the second coordinate information and the third coordinate information corresponding to each feature point, and then establish the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system based on the correspondence between the second coordinate information and the third coordinate information corresponding to all feature points.

[0107] As a preferred embodiment, the extraction module 44 specifically includes a partitioning unit 441 and an extraction unit 442, with each unit as follows:

[0108] The segmentation unit 441 is used to perform object recognition on the first image and to segment the region in the first image containing the reference target into a target region image.

[0109] The extraction unit 442 is used to preprocess the target region image, and then extract features from the preprocessed target region image to obtain several feature points of the reference target, and obtain the second coordinate information of each feature point in the pixel coordinate system; wherein, the preprocessing of the target region image includes image enhancement, edge detection and binarization.

[0110] As a preferred solution, the external damage hazard location system M3 specifically includes a first coordinate transformation module 31, a second coordinate transformation module 32, and a third coordinate transformation module 33, with each module described in detail below:

[0111] The first coordinate transformation module 31 is used to transform the first coordinate information of the pixel with external damage risk in the pixel coordinate system into the second coordinate information in the camera coordinate system according to the intrinsic parameters of the camera when the detection result is that there are pixels with external damage risk in the image to be identified. The camera is used to capture the area to be detected in real time to obtain the image to be identified, and the camera coordinate system is a coordinate system with the optical center of the camera as the origin.

[0112] The second coordinate transformation module 32 is used to transform the second coordinate information into the third coordinate information in the world coordinate system based on the camera's position information and shooting angle.

[0113] The third coordinate transformation module 33 is used to convert the third coordinate information into first point cloud data in the spatial coordinate system using three-dimensional point cloud technology, so as to complete the location of external damage hazards of transmission towers.

[0114] As a preferred embodiment, the area shooting system M1 specifically includes a shooting module 11 and a preprocessing module 12, the details of which are as follows:

[0115] The imaging module 11 is used to take real-time pictures of the area to be detected using a camera installed at the transmission tower to obtain an initial image;

[0116] The preprocessing module 12 is used to preprocess the initial image to obtain the image to be identified of the region to be detected; wherein, the preprocessing of the initial image includes image denoising, image enhancement and edge detection.

[0117] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0118] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0119] This invention provides a method and apparatus for detecting and locating external damage hazards on power transmission towers. It utilizes the YOLOv4-tiny algorithm, which boasts small scale and high detection accuracy, to construct an initial model. This initial model is then trained to obtain a target detection model for detecting external damage hazards in the image to be identified. The image to be identified, captured in real-time in the area to be identified, is then input into the target detection model. This allows the target detection model to quickly detect external damage hazards in the image with fewer parameters and less computational resources. Furthermore, it avoids the loss of detection data and storage space caused by complex computational processes during the detection process, thereby improving the accuracy and efficiency of external damage hazard detection. Additionally, by establishing a pixel coordinate system, the positional information of the detected external damage hazard pixels in the image to be identified can be accurately represented. Based on the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, the first coordinate information in the pixel coordinate system is transformed into first point cloud data in the spatial coordinate system. This enables precise location of the external damage hazard in actual space, facilitating targeted repair and maintenance of power transmission towers.

[0120] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for detecting and locating potential external damage to power transmission towers, characterized in that, include: The area to be detected is captured in real time to obtain an image of the area to be detected; wherein the area to be detected contains the power transmission tower, or the area to be detected is adjacent to the power transmission tower; The image to be identified is input into the target detection model, so that the target detection model can detect external damage hazards in the image to be identified and output the detection result of whether there are pixels with external damage hazards in the image to be identified; wherein, the target detection model is obtained by training an initial model pre-built using the YOLOv4-tiny algorithm; When the detection result indicates that there are pixels with potential external damage in the image to be identified, the first coordinate information of the pixels with potential external damage in the pixel coordinate system is converted into the first point cloud data in the spatial coordinate system based on the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, so as to complete the location of the potential external damage of the transmission tower; wherein, the pixel coordinate system is established based on the image to be identified; In the initial model, CSPD DarkNet et 5 3 _ tiny is used as the backbone feature extraction network layer. The backbone feature extraction network layer consists of a first DarkNet convolution, a second DarkNet convolution, a first residual block, a second residual block, and a third DarkNet convolution; the LeakyReLU activation function is used; the neck network layer uses a feature pyramid structure, which upsamples the last effective feature layer after convolution, and then stacks and convolves it with the previous effective feature layer. Pixels with potential for external damage are those that are susceptible to damage from external forces.

2. The method for detecting and locating potential external damage to transmission towers according to claim 1, characterized in that, The coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system is obtained as follows: The camera is calibrated to obtain its intrinsic parameters; The area to be detected is scanned in real time using a laser scanner to obtain the second point cloud data of the area to be detected, and the area to be detected is captured in real time using a camera to obtain the first image; Using the PNP algorithm, the extrinsic parameters of the camera are calculated based on the second point cloud data and the first image; wherein, the extrinsic parameters of the camera include the camera's rotation matrix and translation vector; The target region image to be identified is extracted from the first image, and then the target region image is subjected to feature extraction to obtain a number of corresponding feature points, and the second coordinate information of each feature point in the pixel coordinate system is obtained. Based on the intrinsic parameters, the extrinsic parameters, the second coordinate information corresponding to each feature point, and the third coordinate information of each feature point in the spatial coordinate system, each feature point is matched to obtain the correspondence between the second coordinate information and the third coordinate information corresponding to each feature point. Then, based on the correspondence between the second coordinate information and the third coordinate information corresponding to all feature points, a coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system is established.

3. The method for detecting and locating potential external damage to transmission towers according to claim 2, characterized in that, The process involves extracting the target region image to be identified from the first image, then performing feature extraction on the target region image to obtain several corresponding feature points, and acquiring the second coordinate information of each feature point in the pixel coordinate system. Specifically, this process includes: Object recognition is performed on the first image, and the region in the first image containing the reference target is divided into a target region image; The target region image is preprocessed, and then features are extracted from the preprocessed target region image to obtain several feature points of the reference target, and the second coordinate information of each feature point in the pixel coordinate system is obtained; wherein, the preprocessing of the target region image includes image enhancement, edge detection and binarization.

4. The method for detecting and locating potential external damage to transmission towers according to claim 1, characterized in that, When the detection result indicates the presence of a pixel with potential external damage in the image to be identified, based on the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, the first coordinate information of the pixel with potential external damage in the pixel coordinate system is converted into the first point cloud data in the spatial coordinate system to complete the location of the potential external damage to the transmission tower. Specifically: When the detection result indicates that there are pixels with potential external damage in the image to be identified, the first coordinate information of the pixels with potential external damage in the pixel coordinate system is converted into the second coordinate information in the camera coordinate system according to the camera's intrinsic parameters. Based on the camera's position information and shooting angle, the second coordinate information is transformed into a third coordinate information in the world coordinate system; Using 3D point cloud technology, the third coordinate information is converted into first point cloud data in the spatial coordinate system to locate the external damage hazard of the transmission tower. The camera is used to capture images of the area to be detected in real time to obtain the image to be identified, and the camera coordinate system is a coordinate system with the optical center of the camera as the origin.

5. The method for detecting and locating potential external damage to transmission towers according to claim 1, characterized in that, The process of capturing real-time images of the area to be detected to obtain an image of the area to be identified is as follows: Using a camera installed on the transmission tower, the area to be detected is photographed in real time to obtain an initial image; The initial image is preprocessed to obtain the image to be identified for the region to be detected; The preprocessing of the initial image includes image denoising, image enhancement, and edge detection.

6. A device for detecting and locating potential external damage to transmission towers, characterized in that, include: An area imaging system is used to capture images of an area to be detected in real time to obtain an image of the area to be identified; wherein the area to be detected includes the power transmission tower, or the area to be detected is adjacent to the power transmission tower; An external damage hazard detection system is used to input the image to be identified into a target detection model, so that the target detection model can detect external damage hazards in the image to be identified and output the detection result of whether there are external damage hazard pixels in the image to be identified; wherein, the target detection model is obtained by training an initial model pre-built using the YOLOv4-tiny algorithm; An external damage hazard location system is used to, when the detection result indicates the presence of external damage hazard pixels in the image to be identified, convert the first coordinate information of the external damage hazard pixels in the pixel coordinate system into first point cloud data in the spatial coordinate system based on the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system, so as to complete the location of external damage hazards of the transmission tower; wherein, the pixel coordinate system is established based on the image to be identified; In the initial model, CSPD DarkNet et 5 3 _ tiny is used as the backbone feature extraction network layer. The backbone feature extraction network layer consists of a first DarkNet convolution, a second DarkNet convolution, a first residual block, a second residual block, and a third DarkNet convolution; the LeakyReLU activation function is used; the neck network layer uses a feature pyramid structure, which upsamples the last effective feature layer after convolution, and then stacks and convolves it with the previous effective feature layer. Pixels with potential for external damage are those that are susceptible to damage from external forces.

7. The device for detecting and locating potential external damage to transmission towers according to claim 6, characterized in that, It also includes a data acquisition system; wherein, the data acquisition system specifically includes: The calibration module is used to calibrate the camera and obtain its intrinsic parameters. The scanning and imaging module is used to scan the area to be detected in real time using a laser scanner to obtain the second point cloud data of the area to be detected, and to capture the area to be detected in real time using the camera to obtain the first image; The calculation module is used to calculate the extrinsic parameters of the camera based on the second point cloud data and the first image using the PNP algorithm; wherein the extrinsic parameters of the camera include the camera's rotation matrix and translation vector; The extraction module is used to extract the target region image to be identified from the first image, then perform feature extraction on the target region image to obtain a number of corresponding feature points, and obtain the second coordinate information of each feature point in the pixel coordinate system. The matching module is used to match each feature point according to the intrinsic parameters, the extrinsic parameters, the second coordinate information corresponding to each feature point, and the third coordinate information of each feature point in the spatial coordinate system, to obtain the correspondence between the second coordinate information and the third coordinate information corresponding to each feature point, and then establish the coordinate mapping relationship between the pixel coordinate system and the spatial coordinate system based on the correspondence between the second coordinate information and the third coordinate information corresponding to all feature points.

8. The device for detecting and locating potential external damage to transmission towers according to claim 7, characterized in that, The extraction module specifically includes: The segmentation unit is used to perform object recognition on the first image and to segment the region in the first image containing the reference target into a target region image. The extraction unit is used to preprocess the target region image, then extract features from the preprocessed target region image to obtain several feature points of the reference target, and obtain the second coordinate information of each feature point in the pixel coordinate system; wherein, the preprocessing of the target region image includes image enhancement, edge detection and binarization.

9. The device for detecting and locating potential external damage to transmission towers as described in claim 6, characterized in that, The external damage hazard location system specifically includes: The first coordinate transformation module is used to transform the first coordinate information of the pixel with external damage risk in the pixel coordinate system into the second coordinate information in the camera coordinate system according to the camera's intrinsic parameters when the detection result indicates that there is a pixel with external damage risk in the image to be identified. The camera is used to capture the area to be detected in real time to obtain the image to be identified, and the camera coordinate system is a coordinate system with the optical center of the camera as the origin. The second coordinate transformation module is used to transform the second coordinate information into third coordinate information in the world coordinate system based on the position information and shooting angle of the camera. The third coordinate transformation module is used to convert the third coordinate information into first point cloud data in the spatial coordinate system using three-dimensional point cloud technology, so as to complete the location of external damage hazards of the transmission tower.

10. The device for detecting and locating potential external damage to transmission towers as described in claim 6, characterized in that, The area shooting system specifically includes: The imaging module is used to capture images of the area to be detected in real time using a camera installed on the power transmission tower, so as to obtain an initial image; The preprocessing module is used to preprocess the initial image to obtain the image to be identified of the region to be detected; wherein, the preprocessing of the initial image includes image denoising, image enhancement and edge detection.

Images