A power transmission line detection method and related device

Abstract

This invention provides a method and related equipment for detecting power transmission lines, comprising: acquiring laser data and image data of a target power transmission line; extracting local and global features of the laser data using a point cloud encoder module; fusing the local and global features of the laser data with the features of the image data to obtain fused features; inputting the fused features into a YOLOv5 framework for target detection to obtain detection results; the point cloud encoder module includes a point cloud preprocessing unit, an encoder, and a decoder; the output of the point cloud preprocessing unit is connected to the input of the encoder, the output of the encoder is connected to the input of the decoder, and the output of the decoder is connected to the input of the YOLOv5 framework; this method effectively fuses laser data and image data, reduces the sensitivity of environmental factors around the line to enhance the robustness of line detection, and improves the accuracy and reliability of line detection.

Description

Technical Field

[0001] This invention relates to the field of power monitoring technology, and in particular to a method and related equipment for detecting power transmission lines. Background Technology

[0002] Because power transmission lines are widely distributed and operate in open-air environments for extended periods, they are frequently affected by changes in the surrounding environment and nature. Therefore, the inspection of power transmission lines plays a crucial role in ensuring the safe and reliable operation of the power grid. Regular line inspections are necessary to promptly detect and address any anomalies, ensuring the safe operation of the transmission lines. Current power transmission line monitoring technologies, such as 4G wireless video monitoring, single-point infrared or laser beam blocking alarm detection, and millimeter-wave radar monitoring, share some common drawbacks. These include missed timed snapshots, inability to provide on-site alarms, inability to accurately identify external damage sources, high false alarm rates, numerous blind spots, susceptibility to environmental factors and birds, cumbersome equipment installation, low resolution, and limited detection range.

[0003] In target detection, existing technologies typically employ the YOLOv5 framework, which learns image features through convolutional layers to automatically identify and extract features of different objects or components, thus achieving target detection. However, relying solely on image data acquired by camera devices for power line detection can be affected by factors such as changes in lighting, occlusion, and background interference. Especially in complex environmental conditions, such as the presence of trees, buildings, and clouds, the visibility of the power line may be limited, leading to decreased detection accuracy. Due to the complex structure and variations of power transmission lines, image data acquired solely by camera devices is insufficient to accurately identify the shape, orientation, and details of the line. This can result in inaccurate detection results or even misclassification as other objects. The resolution and field of view of image data acquired by camera devices also limit effective detection of distant lines. At greater distances, lines may appear blurry and their edge information indistinct, posing a challenge to power line detection. Summary of the Invention

[0004] This invention provides a method and related equipment for detecting power transmission lines, the purpose of which is to enhance the robustness of line detection, thereby improving the accuracy and reliability of line detection.

[0005] To achieve the above objectives, the present invention provides a method for detecting power transmission lines, comprising:

[0006] Step 1: Acquire laser data collected by laser equipment and image data collected by camera equipment for the target transmission line;

[0007] Step 2: Construct a local similarity domain for each point cloud in the laser data using the point cloud encoder module, and encode the point clouds in the local similarity domain to obtain the local features of the laser data; Divide the laser data into multiple spherical regions using the point cloud encoder module, encode the point clouds in each spherical region to obtain the global features of the laser data, and fuse the local features of the laser data, the global features of the laser data, and the features of the image data to obtain the fused features;

[0008] Step 3: Input the fused features into the Yolov5 framework for object detection to obtain the detection results;

[0009] The point cloud encoder module includes a point cloud preprocessing unit, an encoder, and a decoder;

[0010] The output of the point cloud preprocessing unit is connected to the input of the encoder, the output of the encoder is connected to the input of the decoder, and the output of the decoder is connected to the input of the Yolov5 framework.

[0011] Furthermore, prior to step 2, the following steps are also included:

[0012] The image data is preprocessed to obtain preprocessed image data;

[0013] The preprocessed image data is input into the constructed point cloud encoder module.

[0014] Furthermore, step 2 includes:

[0015] Step 21: Input the laser data into the point cloud preprocessing unit for preprocessing to obtain point cloud data;

[0016] Step 22: Input the point cloud data into the encoder for feature extraction. Construct a local similarity domain for each point cloud in the point cloud data through the encoder, and encode the point clouds in the local similarity domain to obtain the local features of the laser data.

[0017] Step 23: Divide the laser data into multiple spherical regions using an encoder, encode the point cloud in each spherical region, and obtain the global features of the laser data;

[0018] Step 24: Input the preprocessed image data into the encoder for feature extraction to obtain a low-level feature map of the image data. The low-level feature map is used to characterize the edge features and texture features of the image data.

[0019] Step 25: Input the local features of the point cloud data, the global features of the point cloud data, and the low-level feature map of the image data into the decoder for feature fusion to obtain the fused features.

[0020] Furthermore, step 21 includes:

[0021] The laser data is input into the point cloud preprocessing unit for smoothing to obtain effective data;

[0022] Distribute the valid data into multiple clusters and set up a grid within each cluster;

[0023] The valid data located at the center of the grid is retained to obtain point cloud data.

[0024] Furthermore, the point cloud data is input into the encoder for feature extraction, including:

[0025] In the encoder, for each point cloud in the point cloud data, a local similarity domain is constructed based on the coordinate information and clustering information of the point cloud, and the clustering weight is calculated.

[0026] Feature encoding is performed on the point cloud in each local similarity domain to obtain the features of each point cloud;

[0027] The features of each point cloud are mapped to a high-dimensional space to obtain the relational features within each local similarity domain;

[0028] By aggregating the relational features within all local similarity domains, a local feature representation of the point cloud data is obtained, which is used to characterize the local feature information of the point cloud data.

[0029] Point cloud data is divided into multiple spherical regions with different resolutions by using a hierarchical sampling method.

[0030] Feature encoding is performed on the point cloud of each spherical region to obtain the point cloud features of each spherical region;

[0031] The point cloud features of each spherical region are aggregated to obtain the global feature representation of each spherical region;

[0032] The global feature representations of all spherical regions are aggregated to obtain the global features of the point cloud data.

[0033] The global features of point cloud data are mapped and encoded to obtain the global feature representation of the point cloud data;

[0034] By fusing local and global feature representations, a comprehensive feature of the point cloud data is obtained.

[0035] Furthermore, the image data is input into the encoder for feature extraction, including:

[0036] The encoder uses a pre-trained convolutional neural network to extract features from image data, resulting in low-level feature maps that represent local details of the image data.

[0037] Furthermore, step 25 includes:

[0038] The comprehensive features of the point cloud data and the low-level feature map are input into the decoder;

[0039] The comprehensive features of point cloud data are initially fused with low-level feature maps using a convolutional neural network to obtain the first fused feature.

[0040] The first fused feature is upsampled to expand the low-resolution feature representation into a high-resolution feature representation;

[0041] The high-resolution feature representations are fused and refined to obtain the fused feature representations.

[0042] Furthermore, the loss function of the Yolov5 framework is:

[0043] Total Loss=Box Loss+Class Loss+Object Loss+Laser Loss

[0044]

[0045] Where Box Loss represents the bounding box loss term, Class Loss represents the classification loss term, Object Loss represents the target loss term, Laser Loss represents the laser loss term, and λ laser The weights used to balance laser loss are represented by S, the size of the feature map, B, and the number of bounding boxes predicted for each cell. i Represents laser data, This represents the laser data corresponding to the predicted bounding box.

[0046] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a transmission line detection method.

[0047] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a power transmission line detection method.

[0048] The above-described solution of the present invention has the following beneficial effects:

[0049] This invention acquires laser data and image data of a target transmission line; inputs the laser data and image data into a constructed point cloud encoder module, fuses the features of the laser data and the image data to obtain fused features; inputs the fused features into a YOLOv5 framework for target detection to obtain detection results. Compared with existing technologies, this invention introduces laser data to provide geometric information of the transmission line, and laser data is highly sensitive to changes in illumination and occlusion, capable of penetrating a certain degree of obstruction, thus overcoming the shortcomings of traditional YOLOv5 algorithms that only use image data. This invention uses a point cloud encoder module to transform the point cloud information in the laser data into a feature representation form compatible with image data, enabling effective fusion of laser data and image data, reducing the sensitivity to environmental factors around the line to enhance the robustness of line detection, thereby improving the accuracy and reliability of line detection.

[0050] Other beneficial effects of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating an embodiment of the present invention. Detailed Implementation

[0052] To make the technical problems, solutions, and advantages of this invention clearer, a detailed description will be provided below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0053] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0054] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a locking connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0055] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0056] This invention addresses existing problems by providing a method and related equipment for detecting power transmission lines.

[0057] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting power transmission lines, including:

[0058] Step 1: Acquire laser data and image data of the target transmission line;

[0059] Step 2: Construct a local similarity domain for each point cloud in the laser data using the point cloud encoder module, and encode the point clouds in the local similarity domain to obtain the local features of the laser data; Divide the laser data into multiple spherical regions using the point cloud encoder module, encode the point clouds in each spherical region to obtain the global features of the laser data, and fuse the local features of the laser data, the global features of the laser data, and the features of the image data to obtain the fused features;

[0060] Step 3: Input the fused features into the Yolov5 framework for object detection to obtain the detection results;

[0061] The point cloud encoder module includes a point cloud preprocessing unit, an encoder, and a decoder;

[0062] The output of the point cloud preprocessing unit is connected to the input of the encoder, the output of the encoder is connected to the input of the decoder, and the output of the decoder is connected to the input of the Yolov5 framework.

[0063] In this embodiment of the invention, the laser data of the target transmission line is the three-dimensional coordinate information obtained by a laser scanner through the emission and reception of a laser beam, which is usually represented as a set of points (x, y, z); the image data of the target transmission line is a set of RGB images captured by a camera.

[0064] Specifically, before step 2, the following is also included:

[0065] The image data is preprocessed to obtain preprocessed image data;

[0066] The preprocessed image data is input into the constructed point cloud encoder module.

[0067] In this embodiment of the invention, the preprocessing methods include adjusting the image size and normalizing the image data to adapt it to the input requirements of the model.

[0068] Specifically, step 2 includes:

[0069] Step 21: Input the laser data into the point cloud preprocessing unit for preprocessing to obtain point cloud data;

[0070] Step 22: Input the point cloud data into the encoder for feature extraction. Construct a local similarity domain for each point cloud in the point cloud data through the encoder, and encode the point clouds in the local similarity domain to obtain the local features of the laser data.

[0071] Step 23: Divide the laser data into multiple spherical regions using an encoder, encode the point cloud in each spherical region, and obtain the global features of the laser data;

[0072] Step 24: Input the preprocessed image data into the encoder for feature extraction to obtain a low-level feature map of the image data. The low-level feature map is used for the edge features and texture features of the image data.

[0073] Step 25: Input the local features of the point cloud data, the global features of the point cloud data, and the low-level feature map of the image data into the decoder for feature fusion to obtain the fused features.

[0074] In the prior art, the structure of a point cloud encoder module is usually an encoder-decoder structure. In this embodiment of the invention, a point cloud preprocessing unit is designed before the encoder. The laser data is preprocessed by the point cloud preprocessing unit and then input into the encoder for feature encoding.

[0075] Specifically, step 21 includes:

[0076] The laser data is input into the point cloud preprocessing unit for smoothing to obtain effective data;

[0077] Distribute the valid data into multiple clusters and set up a grid within each cluster;

[0078] The valid data located at the center of the grid is retained to obtain point cloud data.

[0079] In this embodiment of the invention, the point cloud preprocessing unit uses a Gaussian filtering algorithm to smooth the laser data, removing noise and outliers. Gaussian filtering smooths the points by applying a Gaussian kernel function around each point cloud in the laser data, thus obtaining effective data.

[0080] In order to group point cloud data into sets with similar features, thereby identifying different targets or local structures and providing more accurate regions for subsequent processing, this embodiment of the invention uses the K-means clustering algorithm to distribute effective data into multiple different clusters. In order to reduce the density of effective data, thereby reducing computational complexity and improving processing efficiency, this embodiment of the invention sets a grid within each cluster.

[0081] Since the grid center point is a point with important features, only the grid center point is retained, while redundant points are reduced, and the retained grid center point is used as the point cloud data.

[0082] Specifically, point cloud data is input into the encoder for feature extraction, including:

[0083] In the encoder, for each point cloud in the point cloud data, a local similarity domain is constructed based on the coordinate information and clustering information of the point cloud, and the clustering weight is calculated.

[0084] Feature encoding is performed on the point cloud in each local similarity domain to obtain the features of each point cloud;

[0085] The features of each point cloud are mapped to a high-dimensional space to obtain the relational features within each local similarity domain;

[0086] By aggregating the relational features within all local similarity domains, a local feature representation of the point cloud data is obtained, which is used to characterize the local feature information of the point cloud data.

[0087] Point cloud data is divided into multiple spherical regions with different resolutions by using a hierarchical sampling method.

[0088] Feature encoding is performed on the point cloud of each spherical region to obtain the point cloud features of each spherical region;

[0089] The point cloud features of each spherical region are aggregated to obtain the global feature representation of each spherical region;

[0090] The global feature representations of all spherical regions are aggregated to obtain the global features of the point cloud data.

[0091] The global features of point cloud data are mapped and encoded to obtain the global feature representation of the point cloud data;

[0092] By fusing local and global feature representations, a comprehensive feature of the point cloud data is obtained.

[0093] In this embodiment of the invention, for each point cloud, a local similarity region is first calculated based on the coordinate information and clustering information of each point cloud. The specific process is as follows:

[0094] The K-means algorithm is used to cluster the point cloud data, and the clustering results are then divided into multiple distinct clusters. Assume the clustering results are C = C1, C2, ..., C... K C K This represents the k-th cluster;

[0095] For each cluster C K Choose a representative point as the center point of the cluster, denoted as c. k ;

[0096] For each center point c k According to each center point c k The number and location of point clouds in the local similarity region are used to construct the local similarity region for each point cloud, denoted as N(c k Simultaneously calculate C for each cluster. K Clustering weights w of all point clouds k , is represented as:

[0097]

[0098] Among them, |C K | represents cluster C K The number of midpoint clouds, This indicates the total number of point clouds.

[0099] For point clouds within locally similar domains, spherical convolution and multi-layer perceptron are used to implement feature encoding. The spherical convolution operation utilizes the topological results and adjacency relationships of the point cloud data. The convolution kernel is used to perform a convolution operation with the features of each point cloud, thereby updating the feature representation of the point cloud. The expression is as follows:

[0100]

[0101] Where, f′ i Let g(c) represent the features of the i-th point cloud after convolution, N represent the number of point clouds in the local similarity domain, K represent the normalization constant, g(·) represent the kernel function, and dist(c) represent the features of the i-th point cloud after convolution. i c k ) represents point cloud c i With center point c k The distance between them, f jRepresenting the features of the j-th point cloud in a local similarity domain, the spherical convolution kernel uses a Gaussian weight function and clustering weights to define the kernel weights w. ik where the weight w ik From point cloud c i With center point c k The weights w of the spherical convolution kernel are determined by the distance between them and the clustering weights. ik It can be calculated using the following formula:

[0102]

[0103] Where σ represents a parameter controlling the range of the convolution kernel, dist(c j c k ) represents point cloud c j To the center point c k The distance between them.

[0104] The features of the point cloud are mapped to a high-dimensional space through a fully connected layer in a multilayer perceptron to capture relational features and local structures within local similarity domains. The formula is as follows:

[0105] f″ i =ReLU(W1·f′) i +b1)

[0106] Where, f″ i The features are represented by the multilayer perceptron transformation, W1 represents the weights of the multilayer perceptron, b1 represents the bias of the multilayer perceptron, and ReLU(·) represents the activation function.

[0107] Next, max pooling is used to aggregate the captured relational features within the local similarity domain, forming a fixed-length local feature vector. This vector represents the local feature information of the point cloud, as shown in the following expression:

[0108] F local =max(f″ i )

[0109] Among them, F local This represents the local feature representation after pooling.

[0110] Specifically, this embodiment of the invention further performs global feature extraction on the point cloud data, as follows:

[0111] First, the point cloud data is divided into multiple spherical regions of different resolutions using hierarchical sampling. Different resolution regions represent different receptive field ranges. Feature encoding is performed on the point cloud data within each spherical region, similar to local feature extraction. Spherical convolution and a multilayer perceptron (MLP) are used to extract and encode features for each region. Then, max pooling is used to aggregate the features from each region into a single feature vector, representing the global features of that region. Subsequently, features from regions of different resolutions are aggregated. This typically involves upsampling lower-resolution features to higher resolutions and fusing features from all regions to obtain the global features of the entire point cloud data. Finally, the global features are mapped and encoded using a MLP to obtain the global feature vector of the point cloud data. Let the input point cloud data be P, and the global features obtained after passing through the MLP are F. global The expression is as follows:

[0112] F global =MLP global (P)

[0113] During the global feature aggregation stage, global feature F glocal Will be related to local feature F local The data is fused to obtain the comprehensive features of the point cloud data, taking into account both the local and global features of the point cloud data.

[0114] Specifically, the image data is input into the encoder for feature extraction, including:

[0115] The encoder uses a pre-trained convolutional neural network to extract features from image data, resulting in low-level feature maps that represent local details of the image data.

[0116] In this embodiment of the invention, a pre-trained convolutional neural network (ResNet) is used to extract features from the preprocessed image data. A series of convolution and pooling operations are performed on the image through convolutional layers to extract low-level feature maps from the image data. The reason for usually choosing to extract lower-level feature maps is that these lower-level features have more local detail information.

[0117] Specifically, step 25 includes:

[0118] The comprehensive features of the point cloud data and the low-level feature map are input into the decoder;

[0119] The comprehensive features of point cloud data are initially fused with low-level feature maps using a convolutional neural network to obtain the first fused feature.

[0120] The first fused feature is upsampled to expand the low-resolution feature representation into a high-resolution feature representation;

[0121] The high-resolution feature representations are fused and refined to obtain the fused feature representations.

[0122] In the decoder of this embodiment, multiple fusion modules are introduced, and each fusion module consists of multiple layers. The features of each layer come from the feature representations of the encoder and image data at different layers. In each layer, a convolutional neural network is used to fuse the point cloud features extracted by the encoder and the features of the image data to obtain a more comprehensive and richer first fusion result. In the decoder, feature upsampling is performed layer by layer to gradually expand the low-resolution feature mapping to the high-resolution feature mapping in order to recover the detailed information of the transmission line. In the fusion module at each layer, the feature representation is gradually fused and enhanced through multiple fusion and refinement operations to obtain the fused feature representation, thereby improving the accuracy and robustness of line detection. Finally, the fused feature representation is input into the Yolov5 framework.

[0123] Through the processing steps of the encoder and decoder described above, the point cloud encoder module in this invention can extract key features from laser data. These features are typically multi-dimensional vectors with a structure and format similar to image data. These vectors are then fused with features from the image data to generate comprehensive features that can be seamlessly integrated with the Yolov5 framework to achieve line detection and recognition tasks. This fused feature representation provides more comprehensive and richer information. Through the design and optimization of the point cloud encoder module, laser data can be effectively converted into feature representations compatible with image data, thus providing more comprehensive and accurate information for subsequent line detection and recognition tasks. The introduction of the point cloud encoder module can compensate for the shortcomings of traditional image data, fully utilize the geometric information of laser data, and improve the accuracy and robustness of line detection.

[0124] Because the traditional YOLOv5 framework primarily relies on image data for object detection, it may fail to accurately detect targets lacking obvious texture or with low contrast in certain scenarios. Furthermore, it doesn't fully utilize information from other data sources such as laser data, potentially missing crucial object detection information in some situations. Under conditions of significant environmental variation, such as birds in flight or changes in lighting, the traditional YOLOv5 framework is prone to false positives or false negatives. Introducing a new loss function and combining it with laser data can overcome some of the shortcomings of the traditional YOLOv5 framework, improving the accuracy and robustness of input line detection. The loss function of the traditional YOLOv5 framework mainly consists of three parts: Bounding Box Loss, Classification Loss, and Objectness Loss. The following is a detailed explanation and formula representation of each loss function:

[0125] (1) Bounding box loss:

[0126] The bounding box loss measures the difference between the predicted bounding box's position and size and the true bounding box. The Yolov5 framework uses Mean Square Error (MSE) as the bounding box loss function, expressed as:

[0127]

[0128] Where, λ coord Let S represent the weights used to balance the bounding box loss, S represent the size of the feature map, and B represent the number of bounding boxes predicted per cell. i y i ω i h i () represents the predicted bounding box coordinates and dimensions. This represents the actual bounding box coordinates and dimensions.

[0129] (2) Classification loss:

[0130] The classification loss measures the accuracy of the target classification. The YOLOv5 framework uses cross-entropy loss as the classification loss function, expressed as:

[0131]

[0132] Where, λ clS represents the weighting coefficients used to balance the classification loss, c represents the number of target classes, and p i (c) represents the predicted probability of the target class c. This represents the probability of the true target category c.

[0133] (3) Target loss

[0134] The target loss measures the accuracy of predicting whether the bounding box contains the target object. The Yolov5 framework uses binary cross-entropy loss as the target loss function, expressed as:

[0135]

[0136] Where, λ obj C represents the weighting coefficients used to balance the target loss. i This indicates the confidence level of whether the predicted bounding box i contains the target. Indicates whether the actual bounding box i contains the target's label.

[0137] The loss function of the traditional Yolov5 framework is the weighted sum of the three loss functions mentioned above.

[0138] This invention embodiment is based on the traditional Yolov5 framework loss function, incorporating a laser data loss term to constrain the matching degree between the predicted bounding box and the laser data. Assuming the laser data contains target location information, this invention embodiment can use mean squared error (MSE) as a measure of laser data loss. Specifically, assuming the laser data is L... i Then the laser data loss term can be defined as:

[0139]

[0140] Where, λ laser L represents the weighting coefficient used to balance laser data loss. i Represents laser data, This represents the laser data corresponding to the predicted bounding box. The loss function constrains the distance between the center of the predicted bounding box and the center of the laser data, making the predicted bounding box closer to the position of the laser data.

[0141] The final loss function can be defined as a weighted sum of the traditional loss function of the Yolov5 framework and the laser data loss:

[0142] Total Loss=Box Loss+Class Loss+Object Loss+Laser Loss

[0143] Where Box Loss represents the bounding box loss term, Class Loss represents the classification loss term, and Object Loss represents the target loss term.

[0144] This invention, by incorporating a laser data loss term, improves the loss function by better constraining the matching degree between the predicted bounding box and the laser data, thereby enhancing the detection accuracy of external damage hazards to transmission lines. Compared to the traditional YOLOv5, the improved loss function more effectively utilizes the information from the laser data, helping to improve the detection accuracy of external damage hazards to transmission lines, reduce false positive and false negative rates, and improve the model's performance and robustness.

[0145] This invention combines and optimizes the traditional loss function of the YOLOv5 framework, the laser data loss, and the loss of the point cloud encoder module; it uses optimization algorithms such as gradient descent for training and updates network parameters through backpropagation; during training, the fused data is provided as input to the network, and corresponding label data is provided for calculating the loss function; based on the feedback of the loss function, the network parameters are iteratively optimized until a preset stopping condition is reached.

[0146] This invention acquires laser data and image data of a target transmission line; inputs the laser data and image data into a point cloud encoder module, fuses the features of the laser data and the image data to obtain fused features; inputs the fused features into a YOLOv5 framework for target detection to obtain detection results. Compared with existing technologies, this invention introduces laser data to provide geometric information of the transmission line, and laser data is highly sensitive to changes in illumination and occlusion, capable of penetrating a certain degree of obstruction, thus overcoming the shortcomings of traditional YOLOv5 algorithms that only use image data. This invention uses a point cloud encoder module to transform the point cloud information in the laser data into a feature representation compatible with image data, enabling effective fusion of laser data and image data, reducing the sensitivity to environmental factors around the line to enhance the robustness of line detection, thereby improving the accuracy and reliability of line detection.

[0147] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements a power transmission line detection method.

[0148] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the embodiments of the present invention can implement all or part of the processes in the methods described above by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a building device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0149] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0150] This invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a power transmission line detection method.

[0151] It should be noted that the terminal device can be a mobile phone, tablet computer, laptop computer, Ultra-mobile Personal Computer (UMPC), netbook, Personal Digital Assistant (PDA), etc. For example, the terminal device can be a station (ST) in a WLAN, a cellular phone, cordless phone, Session Initiation Protocol (SIP) phone, Wireless Local Loop (WLL) station, PDA, handheld device with wireless communication capabilities, computing device or other processing device connected to a wireless modem, computer, laptop computer, handheld communication device, handheld computing device, satellite wireless device, etc. The embodiments of this invention do not limit the specific type of terminal device.

[0152] The processor referred to can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0153] In some embodiments, the memory may be an internal storage unit of the terminal device, such as a hard drive or RAM. In other embodiments, the memory may be an external storage device of the terminal device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital Card (SD), or Flash Card. Furthermore, the memory may include both internal and external storage units of the terminal device. The memory is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or will be output.

[0154] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting transmission lines, characterized in that, include: Step 1: Acquire laser data collected by laser equipment and image data collected by camera equipment for the target transmission line; Step 2: Construct a local similarity domain for each point cloud in the laser data using the point cloud encoder module, and encode the point clouds in the local similarity domains to obtain the local features of the laser data; divide the laser data into multiple spherical regions using the point cloud encoder module, encode the point clouds in each spherical region to obtain the global features of the laser data; fuse the local features of the laser data, the global features of the laser data, and the features of the image data to obtain the fused features, including: Step 21: Input the laser data into the point cloud preprocessing unit for preprocessing to obtain point cloud data; Step 22: Input the point cloud data into the encoder for feature extraction, construct a local similarity domain for each point cloud in the point cloud data through the encoder, and encode the point clouds in the local similarity domain to obtain the local features of the laser data; Step 23: Divide the laser data into multiple spherical regions using the encoder, and encode the point cloud in each spherical region to obtain the global features of the laser data; Step 24: Input the preprocessed image data into the encoder for feature extraction to obtain a low-level feature map of the image data. The low-level feature map is used to characterize the edge features and texture features of the image data, including: In the encoder, for each point cloud in the point cloud data, a local similarity domain of each point cloud is constructed based on the coordinate information and clustering information of the point cloud, and the clustering weight is calculated. Feature encoding is performed on the point cloud in each of the local similarity domains to obtain the features of each point cloud; The features of each point cloud are mapped to a high-dimensional space to obtain the relational features within each local similarity domain; By aggregating the relational features within all local similarity domains, a local feature representation of the point cloud data is obtained, which is used to characterize the local feature information of the point cloud data. The point cloud data is divided into multiple spherical regions with different resolutions by using a hierarchical sampling method. Feature encoding is performed on the point cloud of each spherical region to obtain the point cloud features of each spherical region; The point cloud features of each spherical region are aggregated to obtain the global feature representation of each spherical region; The global feature representations of all spherical regions are aggregated to obtain the global features of the point cloud data. The global features of the point cloud data are mapped and encoded to obtain a global feature representation of the point cloud data; The local feature representation and the global feature representation are fused to obtain the comprehensive features of the point cloud data; Step 25: Input the local features of the point cloud data, the global features of the point cloud data, and the low-level feature map of the image data into the decoder for feature fusion to obtain the fused features; Step 3: Input the fused features into the Yolov5 framework for target detection to obtain the detection results.

2. The transmission line testing method according to claim 1, characterized in that, Before step 2, the following is also included: The image data is preprocessed to obtain preprocessed image data; The preprocessed image data is input into the constructed point cloud encoder module.

3. The transmission line testing method according to claim 1, characterized in that, Step 21 includes: The laser data is input into the point cloud preprocessing unit for smoothing to obtain valid data; The valid data is distributed into multiple clusters, and a grid is set within each cluster; The valid data located at the center of the grid is retained to obtain point cloud data.

4. The transmission line testing method according to claim 1, characterized in that, The image data is input into the encoder for feature extraction, including: The encoder uses a pre-trained convolutional neural network to extract features from the image data, obtaining a low-level feature map to characterize the local details of the image data.

5. The transmission line testing method according to claim 4, characterized in that, Step 25 includes: The comprehensive features of the point cloud data and the low-level feature map are input into the decoder; The comprehensive features of the point cloud data are initially fused with the low-level feature map using a convolutional neural network to obtain the first fused feature; The first fused feature is upsampled to expand the low-resolution feature representation into a high-resolution feature representation; The high-resolution feature representations are fused and refined to obtain the fused feature representations.

6. The transmission line testing method according to claim 1, characterized in that, The loss function of the Yolov5 framework is: ； ； in, Represents the bounding box loss term. Represents the classification loss term. Represents the target loss term. Represents the laser loss term. This represents the weighting coefficient used to balance laser loss. Indicates the size of the feature map. This indicates the number of bounding boxes predicted for each cell. Represents laser data, This represents the laser data corresponding to the predicted bounding box.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the transmission line detection method as described in any one of claims 1 to 6.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the transmission line detection method as described in any one of claims 1 to 6.

Images