Lightweight power transmission channel hidden danger ranging method and hidden danger early warning device

Abstract

A light-weight power transmission channel hidden danger ranging method and a hidden danger early warning device, the steps are: S1, obtain the two-dimensional image data and three-dimensional point cloud data of the power transmission channel background, construct the coordinate transformation and mapping relationship of each pixel coordinate in the two-dimensional image and each three-dimensional point coordinate in the background point cloud, and select at least five groups of feature points to solve the camera pose using the EPnP algorithm; S2, collect and preprocess the external damage hidden danger target image data set, train the improved YOLOv4-tiny real-time detection model, which is used for detecting and identifying the external damage hidden danger target that may appear in the power transmission channel; S3, receive the real-time image of the power transmission channel and detect the corresponding hidden danger target type, calculate the Euclidean distance between the target object and the power transmission line according to the three-dimensional point coordinates of the target object and the three-dimensional point coordinates of the power transmission line, and determine whether to issue a warning according to the safety distance in the corresponding national standard. The method effectively improves the accuracy of the hidden danger target recognition of the power transmission channel, improves the detection and recognition efficiency, reasonably utilizes the operation resources, and is suitable for wide promotion.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of power transmission line hidden danger target inspection, and particularly relates to a lightweight power transmission channel hidden danger ranging method and a hidden danger early warning device. BACKGROUND

[0002] As an important part of the power system, the safe and stable operation of the power transmission line is crucial for the reliable power supply of the power grid. In the power grid construction project, there are problems related to the "three spans" of the power transmission line, i.e., the power transmission and distribution line crossing the railway, the first-class and above highway and the important power transmission channel area. The "three span" problem is a key control index problem of the project, and if not handled properly, it may lead to a major power grid safety event, even a public safety event. At the same time, with the large-scale development of infrastructure construction, the incidence of power grid accidents caused by the external damage of construction machinery and equipment to the power transmission line has also gradually increased. In order to effectively protect the safety of the power grid and its crossing (adjacent) facilities, the power grid and the railway, highway and other departments have successively introduced regulations, specifications and industry regulations for the above-mentioned related problems. Therefore, effectively identifying hidden danger targets and quantitatively and qualitatively determining the risk level according to the specific situation are the key work in the operation and maintenance process of the power transmission line.

[0003] The traditional inspection and maintenance of the power transmission line mainly relies on regular manual inspection, and this method has many drawbacks such as weak sense of responsibility of the inspection personnel, limited equipment level, and blind area of inspection caused by empiricism misjudgment. With the development of science and technology and the promotion of intelligent inspection, the use of unmanned aerial vehicle inspection can solve some shortcomings of the traditional inspection method limited by manual work. However, the unmanned aerial vehicle inspection also has limitations such as high economic cost, high operation difficulty, limited flight distance, and many uncertain factors in the wild, which is not suitable for large-scale promotion. Under this background, using a monocular camera as a visual terminal and using computer vision technology to extract information from the detected image to accurately identify obstacles and solve the distance between the obstacles and the power transmission channel is an important direction of the research on the prevention of external damage of the power transmission channel.

[0004] With the development of deep learning, the intelligent early warning method for preventing external damage based on deep learning has also gradually begun to stand out. While the algorithm performance of convolutional neural network is gradually improved, the number of network layers is also increasing, and the network model contains a large number of weight parameters, which requires high device storage; in the actual application process, in order to meet the application standard, the processor performance requirement is also very high; at the same time, there are problems such as slow detection speed and poor detection accuracy. SUMMARY

[0005] In view of the technical problems existing in the background art, the lightweight power transmission channel hidden danger ranging method provided by the present application constructs the coordinate transformation relationship of the power transmission channel background image coordinates and the point cloud data coordinates, receives the real-time image of the power transmission channel shot by the monocular camera, and uses a lightweight artificial intelligence algorithm to identify and detect the hidden danger targets that may appear in the power transmission channel, so as to calculate the distance between the target hidden danger object and the power transmission line, judge whether safety warning is needed, has universality and effectiveness, and can be applied to various scenes such as cities and mountainous areas, and ensures the safe and effective operation of the power system.

[0006] In order to solve the above technical problems, the present application adopts the following technical solutions to realize:

[0007] The lightweight power transmission channel hidden danger ranging method and hidden danger warning device, the steps are:

[0008] S1, obtain the two-dimensional image data and three-dimensional point cloud data of the power transmission channel background, construct the coordinate transformation and mapping relationship of each pixel coordinate in the two-dimensional image and each three-dimensional point coordinate in the background point cloud, and select at least five groups of feature points to solve the camera pose using the EPnP algorithm;

[0009] S2, collect and preprocess the external damage hidden danger target image data set, train an improved YOLOv4-tiny real-time detection model for detecting and identifying the external damage hidden danger targets that may appear in the power transmission channel;

[0010] S3, receive the real-time image of the power transmission channel and detect the corresponding hidden danger target type, and calculate the Euclidean distance between the target object and the power transmission line according to the three-dimensional point coordinates of the target object and the three-dimensional point coordinates of the power transmission line.

[0011] The camera equipment manufactured by using the monocular real-time ranging method provided by the present application can be provided with an alarm prompt module for judging whether to issue a warning according to the safety distance in the corresponding national standard. For example, if the calculated Euclidean distance is lower than the national standard corresponding to the geographical environment and voltage level of the power transmission channel, an alarm is given; or the target hidden danger object can be manually selected to further calculate the distance between the target hidden danger object and the power transmission line.

[0012] Preferably, the decomposition steps of step S1 are as follows:

[0013] S101, use a monocular camera to shoot and obtain two-dimensional image data of the power transmission channel;

[0014] S102, use a UAV loaded with a laser radar to obtain three-dimensional point cloud data of the power transmission channel background;

[0015] S103, construct the coordinate transformation relationship of the two-dimensional image and the background three-dimensional point cloud;

[0016] S104, select at least five groups of feature points, and use the EPnP algorithm to solve the pose of the monocular camera.

[0017] Preferably, in step S104, in order to ensure that the mapping relationship after coordinate transformation is good, at least five groups of positions with large turning degrees are selected as feature points, and an EPnP algorithm is used to solve the monocular camera external parameter matrix.

[0018] Specifically, an EPnP algorithm is used to solve the rotation matrix and the translation matrix in the coordinate correspondence relationship. Discrete points of the lead wire in the three-dimensional point cloud data are fitted into a three-dimensional curve, and a corresponding lead wire data mapping model is established according to the three-dimensional curve; wherein the lead wire should be an edge lead wire of a power transmission line channel; and three-dimensional coordinate position information of the lead wire is obtained according to the lead wire data mapping model.

[0019] Preferably, the decomposition step of step S2 is as follows:

[0020] S201, an image of an external damage hidden danger object is captured by using a monocular camera, and the captured image is processed into a data set suitable for a deep learning model detection;

[0021] S202, a Labelme software is used to label multiple types of external damage hidden danger targets in the data set, and the labeled data set is arranged according to a COCO data set format;

[0022] S203, on the basis of an existing data set, data images related to external damage hidden danger objects in a COCO public data set are added, and a training set, a verification set and a test set are arranged and generated;

[0023] S204, an improved YOLOv4-tiny real-time detection model is constructed according to the characteristics of the hidden danger target;

[0024] S205, the training set and the verification set are used to train and test the model.

[0025] Preferably, in step S201, the data set used should be a two-dimensional image of a power transmission channel captured by using a monocular camera in step S101, and a Labelme software is used for labeling, and the hidden danger target is divided into three types of car, truck and crane.

[0026] Preferably, in step S204, the improved YOLOv4-tiny real-time detection model uses a MobileNetV3-small network as a backbone network, instead of a CSPDarknet53-tiny network in the original YOLOv4-tiny model for feature extraction.

[0027] Preferably, in step S204, an improved SE (Squeeze-and-excitation) module is introduced in the backbone network MobileNetV3-small, and the network features are adjusted through a lightweight attention mechanism module to increase the weight of the effective channel.

[0028] Preferably, in step S204, the extracted feature information of different scales is fused by using a Spatial Pyramid Pooling (SPP) network and a Path Aggregation Network (PANet) to obtain three more effective feature fusion layers.

[0029] Among them, MobileNetV3-small is a version of MobileNetV3 network suitable for platforms with lower hardware performance; it is found during the calculation of the MobileNetV3-small network structure using Network Architecture Search (NAS) technology that using 5 5 depth convolution kernels instead of part of 3 3 depth convolution kernels can make the network obtain higher accuracy; in the structure of the bottleneck part of the network, the improved SE (Squeeze-and-excitation) module is introduced after the depth convolution, and the SE operation is performed before the point-by-point convolution, compared with the traditional SE module, which ensures the accuracy, improves the model performance, and reduces the parameter amount and calculation amount; using h-swish (hard version of swish) activation function instead of part of ReLU6 activation function, which improves the model precision while reducing the calculation amount. The h-swish function used is as follows:

[0030]

[0031] Preferably, in S3, the Euclidean distance is calculated by the following formula:

[0032]

[0033] Among them, and are two three-dimensional point coordinates, and d is the Euclidean distance.

[0034] According to the obtained Euclidean distance, it is judged whether it is lower than the minimum safety distance specified in the national standard under the voltage level of the power transmission line;

[0035] If the Euclidean distance is lower than the minimum safety distance, the server sends a warning signal;

[0036] The target hidden object can also be manually selected, and a minimum distance value between the hidden object and the power transmission line is output.

[0037] The monocular real-time ranging device adopts the light-weight power transmission channel hidden danger ranging method, and comprises an alarm prompt module for judging whether to issue a warning according to a safety distance in the corresponding national standard, that is, if the calculated Euclidean distance is lower than the corresponding national standard of the geographical environment and voltage level of the power transmission channel, the alarm is triggered.

[0038] The patent can achieve the following beneficial effects:

[0039] 1. By constructing the coordinate transformation and mapping relationship between the two-dimensional image and the three-dimensional point cloud, the real-time recognition and ranging of the hidden danger target of the power transmission channel can be realized by using a monocular camera. On the premise of meeting the detection accuracy requirement, the overall cost of the device is low, the installation difficulty is low, and the device has good economic benefits and promotion value.

[0040] 2. The target detection part uses an improved AttYOLOv4-tiny model. The model is compressed on the basis of the original YOLOv4, so that the model parameter quantity is one-tenth of that of YOLOv4, thereby solving the memory problem and further solving the speed problem without losing network performance. At the same time, the attention mechanism is introduced to dynamically weight the feature data of the region of interest by a weight coefficient, improve the network's attention to the key region, effectively solve the identification difficulty problem of small target and target being partially blocked, and improve the model detection efficiency.

[0041] 3. According to the corresponding national standard of the geographical environment and voltage level of the power transmission channel, the safety distance can be set. When the distance between the target hidden object and the power transmission line is lower than the safety distance, an early warning is given; the target hidden object can also be manually selected to calculate the distance between the target hidden object and the power transmission line, thereby meeting various needs in the inspection and maintenance work. BRIEF DESCRIPTION OF DRAWINGS

[0042] The application will be further described below in combination with the drawings and embodiments:

[0043] Figure 1 It is an operation flowchart of the application;

[0044] Figure 2 It is a conversion schematic diagram of the spatial mapping conversion relationship provided by the embodiment of the application;

[0045] Figure 3 It is a target detection model structure diagram of the improved AttYOLOv4-tiny provided by the embodiment of the application;

[0046] Figure 4 It is a MobileNetV3-small backbone feature extraction network structure diagram provided by the embodiment of the application;

[0047] Figure 5 A schematic diagram of the depth separable convolution structure in the MobileNetV3-small backbone feature extraction network provided by the embodiment of the application is shown. DETAILED DESCRIPTION

[0048] Embodiment 1

[0049] As shown in the preferred scheme, Figures 1 to 5 The light-weight power transmission channel hidden danger ranging method and hidden danger early warning device, the steps are:

[0050] S1, using a UAV loaded with a laser radar to obtain three-dimensional point cloud information of the power transmission channel, and using a monocular camera to shoot two-dimensional image information of the power transmission channel;

[0051] The coordinate transformation and mapping relationship between each pixel coordinate in the two-dimensional image and each three-dimensional point coordinate in the background point cloud is constructed, and at least five groups of feature points are selected, and the EPnP algorithm is used to solve the camera pose;

[0052] S2, collect and process the external damage hidden danger target image dataset, train the improved AttYOLOv4-tiny real-time detection model, and use it to detect and identify the external damage hidden danger targets that may appear in the power transmission channel;

[0053] S3, receiving the real-time image of the power transmission channel and detecting the corresponding hidden danger target type, calculating the Euclidean distance between the target object and the power transmission line according to the three-dimensional point coordinates of the target object and the three-dimensional point coordinates of the power transmission line, and judging whether to issue a warning according to the safety distance in the corresponding national standard;

[0054] The target hidden danger object can also be manually selected to calculate the distance between the target hidden danger object and the power transmission line.

[0055] The above steps realize monocular real-time ranging of hidden danger targets in the power transmission channel.

[0056] As Figure 2 In the monocular camera and laser point cloud joint calibration, the coordinate transformation relationship is as follows:

[0057]

[0058] Wherein, and represent the physical size of each pixel on the horizontal axis and the vertical axis of the image, is the pixel coordinate of the focal point of the camera optical axis and the image plane, represents the focal length of the camera, is the pixel coordinate, is the three-dimensional point coordinate.

[0059] Further, in the calibration of the monocular camera, Zhang Zhengyou calibration algorithm is used to obtain the internal parameter matrix of the camera wherein , is a camera focal length parameter, , is a camera optical center parameter.

[0060] By combining the three-dimensional point cloud coordinates with the pixel coordinates of the monocular camera through the camera pose estimation algorithm, the EPnP algorithm can be used to obtain the external parameters of the three-dimensional point cloud and the monocular camera: the rotation matrix and the translation matrix . Both constitute the external parameter matrix of the camera .

[0061] As Figure 3 , the improved YOLOv4-tiny used is a compressed version of YOLOv4, the backbone feature extraction network uses MobileNetV3-small structure, the enhanced feature extraction network uses SPP and PANet structure, and the prediction network uses YOLOHead structure.

[0062] wherein the MobileNetV3-small network structure used is a version of MobileNetV3 suitable for platforms with lower hardware performance;

[0063] The function of the backbone feature extraction network is to perform preliminary feature extraction to obtain three preliminary effective feature layers;

[0064] The function of the enhanced feature extraction network is to perform enhanced feature extraction, perform feature fusion on the three preliminary effective feature layers, extract better features, and obtain three more effective effective feature layers;

[0065] The function of the prediction network is to obtain the prediction result using the more effective effective feature layers.

[0066] As Figure 4 , the MobileNetV3-small structure has the following characteristics:

[0067] using an inverse residual structure with a linear bottleneck,

[0068] Among them, the linear bottleneck structure relatively standard convolution, before the depthwise convolution operation increases a new pointwise convolution associated with each channel, get more features, while removing the second PW convolution of ReLU activation function, reduce information loss, while increasing a layer of Linear in low-dimensional space, can reduce the number of parameters while reducing the amount of convolution calculation, optimize the network from space and time,

[0069] Further, on the basis of the original ResNet residual structure, the first layer PW convolution is replaced by the ReLU6 activation function after the dimension is increased, and the subsequent DW convolution is also replaced by the ReLU6 activation function, and the Linear activation function is used after the PW convolution is reduced in dimension, which is beneficial to mobile terminal use (helpful to reduce the parameter and M-Adds calculation amount), because the dimension increasing and reducing method is just opposite to the residual structure in ResNet, it is called inverted residuals.

[0070] Using depthwise separable convolution structure;

[0071] Using improved SE module, that is, the SE module is used in the last layer of the linear bottleneck structure, and then the PW convolution is performed, so as to keep the input and output of each layer of the network structure, and not to disturb the original main structure of the network;

[0072] Using h-swish (hard version of swish) activation function instead of part of ReLU6 activation function, improving the model precision while reducing the calculation amount. The h-swish function used is as follows:

[0073]

[0074] As shown in Table 1, in the MobileNetV3-small backbone feature extraction network parameter table, INPUT represents the shape change of each feature layer of MobileNetV3-small;

[0075]

[0076] Table 1 MobileNetV3-small backbone feature extraction network parameter table

[0077] OPERATOR represents the block change that each feature layer will experience;

[0078] #OUT represents the number of channels of the feature layer when input to bneck;

[0079] SE represents whether to introduce an attention mechanism in this layer;

[0080] NL represents the kind of activation function, HS represents h-swish, and RE represents ReLU

[0081] S represents the step length used in each block structure;

[0082] As Figure 5 In the prior art, a depth separable convolution is composed of a depth convolution and a point convolution.

[0083] For example, taking an input feature map size (5, 5, 3) and an output feature map size (5, 5, 4) as an example, when padding = 1, stride = 1, and a depth convolution kernel size of 3x3 is used, the parameter quantity is 39, and the calculation quantity is 975 times; when a standard convolution kernel size of 3x3 is used, the parameter quantity is 108, and the calculation quantity is 2700 times.

[0084] As can be seen, the depth separable convolution uses a smaller spatial cost (reduced parameters) and a smaller time cost (reduced calculation quantity), and achieves the same effect (extracting features) as a standard convolution layer.

[0085] After the target object appears, the target detection algorithm frames the target object.

[0086] Further, the spatial distance from the top corner to the bottom corner on the same side is calculated.

[0087] The bottom corner pixel coordinates are selected.

[0088] According to the mapping relationship in step S2, the three-dimensional point coordinates corresponding to the bottom corner pixel coordinates are determined.

[0089] The Z-axis coordinates of the three-dimensional point coordinates of each bottom corner are added to the spatial distance to obtain the three-dimensional point coordinates of each top corner.

[0090] According to the three-dimensional point coordinates of each top corner and the three-dimensional point coordinates of the power transmission line obtained in step S2, the Euclidean distances between the top corners of the target object frame and the power transmission line can be calculated, and the shortest Euclidean distance obtained by calculation is selected as the distance between the target object and the power transmission line.

[0091] Further, the Euclidean distance is calculated by the following formula:

[0092]

[0093] wherein, and are two three-dimensional point coordinates, and d is the Euclidean distance.

[0094] Further, it is judged whether the Euclidean distance is lower than the minimum safety distance provided in the national standard under the voltage level of the power transmission line.

[0095] If the Euclidean distance is lower than the minimum safety distance, the server sends a pre-warning signal.

[0096] On the contrary, the target hidden danger object can be manually selected, and the corresponding Euclidean distance is output.

[0097] It is proved by actual measurement that the method can better complete the identification and distance measurement of the hidden danger target in the bookstore under the complex background, has good detection speed and detection precision, and provides a reliable selection for the patrol and maintenance of the power transmission channel.

[0098] The above-described embodiments are only preferred technical solutions of the present application, and should not be regarded as limitations of the present application. The protection scope of the present application should be the technical solutions recited in the claims, including equivalent replacement solutions of the technical features recited in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present application.

Claims

1. A method for lightening transmission channel hidden danger ranging, characterized in that The method comprises the following steps: S1, obtaining two-dimensional image data and three-dimensional point cloud data of the power transmission channel background, constructing coordinate transformation and mapping relationship of each pixel coordinate in the two-dimensional image and each three-dimensional point coordinate in the background point cloud, and selecting at least five groups of feature points to solve the camera pose using the EPnP algorithm; the two-dimensional image data is obtained by shooting with a monocular camera; the three-dimensional point cloud data is obtained by a UAV loaded with a laser radar; S2, collecting and preprocessing the external damage hidden danger target image dataset, training an improved YOLOv4-tiny real-time detection model for detecting and identifying possible external damage hidden danger targets in the power transmission channel; S3, receiving real-time images of the power transmission channel and detecting corresponding hidden danger target types, calculating the Euclidean distance between the target and the power transmission line according to the three-dimensional point coordinates of the target and the three-dimensional point coordinates of the power transmission line; The improved YOLOv4-tiny real-time detection model uses a MobileNetV3-small network as the backbone network, instead of the CSPDarknet53-tiny network in the original YOLOv4-tiny model for feature extraction; An improved SE module is introduced in the MobileNetV3-small backbone network, which adjusts the network features through a lightweight attention mechanism module to increase the weight of the effective channels; Spatial Pyramid Pooling and PathAggregation Network are used to fuse the extracted feature information of different scales to obtain three more effective feature fusion layers.

2. The method of claim 1, wherein: In step S104, to ensure that the mapping relationship after coordinate transformation is good, at least five groups of positions with large turning degrees are selected as feature points, and the EPnP algorithm is used to solve the monocular camera external parameter matrix.

3. The method of claim 2, wherein: The decomposition steps of step S2 are as follows: S201, use a monocular camera to shoot an external damage hidden danger object image, and process the shot image into a dataset suitable for deep learning model detection; S202, use Labelme software to label the multiple external damage hidden danger targets in the dataset, and organize the labeled dataset according to the COCO dataset format; S203, on the basis of the existing dataset, add data images related to external damage hidden danger objects in the COCO public dataset, organize and generate training set, validation set and test set; S204, preliminarily construct an improved YOLOv4-tiny real-time detection model according to the characteristics of the hidden danger target; S205, use the set training set and validation set to train and test the model.

4. The method of claim 1, wherein: In step S201, the dataset used should be the two-dimensional image of the power transmission channel shot by the monocular camera in step S101, and the Labelme software is used for labeling, and the hidden danger targets are divided into three categories: cars, trucks and cranes.

5. The method of claim 1, wherein: In S3, the Euclidean distance is calculated by the following formula: wherein, and are two three-dimensional point coordinates, respectively, and d is the Euclidean distance.

6. A power transmission channel hazard early warning device characterized by: The light-weight power transmission channel hidden danger ranging method according to any one of claims 1-5 is adopted, including an alarm prompt module, which is used for judging whether to issue a warning according to a safety distance in a corresponding national standard; that is, if the Euclidean distance is lower than the national standard corresponding to the geographical environment and voltage level of the power transmission channel, an alarm is given.

Images