Control method and system for centering lead throwing frame based on unmanned aerial vehicle vision

Abstract

The invention discloses an unmanned aerial vehicle vision-based centering lead casting control method and system. The method comprises the following steps of: acquiring an image of a power transmission lead by using an unmanned aerial vehicle; segmenting the image by using a DeepLabV3 + semantic segmentation algorithm, and carrying out binarization processing; performing linear fitting on the binarized image, and calculating a center line offset according to a position relation between the center line of the unmanned aerial vehicle and the center line of the power transmission conductor; and according to the offset, the unmanned aerial vehicle is controlled to be aligned with the center line of the wire, and the suspension wire support is put within the safe distance. Accurate navigation and positioning of the unmanned aerial vehicle in a complex environment are realized, and the automation level of the unmanned aerial vehicle in power transmission conductor maintenance is greatly improved. The problems that traditional manual operation is high in difficulty and low in efficiency are effectively solved. The safety of the unmanned aerial vehicle in a high-voltage environment is ensured, and the discharge risk is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle control, and in particular to a control method and system for centering wire racking based on unmanned aerial vehicle vision. Background Art

[0002] With the continuous advancement of science and technology, drone and robot technologies have been widely used in various fields. Among them, the technology of collaborative operation of drones and robots is particularly important in power inspection and maintenance.

[0003] Before the operating robot goes online, a drone is needed to assist in placing the suspension rope bracket. However, the traditional manual operation method, that is, using cameras to observe and control the placement of the transmission line, is difficult and inefficient.

[0004] Therefore, vision-based UAV automatic control technology has become the focus of research. By introducing advanced image processing algorithms and control strategies, accurate navigation and positioning of UAVs in complex environments can be achieved. Summary of the invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by the present invention is that the existing manual operation method has the problems of high difficulty and low efficiency.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a control method for centering wire rack based on UAV vision, comprising:

[0008] Use drones to collect images of transmission lines;

[0009] Use DeepLabV3+ semantic segmentation algorithm to segment the image and perform binarization processing;

[0010] Perform straight line fitting on the binary processed image, and calculate the centerline offset based on the positional relationship between the centerline of the UAV and the centerline of the transmission line;

[0011] According to the offset, the drone is controlled to align with the center line of the conductor and the suspension wire bracket is dropped at a safe distance.

[0012] As a preferred solution of the control method for the central wire racking based on UAV vision of the present invention, wherein: the image segmentation using the DeepLabV3+ semantic segmentation algorithm includes labeling the transmission wire picture using the labeling tool Labelme to obtain a transmission wire data set, which is input into the DeepLabV3+ network for training to obtain a transmission wire segmentation model;

[0013] The obtained model is used to process the transmission line images collected by the UAV;

[0014] The feature map segmented by DeepLabV3+ is gray-scaled and then binarized with an adaptive threshold. The binarized image is divided into a white area and a black area, with the white area as the foreground of the image and the black area as the background of the image.

[0015] The calculation process of adaptive threshold binarization:

[0016] Take the size of the image to be processed as m×n, and calculate the proportion of background pixels and foreground pixels respectively:

[0017]

[0018] Among them, n a is the number of pixels belonging to the background, n b is the number of pixels belonging to the foreground;

[0019] According to p a , p b Calculate the average gray value μ of the background a , the gray value of the foreground μ b And the global average gray value μ of the image c ;

[0020]

[0021] According to the concept of variance, the inter-class variance σ 2 The expression of (t) is:

[0022]

[0023] Simplified:

[0024]

[0025] Where t is the threshold for dividing the foreground and background, L is the total number of gray levels, i is the index value of the gray level, i = 0, 1, ..., 255; p i It represents the number of times a point with pixel value i appears in the image divided by the sum of the number of times all pixels appear;

[0026] Select the optimal threshold T to achieve the best image binarization segmentation effect;

[0027] σ 2 (T) = max[σ 2 (t)]

[0028] The discriminant of adaptive threshold binarization is obtained:

[0029]

[0030] Among them, f(x,y) is the grayscale value and (x,y) is the pixel point.

[0031] As a preferred solution of the control method for centering wire racking based on UAV vision of the present invention, wherein: the straight line fitting of the binarized image includes extracting the center point of each row of the transmission wire in the binarized feature image, using a straight line fitting algorithm based on RANSAC estimation, repeatedly selecting data to estimate the model, continuously iterating and estimating the optimal model, thereby eliminating noise data in the feature image;

[0032] The specific process of the algorithm includes:

[0033] Step 1: Find the wire position in the binarized image and randomly select two points at the wire position to determine a straight line l;

[0034] Step 2: According to the threshold t, determine the set of data points S(l) whose geometric distance to the line l is less than the threshold t, which represents the consistent set of the line l;

[0035] Step 3: Repeat steps 1 and 2, and after A times of random sampling, we get the straight line l 1 ,l 2 ,.....,l a And the corresponding consistent set S(l 1 ),S(l 2 ),......,S(l a );

[0036] Step 4: Through continuous iterative calculation, the best fitting straight line of the largest consistent set is solved as the best matching straight line of the data points;

[0037] Assume that the proportion of the consistent set to the entire data set is e. Then, when the number of selected points is 2 each time, the probability of selecting a non-consistent set is 1-e. 2 ;

[0038] When the number of iterations is A, the probability of selecting a non-consistent set in the A-iteration model is (1-e 2 ) A , then the probability E of choosing the exact two points to calculate the correct model is:

[0039] E=1-(1-e 2 ) A

[0040] By sorting, the number of iterations m can be obtained as:

[0041]

[0042] Where A represents the number of sampling times, a represents the number of consistent sets obtained by sampling, and A≥a.

[0043] As a preferred solution of the control method for centering wire rack based on UAV vision of the present invention, the center line offset includes: 0 (0.5m, 0.5n) is used as the origin to establish the coordinate axis, and the midline expression of the two transmission lines after fitting is assumed to be:

[0044]

[0045] The center line expression of the two transmission lines is obtained by formula derivation:

[0046]

[0047] Take the midline y 3 The average coordinate point on the x-axis in the y-th row of the image is Calculate C I Point distance C 0 The weighted average offset d of the points is:

[0048]

[0049] The expression of the deviation angle θ is:

[0050]

[0051] When the offset is less than zero, the UAV is located on the right side of the centerline of the transmission line, and the offset angle is a negative value; when the offset is greater than zero, the UAV is located on the left side of the centerline of the transmission line, and the offset angle is a positive value.

[0052] As a preferred solution of the control method for centering wire rack based on UAV vision of the present invention, the center line offset includes: 0 (0.5m, 0.5n) is used as the origin to establish the coordinate axis, and the midline expression of the two transmission lines after fitting is assumed to be:

[0053]

[0054] The center line expression of the two transmission lines is obtained by formula derivation:

[0055]

[0056] Take the midline y 3 The average coordinate point on the x-axis in the y-th row of the image is Calculate C I Point distance C 0The weighted average offset d of the points is:

[0057]

[0058] The expression of the deviation angle θ is:

[0059]

[0060] When the offset is less than zero, the UAV is located on the right side of the centerline of the transmission line, and the offset angle is a negative value; when the offset is greater than zero, the UAV is located on the left side of the centerline of the transmission line, and the offset angle is a positive value.

[0061] As a preferred solution of the control method for centering wire rack based on UAV vision of the present invention, wherein: the UAV is visually positioned, and a pixel plane coordinate system is set. i -uv, the u axis corresponds to the width of the image, the ν axis corresponds to the height of the image, and the u 0 and v 0 Represents the pixel difference between the origin of the pixel coordinate system and the origin of the image coordinate system on the u-axis and ν-axis;

[0062] The conversion expression from pixel to image coordinate system is:

[0063]

[0064] Where a and b are x c Axis and y c The scale factor of the axis;

[0065] The expression for mapping a three-dimensional point in the actual space to a two-dimensional point on the image pixel is:

[0066]

[0067] The actual position of the image points is:

[0068]

[0069] The actual offset D between the transmission line and the center point of the UAV is:

[0070]

[0071] Assuming the roll angle of the drone is β, the offset D0 after correction and compensation is expressed as:

[0072]

[0073] Among them, X, Y, and Z are the coordinates of the point in the image in the horizontal, vertical, and vertical axes in the actual space, respectively. U represents the coordinate on the u-axis, and V represents the coordinate on the v-axis.

[0074] As a preferred solution of the control method for centering wire rack based on UAV vision of the present invention, wherein: the control of the UAV to align the center line of the wire includes: controlling the center line of the UAV θ Coincident with the center line of the two conductors l 1 , through translation, so that l θ With l 1 The deviation between them is zero;

[0075] Rotate the drone counterclockwise by an angle of θ so that the center line of the drone is parallel to the transmission line, calculate the offset D between the two lines, and control the drone to move left until the offset D becomes 0, completing the centering control;

[0076] Design the yaw axis visual angle controller and the roll axis visual position controller;

[0077] The yaw axis visual angle controller uses the deviation angle obtained by the visual positioning system as the outer loop PID and the angular velocity as the inner loop PID. The two are connected in series to control the yaw angle of the drone.

[0078] The error angle e1 is obtained by subtracting the expected rotation angle from the angle calculated by the current visual positioning system. This is used as the input of the outer loop. The expected value of the inner loop is calculated and output through the angle P controller. The error e2 is then compared with the current deviation angular velocity obtained by the visual positioning system. This is input into the inner loop angular velocity PID controller for calculation and the motor speed control value is output. The yaw angle of the drone is changed by controlling the drone motor.

[0079] The roll axis visual position controller uses the deviation distance vision obtained by the visual positioning system as the outer loop PID and the deviation change rate as the inner loop PID. The two are cascaded to control the yaw angle of the drone.

[0080] The expected deviation value is subtracted from the deviation distance calculated by the current visual positioning system to obtain the error value e1, which is used as the outer loop input. The expected value of the inner loop is calculated and output through the position P controller, and then compared with the current deviation change rate obtained by the visual positioning system to obtain the error e2, which is input into the inner loop rate PID controller for calculation and outputs the motor speed control value. The roll axis position of the drone is controlled by controlling the drone motor.

[0081] As a preferred solution of the control method for centering wire racking based on UAV vision of the present invention, the safety distance includes simulating the process of live working robot going up and down the line by ANSYS electromagnetic simulation software to obtain the relationship between the capacitance parameter and the robot distance;

[0082] Thus, the safe distance d is obtained, and the UAV is controlled to drop the suspension bracket to the transmission line at the safe distance.

[0083] A control system for centering wire racking based on drone vision using the method of the present invention is characterized by:

[0084] A collection unit, which uses a drone to collect images of the transmission lines;

[0085] The processing unit uses the DeepLabV3+ semantic segmentation algorithm to segment the image and perform binarization processing;

[0086] The fitting unit performs straight line fitting on the binary processed image and calculates the centerline offset according to the positional relationship between the centerline of the UAV and the centerline of the transmission line;

[0087] The control unit controls the drone to align with the center line of the conductor according to the offset, and drops the wire support at a safe distance.

[0088] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein: the processor implements the steps of any one of the methods of the present invention when executing the computer program.

[0089] A computer-readable storage medium stores a computer program, wherein: when the computer program is executed by a processor, the steps of any one of the methods of the present invention are implemented.

[0090] Beneficial effects of the present invention: The control method for centering conductor delivery based on UAV vision provided by the present invention realizes the precise navigation and positioning of UAVs in complex environments, greatly improving the automation level of UAVs in the maintenance of power transmission lines. The use of vision-based automatic centering control of UAVs effectively solves the problems of high difficulty and low efficiency of traditional manual operations. Through adaptive threshold binarization processing and precise offset calculation, the UAV can adjust its position autonomously to ensure the accurate delivery of the suspension rope bracket, significantly improving the operation accuracy and efficiency. Electric field analysis and safety distance calculation ensure the safety of UAVs in high-voltage environments and reduce the risk of discharge. It not only improves the safety and efficiency of power inspection and maintenance operations, but also promotes the intelligent application of UAV and robot technology in the power industry, and has broad promotion value and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0092] Figure 1An overall flow chart of a control method for centering wire racking based on drone vision provided in the first embodiment of the present invention;

[0093] Figure 2 A control method for centering wire racking based on UAV vision provided by the first embodiment of the present invention; image segmentation processing result of UAV centering control; (a) is the original image, (b) is the segmentation processing result;

[0094] Figure 3 A binary diagram of a control method for centering wire racking based on drone vision provided by the first embodiment of the present invention;

[0095] Figure 4 The result of straight line fitting of a control method for centering wire racking based on UAV vision provided by the first embodiment of the present invention;

[0096] Figure 5 A control method for centering wire racking based on UAV vision provided in the first embodiment of the present invention is provided, wherein two wires are fitted to obtain a processing result of the center line;

[0097] Figure 6 A schematic diagram of calculating the centerline offset of a control method for centering wire racking based on drone vision provided in the first embodiment of the present invention;

[0098] Figure 7 A pinhole imaging principle diagram of a control method for centering wire racking based on drone vision provided in the first embodiment of the present invention;

[0099] Figure 8 An image coordinate system and a pixel coordinate system of a control method for centering wire racking based on drone vision provided in the first embodiment of the present invention;

[0100] Fig. 9 A compensation analysis diagram of the offset of a control method for centering wire racking based on drone vision provided in the first embodiment of the present invention;

[0101] Fig.10 The equivalent circuit before the transfer of the three-phase potential of A, B, and C of a control method for centering wire racking based on drone vision provided in the second embodiment of the present invention;

[0102] Fig.11 A curve showing the relationship between the capacitance parameter of capacitor CC1 and the equipotential safety distance in a control method for centering wire racking based on drone vision provided in the second embodiment of the present invention;

[0103] Fig.12A curve showing the relationship between the capacitance parameter of capacitor CC2 and the equipotential safety distance in a control method for centering wire racking based on drone vision provided in the second embodiment of the present invention;

[0104] Fig.13 A curve showing the relationship between the electric field strength and the spacing of the C-phase transmission line in a control method for centering conductor racking based on drone vision provided in the second embodiment of the present invention;

[0105] Fig.14 A second embodiment of the present invention provides a control method for centering conductor racking based on UAV vision, which provides the electric field distribution between the surface of the UAV and the transmission line at 500kV. DETAILED DESCRIPTION

[0106] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0107] Example 1, reference Figure 1-Figure 9 , which is an embodiment of the present invention, provides a control method for centering wire rack based on drone vision, comprising:

[0108] S1: Use drones to collect images of power transmission lines. Use DeepLabV3+ semantic segmentation algorithm to segment the images and perform binarization processing.

[0109] The image segmentation using the DeepLabV3+ semantic segmentation algorithm includes labeling the transmission line picture using the labeling tool Labelme to obtain a transmission line data set, which is input into the DeepLabV3+ network for training to obtain a transmission line segmentation model.

[0110] The obtained model is used to process the transmission line images collected by UAV.

[0111] The feature map segmented by DeepLabV3+ is gray-scaled and then binarized with an adaptive threshold. The binarized image is divided into a white area and a black area, with the white area as the foreground of the image and the black area as the background of the image.

[0112] It should be said that the network of DeepLabV3+ is mainly divided into two parts, namely the encoder part and the decoder part. The encoder part is mainly used to extract image features, which is composed of the Xception module and the ASPP module. First, in order to reduce the parameters of the model, a deep separable convolution is added to the Xception network, and then the features extracted at the beginning are input into the ASPP module. The ASPP module contains a 1×1 convolution, three dilated convolutions with dilated rates of 6, 12, and 18, and a global pooling layer. They act as five parallel branches to further extract the input features, obtain multi-scale feature information and fuse and splice them, and finally output them through a 1×1 convolution.

[0113] The decoder part mainly restores the underlying features and the high-level features output by the encoder step by step, and restores various feature information to the original image size. In the encoder, the underlying features output by the Xception network enter the decoder, undergo a 1×1 convolution, and are fused with the high-level features output by the encoder after 4 times upsampling. Finally, after a 3×3 convolution and 4 times upsampling, the final prediction image is output.

[0114] Compared with conventional convolution, the number of parameters of depthwise separable convolution is reduced and the computational efficiency is significantly improved. Depthwise separable convolution mainly consists of two parts, namely channel-by-channel convolution and point-by-point convolution. Channel-by-channel convolution operation means that each channel is responsible for one convolution. Since the number of convolution kernels has not changed compared with the previous channel, the number of channels of the feature map of the first convolution operation output remains unchanged. In addition, each channel is operated independently, and the same spatial information on different channels cannot be effectively used. To this end, point-by-point convolution is introduced to weightedly combine these feature information to obtain a new feature map.

[0115] The semantic segmentation network does not have a fully connected layer and is mainly composed of convolutions. The semantic segmentation network of DeepLabV3+ uses dilated convolutions. In the DeepLabV3+ network structure, due to the reduction of pooling layers, feature information cannot be well aggregated and the range of the receptive field is small. For this reason, a dilated convolution layer is introduced to increase the range of the receptive field while effectively reducing information loss. The dilated convolution layer can also reduce the amount of model training because its parameter quantity and computational complexity will not change due to the different output strides used during training. There will be a certain interval between the pixels of the feature map input by the dilated convolution, and they are not tightly arranged together. The size of these intervals is called the dilation rate or expansion coefficient.

[0116] The transmission line images were annotated using the annotation tool Labelme to obtain a transmission line dataset, which was then input into the DeepLabV3+ network for training to obtain a transmission line segmentation model. The obtained model was used to process the transmission line images collected by the drone. The processing results are shown in the figure below. Figure 2 As shown, Figure (a) is the original image, and Figure (b) is the processed segmentation result.

[0117] In order to further extract the features of the transmission line, it is necessary to binarize the feature map segmented by DeepLabV3+ after grayscale processing. The binarized image is divided into white and black areas. This article calls the white area the foreground of the image and the black area the background of the image. The basis for the centering control of the drone is determined by the foreground. During the centering control of the drone, the picture is dynamic and continuous, and the background in the picture is constantly changing. In addition, the threshold used for judgment is different in different environments, different weather, and different light intensities. In order to achieve better extraction effects, an adaptive threshold is used to binarize the feature map.

[0118] Adaptive threshold binarization is also called maximum variance binarization. It can select an optimal threshold based on the current feature information of the image, that is, calculate a threshold that can maximize the variance between the average grayscale value of the foreground and the average grayscale value of the global image, as well as the average grayscale value of the background and the average grayscale value of the global image, to ensure the best image distinction effect.

[0119] The calculation process of adaptive threshold binarization:

[0120] Take the size of the image to be processed as m×n, and calculate the proportion of background pixels and foreground pixels respectively:

[0121]

[0122] Among them, n a is the number of pixels belonging to the background, n b is the number of pixels belonging to the foreground.

[0123] According to p a , p b Calculate the average gray value μ of the background a , the gray value of the foreground μ b And the global average gray value μ of the image c .

[0124]

[0125] μ c =p a μ a +pb μ b

[0126] According to the concept of variance, the inter-class variance σ 2 The expression of (t) is:

[0127]

[0128] Simplified:

[0129]

[0130] Where t is the threshold for dividing the foreground and background, L is the total number of gray levels, i is the index value of the gray level, i = 0, 1, ..., 255; p i It represents the number of times a point with pixel value i appears in the image divided by the sum of the number of times all pixels appear. Select the optimal threshold T to achieve the best segmentation effect of image binarization.

[0131] σ 2 (T) = max[σ 2 (t)]

[0132] The discriminant of adaptive threshold binarization is obtained:

[0133]

[0134] Among them, f(x,y) is the grayscale value and (x,y) is the pixel point.

[0135] In order to improve the real-time performance of the UAV centering control and speed up the processing of image features, it is necessary to adjust the relevant parameters of the adaptive threshold binarization algorithm. Quantization histogram processing: To reduce the amount of calculation, the step size of the gray value is selected as: 2 a , the grayscale level of the grayscale image is L, that is, the number of grayscale levels of the histogram after quantization becomes ΔL=L / 2 a , the number has been significantly reduced. In order to ensure the integrity of feature information while improving the operation speed, a=2 is selected, that is, the final number of gray levels is ΔL=L / 4.

[0136] Processing of jump calculation of grayscale images: In the process of binarization, each pixel on the image is generally classified and judged by a given threshold, which is very time-consuming. For this reason, jump processing is required, that is, processing every b pixels. So when processing a grayscale image of size m×n, select the step size b of the rows and columns as 2, then the total number of operations after the final processing becomes 1 / 4 of the original, and the final output judgment result will not be affected. Processing of interval constraints: Before the binarization process, the features of the image have been segmented more clearly, that is, the range of the threshold will also be in a relatively fixed interval. Through a large number of experimental tests in the corresponding environment, the range of the threshold is generally [90,160]. Considering the accuracy of the processing results and the speed of processing, the selection range of the threshold is limited to [55,205]. The results of adaptive threshold binarization processing are as follows: Figure 3 .

[0137] S2: Perform straight line fitting on the binary processed image and calculate the centerline offset based on the positional relationship between the centerline of the drone and the centerline of the transmission line.

[0138] The straight line fitting of the binarized image includes extracting the center point of each row of the transmission wire in the binarized feature image, using a straight line fitting algorithm based on RANSAC estimation, repeatedly selecting data to estimate the model, continuously iterating and estimating the optimal model, thereby eliminating noise data in the feature image.

[0139] The specific process of the algorithm includes:

[0140] Step 1: Find the wire position in the binarized image and randomly select two points at the wire position to determine a straight line l, that is, select the minimum data set.

[0141] Step 2: According to the threshold t, determine the data point set S(l) whose geometric distance to the line l is less than the threshold t, which represents the consistent set of the line l.

[0142] Step 3: Repeat steps 1 and 2, and after A times of random sampling, we get the straight line l 1 ,l 2 ,.....,l a And the corresponding consistent set S(l 1 ),S(l 2 ),......,S(l a ).

[0143] Step 4: Through continuous iterative calculation, the best fitting straight line of the largest consistent set is solved as the best matching straight line of the data points.

[0144] Assume that the proportion of the consistent set to the entire data set is e. Then, when the number of selected points is 2 each time, the probability of selecting a non-consistent set is 1-e. 2 .

[0145] When the number of iterations is A, the probability of selecting a non-consistent set in the A-iteration model is (1-e 2 ) A , then the probability E of choosing the exact two points to calculate the correct model is:

[0146] E=1-(1-e 2 ) A

[0147] By sorting, the number of iterations m can be obtained as:

[0148]

[0149] Where A represents the number of sampling times, a represents the number of consistent sets obtained by sampling, and A≥a.

[0150] By extracting the center point of each row of transmission lines in the binary feature map, the image after straight line fitting based on RANSAC estimation is shown as follows Figure 4 shown.

[0151] The purpose of drone centering is to accurately place the suspension rope bracket on the transmission line. To achieve this operation, it is necessary to fit the center lines of the two transmission lines and calculate the offset based on the positional relationship between the center line of the drone and the center line of the transmission line. The center line offset includes: 0 (0.5m, 0.5n) is used as the origin to establish the coordinate axis, and the midline expression of the two transmission lines after fitting is assumed to be:

[0152]

[0153] The center line expression of the two transmission lines is obtained by formula derivation:

[0154]

[0155] The center lines y of the two transmission lines are obtained 3 As the reference line of the UAV centering deviation, the midline diagram of the two fitted straight lines is as follows: Figure 5 shown.

[0156] Although the transmission line image has undergone a series of feature extraction, noise will still appear in the background area. If the offset is calculated directly using the center point of the image and the center line of the wire without processing, the effect of the UAV's centering control will be greatly reduced. In order to reduce the occurrence of this error, a global centerline pixel offset calculation method is used in the offset calculation.

[0157] The calculation diagram of the offset is as follows: Figure 6 Take the midline y 3 The average coordinate point on the x-axis in the y-th row of the image is Calculate C I Point distance C 0 The weighted average offset d of the points is:

[0158]

[0159] The expression of the deviation angle θ is:

[0160]

[0161] In summary, when the offset is less than zero, the drone is located on the right side of the centerline of the transmission line, and the offset angle is negative; when the offset is greater than zero, the drone is located on the left side of the centerline of the transmission line, and the offset angle is positive. The absolute value of the offset reflects the length of the drone's offset. By inputting the offset and offset angle into the controller, it lays the foundation for the subsequent drone centering control.

[0162] S3: According to the offset, the drone is controlled to align with the center line of the conductor and a suspension wire bracket is placed at a safe distance.

[0163] The relative position of the drone and the center line between the two conductors is mainly determined by the onboard camera to collect data. In the actual environment, the distance from the transmission line to the optical center of the camera must be far greater than the focal length of the camera, so the relative position analysis needs to be carried out through the principle of pinhole imaging. The principle diagram of pinhole imaging is shown in the figure. Figure 7 shown.

[0164] o-xyz is the camera coordinate system, in mm, oc-x c y c is a two-dimensional coordinate system parallel to the camera coordinate system o-xy and coplanar with the image pixel coordinate system, with the unit of mm, image plane midpoint, o is the camera midpoint, o c o is the focal length f of the camera. Assuming the position of the target point in the camera coordinate system is Q, then Q c is the position of the target point on the image. c The two points Q are perpendicular to o c The z axis intersects at P c, the auxiliary line of point P, and the expression of image-camera coordinate system conversion is calculated as:

[0165]

[0166] In order to further obtain the relative position of the drone to the centerline of the wire, it is necessary to transform the points on the image coordinate system into the pixel plane coordinate system. i -uv, the image-pixel coordinate system is as follows Figure 8 shown.

[0167] Assume pixel plane coordinate system o i -uv unit is Pixel, the u axis corresponds to the width of the image, the ν axis corresponds to the height of the image, and the u 0 and v 0 Represents the pixel difference between the origin of the pixel coordinate system and the origin of the image coordinate system on the u-axis and the v-axis; the conversion expression of the pixel-image coordinate system is:

[0168]

[0169] Where a and b are x c Axis and y c The scale factor of the axis.

[0170] The expression for mapping a three-dimensional point in the actual space to a two-dimensional point on the image pixel is:

[0171]

[0172] The actual position of the image points is:

[0173]

[0174] The actual offset D between the transmission line and the center point of the UAV is:

[0175]

[0176] Since the drone's onboard camera does not have an anti-shake gimbal, the camera's viewing angle will change with the drone's attitude angle during the actual flight, resulting in the image center not being directly below the drone, which will produce a certain deviation. At the same time, the camera is not installed on the center line of the drone, so considering the existence of these two errors, the offset needs to be compensated. Assuming that the drone's roll angle is β, the offset compensation analysis is as follows: Fig. 9 shown.

[0177] According to the compensation analysis diagram of the offset, it can be concluded that the deviation compensation distance l is:

[0178] l=Ztanβ

[0179] In order to eliminate the deviation caused by the camera not being at the center line of the drone, the camera position compensation x is introduced. 0 The offset D0 after correction and compensation is expressed as:

[0180]

[0181] Among them, X, Y, and Z are the coordinates of the point in the image in the horizontal, vertical, and vertical axes in the actual space, respectively. U represents the coordinate on the u-axis, and V represents the coordinate on the v-axis.

[0182] Since the traditional manual operation for centering control has high requirements for the UAV operator and the centering rate is low, which seriously affects the work efficiency, this paper adopts the automatic centering control of the UAV based on vision, that is, the UAV can autonomously find the line and adjust the control when hovering near the wire. The control is mainly divided into the following steps:

[0183] (1) A drone equipped with a suspension rope bracket and an insulating traction rope is flown above the wires, and the drone is turned horizontally in a direction close to the transmission wires until the two wires are completely in the camera view.

[0184] (2) The drone processes the image and uses the DeepLabV3+ algorithm to segment the transmission lines. Then, the adaptive threshold binarization algorithm is used to effectively separate the foreground and background. Finally, the center points of the two wires are fitted into two center straight lines, and the center lines of the two wires are obtained.

[0185] (3) By comparing the positional relationship between the center lines of the two wires obtained in the image and the center line of the image, the deviation distance and deviation angle between the UAV and the center line of the wire are solved.

[0186] (4) The UAV is controlled to rotate and move horizontally by the deviation distance and deviation angle until the deviation amount reaches the allowable error range, thus completing the centering control.

[0187] Control the center line of the drone θ Coincident with the center line of the two conductors l 1 , through translation, so that l θ With l 1 The deviation between them is zero. Rotate the drone counterclockwise by angle θ so that the center line of the drone is parallel to the transmission line, calculate the offset D between the two lines, and control the drone to move left until the offset D becomes 0, completing the centering control.

[0188] In the translation and centering position control of the center line of the UAV and the center lines of the two conductors, it is necessary to design a yaw axis visual angle controller and a roll axis visual position controller, and both controllers adopt PID control.

[0189] The yaw axis vision angle controller uses the deviation angle obtained by the visual positioning system as the outer loop PID and the angular velocity as the inner loop PID. The two are connected in cascade to control the yaw angle of the drone.

[0190] The error angle e1 is obtained by subtracting the expected rotation angle from the angle calculated by the current visual positioning system. This is used as the input of the outer loop. The expected value of the inner loop is calculated and output through the angle P controller. The error e2 is then compared with the current deviation angular velocity obtained by the visual positioning system and input into the inner loop angular velocity PID controller for calculation. The motor speed control value is output and the yaw angle of the drone is changed by controlling the drone motor.

[0191] The roll axis visual position controller uses the deviation distance vision obtained by the visual positioning system as the outer loop PID and the deviation change rate as the inner loop PID. The two are connected in cascade to control the yaw angle of the drone.

[0192] The expected deviation value is subtracted from the deviation distance calculated by the current visual positioning system to obtain the error value e1, which is used as the outer loop input. The expected value of the inner loop is calculated and output through the position P controller, and then compared with the current deviation change rate obtained by the visual positioning system to obtain the error e2, which is input into the inner loop rate PID controller for calculation and outputs the motor speed control value. The roll axis position of the drone is controlled by controlling the drone motor.

[0193] After the drone is aligned with the conductor, it needs to perform a drop operation. However, since the drone generates tip discharge when approaching the live transmission line, it affects the internal electronic components of the drone and hinders the control of the drone to the center. It is necessary to analyze the safe distance for the drone to hover on the transmission line to drop the wire. The process of the live working robot going up and down the line is simulated by ANSYS electromagnetic simulation software, and the relationship between the capacitance parameter and the robot distance is obtained. In this way, the safe distance D is obtained, and the drone is controlled to drop the wire support on the transmission line at a safe distance.

[0194] On the other hand, this embodiment also provides a control system for centering wire racking based on drone vision, which includes:

[0195] A collection unit, which uses a drone to collect images of the transmission lines;

[0196] The processing unit uses the DeepLabV3+ semantic segmentation algorithm to segment the image and perform binarization processing.

[0197] The fitting unit performs straight line fitting on the binarized image and calculates the centerline offset based on the positional relationship between the centerline of the UAV and the centerline of the transmission line.

[0198] The control unit controls the drone to align with the center line of the conductor according to the offset, and drops the wire support at a safe distance.

[0199] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0200] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0201] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0202] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0203] Example 2, reference Figure 10-Figure 14 , which is an embodiment of the present invention, provides a control method for centering wire racking based on UAV vision. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0204] After the UAV is aligned with the center conductor, it needs to perform a launch operation. However, because the UAV generates tip discharge when approaching the energized transmission line, it affects the internal electronic components of the UAV and hinders the UAV's centering launch control. Therefore, it is necessary to analyze the safe distance for the UAV to hover over the transmission line to launch the launch.

[0205] For the 500kV transmission line, the A, B, and C phase transmission lines are analyzed. The equivalent circuits before the three-phase potential transfer are as follows: Fig.10 shown.

[0206] Taking the C phase circuit as an example, when the operating circuit is the C phase transmission line, point Q on the C phase transmission line 3 Through capacitor C c1 , capacitor C C2 Ground, capacitor C c1 , capacitor C c2 Point O 3 is the position of the live working robot, capacitor C C1 is the capacitance between the live working robot and the C-phase transmission line, capacitance C c2 is the ground capacitance of the live working robot, and the phase voltage U C As point Q 3 The equivalent power supply between the capacitor C and the ground. By simulating the process of the live working robot on and off the line with ansys electromagnetic simulation software, the capacitance C cl and capacitor The relationship between the capacitance parameters and the robot distance is shown in the figure below. Fig.11 , Fig.12 shown.

[0207] The corresponding fitting function expression is obtained by fitting the data:

[0208] C C1 =257.6d -0.406 -3.109

[0209] C C2 =283.1d -0.435 -5.632

[0210] According to the average breakdown field strength formula of the rod-plate gap:

[0211]

[0212] Where r is the distance from a single conductor to the neutral line of the four conductors, and d is the distance between the transmission line and the drone. The expression of the safety distance d can be obtained as follows:

[0213]

[0214] In actual situations, the average breakdown field strength of the uneven electric field between the rod and the plate is 5kV / cm, r is 20cm, and the theoretical value of the safe distance calculated is 59cm. Taking the breakdown field strength of 5kV / cm as the limit, ansys is used to simulate the change of the electric field during the approach of the drone and the transmission line, and the final safe distance is 68cm. The relationship curve between the electric field strength and the spacing of the C-phase transmission line is shown in the figure below. Fig.13 shown.

[0215] Verified by theoretical formula, the value calculated theoretically is slightly smaller than the simulation value. Considering the complex electromagnetic environment, the larger value shall prevail. Considering the actual safety margin, the C phase rack safety distance is 80cm. The simulation diagram is as follows: Fig.14 As shown in the figure, when the distance between the UAV and the transmission line is 80cm, there is no tip discharge on the surface of the UAV, and it has no impact on the UAV. Similarly, the safe distance between the A phase and the B phase is 80cm and 90cm.

[0216] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A control method for centering wire rack based on drone vision, characterized in that: include: Use drones to collect images of transmission lines; Use DeepLabV3+ semantic segmentation algorithm to segment the image and perform binarization processing; Perform straight line fitting on the binary processed image, and calculate the centerline offset based on the positional relationship between the centerline of the UAV and the centerline of the transmission line; According to the offset, the drone is controlled to align with the center line of the conductor and the suspension wire bracket is dropped at a safe distance.

2. The control method for centering wire rack based on drone vision as claimed in claim 1, characterized in that: The use of the DeepLabV3+ semantic segmentation algorithm to segment the image includes: using the labeling tool Labelme to label the transmission line picture to obtain a transmission line data set, inputting the data set into the DeepLabV3+ network for training, and obtaining a transmission line segmentation model; The obtained model is used to process the transmission line images collected by the UAV; The feature map segmented by DeepLabV3+ is processed in grayscale and then binarized by adaptive threshold; The image after binarization is divided into white area and black area, the white area is used as the foreground of the image, and the black area is used as the background of the image; The calculation process of adaptive threshold binarization: Take the size of the image to be processed as m×n, and calculate the proportion of background pixels and foreground pixels respectively: Among them, n a is the number of pixels belonging to the background, n b is the number of pixels belonging to the foreground; According to p a , p b Calculate the average gray value μ of the background a , the gray value of the foreground μ b And the global average gray value μ of the image c ; m c =p a m a +p b m b According to the concept of variance, the inter-class variance σ 2 The expression of (t) is: s 2 (t)=p a (m a -m c ) 2 +p b (m b -m c ) 2 Simplified: Where t is the threshold for dividing the foreground and background, L is the total number of gray levels, i is the index value of the gray level, i = 0, 1, ..., 255; p i It represents the number of times a point with pixel value i appears in the image divided by the sum of the number of times all pixels appear; Select the optimal threshold T to achieve the best image binarization segmentation effect; s 2 (T)=max[σ 2 (t)] The discriminant of adaptive threshold binarization is obtained: Among them, f(x,y) is the grayscale value and (x,y) is the pixel point.

3. The control method for centering wire rack based on UAV vision as claimed in claim 2, characterized in that: The linear fitting of the binarized image includes extracting the center point of each row of the power transmission wire in the binarized feature image, using a linear fitting algorithm based on RANSAC estimation, repeatedly selecting data to estimate the model, and continuously iterating and estimating the optimal model, thereby eliminating noise data in the feature image; The specific process of the algorithm includes: Step 1: Find the wire position in the binarized image and randomly select two points at the wire position to determine a straight line l; Step 2: According to the threshold t, determine the set of data points S(l) whose geometric distance to the line l is less than the threshold t, which represents the consistent set of the line l; Step 3: Repeat steps 1 and 2, and after A times of random sampling, we get the straight lines l1, l2, ..., l a And the corresponding consistent sets S(l1),S(l2),...,S(l a ); Step 4: Through continuous iterative calculation, the best fitting straight line of the largest consistent set is solved as the best matching straight line of the data points; Assume that the proportion of the consistent set to the entire data set is e. Then, when the number of selected points is 2 each time, the probability of selecting a non-consistent set is 1-e. 2 ; When the number of iterations is A, the probability of selecting a non-consistent set in the A-iteration model is (1-e 2 ) A , then the probability E of choosing the exact two points to calculate the correct model is: E=1-(1-e 2 ) A By sorting, the number of iterations m can be obtained as: Where A represents the number of sampling times, a represents the number of consistent sets obtained by sampling, and A≥a.

4. The control method for centering wire rack based on UAV vision as claimed in claim 3 is characterized by: The centerline offset includes establishing a coordinate axis with the image midpoint f0 (0.5m, 0.5n) as the origin, and assuming that the centerline expression of the two transmission lines after fitting is: The center line expression of the two transmission lines is obtained by formula derivation: Take the average coordinate point of the midline y3 on the x-axis in the y-th row of the image as Calculate C I The weighted average offset d of point C0 is: The expression of the deviation angle θ is: When the offset is less than zero, the UAV is located on the right side of the centerline of the transmission line, and the offset angle is a negative value; when the offset is greater than zero, the UAV is located on the left side of the centerline of the transmission line, and the offset angle is a positive value.

5. The control method for centering wire rack based on UAV vision as claimed in claim 4, characterized in that: Perform visual positioning on the drone and set the pixel plane coordinate system o i -uv, the u axis corresponds to the width of the image, the ν axis corresponds to the height of the image, and u0 and v0 represent the pixel difference between the origin of the pixel coordinate system and the origin of the image coordinate system on the u axis and the ν axis; The conversion expression from pixel to image coordinate system is: Where a and b are x c Axis and y c The scale factor of the axis; The expression for mapping a three-dimensional point in the actual space to a two-dimensional point on the image pixel is: The actual position of the image points is: The actual offset D between the transmission line and the center point of the UAV is: Assuming the roll angle of the drone is β, the offset D0 after correction and compensation is expressed as: Among them, X, Y, and Z are the coordinates of the point in the image in the horizontal, vertical, and vertical axes in the actual space, respectively. U represents the coordinate on the u-axis, and V represents the coordinate on the v-axis.

6. The control method for centering wire rack based on UAV vision as claimed in claim 5, characterized in that: The controlling the drone to align with the center line of the wire comprises: controlling the center line of the drone θ Coincident with the midline l1 of the two wires, through translation, so that l θ The deviation from l1 is zero; Rotate the drone counterclockwise by an angle of θ so that the center line of the drone is parallel to the transmission line, calculate the offset D between the two lines, and control the drone to move left until the offset D becomes 0, completing the centering control; Design the yaw axis visual angle controller and the roll axis visual position controller; The yaw axis visual angle controller uses the deviation angle obtained by the visual positioning system as the outer loop PID and the angular velocity as the inner loop PID. The two are connected in series to control the yaw angle of the drone. The error angle e1 is obtained by subtracting the expected rotation angle from the angle calculated by the current visual positioning system. This is used as the input of the outer loop. The expected value of the inner loop is calculated and output through the angle P controller. The error e2 is then compared with the current deviation angular velocity obtained by the visual positioning system. This is input into the inner loop angular velocity PID controller for calculation and the motor speed control value is output. The yaw angle of the drone is changed by controlling the drone motor. The roll axis visual position controller uses the deviation distance vision obtained by the visual positioning system as the outer loop PID and the deviation change rate as the inner loop PID. The two are cascaded to control the yaw angle of the drone. The expected deviation value is subtracted from the deviation distance calculated by the current visual positioning system to obtain the error value e1, which is used as the outer loop input. The expected value of the inner loop is calculated and output through the position P controller, and then compared with the current deviation change rate obtained by the visual positioning system to obtain the error e2, which is input into the inner loop rate PID controller for calculation and outputs the motor speed control value. The roll axis position of the drone is controlled by controlling the drone motor.

7. The control method for centering wire rack based on UAV vision as claimed in claim 6, characterized in that: The safety distance includes simulating the process of the live working robot going online and offline through ANSYS electromagnetic simulation software to obtain the relationship between the capacitance parameter and the robot distance; Thus, the safe distance d is obtained, and the UAV is controlled to drop the suspension bracket to the transmission line at the safe distance.

8. A control system for centering wire racking based on drone vision using the method as described in any one of claims 1 to 7, characterized in that: A collection unit, which uses a drone to collect images of the transmission lines; The processing unit uses the DeepLabV3+ semantic segmentation algorithm to segment the image and perform binarization processing; The fitting unit performs straight line fitting on the binary processed image and calculates the centerline offset according to the positional relationship between the centerline of the UAV and the centerline of the transmission line; The control unit controls the drone to align with the center line of the conductor according to the offset, and drops the wire support at a safe distance.

9. A computer device comprising: Memory and processor; The memory stores a computer program, which is characterized in that when the processor executes the computer program, the steps of the control method for centering wire racking based on drone vision are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the steps of the control method for centering wire racking based on drone vision are implemented.

Images