Power transmission line tower refined route planning method

Abstract

The invention relates to the technical field of unmanned aerial vehicle route planning, in particular to a power transmission line tower refined route planning method, which comprises the following steps of: in three-dimensional route planning software, creating or opening an inspection task, opening a point cloud for picking up a tower pole, opening tower pole simulation, and then drawing a target; a route is produced after the target is drawn, and parameters are set; and then flight preview is carried out, and a route is published or exported after preview. According to the invention, the unmanned aerial vehicle-mounted binocular camera measurement platform and the image measurement system realize digital optics-based power grid airborne high-precision three-dimensional space measurement of the power transmission line, and the error in measurement is superior to 1cm. The automatic operation of the unmanned aerial vehicle binocular camera measurement platform is realized by three-dimensional route planning software. The operation efficiency of power grid equipment key position measurement is improved, the measurement precision is improved, the operation risks of induced electricity and the like are reduced, the operation cost is saved, and intelligent standard operation is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) route planning, and in particular to a method for fine-grained route planning of transmission line towers. Background Art

[0002] The inspection of transmission lines by drones has become a normal practice, but it is currently limited to the naked eye recognition of visible light photos, or the analysis of spatial distance through certain image processing. The spatial distance analysis of some power grid equipment needs to reach the centimeter level to meet the requirements, such as arcing angle measurement, ice thickness measurement, anti-vibration hammer displacement measurement, wire clamp offset discharge gap measurement, lightning strike position measurement, tower material size measurement, jumper and tower distance measurement, etc. For example, the discharge gap needs to be measured before the ice melting test in the plateau area every year, and each tower needs to be measured during the power outage and maintenance every year. In the daily operation process, the overhead ground wire is affected by factors such as unbalanced tension of the ground wire, its own technical process, and ice covering, which can easily cause the discharge gap plate electrode or rod electrode to loosen, causing the discharge gap to deform, enlarge or shrink. When the discharge gap is too small, it is easy to cause the gap to be broken down by the ice melting current, resulting in ice melting failure; when the discharge gap is too large, it affects the lightning protection effect of the line. The same situation also exists in the arcing angle in daily operation projects. In order to ensure that the discharge gap and arcing angle meet the requirements of the regulations, operators need to climb the tower for inspection and measurement repeatedly every year.

[0003] The current job modes are:

[0004] The traditional measurement method uses a steel tape measure, which is affected by factors such as personnel skill level and measurement method, and has different degrees of error. During the measurement process, some key positions require measurement personnel to measure by wire, which increases the measurement error due to human weight and balance conditions. In addition to being affected by the plateau environment, tower measurement also involves operational risks such as induction electricity. Usually, each person completes the measurement of 1 to 2 towers per day, which is time-consuming, labor-intensive, and inefficient. In addition, the high-frequency tower climbing and high-altitude operations greatly increase the risk of personal injury and death. Traditional photogrammetry methods have many errors in data acquisition, data processing, and results measurement, and cannot meet the requirement of a mean error better than 1 cm. Airborne laser radar technology mainly obtains large-scale decimeter-level laser point clouds, and cannot meet the requirement of a small-scale mean error better than 1 cm. Three-dimensional modeling based on traditional photogrammetry or airborne laser radar technology, according to the principle of error accumulation, its model accuracy is lower than the data accuracy, and cannot meet the requirements. High-precision ground-mounted three-dimensional laser scanners can theoretically meet millimeter-level measurement accuracy, but have high requirements for measurement distance and visibility, and low operating efficiency.

[0005] In this context, there is an urgent need for an indirect measurement device and technical solution for power grid equipment that is fast, efficient, safe, highly accurate, standardized, and scalable. Summary of the invention

[0006] The object of the present invention is to provide a method for fine-grained route planning of transmission line towers to solve the problems raised in the above-mentioned background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A transmission line tower refined route planning method, comprising:

[0009] In the 3D route planning software, according to the newly created or opened inspection task, open the point cloud to pick up the tower, open the tower simulation, and then draw the target;

[0010] After the target is drawn, the route is produced and the parameters are set;

[0011] Then preview the flight, publish the route or export the route after previewing.

[0012] Furthermore, a UAV equipped with a binocular camera measurement platform is used to pick up the point cloud of the tower.

[0013] Furthermore, the binocular camera measurement platform picks up the point cloud of the tower pole, which specifically includes image acquisition, camera calibration, stereo matching, three-dimensional coordinate solution and post-processing.

[0014] Furthermore, image acquisition includes using two cameras to simultaneously acquire digital images of targets within overlapping fields of view, or moving a single camera to acquire digital images of targets at different positions in a time-sharing manner, and processing the images through projection transformation and other related theories to reconstruct the target position and posture.

[0015] Furthermore, camera calibration includes solving the parameters of the imaging model, which mainly include the intrinsic parameters and extrinsic parameters of the camera. The intrinsic parameters include the camera focal length, image principal point, radial distortion coefficient and tangential distortion coefficient, and the extrinsic parameters include the rotation matrix and the translation matrix.

[0016] Furthermore, the image correction includes first using the distortion coefficients in the intrinsic parameters obtained by camera calibration to remove the radial and tangential distortions of the image to obtain an undistorted image, and then transforming the undistorted image through geometric transformation relationships so that two cameras that are not in the same plane are strictly aligned, and the epipolar lines of the two images can be on the same horizontal line. During stereo matching, only one-dimensional search needs to be performed in a certain row to match the corresponding points.

[0017] Furthermore, stereo matching includes finding matching corresponding points from images of different viewpoints, calculating the matching costs of the left and right images through one or more matching primitives, clustering the matching costs to obtain an initial disparity map, and then performing post-processing of interpolation and fitting on the initial disparity map to obtain a final disparity map.

[0018] Furthermore, three-dimensional coordinate solution and post-processing include using the principle of triangulation to calculate the depth information of objects in the scene.

[0019] Furthermore, typical parts recognition is performed on the acquired images based on Kinect depth images and point cloud edge features.

[0020] Furthermore, the acquired depth image is first processed using a nonlinear filtering optimization algorithm to obtain an optimized target point cloud and an eight-neighborhood depth difference algorithm is proposed to extract the edge of the point cloud;

[0021] Then, the random sampling consensus algorithm is used to detect the edges of the segmented point cloud and extract the defined edge features to identify the parts;

[0022] Based on the set parameters such as camera focal length, shooting distance and angle, the system automatically generates photo points and drone flight assistance points for key inspection components, and generates a high-precision and optimal flight route for the flight.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention is based on a binocular camera measurement platform and an image measurement system carried by an unmanned aerial vehicle, and realizes high-precision airborne three-dimensional spatial measurement of power transmission lines based on digital optics, with a measurement error of less than 1 cm.

[0025] The present invention uses three-dimensional route planning software to realize the automated operation of the unmanned aerial vehicle-mounted binocular camera measurement platform.

[0026] The present invention improves the operating efficiency of measuring key positions of power grid equipment, improves measurement accuracy, reduces operating risks such as induced electricity, saves operating costs, and realizes intelligent and standardized operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is the relationship diagram of the world coordinates, image coordinates and camera coordinates of Zhang Zhengyou's algorithm.

[0028] Figure 2 It is the overall framework design diagram of the present invention.

[0029] Figure 3 Schematic diagram of an implementation scheme of the present invention.

[0030] Figure 4 A flowchart for three-dimensional route planning.

[0031] Figure 5 This is a schematic diagram of the tower route planning.

[0032] Figure 6 Research roadmap for binocular camera measurement platform and image measurement system.

[0033] Figure 7 is the image correction map.

[0034] Figure 8 This is a structural diagram of the three-axis stabilized cloud rack.

[0035] Fig. 9 It is the space forward intersection diagram.

[0036] Fig.10 It is a spatial resection diagram.

[0037] Fig.11 Schematic diagram of global optimization. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] See also Figures 1 to 11 , the present invention provides a technical solution:

[0040] The present invention (project) uses the binocular stereo measurement technology of unmanned aerial vehicles to measure arcing angles, discharge gaps and other spatial distances. The unmanned aerial vehicle is equipped with a stereo measurement binocular camera to perform one-time imaging of the part to be measured according to the specified route, and the data is transmitted back in real time through the 4G / 5G network. After the data is acquired, the PC-side image measurement system is used to perform intelligent data analysis to complete the analysis of the spatial distance in real time. The measurement accuracy can reach a mean error better than 1cm, which solves the problems of efficiency, safety, and reliability of traditional measurement methods. Based on the image measurement system, it supports point-to-point and point-to-line distance measurement, line-to-line angle measurement, line-to-surface angle measurement, and three-point area measurement. The project results can be widely used in various measurement needs of power equipment.

[0041] This project realizes the long-distance, high-precision intelligent measurement of power grid equipment. Through the drone stereo measurement technology, it can greatly improve the efficiency of measuring key positions of power grid equipment, improve measurement accuracy, reduce operational risks such as induced electricity, save operating costs, and realize intelligent and standardized operations. It is suitable for work scenes that are difficult for manpower to reach, such as plateaus. Let's take the measurement of arcing angle and discharge gap as an example: the measurement operations that originally had to be carried out during power outage maintenance will be able to be integrated into daily live patrols, and the original 2 people (a team) can measure 6 towers a day, which is improved to 2 people can measure 20 towers a day, which is 2.3 times more efficient, which has a significant effect on reducing power grid failures, reducing failure losses, and comprehensively eliminating safety hazards in the power system.

[0042] The traditional measurement method uses a steel tape measure, which is affected by factors such as the skill level of the personnel and the measurement method, and has different degrees of error. During the measurement process, some key positions require the measurement personnel to measure the wire, which increases the measurement error due to the weight and balance conditions of the human body. There are operational risks such as induction electricity when personnel climb towers to measure. Usually, each person completes the measurement of 1 to 2 towers per day, which is time-consuming, labor-intensive, and inefficient. In addition, the high-frequency tower climbing and high-altitude operations greatly increase the risk of personal injury and death. Traditional photogrammetry methods have many errors in data acquisition, data processing, and results measurement, and cannot meet the requirements of absolute error centimeters (1-2cm). Airborne laser radar technology mainly obtains large-scale decimeter-level laser point clouds, and cannot meet the requirements of absolute error centimeters (1-2cm) in a small range. Three-dimensional modeling based on traditional photogrammetry or airborne laser radar technology, according to the principle of error accumulation, its model accuracy is lower than the data accuracy, and it cannot meet the requirements. High-precision ground-mounted three-dimensional laser scanners can theoretically meet millimeter-level measurement accuracy, but have high requirements for measurement distance and visibility, and low operation efficiency.

[0043] The binocular stereo vision measurement method has the advantages of high efficiency, non-contact measurement, simple system structure and low cost.

[0044] Research on intelligent route planning technology based on laser point cloud or tower model

[0045] The research content mainly includes point cloud classification technology, tower key component identification technology, tower refined inspection route planning, etc., to realize the control of drones to perform high-precision and intelligent tower refined inspections according to the planned routes.

[0046] Research on high-precision measurement technology based on stereo vision

[0047] Image processing technology and machine vision theory are used to study the long-distance measurement method based on the binocular stereo vision system, design a long-distance distance measurement scheme with high measurement accuracy, study the basic model and theory of the existing binocular vision system, study the binocular convergence distance measurement model and the binocular parallel distance measurement model, and select the distance measurement model of this article according to the actual application scenario. Secondly, analyze the influence of the system's structural parameters and image coordinates on the error, and study the influence trend of the system's structural parameters on the error, so as to improve the measurement accuracy of stereo vision. By studying the relevant theories of the binocular system, analyzing the structural characteristics to determine the optimal system structure design, analyzing the detector discretization, the most fundamental factor restricting the visual measurement system, determining the theoretical accuracy limit of the system, analyzing the influence mechanism of the model parameter calibration error on the measurement accuracy, and determining the way to improve the accuracy.

[0048] Research on integration of UAV and high-precision measurement platform

[0049] Through SDK, binocular cameras and drones are combined to realize the integration of drones and high-precision measurement platforms. The stability of the integrated system, the coordination between the camera and image transmission signals, and the shooting angle control of the binocular camera are studied.

[0050] Research on stability technology of high-precision measurement platform

[0051] The stability of the binocular camera mounted on the UAV is studied, the mounting structure is reasonably designed, the installation is simple, and the stability of the entire measurement system is improved. The binocular camera structure can be rotated during operation to facilitate finding the most suitable shooting angle and capturing clear images.

[0052] Research on high-precision internal parameter calculation of measurement system

[0053] The distance of the space target is hundreds or even thousands of times the effective focal length of the camera. A small deviation in the internal and external parameters of the visual measurement system or the image imaging point will cause the error of the measurement result to increase exponentially, so it is crucial to improve the calibration accuracy. The calibration accuracy of the binocular system can be improved through reasonable algorithm design, and the influence of factors such as feature point extraction error on the calibration accuracy of the binocular vision system can be reduced. The calibration method of the binocular vision measurement system is studied to estimate the system parameters, and then realize the correspondence between the three-dimensional space and the matching image pair, so that the binocular vision measurement system can reversely infer the accurate position and posture information of the target based on the acquired image.

[0054] Research on Optimization of Dimension Measurement of Measuring System

[0055] This scheme uses camera extrinsic parameters to establish the measurement coordinate system. In order to further eliminate the system error introduced by parameter calibration error, a model-based optimization method is used to compensate for the system parameter calibration error using camera extrinsic parameters.

[0056] Research on high-precision image distortion correction technology

[0057] For a given system, the idea of ​​improving measurement accuracy is to improve the fitting accuracy of model parameters to the system. The fitting accuracy is divided into two parts: one is the fitting accuracy of the model to the system itself, and the other is the estimation accuracy of model parameters. Fitting accuracy mainly refers to the fitting accuracy of a single camera model. For an optical system, the adjustment error destroys the consistency of the optical axis of the optical system and the ideal imaging characteristics. On the one hand, it leads to unstable calibration of camera parameters, which may converge to a local optimum during iterative optimization; on the other hand, it affects the fitting accuracy of the camera model. The camera model also includes a distortion model. The factors that cause camera distortion are complex and diverse, so almost all distortions are only approximate fits. By analyzing the distortion characteristics of the optical system based on aberration theory, the distortion model of the binocular camera is studied, so as to determine the high-precision distortion calibration technology.

[0058] Research on high-precision measurement management system

[0059] This solution is based on the principle of stereoscopic vision measurement and designs a high-precision measurement management system. The design of the system software mainly includes: structured grating projection, image display, camera stereo calibration and calculation, three-dimensional reconstruction, tower information visualization, two-dimensional detection, result display, configuration file saving and loading, binocular camera parameter setting and other functional modules integrated development.

[0060] Intelligent route planning and design for drones

[0061] In order to improve the efficiency and intelligence of the refined route planning of transmission line towers, find the best shooting point to take a clear image of the object to be tested, and reasonably set the shooting distance and angle and other parameters. The system automatically generates the shooting points of the key components of the inspection and the UAV flight auxiliary points, and generates the high-precision and optimal flight route for this sortie. Route planning needs to comprehensively consider the characteristics of the UAV model, endurance time, operating efficiency, inspection data collection quality, tower type and other conditions, so as to achieve the maximum guarantee of shooting quality and efficiency under the condition of automatic refined inspection of the tower.

[0062] Parameter calculation within the measurement platform

[0063] In 1999, Dr. Zhengyou Zhang of Microsoft Research innovatively proposed a camera calibration algorithm based on a two-dimensional chessboard image calibration plate. This method combines the advantages of good robustness of traditional calibration algorithms and low dependence of self-calibration algorithms on calibration objects. It does not require the calibration plate to move parameters, but only requires the calibration template to move in two directions for calibration. This method has high calibration accuracy and low cost. It has been widely used in desktop vision systems.

[0064] High-precision image distortion correction

[0065] The solution based on the collinearity condition equation is the most important and most widely used method, which includes the image point coordinate error equation method based on the collinearity condition equation, the multi-piece space forward intersection solution, the space rear intersection solution and various light beam solutions.

[0066] The spatial resection solution based on collinear equations uses the image coordinates of the control points (including the object coordinates when necessary) as observation values ​​to solve the camera's internal and external orientation elements, distortion differences, and other additional parameters. It is a calibration method that can be divided into single-chip spatial resection solution and multi-chip spatial resection solution.

[0067] Automatic or semi-automatic measurement technology for specific components

[0068] Analyze and solve the shape of specific components and build a physical model. Through the specific physical model, the measurement software can perform deep learning and recognition. A database is established to provide underlying support for automation technology.

[0069] Project technical route and implementation plan

[0070] The overall structural design of this project:

[0071] This project plan consists of three parts: 3D route planning software, binocular camera measurement platform and image measurement system. First, the route is planned by 3D route planning software, and then the drone is equipped with binocular camera measurement platform to perform image acquisition tasks according to the track file. During the operation, the onboard computer on the binocular camera measurement platform can transmit the captured data back to the server in real time through the 4 / 5G network. The ground operators can analyze and measure the images downloaded from the cloud through the PC-side image measurement system and output the measurement report. The project implementation plan is as follows:

[0072] Research on Intelligent Route Planning Technology

[0073] In order to improve the efficiency and intelligence of fine route planning of transmission line towers, a typical component recognition method based on Kinect depth image and point cloud edge features is proposed. First, the acquired depth image is processed by nonlinear filtering optimization algorithm to obtain the optimized target point cloud and the eight-neighborhood depth difference algorithm is proposed to extract the edge of the point cloud; then the random sampling consensus (RANSAC) algorithm is used to detect the edge of the segmented point cloud and extract the defined edge features to identify the components. According to the set parameters such as camera focal length, shooting distance and angle, the system automatically generates the photo points of the key components of the inspection and the UAV flight auxiliary points, and generates the high-precision and optimal flight route for the sortie. Route planning needs to comprehensively consider the characteristics of the UAV model, endurance time, operation efficiency, inspection data collection quality, tower type and other conditions, so as to achieve the maximum guarantee of inspection quality and efficiency under the condition of automatic fine inspection of towers.

[0074] Research on high-precision measurement technology based on stereo vision

[0075] The research route of binocular camera measurement platform and image measurement system shows that high-precision measurement technology based on stereo vision, integration of drone and high-precision measurement platform, and high-precision measurement platform stability technology are research carried out on the hardware of binocular camera measurement platform; high-precision internal parameter solution of measurement system, dimensional measurement optimization of measurement system, high-precision image distortion calibration technology and high-precision measurement management system are research carried out on image measurement system (software).

[0076] Visual measurement technology has become the preferred measurement method for close-range space operations due to its outstanding advantages such as high precision, strong concealment, low cost, simple equipment, small size, and low power consumption. Visual measurement technology can not only provide geometric information and color information of the target spacecraft, but also the collected target image information is more in line with human information acquisition needs. It can be seen that the study of measurement methods, related technologies and theories involved in visual measurement systems is of great significance for promoting the progress of space operation technology. According to different visual measurement methods, the targets to be operated can be generally divided into cooperative targets and non-cooperative targets. Cooperative targets refer to targets with some known pattern information that can be used as prior knowledge for the visual system, such as cooperative targets with specific shape markers installed on the target spacecraft. When the spacecraft pattern shape and other information are known, the information collection of cooperative targets is easier to achieve. Non-cooperative targets refer to target spacecrafts whose own information is unknown and cannot provide strong information support for the executing spacecraft, such as scrapped spacecrafts and space debris. At present, various types of space activities are frequent, and it is particularly important to achieve on-orbit maintenance and the removal of space debris such as faulty aircraft, which can not only improve the efficiency of space development but also save the cost of space activities. Space technology research still faces many problems that need to be solved. The solution of such problems involves key technologies such as high-precision posture measurement of non-cooperative targets.

[0077] The main components of the visual measurement system are cameras. The target reflects or emits light beams through the optical system to reach the visual sensor. The electronic components complete the real-time and high-speed acquisition of the collected information, pre-process the collected grayscale image or color image data, transmit part of it to the display device, and transmit the other part to the image processing system to complete the effective processing of the image, complete the tasks of target extraction and posture measurement. The visual measurement methods are complex and diverse, corresponding to different visual system models, among which the two main visual systems are monocular model and binocular (multi-eye) model. The monocular model is simple, so the monocular method technology is the most, this type of method only uses the image sequence obtained by one camera, combined with the prior knowledge of the target to solve the posture parameters. Due to the accuracy of the monocular measurement method and the trouble of three-dimensional measurement, these factors limit some applications of monocular in the measurement field or bring some difficulties, especially for the case where the target geometric pattern or geometric constraint characteristics are completely unknown. For non-cooperative targets, the three-dimensional reconstruction method based on binocular (multi-eye) shows its superiority. Binocular vision measurement system is divided into two forms. One is that two cameras simultaneously obtain digital images of the target within the overlapping field of view, and the other is that a single mobile camera obtains digital images of the target at different positions in time, and processes the image through related theories such as projection transformation to reconstruct the target position and posture. Binocular vision has many advantages. Compared with the monocular measurement system, it is the simplest visual model that can achieve three-dimensional reconstruction.

[0078] Points in the objective world are three-dimensional points, and points in the image plane are two-dimensional points. The transformation from the objective world to the image plane is completed through the camera imaging model. Camera calibration is to solve the parameters of the imaging model, mainly including the camera's internal parameters (camera focal length, image principal point, radial distortion coefficient and tangential distortion coefficient) and external parameters (rotation matrix, translation matrix). Solving these parameters lays a solid foundation for the next step of image correction.

[0079] After completing the camera calibration, the next step is image correction. This module has two functions. First, the distortion coefficients in the intrinsic parameters obtained by camera calibration are used to remove the radial and tangential distortion of the image to obtain a distortion-free image. Second, the distortion-free image is transformed through geometric figures so that two cameras that are not in the same plane can strictly correspond to each other. The epipolar lines of the two images can be on the same horizontal line. In stereo matching, only one-dimensional search in a certain line is required to match the corresponding points.

[0080] Stereo matching is to find matching corresponding points from images with different viewpoints. Therefore, the matching problem in stereo matching can be seen as a process of finding the degree of correlation between two sets of data. Stereo matching is to calculate the matching cost of the left and right images through one or more matching primitives, cluster the matching costs, obtain the initial disparity map, and then perform post-processing of interpolation and fitting on the initial disparity map to obtain the final disparity map. The sixth step is to use the principle of triangulation to calculate the depth information of objects in the scene. The above are the five steps of a complete binocular stereo vision system.

[0081] The most basic goal of stereo vision research is to obtain three-dimensional information of the scene. To achieve this goal, a complete stereo vision process usually includes six parts: image acquisition, camera calibration, feature extraction, stereo matching, three-dimensional coordinate solution and post-processing. Image acquisition is the basis of stereo vision technology research. There are many ways to acquire images, which are determined by the specific application occasions and purposes. The acquisition of stereo images must not only meet the application requirements, but also consider the influence of viewpoint differences, lighting conditions, camera performance and scene characteristics. After the acquired image is matched with the target point through feature extraction, combined with the internal and external parameters obtained by camera calibration, the coordinates of the target point in the world coordinate system can be calculated to achieve the purpose of measurement.

[0082] Research on integration of UAV and high-precision measurement platform

[0083] Because high-precision stereoscopic vision measurement requires multiple cameras to shoot at multiple angles, the camera needs to read and store the drone's own GPS information, calibrate the photo shooting location, and the camera needs to shoot at different angles. Therefore, how to offset the impact of environmental factors on camera shooting quality and take measures to reduce or eliminate such errors is the key technology for the integration of drones and high-precision measurement platforms.

[0084] Use a commercial binocular camera. If the test accuracy does not meet the requirements, consider customizing the binocular camera. Based on the SDK software development kit of the drone, expand the application field of the drone, realize the integrated control of software and hardware, and expand the flight operation function. Based on the open API interface of the drone, the customized load can communicate with the internal systems such as the drone flight control, GPS and image transmission to realize functions such as control and data acquisition.

[0085] Research on stability technology of high-precision measurement platform

[0086] The purpose of the three-axis stabilized gimbal control system design is to ensure that the gimbal can still shoot and search for targets when there is wind resistance disturbance outside the system, to respond to the system quickly and to maintain high control accuracy, so that the shooting picture can still remain stable when the optoelectronic device shoots the target. In order to achieve these goals, the control system of the three-axis stabilized gimbal consists of two parts: hardware circuit drive part, software program design part, and neural network algorithm control part.

[0087] First, the overall control system of the three-axis stabilized gimbal is studied, and the overall design process of the entire control system and the functions that each module should realize are designed. Then, the overall control model block diagram of the gimbal is analyzed, and the mathematical model of the three-axis stabilized gimbal is established. Finally, the transfer function of the gimbal control system is obtained, and the factors affecting the stability control of the gimbal are analyzed, and a solution is given. The hardware of the gimbal control system is studied, the type of motor is selected, and the sensors required by the hardware system are selected to ensure that the gimbal control system can obtain accurate attitude information, paving the way for the subsequent stability control. Then, the main modules of the entire hardware system of the three-axis stabilized gimbal are studied to ensure the rationality of the design of each hardware module.

[0088] The algorithm part of the gimbal control system is studied, the learning and training method of the RBF artificial neural network is studied, the RBF artificial neural network algorithm control structure of the three-axis stabilized gimbal control system and the three-axis stabilized gimbal is combined, and finally, through simulation experiments, it is verified that compared with traditional control, the RBF artificial neural network control response and anti-interference aspects are more advantageous.

[0089] Study the PTZ control system software, the overall process of the software framework, serial port, communication module, attitude settlement module, and neural network algorithm implementation module. Build the entire three-axis stabilized PTZ control system for debugging and experimentation and analyze the experimental results, so that each hardware module can work normally and meet the design requirements, the software can achieve the established functions, and the algorithm part can also achieve a certain degree of anti-interference control.

[0090] The purpose of designing the three-axis stabilized gimbal control system is to ensure that the gimbal can still shoot and search for targets when there is wind resistance disturbance outside the system, to respond to the system quickly and to maintain high control accuracy, so that the shooting picture can still remain stable when the optoelectronic device shoots the target. In order to achieve these goals, the control system of the three-axis stabilized gimbal consists of two parts: the hardware circuit drive part, the software program design part, and the neural network algorithm control part. First, the two-axis stabilized gimbal control system is introduced as a whole, then the main components of the entire control system are selected, and finally the mathematical model of the gimbal motor is established.

[0091] Research on high-precision internal parameter calculation of measurement system

[0092] The calibration algorithm of the binocular vision measurement system is studied to estimate the system parameters and then realize the correspondence between the three-dimensional space and the matching image pairs. Only in this way can the binocular vision measurement system infer the accurate position and posture information of the target based on the acquired images.

[0093] Direct linear transformation solution: Camera calibration uses direct linear transformation solution, which is a solution to establish a direct linear relationship between the image coordinates and the corresponding object point object space coordinates. The image coordinates here refer to the image coordinates that do not use the image principal point as the coordinate origin. This method does not require the initial values ​​of the internal and external orientation elements, so this method is particularly suitable for solving the calibration parameters of non-measurement cameras. However, this method is a single model solution, and has high requirements on the number and distribution of control points, otherwise it is easy to be ill-conditioned during the solution process.

[0094] Collinearity equation solution: The solution based on collinearity equation is the most important and most widely used method, including the image point coordinate error equation method based on collinearity equation, multi-chip spatial intersection solution, spatial resection solution and various light beam solutions. The spatial resection solution based on collinearity equation uses the image coordinates of the control points (including the object coordinates when necessary) as observation values ​​to solve the camera's internal and external orientation elements, distortion and other additional parameters. It can be divided into a single-chip spatial resection solution and a multi-chip spatial resection solution.

[0095] In principle, the direct linear transformation solution is derived from the collinearity condition equation. The commonly used collinearity condition equation is expressed as:

[0096]

[0097] Where: x, y are image plane coordinates;

[0098] x 0 ,y 0 , f is the orientation element in the image;

[0099] X s , Y s , Z s , ψ, ω, κ are the exterior orientation elements of the image;

[0100] X, Y, and Z are object coordinates;

[0101] a i , b i , c i (i=1,2,3) are the direction cosines formed by the external azimuth elements of the image.

[0102] Distortion correction: The geometric distortion of an image refers to the error in the measured geometric position of an image point at a target point on the image plane, which causes the imaging system to fail to strictly satisfy the pinhole imaging model (or central projection relationship) between the image and the actual scene within the entire field of view, causing the central projection ray to bend. Distortion can be divided into radial distortion and tangential distortion. Radial distortion refers to the distortion difference of an image point deviating from its ideal position in the radial direction, which is only related to the distance of the image point from the image principal point. The radial distortion is symmetrical, but its center of symmetry does not completely coincide with the image principal point. For simplicity, the image principal point is usually regarded as the center of symmetry. The positive or negative value of radial distortion is related to its offset direction. The offset away from the image principal point is positive, and the offset close to the image principal point is negative.

[0103] According to the theory of geometric optics, the radial distortion error △r of the lens can be expressed by the following odd-order polynomial:

[0104] △r=k 1 r 3 +k 2 r 5 +k 3 r 7 +L type 1

[0105]

[0106] Where: r——image point radius;

[0107] x, y——image point coordinates;

[0108] x 0 ,y 0 ——Image principal point coordinates;

[0109] ki (i=1,2,3L)——radial distortion parameter;

[0110] The eccentric distortion error is caused by the manufacturing, installation and vibration of the lens, which causes the lens units to deviate from the lens axis or tilt, causing the image point to deviate from its theoretical position in both the radial and tangential directions. Under normal circumstances, the eccentric distortion error is much smaller than the radial distortion error, and the total error caused by non-radial distortion is about 1 / 7 to 1 / 8 of the error caused by radial distortion. Similar to radial distortion, in close-range photogrammetry, eccentric distortion changes with the change of focal length D, and the ground objects (distance D') at different focal lengths D also have changes in eccentric distortion. Taking into account the eccentric distortion correction of the focal length D under close-range conditions is shown in the following formula:

[0111]

[0112] Where: △x D , △y D ——Eccentric distortion difference component when the focal length is D;

[0113] f——the principal distance when the focal length is D;

[0114] D——focus adjustment;

[0115] p 1 , p 2 ——Eccentric distortion parameter.

[0116] Space forward intersection:

[0117] The work of solving the coordinates of model points (or ground points) by using the internal orientation elements of the two images of a stereo image pair, the coordinates of the image points with the same name and the relative orientation elements (or external orientation elements) of the image pair.

[0118] 1. Use the relative orientation elements of the two images of the stereo image pair to calculate the three-dimensional coordinates of the model point;

[0119] 2. Use the exterior orientation elements of the two images of the stereo pair to calculate the ground coordinates of the ground point.

[0120] Spatial Resection:

[0121] A method for determining the exterior orientation elements of an image using ground control points and their image points on the image. This method requires at least three ground control points that are not in a straight line.

[0122] Pyramid method: using collinearity equations to solve the exterior orientation elements of the image

[0123]

[0124] Research on Optimization of Dimension Measurement of Measuring System

[0125] The coupling of the coaxial error of the optical system and the camera model parameters may add position changes to the camera extrinsic parameters, which is contrary to the camera model theory. Secondly, the position transformation will also be transferred to the structural parameters, resulting in structural parameter errors. In the calibration process, factors such as checkerboard error, feature point extraction error, and image noise determine the inevitable errors in the calibration parameter results, so it is very necessary to further optimize the system. Binocular system calibration refers to the calibration of the system's intrinsic parameters, mainly the left and right camera intrinsic parameter matrices, distortion parameters, and system structural parameters. Usually, the position relationship between the camera and the target is unknown. In the calibration process, the camera's extrinsic parameters are used as intermediate quantities to work together with the system's intrinsic parameters to achieve camera fitting. Camera extrinsic parameters are also very important parameters, but they are usually ignored. For example, the camera extrinsic parameters are discarded and a camera coordinate system in the binocular system is directly set as the measurement coordinate system. If there is no position change component in the camera extrinsic parameters, this processing method is not wrong, otherwise it is likely to introduce system errors.

[0126] Based on the above analysis, this scheme uses camera extrinsic parameters to establish the measurement coordinate system. In order to further eliminate the system error introduced by parameter calibration error, a model-based optimization method is proposed to compensate the system parameter calibration error using camera extrinsic parameters. In theory, this optimization process can be directly performed together with calibration, but the following factors determine the necessity of separate optimization:

[0127] (1) During the calibration process, a target coordinate system is established for each target, so it is not convenient to unify the coordinate system.

[0128] (2) The instability of the calibration process causes differences in the results between different groups. The usual way to deal with this is to take the average of multiple calibration results as the final calibration result, which destroys the results of each calibration process.

[0129] coordination.

[0130] (3) Parameter calibration is for the entire measurement space, but the system has local errors, such as distortion, which are difficult to eliminate through calibration alone.

[0131] Research on high-precision image distortion correction technology

[0132] For a given system, the idea of ​​improving measurement accuracy is to improve the fitting accuracy of model parameters to the system. The fitting accuracy is divided into two parts: one is the fitting accuracy of the model to the system itself, and the other is the estimation accuracy of model parameters. Fitting accuracy mainly refers to the fitting accuracy of a single camera model. For an optical system, the adjustment error destroys the consistency of the optical axis of the optical system and the ideal imaging characteristics. On the one hand, it leads to the instability of the calibration of the camera parameters, which may converge to the local optimum during the iterative optimization process; on the other hand, it affects the fitting accuracy of the camera model. The camera model also includes a distortion model. The factors that cause camera distortion are complex and diverse, so almost all distortions are only approximate fits. In order to improve the fitting accuracy of single camera model parameters, the aberration theory of optical systems is first studied, and then related research is carried out on three aspects, including the coupling of internal and external parameters of the camera model, non-coaxiality and distortion of the optical system, and targeted solutions are proposed or improved.

[0133] Image distortion refers to the errors in spectral characteristics and geometric characteristics between remote sensing images and the real surface images they reflect. That is, radiation error and geometric error. The former is manifested as the distortion of the image in grayscale; the latter is manifested as the deformation of the geometric relationship. Image distortion is an important indicator for testing the interpretation and mapping performance of remote sensing images. If you do not understand the nature and degree of image distortion and use uncorrected distorted remote sensing images, it is impossible to obtain the correct application effect and may even be counterproductive. Therefore, the original data of remote sensing images usually need to be corrected by radiation and geometry.

[0134] The pose between two coordinate systems is determined by representing a set of points in two coordinate systems. The main source of error in the three-dimensional point cloud pose measurement algorithm based on the binocular measurement system is the reconstruction error. Since the target coordinate system is self-built, the coordinates of the points in the two coordinate systems are all observation values, which further increases the impact of the point cloud reconstruction error on the accuracy of the pose solution. Here, the reconstruction error refers to the random error caused by image noise, which reduces the impact of random errors on the accuracy of the pose estimation algorithm.

[0135] The binocular vision system collects images at M different positions, obtains the coordinates of the point cloud in the target coordinate system at each position, and obtains the pose transformation matrix of the binocular vision system relative to the target coordinate system at the final position M based on the mutual constraints of multiple positions.

[0136] The camera uses the pinhole imaging principle to convert the scene into an image, and the pinhole imaging is achieved through the lens during the imaging process. The scene passes through the lens and forms a reduced inverted image in the photosensitive component of the camera. In this process, the light is refracted through the lens to form an image. However, the refraction of the lens is related to the crystal structure of the lens. Its physical properties make the light propagate in a straight line at the optical center. As the distance between the light and the optical axis of the optical center increases, the refraction effect caused by the lens becomes more obvious, which causes the light farther away from the optical axis to deviate further from the optical center. This forms lens distortion. In general, lens distortion is actually a general term for the perspective distortion effect inherent in optical lenses, which makes it impossible for the imaging system to strictly satisfy the pinhole imaging model for the actual scene and image in the full field of view, and causes the central projection ray to bend. Generally speaking, lens distortion is mainly nonlinear distortion, including radial distortion and tangential distortion. Radial distortion refers to the change in length of the vector end along the ray direction, that is, the change in the radius vector. Common barrel distortion and pincushion distortion are caused by radial distortion. Tangential distortion is caused by the change of the ideal point along the tangent direction, which can also be understood as a change in the angle direction. Ideal lens distortion is caused by the inherent distortion of the ideal lens. In fact, the cause of lens distortion is not only due to the inherent distortion of the lens. The accuracy and quality of the lens also have a certain degree of influence on lens distortion. This is manifested in the "lens processed by the lens manufacturer is not an ideal smooth lens" model. Two lenses of the same type processed by the same manufacturer will inevitably have some differences due to processing reasons. In addition, during the installation of the lens, due to various objective mechanical position reasons and manual installation operation reasons, it is impossible for the camera to perform visual inspection in a completely ideal environment, which makes the lens distortion have more random influence factors. Under normal circumstances, the effect of lens distortion on the image is minimal. However, in machine vision inspection, especially high-precision machine vision inspection, lens distortion is a major factor affecting accuracy. Therefore, this solution adds distortion correction to the camera lens in the currently used visual inspection system, which greatly reduces the influence of lens distortion on the image, thereby improving the inspection accuracy.

[0137] 4.1.8 Research on high-precision measurement management system

[0138] During measurement, the camera collects data quickly, and performs feature extraction and recognition, stereo matching, three-dimensional reconstruction, and motion analysis on the data stream in the buffer. Parallel computing is used to accelerate the entire algorithm process to ensure measurement efficiency. After the measurement is completed, the deformation field, velocity field, and acceleration field of the measured surface in different time periods are intuitively displayed on the software interface, and an analysis report is generated for archiving. Therefore, the software design mainly includes top-level design and bottom-level design. The top-level design includes UI interface design, software management, and data management. The bottom-level design includes system calibration, data acquisition, feature extraction and recognition, stereo matching, three-dimensional reconstruction, and deformation analysis. It is necessary to design the overall framework of the software and reasonably divide the various functional modules. Formulate software code specifications and design test methods to ensure that the entire software project is easy to modify and maintain.

[0139] 4.2 Project Innovations

[0140] (1) Develop an unmanned aerial vehicle (UAV) binocular camera measurement platform and image measurement system to achieve high-precision three-dimensional spatial measurement of power transmission lines based on digital optics, with a measurement error of less than 1 cm.

[0141] (2) Develop a 3D route planning software to automate the operation of the drone-mounted binocular camera measurement platform.

[0142] (3) Improve the efficiency of key position measurements of power grid equipment, improve measurement accuracy, reduce operational risks such as induced electricity, save operating costs, and realize intelligent and standardized operations.

[0143] Project results to achieve function

[0144] (1) Intelligent route planning and design system function

[0145] Design the flight route according to the needs to realize the automatic flight operation of the UAV-mounted binocular camera measurement platform.

[0146] (2) Binocular camera measurement platform function

[0147] ① Real-time transmission of captured images, support for real-time data transmission over 4G / 5G wireless networks, and real-time data analysis;

[0148] ②Support binocular camera 0°~360° horizontal rotation and 0°~90° vertical rotation;

[0149] ③ During operation, the inspection screen is transmitted by an intelligent camera, which is convenient for taking pictures.

[0150] (3) The image measurement system is used to analyze the images taken by the "binocular camera measurement platform" and has the following functions:

[0151] ①Support point-to-point, point-to-line distance measurement, line-to-line angle measurement, line-to-surface angle measurement, and three-point area measurement;

[0152] ②Support image distortion correction;

[0153] ③Support one-click output of measurement report;

[0154] ④Support free import of camera parameters;

[0155] ⑤Support automatic image classification;

[0156] ⑥Support the selection of points with the same name;

[0157] ⑦Supports the matching of binocular photos and tower coordinates, and can display the line and tower number at the corresponding position in the photo.

[0158] Product Technical Parameters

[0159]

[0160]

[0161] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for fine-grained route planning of a transmission line tower, characterized in that: include: In the 3D route planning software, according to the newly created or opened inspection task, open the point cloud to pick up the tower, open the tower simulation, and then draw the target; After the target is drawn, the route is produced and the parameters are set; Then preview the flight, publish the route or export the route after previewing.

2. A transmission line tower refined route planning method as claimed in claim 1, characterized in that: Use a drone equipped with a binocular camera measurement platform to pick up the point cloud of the tower.

3. A transmission line tower refined route planning method as claimed in claim 2, characterized in that: The binocular camera measurement platform picks up the point cloud of the tower pole, which specifically includes image acquisition, camera calibration, stereo matching, three-dimensional coordinate solution and post-processing.

4. A transmission line tower refined route planning method as claimed in claim 3, characterized in that: Image acquisition includes using two cameras to simultaneously acquire digital images of targets within overlapping fields of view, or moving a single camera to acquire digital images of targets at different locations in different time periods, and processing the images through projection transformation and other related theories to reconstruct the target position and posture.

5. A transmission line tower refined route planning method as claimed in claim 3, characterized in that: Camera calibration includes solving the parameters of the imaging model, which mainly include the intrinsic and extrinsic parameters of the camera. The intrinsic parameters include the camera focal length, image principal point, radial distortion coefficient and tangential distortion coefficient, and the extrinsic parameters include the rotation matrix and translation matrix.

6. A transmission line tower refined route planning method as claimed in claim 3, characterized in that: Image correction includes first using the distortion coefficients in the intrinsic parameters obtained by camera calibration to remove the radial and tangential distortions of the image to obtain an undistorted image. Secondly, the undistorted image is transformed through geometric figures so that the two cameras that are not in the same plane are strictly aligned, and the epipolar lines of the two images can be on the same horizontal line. During stereo matching, only one-dimensional search in a certain row is required to match the corresponding points.

7. A transmission line tower refined route planning method as claimed in claim 3, characterized in that: Stereo matching includes finding matching corresponding points from images of different viewpoints, calculating the matching cost of the left and right images through one or more matching primitives, clustering the matching costs to obtain the initial disparity map, and then performing post-processing of interpolation and fitting on the initial disparity map to obtain the final disparity map.

8. A transmission line tower refined route planning method as claimed in claim 3, characterized in that: Three-dimensional coordinate solution and post-processing include using the principle of triangulation to calculate the depth information of objects in the scene.

9. A transmission line tower refined route planning method as claimed in claim 3, characterized in that: Typical parts recognition based on Kinect depth image and point cloud edge features of the acquired images.

10. A transmission line tower refined route planning method as claimed in claim 9, characterized in that: Firstly, the acquired depth image is processed by using nonlinear filtering optimization algorithm to obtain the optimized target point cloud, and an eight-neighborhood depth difference algorithm is proposed to extract the edge of the point cloud. Then, the random sampling consensus algorithm is used to detect the edges of the segmented point cloud and extract the defined edge features to identify the parts; Based on the set parameters such as camera focal length, shooting distance and angle, the system automatically generates photo points and drone flight assistance points for key inspection components, and generates a high-precision and optimal flight route for the flight.

Images