A method for verifying the validity of UAV routes based on laser point cloud
Verifying the drone route through laser point cloud technology, solving the risk of tower crashes and unqualified photos caused by the pilot's track deviation, realizing the safety and accuracy of the track, and improving the pilot's control ability .
Patent Information
- Application Number
- CN202210450832.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-26
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-04-26
AI Technical Summary
During the drone pole tower inspection mission, the pilot's track deviation leads to the risk of the drone hitting the tower and the unqualified photos. It is difficult for the existing technology to effectively verify and improve the pilot's fine control ability.
Using the drone route validity verification method based on laser point cloud, the real flight track latitude and longitude coordinates are converted to UTM coordinates, match the three-dimensional point cloud tower model, determine the photo imaging range, and simulate track flight in the three-dimensional model for effective effectiveness Sex verification, and an evaluation report is generated.
It improves the safety and accuracy of the track, tests the pilot's refined patrol capabilities, avoids inefficient or unsafe tracks, and ensures the effectiveness and completeness of the photos taken.
Smart Images

Figure CN114842358B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) track verification, and in particular to a method for verifying the validity of an UAV route based on laser point cloud. Background Art
[0002] As an emerging technology, low-altitude UAV control technology has demonstrated its unique role in various industries. Due to the low cost, no risk of casualties, good maneuverability, beyond visual range flight, easy and efficient use, it is widely used in various industries. However, with the popularization of UAV applications, people's requirements for the technical level of UAV control are getting higher and higher, and the demand for UAV pilots is also increasing. Pilots have emerged. Pilots control UAVs to perform more detailed tasks, such as surveying and mapping, electricity, meteorology, environmental protection, and marine surveillance. Since the pilots' tasks are completely dependent on the pilots' skills and experience, it is inevitable that there will be deviations from the track and the collected photos will not meet the task requirements. In the pole tower inspection flight mission, the deviation from the track will lead to the risk of the UAV crashing into the tower and the pole tower inspection photos taken are unqualified, and the risk of re-executing the mission. Summary of the invention
[0003] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method for verifying the validity of a UAV route based on laser point cloud.
[0004] In order to achieve the above object, the present invention provides the following technical solutions:
[0005] A method for verifying the effectiveness of a UAV route based on laser point cloud comprises the following steps:
[0006] Convert the actual flight track longitude and latitude coordinates to UTM coordinates;
[0007] Match the 3D point cloud tower that corresponds to the tower type in the actual flight track;
[0008] Determine the photo imaging range of the 3D point cloud tower;
[0009] Verify the validity of the simulated trajectory.
[0010] In the present invention, preferably, the actual flight track adopts json format, and the longitude and latitude coordinates in json format are converted into UTM coordinates.
[0011] In the present invention, preferably, the matching of the three-dimensional point cloud tower corresponding to the tower type in the actual flight track specifically includes the following steps:
[0012] Convert the latitude and longitude coordinates of the actual tower records and track files to UTM coordinates, which are the same as the coordinate format of the point cloud tower model and point cloud tower model records;
[0013] Calculate the offset data between the point cloud tower and the actual tower position, and translate the point cloud tower position according to the offset data;
[0014] Rotate the point cloud tower to match the actual tower channel orientation.
[0015] In the present invention, preferably, the rotating point cloud tower to match the actual tower channel direction specifically includes the following steps:
[0016] Calculate the actual tower channel vector based on the actual large side coordinates and the actual small side coordinates;
[0017] Calculate the point cloud tower channel vector based on the point cloud large side coordinates and the point cloud small side coordinates;
[0018] Calculate the actual tower and point cloud tower channel vector length;
[0019] Calculate the cosine value of the angle α between the actual tower vector and the point cloud tower vector, and use the arccosine formula to obtain the final angle angle1;
[0020] Combined with the graphics rotation matrix, the entire point cloud model is rotated according to the angle so that the three-dimensional point cloud tower matches the actual track direction and position.
[0021] In the present invention, preferably, the step of determining the photographic imaging range of the three-dimensional point cloud tower comprises the following steps:
[0022] The actual shooting position, the horizontal angle between the camera and true north, the camera parameters and the shooting pitch angle are known;
[0023] According to the graphics transformation matrix algorithm, combined with the horizontal angle between the camera and the true north, the shooting pitch angle is calculated to obtain the transformation matrix matrix, which is rotated relative to the true north (0,1,0) to obtain the final camera orientation vector;
[0024] Calculate the cosine of the angle between the camera's orientation vector and the vector pointing to the shooting point, and use the inverse cosine formula to get the angle angle2;
[0025] Calculate the distance from the target point to the shooting center and the depth value of the target point from the camera position.
[0026] In the present invention, preferably, after determining the photographic imaging range of the three-dimensional point cloud tower, it also includes the individual naming of the tower components. Based on the point cloud automatic classification technology, the tower components are named individually, and the named components include the tower, ground wire, insulator, insulator conductor end, and insulator crossarm end.
[0027] In the present invention, preferably, by performing individual naming on the tower components, the shooting target point positions of the tower, tower head, tower body and tower base are calculated, which specifically includes the following steps:
[0028] Calculate the classified point cloud of the tower to obtain the center point of the tower (centerX, centerY), and take the highest point as the topZ of the tower and the lowest point as the bottomZ of the tower;
[0029] The coordinates of the location for taking the full view of the tower are: (centerX, centerY, (bottomZ+topZ) / 2);
[0030] The coordinates of the tower head shooting position are: (centerX, centerY, t_maxZ-(t_maxZ-t_minZ) / 4)
[0031] The coordinates of the tower shooting position are: (centerX, centerY, ((bottomZ+topZ) / 2-bottomZ) / 3*2+bottomZ)
[0032] The coordinates of the tower base shooting position are: (centerX, centerY, bottomZ).
[0033] In the present invention, preferably, the positions of the shooting target points of the left and right ground wires are determined by the ground wire individualization results; and the shooting positions of the insulators, insulator wire ends and insulator crossarm ends are determined by the insulator individualization results.
[0034] In the present invention, preferably, the validity verification of the simulated track is performed. If the track hits the tower, the actual inspection track of the pilot is unqualified and the simulation verification is stopped. If the route does not hit the tower, the tower point cloud is simulated inspected and photographed based on the imported inspection route to obtain verification results photos. The results photos are named according to the main monomers within the imaging range.
[0035] In the present invention, preferably, the number of simulated shots is compared with the number of standard shots, and a report on the missed shooting rate of shooting points is output; the validity of the taken photos is judged, the effective ratio of the inspection photos is calculated, and a photo effectiveness evaluation report is output; it is compared whether the single body parts captured in the taken photos include all the standard shooting parts, the taken photos are named according to the captured single body content, and a shooting target name report is output.
[0036] Compared with the prior art, the present invention has the following beneficial effects:
[0037] The method of the present invention establishes a three-dimensional point cloud model that matches the tower in the actual flight track, simulates the flight track inspection in the three-dimensional point cloud model, takes photos of the model, effectively evaluates the photos, and generates an evaluation report. By simulating the flight track inspection in the three-dimensional model and analyzing the effectiveness of the simulated photos, the safety and accuracy of the actual flight track can be judged, and the pilot's refined inspection and actual flight ability can be tested at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 The present invention is a flowchart of a method for verifying the effectiveness of a UAV route based on laser point cloud.
[0039] Figure 2 The present invention provides a schematic flow chart of the simulation track validity verification process of the UAV route validity verification method based on laser point cloud.
[0040] Figure 3 This is a schematic diagram of the classification of tower components in the method for verifying the effectiveness of a UAV route based on laser point cloud described in the present invention.
[0041] Figure 4 This is a schematic diagram of the classification of insulator components in the UAV route validity verification method based on laser point cloud described in the present invention. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0043] It should be noted that when a component is referred to as being "fixed to" another component, it may be directly on the other component or there may also be a component centered. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may also be a component centered. When a component is considered to be "set on" another component, it may be directly set on the other component or there may also be a component centered. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.
[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.
[0045] Please also see Figure 1 to Figure 2A preferred embodiment of the present invention provides a method for verifying the validity of a UAV route based on laser point cloud. By matching the three-dimensional point cloud model data of a tower with a similar tower type, the flight path actually planned by the pilot is simulated and reproduced in the three-dimensional point cloud model to verify the safety of the flight path and the validity of the photos taken of the flight path, thereby checking the operability of the flight path and the pilot's refined inspection and actual flight ability, and avoiding inefficient or unsafe flight paths. The method mainly includes the following steps:
[0046] Convert the actual flight track longitude and latitude coordinates to UTM coordinates;
[0047] Match the 3D point cloud tower that corresponds to the tower type in the actual flight track;
[0048] Determine the photo imaging range of the 3D point cloud tower;
[0049] Verify the validity of the simulated trajectory.
[0050] Specifically, based on the pole tower ledger, the actual flight track is matched with the three-dimensional point cloud model pole tower. The actual flight track is in json format, and the latitude and longitude coordinates of the actual flight track in json format are converted into UTM coordinates to keep consistent with the coordinates of the point cloud tower model and the point cloud tower model ledger.
[0051] Specifically, the selection of the three-dimensional point cloud tower needs to be similar to the tower type actually corresponding to the flight track, so as to perform verification based on the similar tower type, so that the verification is more in line with reality and the result is more accurate. After the imported actual flight track is converted into coordinates, the large and small side directions of the tower in the actual flight track can be determined by using the known tower ledger, and the direction of the three-dimensional point cloud tower is adjusted according to the large and small side directions of the tower in the flight track, so that the cloud model tower is consistent with the channel direction of the actual flight track. The matching of the three-dimensional point cloud tower corresponding to the tower type in the actual flight track specifically includes the following steps:
[0052] S01, using a general algorithm (universal transverse Mercator projection) to convert the latitude and longitude coordinates of the actual photographed tower ledger and track file into UTM coordinates to keep consistent with the latitude and longitude coordinates of the point cloud model tower and point cloud model tower ledger. The coordinates are in a unified format (X / Y / Z). Based on the known ledger information, it can be confirmed that:
[0053] Actual tower coordinates: (realTowerX, realTowerY, realTowerZ)
[0054] Real large side coordinates: (realTowerBigX, realTowerBigY, realTowerBigZ)
[0055] Real small side coordinates: (realTowerSmallX, realTowerSmallY, realTowerSmallZ)
[0056] Point cloud tower coordinates: (lasTowerX, lasTowerY, lasTowerZ)
[0057] Point cloud large side coordinates: (lasTowerBigX,lasTowerBigY,lasTowerBigZ)
[0058] Point cloud small side coordinates: (lasTowerSmallX, lasTowerSmallY, lasTowerSmallZ)
[0059] S02, calculate the position offset data between the point cloud tower and the actual tower.
[0060] OffsetX=realTowerX-lasTowerX;
[0061] OffsetY=realTowerY-lasTowerY;
[0062] OffsetZ=realTowerZ-lasTowerZ;
[0063] And translate the point cloud tower position according to the offset data, that is, all point cloud coordinates are added with the offset (OffsetX, OffsetY, OffsetZ), and the translated point cloud tower coordinates are (lasTowerX+OffsetX, lasTowerY+OffsetY, lasTowerZ+OffsetZ);
[0064] S03, rotating the point cloud tower to match the actual tower channel direction, specifically including the following steps:
[0065] S031, calculate the actual tower channel vector based on the actual large side coordinates and the actual small side coordinates:
[0066] realChannel.x=realTowerBigX–realTowerSmallX;
[0067] realChannel.y=realTowerBigY–realTowerSmallY;
[0068] realChannel.z=realTowerBigZ–realTowerSmallZ;
[0069] S032, calculate the point cloud tower channel vector based on the point cloud large side coordinates and the point cloud small side coordinates:
[0070] lasChannel.x=lasTowerBigX–lasTowerSmallX;
[0071] lasChannel.y=lasTowerBigY–lasTowerSmallY;
[0072] lasChannel.z=lasTowerBigZ–lasTowerSmallZ;
[0073] S033, calculate the actual tower and point cloud tower channel vector length:
[0074] According to the actual tower channel vector, the actual tower channel vector length is calculated as realChannel.length.
[0075] The length of the point cloud tower channel vector is calculated based on the point cloud tower channel vector as lasChannel.length;
[0076] S034, calculate the cosine value of the angle α between the actual tower vector and the point cloud tower vector based on the actual tower channel vector and the point cloud tower channel vector, and use the arc cosine formula to obtain the final angle angle1,
[0077] S035, combined with the graphics rotation matrix, rotates the entire point cloud model according to the angle angle1 to complete the matching of the three-dimensional point cloud tower with the actual track direction and position.
[0078] Specifically, a simulated flight is performed according to an imported actual flight track, the track includes parameters such as latitude and longitude coordinates, and the pitch angle of the gimbal. The point cloud model of the tower is verified and photographed based on the parameter information. The determination of the imaging range of the three-dimensional point cloud tower includes the following steps:
[0079] S11, the actual shooting position cameraPos, the horizontal angle between the camera and the true north cameraYaw, the camera parameters and the shooting pitch angle cameraPich are known;
[0080] According to the graphics transformation matrix algorithm, combined with the horizontal angle between the camera and the true north, the shooting pitch angle is calculated to obtain the transformation matrix matrix, which is rotated relative to the true north (0,1,0) to obtain the final camera orientation vector:
[0081] CameraV=matrix*(0,1,0)
[0082] S12, calculate the distance from the shooting point targetPos to the CameraV vector line. The closer the distance is, the closer it is to the center of the photo. The vector pointing to the shooting point is camera2Target = targetPos-cameraPos. Calculate the cosine value of the angle between the camera's orientation vector and the vector pointing to the shooting point based on the graphics formula, and then use the inverse cosine formula to get the angle angle2;
[0083] S13, calculate the distance distanceToCenter from the target point to the shooting center and the depth value deepToTarget from the target point to the camera position according to the distance formula:
[0084] distanceToCenter=sin(angle2) / |camera2Target|;
[0085] deepToTarget=cos(angle2) / |camera2Target|;The smaller the distanceToCenter is, the closer it is to the shooting center, and the smaller the deepToTarget is, the closer the imaging plane of the shooting point is to the camera point.
[0086] In this embodiment, based on the point cloud automatic classification technology, the tower components are named individually, and the named components include the tower, ground wire, insulator, insulator wire end, and insulator cross arm end.
[0087] like Figure 3 and Figure 4 As shown, specifically, by performing individual naming on the tower components, the shooting target point positions of the tower, tower head, tower body, and tower base are calculated, which specifically includes the following steps:
[0088] S21, calculate the classified point cloud of the tower, obtain the center point of the tower (centerX, centerY), and take the highest point as the top of the tower topZ, and the lowest point as the bottom of the tower bottomZ;
[0089] The coordinates of the location for taking the full view of the tower are: (centerX, centerY, (bottomZ+topZ) / 2);
[0090] The coordinates of the tower head shooting position are: (centerX, centerY, t_maxZ-(t_maxZ-t_minZ) / 4)
[0091] The coordinates of the tower shooting position are: (centerX, centerY, ((bottomZ+topZ) / 2-bottomZ) / 3*2+bottomZ)
[0092] The coordinates of the tower base shooting position are: (centerX, centerY, bottomZ).
[0093] S22, judging the positions of the shooting target points of the left and right ground lines according to the ground line monomerization result;
[0094] S23, judging the shooting target point position of the insulator and the shooting positions of the insulator conductor end and the insulator cross arm end through the insulator individualization result.
[0095] In this embodiment, when the point cloud model is matched, the validity of the simulated track is verified. During the verification process, if the track coordinates coincide with the point cloud tower coordinates, it means that the actual track has hit the tower, and the pilot's actual inspection track is unqualified, so the simulation verification is stopped; if the route does not hit the tower, inspection photos are taken one by one according to the imported inspection route, and based on the camera imaging principle, the tower point cloud is simulated and photographed to obtain verification results photos; and the results photos are named according to the main monomers within the imaging range.
[0096] Specifically, for the naming of ground wires, insulator strings, insulator conductor ends and insulator crossarm ends, the naming rule is first to name the monomer according to which target object is displayed within the camera imaging range. When there are multiple monomers in a photo, they are named according to the distance of the monomer from the camera and the distance of the monomer from the center of the photo. The monomer is close to the camera and the center of the camera photo. Since the shooting distance of the full view of the tower, the tower head, the tower base and the tower body is usually far away from the pole tower, these positions are easily confused when matching with the camera shooting point due to the long distance. Therefore, the identified ground wires, insulator strings, insulator conductor ends and insulator crossarm ends are first filtered out, and then the remaining photos are matched according to the distance of the monomer from the camera and the distance of the monomer from the center of the photo, and finally the photo name is determined.
[0097] In this embodiment, the number of shots in the simulated pilot's track is compared with the standard number of shots, the missed shot rate of the pilot's shooting points is calculated, and a missed shot rate report of the shooting points is output; the validity of the taken photos is judged to determine whether the photos are within the effective range of the camera imaging (the effective range is defined as 2 / 3 of the length and width of the photo), and the number of valid photos is calculated, the effective ratio of the inspection photos is calculated, and a photo validity evaluation report is output; it is compared whether the single body parts captured in the taken photos include all the standard shooting parts, the content of the photos taken in the actual simulated inspection is compared with the standard shooting parts, it is determined whether the parts of the actual photos include all the standard shooting parts, the photos are named according to the captured single body content, and a shooting target name report is output.
[0098] Specifically, for example, a total of 26 photos should be taken in the sheep horn tower model, including 26 shooting points, where the photo names include 1 photo of the whole tower, 1 photo of the tower head, 1 photo of the tower body, 1 photo of the tower base, 2 photos of channels, 6 insulator strings, 6 insulator crossarm end hanging points, 6 insulator wire end hanging points, and 2 ground wire hanging points. First, determine whether the total number of photos taken to simulate the actual track is 26, and then determine how many insulator wire ends, insulator crossarm ends, insulator strings and other monomers are included in the photos taken, determine which parts are not photographed or are not fully photographed, and output the track evaluation report based on the judgment results.
[0099] The above description is a detailed description of the preferred feasible embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modified changes completed under the technical spirit suggested by the present invention should fall within the patent scope covered by the present invention.
Claims
1. A method for verifying the effectiveness of a UAV route based on laser point cloud, characterized in that: The steps include: Convert the actual flight track longitude and latitude coordinates to UTM coordinates; Match the 3D point cloud tower that corresponds to the tower type in the actual flight track: Convert the latitude and longitude coordinates of the actual tower records and track files to UTM coordinates, which are the same as the coordinate format of the point cloud tower model and point cloud tower model records; Calculate the offset data between the point cloud tower and the actual tower position, and translate the point cloud tower position according to the offset data; Rotate the point cloud tower to match the actual tower channel direction; Determine the photo imaging range of the 3D point cloud tower: The actual shooting position, the horizontal angle between the camera and true north, the camera parameters and the shooting pitch angle are known; According to the graphics transformation matrix algorithm, combined with the horizontal angle between the camera and the true north, the shooting pitch angle is calculated to obtain the transformation matrix matrix, which is rotated relative to the true north (0,1,0) to obtain the final camera orientation vector; Calculate the cosine of the angle between the camera's orientation vector and the vector pointing to the shooting point, and use the inverse cosine formula to get the angle angle2; Calculate the distance from the target point to the shooting center and the depth value of the target point from the camera position; Verify the validity of the simulated flight path. If the flight path hits the tower, the pilot’s actual inspection flight path is unqualified and the simulation verification is stopped. If the flight path does not hit the tower, simulate the inspection and take photos of the tower point cloud based on the imported inspection route to obtain verification photos. The photos are named according to the main entities within the imaging range.
2. The method for verifying the effectiveness of a UAV route based on laser point cloud according to claim 1, characterized in that: The actual flight track is in json format, and the longitude and latitude coordinates in json format are converted into UTM coordinates.
3. The method for verifying the effectiveness of a UAV route based on laser point cloud according to claim 2, characterized in that: The rotating point cloud tower to match the actual tower channel direction specifically includes the following steps: Calculate the actual tower channel vector based on the actual large side coordinates and the actual small side coordinates; Calculate the point cloud tower channel vector based on the point cloud large side coordinates and the point cloud small side coordinates; Calculate the actual tower and point cloud tower channel vector length; Calculate the cosine value of the angle α between the actual tower vector and the point cloud tower vector, and use the arccosine formula to obtain the final angle angle1; Combined with the graphics rotation matrix, the entire point cloud model is rotated according to the angle so that the three-dimensional point cloud tower matches the actual track direction and position.
4. The method for verifying the effectiveness of a UAV route based on laser point cloud according to claim 1, characterized in that: After determining the photographic imaging range of the three-dimensional point cloud tower, it also includes the individual naming of the tower components. Based on the point cloud automatic classification technology, the tower components are named individually. The named components include the tower, ground wire, insulator, insulator conductor end, and insulator crossarm end.
5. The method for verifying the validity of a UAV route based on laser point cloud according to claim 4, characterized in that: The locations of the shooting targets of the left and right ground wires are determined by the individualization results of the ground wires; the shooting locations of the insulators, the insulator conductor ends, and the insulator crossarm ends are determined by the individualization results of the insulators.
6. The method for verifying the effectiveness of a UAV route based on laser point cloud according to claim 5, characterized in that: Compare the number of simulated shots with the number of standard shots, and output a report on the missed shooting rate of shooting points; determine the effectiveness of the photos taken, calculate the effective ratio of inspection photos, and output a photo effectiveness evaluation report; compare whether the single body parts captured in the photos taken include all standard shooting parts, name the photos according to the captured single body content, and output a report on the shooting target name.
Citation Information
Patent Citations
Electric power tower inspection track generation method based on point cloud matching
CN110619678A
Unmanned aerial vehicle SLAM technology inclination angle measurement system based on extended Kalman filtering
CN112197741A
Unmanned aerial vehicle oblique photography route planning method and computer readable storage medium
CN112484717A
Electric power unmanned aerial vehicle inspection method and system based on electric power tower model matching
CN113296537A
Automatic naming method for unmanned aerial vehicle inspection route planning of power transmission line
CN113641187A
