Heat distribution pipeline rendering deviation correction method and device based on unmanned aerial vehicle picture and medium

Abstract

The invention discloses a heat distribution pipeline rendering deviation correction method and device based on an unmanned aerial vehicle picture and a medium, belongs to the technical field of heat distribution pipeline inspection, and aims to solve the problems that picture information and unmanned aerial vehicle and mounting state information are not synchronized easily when an existing unmanned aerial vehicle inspects a heat distribution pipeline, and the heat distribution pipeline is not corrected. And the position of the underground heat distribution pipeline in a picture is deviated, so that the inspection effect is influenced. The method comprises the following steps: carrying out key point identification and matching on a front frame image and a rear frame image shot by an unmanned aerial vehicle, and determining a perspective transformation matrix based on key point pairs; according to the perspective transformation matrix, performing mapping processing on pixel coordinates selected in the rear frame image on the front frame image to obtain front frame GIS coordinates of the front frame image; performing correction processing on the flight state of the unmanned aerial vehicle under all-directional distance and angle factors to obtain unmanned aerial vehicle parameters; and performing image drawing on the pipeline position to obtain a final pipeline position image.

Description

Technical Field

[0001] The present application relates to the field of thermal pipeline inspection, and in particular to a thermal pipeline rendering correction method, equipment and medium based on drone images. Background Art

[0002] Currently, pipeline real-time rendering products on the market mainly use the internal parameters of the drone-mounted camera, as well as the drone’s latitude, longitude, altitude and other information to convert GIS points into pixel positions in the picture.

[0003] During the construction of thermal pipelines, welding is performed every 12 meters, and GIS information is calibrated at the welding point, which can be used for subsequent maintenance. Thermal pipelines are usually laid at a depth of more than 1 meter underground. Areas that may be passed along the way include roads, fields, bridges, rivers, etc., making ground inspections difficult. Drones are usually used for inspections to ensure pipeline safety. During the inspection process, pilots usually rely on memory and experience to determine the location of the pipeline, which is prone to deviations.

[0004] Using the internal parameters of the drone-mounted camera, as well as the drone's latitude, longitude, altitude and other information, the GIS points can be converted into pixel coordinates in the picture. By connecting the pipeline GIS points near the drone into a line, the location of the pipeline can be drawn in the picture.

[0005] During the flight of the drone, the mounted camera continuously outputs video images, and the flight control system obtains the status information of the drone and the mount in real time. Taking 25FPS as an example, the image is updated at 25 frames per second, while the status information of the drone and the mount is usually much lower than this frequency, causing the image information and the drone and the mount status information to be out of sync, causing the position of the underground pipeline in the image to shift, ultimately affecting the inspection effect. Summary of the invention

[0006] The embodiments of the present application provide a method, device and medium for correcting the rendering of a thermal pipeline based on drone images, which are used to solve the following technical problems: when existing drones are inspecting thermal pipelines, it is easy for the image information and the drone and mounting status information to be out of sync, causing the position of the underground thermal pipeline in the image to shift, affecting the inspection effect.

[0007] The present application embodiment adopts the following technical solutions:

[0008] On the one hand, an embodiment of the present application provides a thermal pipeline rendering correction method based on a drone image, including: identifying and matching key points of a front frame image and a rear frame image taken by a drone, and determining a perspective transformation matrix based on key point pairs; according to the perspective transformation matrix, mapping the pixel coordinates selected in the rear frame image on the front frame image to obtain the front frame GIS coordinates of the front frame image; according to the pixel coordinates in the rear frame image and the front frame GIS coordinates, correcting the drone flight state under all-round distance and angle factors to obtain drone parameters; wherein the drone parameters include: mount pitch angle, mount yaw angle, vertical field of view angle, horizontal field of view angle, drone longitude and drone latitude; using the drone parameters and the pre-acquired pipeline GIS information, the pipeline position is imaged to obtain a final pipeline position image.

[0009] The embodiment of the present application can more accurately determine the relative position relationship between images by identifying and matching key points, thereby improving the accuracy of perspective transformation. The pixel coordinates of the rear frame image are mapped to the GIS coordinates of the front frame image using the perspective transformation matrix, thereby achieving accurate conversion of coordinates. The accuracy of the drone parameters is improved. At the same time, the comprehensive measurement of drone parameters, including pitch angle, yaw angle, field of view angle, longitude and latitude, etc., provides a complete data basis for subsequent data processing. The drone parameters are combined with the pre-acquired GIS information to enhance the accuracy of pipeline location drawing. In addition, the automated data processing process reduces manual intervention and improves work efficiency. It can also reduce errors in data processing through precise image matching and flight status correction. It can also improve the synchronization level of picture information and drone and mounting status information, reduce the problem of position offset of underground pipelines in the picture, and enhance the inspection effect of thermal pipelines.

[0010] In a feasible implementation manner, the key points of the front frame image and the rear frame image taken by the drone are identified and matched to determine the perspective transformation matrix based on the key point pair, which specifically includes: using the drone to inspect and photograph the underground thermal pipeline, and selecting images of any adjacent frames to obtain the front frame image and the rear frame image; identifying and extracting the key points of the front frame image to obtain the front frame key points; and identifying and extracting the key points of the rear frame image to obtain the rear frame key points; using the preset Brute-Force technology, matching the front frame key points with the rear frame key points, and sorting the matched key points to determine the sorted key point pairs; performing matrix conversion on the key point pairs to obtain the perspective transformation matrix.

[0011] In a feasible implementation, according to the perspective transformation matrix, the pixel coordinates selected in the rear frame image are mapped onto the front frame image to obtain the front frame GIS coordinates of the front frame image, specifically including: based on the resolution of the rear frame image, the pixel area of ​​the rear frame image is divided into nine equal parts, and the corresponding pixel coordinates of the center point of each pixel area after division are determined; wherein the number of the pixel coordinates is 9; according to the perspective transformation matrix, the pixel coordinates in the rear frame image are mapped to the front frame image to determine the mapped pixel coordinates in the front frame image; wherein the number of the mapped pixel coordinates is 9; and the GIS coordinates of the mapped pixel coordinates are calculated through the drone status information corresponding to the front frame image to obtain the front frame GIS coordinates of the front frame image.

[0012] In a feasible implementation, according to the pixel coordinates in the rear frame image and the front frame GIS coordinates, the UAV flight state is corrected in all directions under distance and angle factors to obtain UAV parameters, specifically including: determining the rear frame GIS coordinates of the rear frame image based on the matching correction of the front frame GIS coordinates; and extracting the pixel coordinates of the rear frame image; according to the optical center position of the UAV and the rear frame GIS coordinates, calculating the distance between each pixel coordinate to obtain the pixel point spacing; according to the optical center position, performing transmission processing on the pixel coordinates related to the ground distance to obtain the ground projection distance of the pixel coordinates; through preset trigonometric functions and The angle about the optical center position is used to calculate the field of view angle of the pixel point spacing and the ground projection distance to obtain the field of view angle parameters of the UAV; wherein the field of view angle parameters include: the mount pitch angle, the vertical field of view angle and the horizontal field of view angle; based on the longitude and latitude differences corresponding to the pixel coordinates, the longitude and latitude parameters of the UAV are calculated; wherein the longitude and latitude parameters include: the mount yaw angle, the longitude and latitude of the UAV; through the field of view angle parameters and the longitude and latitude parameters, and based on the matching correction between the previous frame GIS coordinates and the subsequent frame GIS coordinates, the flight state of the UAV is corrected to obtain the corrected UAV parameters of the UAV.

[0013] In a feasible implementation manner, the longitude and latitude parameters of the UAV are calculated based on the longitude and latitude differences corresponding to the pixel coordinates, specifically including: based on the longitude and latitude values ​​of the pixel coordinates at both ends of the edge of the rear frame image, the latitude difference and longitude difference of the pixel coordinates at both ends of the edge are obtained; and the mounting yaw angle of the UAV is calculated according to the latitude difference and the longitude difference; the center longitude and latitude difference between the optical center position and the center pixel coordinates is calculated, and the center longitude and latitude difference is compared and judged with the edge longitude and latitude difference of the pixel coordinates at both ends of the edge; wherein the center pixel coordinates and the optical center position are in a vertical relationship; if the edge longitude and latitude difference is greater than the center longitude and latitude difference, the longitude and latitude parameters of the UAV are determined as the longitude and latitude of the center pixel coordinates - the center longitude and latitude difference; if the edge longitude and latitude difference is less than the center longitude and latitude difference, the longitude and latitude parameters of the UAV are determined as the longitude and latitude of the center pixel coordinates + the center longitude and latitude difference.

[0014] In a feasible implementation, the front frame GIS coordinates can perform coordinate correction on the drone status information corresponding to the rear frame image.

[0015] In a feasible implementation, before the pipeline position is imaged by the drone parameters and the pre-acquired pipeline GIS information to obtain the final pipeline position image, the method also includes: based on the drone parameters, ground projecting the four corner points in the current frame image taken to obtain the longitude and latitude of the four corner points; constructing a straight line function for the four longitude and latitude of the corner points to obtain an edge straight line function based on the four edges of the current frame image; through a preset sliding window, sliding processing is performed on the information list of the pre-acquired pipeline GIS information to obtain two adjacent thermal pipeline welding points; wherein the pipeline GIS information includes the point information of the thermal pipeline welding points; based on the two adjacent thermal pipeline welding points, constructing an actual welding straight line function.

[0016] In a feasible implementation manner, the pipeline position is imaged by using the drone parameters and the pre-acquired pipeline GIS information to obtain a final pipeline position image, specifically including: calculating the intersection of the actual welding straight line function and the edge straight line function to obtain two intersection points within the current frame image range; if two adjacent thermal pipeline welding points are both located in the current frame image, the coordinates corresponding to the thermal pipeline welding points are determined as pipeline position endpoints; if two adjacent thermal pipeline welding points are not both located in the current frame image, the coordinates corresponding to the two intersection points are determined as pipeline position endpoints; according to the pipeline position endpoints and through the camera mounted on the drone, the pipeline position is rendered and drawn to obtain a final pipeline position image.

[0017] In the second aspect, an embodiment of the present application also provides a thermal pipeline rendering correction device based on drone images, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute a thermal pipeline rendering correction method based on drone images as described in any of the above embodiments.

[0018] In a third aspect, an embodiment of the present application also provides a non-volatile computer storage medium, characterized in that the storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions, and when the instructions are executed by a terminal, the terminal executes a thermal pipeline rendering correction method based on drone images described in any of the above embodiments.

[0019] The present application provides a method, device and medium for correcting thermal pipeline rendering based on drone images. Compared with the prior art, the embodiments of the present application have the following beneficial technical effects:

[0020] 1. By identifying and matching key points, the relative position relationship between images can be determined more accurately, thereby improving the accuracy of perspective transformation.

[0021] 2. The pixel coordinates of the rear frame image are mapped to the GIS coordinates of the front frame image using the perspective transformation matrix, achieving accurate coordinate conversion.

[0022] 3. The accuracy of UAV parameters is improved by correcting the all-round distance and angle factors of the UAV's flight status.

[0023] 4. Comprehensive measurement of UAV parameters, including pitch angle, yaw angle, field of view angle, longitude and latitude, etc., provides a complete data basis for subsequent data processing.

[0024] 5. Combining UAV parameters with pre-acquired GIS information enhances the accuracy of pipeline location mapping.

[0025] 6. Through image drawing technology, the pipeline position is visualized, making it easier for users to intuitively identify and locate the pipeline.

[0026] 7. Automated data processing processes reduce manual intervention and improve work efficiency.

[0027] 8. Reduce errors in data processing through precise image matching and flight status correction.

[0028] 9. The application of this method can enhance the practicality of thermal pipeline inspection and maintenance, especially in complex environments.

[0029] 10. Automated and precise processing methods may reduce reliance on manual inspections, thereby reducing related costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0031] Figure 1 A flow chart of a thermal pipeline rendering correction method based on drone images provided in an embodiment of the present application;

[0032] Figure 2 A pixel coordinate distribution diagram provided in an embodiment of the present application;

[0033] Figure 3 A schematic diagram of the distance between pixel coordinates provided in an embodiment of the present application;

[0034] Figure 4 A schematic diagram of an optical center position mapping method provided in an embodiment of the present application;

[0035] Figure 5 A schematic diagram of a vertical field of view provided in an embodiment of the present application;

[0036] Figure 6 A schematic diagram of a horizontal field of view provided in an embodiment of the present application;

[0037] Figure 7 A schematic diagram of the structure of a thermal pipeline rendering and correction device based on drone images provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0039] The embodiment of the present application provides a method for correcting the rendering of a thermal pipeline based on drone images, such as Figure 1As shown, the thermal pipeline rendering correction method based on the drone image specifically includes steps S101-S104:

[0040] S101, identifying and matching key points of the front frame image and the rear frame image taken by the drone, and determining a perspective transformation matrix based on the key point pairs.

[0041] Specifically, it is first necessary to use a drone to inspect and photograph the underground thermal pipeline, and select images of any adjacent frames to obtain the previous frame image and the next frame image.

[0042] Furthermore, the key points of the front frame image are identified and extracted to obtain the key points of the front frame, and the key points of the back frame image are identified and extracted to obtain the key points of the back frame.

[0043] Furthermore, the preset Brute-Force technology is used to match the key points of the previous frame with the key points of the next frame, and the matched key points are sorted to determine the sorted key point pairs. Finally, the key point pairs are matrix transformed to obtain the perspective transformation matrix.

[0044] As a feasible implementation method, for the front and back frames of images taken by the drone, the key points are first identified and extracted respectively, and then the key points of the two frames of images are matched using the Brute-Force method. After the matched key points are sorted, the perspective transformation matrix operation is performed on the sorted key point pairs. The perspective transformation matrix is ​​a transformation matrix that maps objects or scenes in three-dimensional space to a two-dimensional plane, and is mainly used to realize three-dimensional graphics rendering and transformation. Through the perspective transformation matrix, points in three-dimensional space can be mapped to a two-dimensional plane while retaining the distance and perspective effects of objects. The perspective transformation matrix is ​​determined by a series of parameters, including the projection center point, the normal vector of the viewing plane, the perspective width and height, etc.

[0045] S102 , mapping pixel coordinates selected from the rear frame image onto the front frame image according to the perspective transformation matrix to obtain front frame GIS coordinates of the front frame image.

[0046] Specifically, firstly, the pixel area of ​​the rear frame image needs to be equally divided into nine equal parts based on the resolution of the rear frame image, and the corresponding pixel coordinates are determined for the center point of each divided pixel area, wherein the number of pixel coordinates is 9.

[0047] Furthermore, according to the perspective transformation matrix, the pixel coordinates in the subsequent frame image are mapped to the previous frame image to determine the mapped pixel coordinates in the previous frame image. The number of mapped pixel coordinates is 9.

[0048] Furthermore, the GIS coordinates of the mapped pixel coordinates are calculated through the drone status information corresponding to the previous frame image to obtain the previous frame GIS coordinates of the previous frame image.

[0049] In one embodiment, Figure 2 A pixel coordinate distribution diagram provided in an embodiment of the present application, such as Figure 2 As shown, when the drone is flying, the images of the front and rear frames are different, and the pixel coordinate systems corresponding to the images need to be converted and unified. The coordinate system of the previous frame (front frame image) is used as the anchor point to convert the coordinates of the rear frame image to the coordinate system of the front frame image. It is necessary to first select 9 points in the rear frame image, taking 1920*1080 resolution as an example: select 0-(320,180), 1-(960,180), 2-(1600,180), 3-(320,540), 4-(960,540), 5-(1600,540), 6-(320,900), 7-(960,900), 8-(1600,900). Then, according to the perspective transformation matrix output above, the 9 points selected in the rear frame image are mapped to the front frame image to obtain 9 points. According to the drone status information of the previous frame image, the previous frame GIS coordinates of the 9 points in the previous frame image are calculated. The 9 previous frame GIS coordinates can be used to correct the flight status information of the next frame.

[0050] S103. According to the pixel coordinates in the rear frame image and the GIS coordinates in the front frame, the UAV flight status is corrected in all directions under distance and angle factors to obtain UAV parameters; wherein the UAV parameters include: mount pitch angle, mount yaw angle, vertical field of view angle, horizontal field of view angle, UAV longitude and UAV latitude.

[0051] Specifically, after matching and correcting the GIS coordinates corresponding to the rear frame image using the front frame GIS coordinates, the rear frame GIS coordinates of the rear frame image are determined. And the pixel coordinates of the rear frame image are extracted. According to the optical center position of the drone and the rear frame GIS coordinates, the distance between each pixel coordinate is calculated to obtain the pixel point spacing. The front frame GIS coordinates can correct the coordinates of the drone status information corresponding to the rear frame image.

[0052] In one embodiment, Figure 3 A schematic diagram of the distance between pixel coordinates provided in an embodiment of the present application; Figure 4 A schematic diagram of an optical center position mapping method provided in an embodiment of the present application is shown in FIG. Figure 3 as well as Figure 4 As shown, firstly, the trigonometric function relationship between the 9 pixel coordinates in the rear frame image and the optical center is established ( Figure 4The numbers 1, 2...8, 9 in the figure are the corresponding pixel coordinates), then record the GIS distance from point 1 to point 4 as d1, the GIS distance from point 4 to point 7 as d2, the GIS distance from point 7 to the drone vertically on the ground as d3, the height of the drone from the ground as alt, the angle between the center of the picture and the plumb line as pitch, the angle between point 1-optical center-point 4 as alpha, the angle between point 3-optical center-point 4 as beta, the distance between point 0 and point 1 projected to the ground as up, the distance between point 3 and point 4 projected to the ground as mid, the distance between point 6 and point 7 projected to the ground as down, the length from point 1 projected to the ground to the optical center is dist1C, the length from point 4 projected to the ground to the optical center is dist4C, the length from point 7 projected to the ground to the optical center is dist7C, and the focal length is L.

[0053] Furthermore, according to the position of the optical center, the pixel coordinates are subjected to transmission processing related to the ground distance to obtain the ground projection distance of the pixel coordinates.

[0054] Furthermore, the field of view angle of the drone is calculated by using the preset trigonometric function and the angle of the optical center position to calculate the field of view angle of the pixel point spacing and the ground projection distance, and the field of view angle parameters of the drone are obtained. The field of view angle parameters include: mounting pitch angle, vertical field of view angle, and horizontal field of view angle.

[0055] In one embodiment, Figure 5 A schematic diagram of a vertical field of view provided in an embodiment of the present application; Figure 6 A schematic diagram of a horizontal field of view provided in an embodiment of the present application is shown in FIG. Figure 5 as well as Figure 6 As shown:

[0056] On the imaging surface: tan(∠0C1)=L*tan(beta) / (L / cos(alpha)), simplified: tan(∠0C1)=tan(beta)*cos(alpha);

[0057] On the ground: tan(∠0C1)=up / dist1C;

[0058] On the imaging surface: tan(∠6C7)=tan(∠0C1);

[0059] On the ground: tan(∠6C7)=down / dist7C;

[0060] On the ground: dist1C = (d1+d2+d3)*sin(pitch+alpha);

[0061] On the ground: dist4C = (d2 + d3) * sin(pitch);

[0062] On the ground: dist7C = d3*sin(pitch-alpha);

[0063] On the ground: tan(pitch-alpha)=d3 / alt;

[0064] In the above 8 equations, the known quantities are alt, up, down, d1, d2, the intermediate quantities are d3, tan(∠0C1), dist1C, dist4C, dist7C, and the quantities to be solved are pitch, alpha, and beta.

[0065] Solve the intermediate quantity through the multi-equation system, which is written as:

[0066] D = (d1 + d2) / alt;

[0067] t = (up-down) / (up+down);

[0068] T = t*t;

[0069] a=2;

[0070] b = D*TD;

[0071] c = 2*T;

[0072] Find: tan(pitch)=(up-down) / ((up+down)*

[0073] tan(alpha)); tan(beta)=mid*cos(pitch) / alt

[0074] Further obtain the field of view angle parameters:

[0075] 1) Vertical field of view: fovV = 2*arctan(3*tan(alpha) / 2);

[0076] 2) Horizontal field of view: fovH = 2*arctan(3*tan(beta) / 2);

[0077] 3) Mounting pitch angle: pitch = arctan ((up-down) / ((up+down) * tan (alpha)));

[0078] Furthermore, it is also necessary to calculate the longitude and latitude parameters of the drone based on the longitude and latitude differences corresponding to the pixel coordinates. The longitude and latitude parameters include: mounting yaw angle, drone longitude, and drone latitude.

[0079] Furthermore, based on the latitude and longitude values ​​of the pixel coordinates at both ends of the edge of the subsequent frame image, the latitude difference and longitude difference of the pixel coordinates at both ends of the edge are obtained. And according to the latitude difference and longitude difference, the mounting yaw angle of the UAV is calculated.

[0080] Furthermore, the central longitude and latitude difference between the optical center position and the central pixel coordinates is calculated, and the central longitude and latitude difference is compared with the edge longitude and latitude difference of the edge pixel coordinates at both ends of the edge. The central pixel coordinates and the optical center position are in a perpendicular relationship.

[0081] If the edge longitude and latitude difference is greater than the center longitude and latitude difference, the longitude and latitude parameters of the drone are determined as the longitude and latitude of the center pixel coordinates minus the longitude and latitude difference of the center.

[0082] If the edge longitude and latitude difference is less than the center longitude and latitude difference, the longitude and latitude parameters of the drone are determined as the longitude and latitude of the center pixel coordinates + the center longitude and latitude difference.

[0083] In one embodiment, Figure 4 As shown, the longitude and latitude differences of point 1 and point 7 on the ground are longitude_17 and latitude_17 (edge ​​longitude and latitude differences), and the mount yaw angle yaw =

[0084] arctan(longitude_17 / latitude_17). On the ground, the distance between point 4 and the drone is: d2+d3=alt*tan(pitch), then the difference between the longitude and latitude of the drone and point 4 (the difference between the center longitude and latitude) is:

[0085] longitude_diff=|(d2+d3)*sin(yaw)|

[0086] latitude_diff=|(d2+d3)*cos(yaw)|

[0087] Then, compare the longitude and latitude of point 1 and point 4:

[0088] 1) If the longitude of point 1 is larger (i.e. the difference between the longitude and longitude of the edge is greater than the difference between the longitude and longitude of the center), the longitude of the drone is longitude = longitude_4-longitude_diff. If the longitude of point 1 is smaller (i.e. the difference between the longitude and longitude of the edge is smaller than the difference between the longitude and longitude of the center), the longitude of the drone is longitude = longitude_4+longitude_diff.

[0089] 2) If the latitude of point 1 is larger (i.e. the edge longitude and latitude difference is greater than the center longitude and latitude difference), the latitude of the drone is latitude = latitude_4-latitude_diff. If the latitude of point 1 is smaller (i.e. the edge longitude and latitude difference is smaller than the center longitude and latitude difference), the latitude of the drone is latitude = latitude_4+latitude_diff.

[0090] Finally, the latitude and longitude parameters of the drone are obtained.

[0091] Furthermore, the UAV flight state is corrected through the field of view angle parameters and the longitude and latitude parameters based on the matching correction between the front frame GIS coordinates and the rear frame GIS coordinates, and finally the corrected UAV parameters of the UAV are obtained.

[0092] S104: Using the UAV parameters and the pre-acquired pipeline GIS information, the pipeline location is imaged to obtain a final pipeline location image.

[0093] Specifically, the four corner points in the current frame image are projected onto the ground using the matched and corrected drone parameters to obtain the longitude and latitude of the four corner points.

[0094] Furthermore, it is also necessary to construct a straight line function for the longitude and latitude of the four corner points to obtain an edge straight line function based on the four edges of the current frame image.

[0095] Furthermore, the pre-acquired pipeline GIS information is processed by sliding the information list through a preset sliding window to obtain two adjacent thermal pipeline welding points; wherein the pipeline GIS information includes the point information of the thermal pipeline welding points. Based on the two adjacent thermal pipeline welding points, the actual welding straight line function is constructed.

[0096] Furthermore, the intersection point of the actual welding straight line function and the edge straight line function is calculated to obtain two intersection points within the range of the current frame image.

[0097] If two adjacent thermal pipeline welding points are both located in the current frame image, the coordinates corresponding to the thermal pipeline welding points are determined as the pipeline position endpoints.

[0098] If two adjacent thermal pipeline welding points are not both located in the current frame image, the coordinates corresponding to the two intersection points are determined as the pipeline position endpoints.

[0099] Furthermore, according to the pipeline position endpoint, the pipeline position is rendered and drawn by a camera mounted on the drone to obtain a final pipeline position image.

[0100] In one embodiment, pipeline GIS information is typically point information, such as Figure 4As shown, according to the order of points and the GIS of points, the pipeline can be drawn on the map. After the drone parameters are corrected, the pipeline location can be rendered in the picture according to the drone parameters and the pipeline GIS information. The specific process is:

[0101] 1) Calculate the latitude and longitude of the four corner points in the current frame image projected on the ground, and obtain the straight line equations of the four edges (edge ​​straight line function).

[0102] 2) Use a window length of 2 to slide in the pipeline GIS information list, and arbitrarily obtain two adjacent thermal pipeline welding points, recorded as p0 and p1, that is, calculate the actual welding straight line function line_p of the two points.

[0103] 3) Calculate the intersection of the actual welding straight line function line_p under p0 and p1 and the four edge straight line functions in step 1), and retain the two intersection points on the edge within the current frame image range, which are recorded as point_list.

[0104] 4) In point_list, the number of points is 2, and then determine whether p0 and p1 are in the current frame image. If they are in the current frame image, the coordinates in the current frame image are taken as the pipeline position endpoints. Otherwise, the corresponding points in point_list are taken as the pipeline position endpoints, a straight line is drawn, and then the pipeline position is rendered and drawn to obtain the final pipeline position image.

[0105] In addition, the embodiment of the present application also provides a thermal pipeline rendering correction device based on drone images, such as Figure 7 As shown, the thermal pipeline rendering correction device based on the drone image specifically includes:

[0106] At least one processor 701; and a memory 702 in communication with the at least one processor 701; wherein the memory 702 stores instructions executable by the at least one processor 701, so that the at least one processor 701 can execute:

[0107] Identify and match the key points of the front frame image and the back frame image taken by the drone, and determine the perspective transformation matrix based on the key point pair;

[0108] According to the perspective transformation matrix, the pixel coordinates selected in the rear frame image are mapped on the front frame image to obtain the front frame GIS coordinates of the front frame image;

[0109] According to the pixel coordinates in the rear frame image and the GIS coordinates in the front frame, the UAV flight status is corrected in all directions and angles to obtain the UAV parameters; the UAV parameters include: mount pitch angle, mount yaw angle, vertical field of view angle, horizontal field of view angle, UAV longitude and UAV latitude;

[0110] The pipeline location is imaged using the drone parameters and pre-acquired pipeline GIS information to obtain the final pipeline location image.

[0111] The embodiment of the present application can more accurately determine the relative position relationship between images by identifying and matching key points, thereby improving the accuracy of perspective transformation. The pixel coordinates of the rear frame image are mapped to the GIS coordinates of the front frame image using the perspective transformation matrix, thereby achieving accurate conversion of coordinates. The accuracy of the drone parameters is improved. At the same time, the comprehensive measurement of drone parameters, including pitch angle, yaw angle, field of view angle, longitude and latitude, etc., provides a complete data basis for subsequent data processing. The drone parameters are combined with the pre-acquired GIS information to enhance the accuracy of pipeline location drawing. In addition, the automated data processing process reduces manual intervention and improves work efficiency. It can also reduce errors in data processing through precise image matching and flight status correction. It can also improve the synchronization level of picture information and drone and mounting status information, reduce the problem of position offset of underground pipelines in the picture, and enhance the inspection effect of thermal pipelines.

[0112] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0113] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0114] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0115] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0116] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0118] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0119] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0120] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0121] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0122] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the specification of the present application.

Claims

1. A thermal pipeline rendering correction method based on drone images, characterized in that: The method comprises: Identify and match the key points of the front frame image and the back frame image taken by the drone, and determine the perspective transformation matrix based on the key point pair; According to the perspective transformation matrix, the pixel coordinates selected from the rear frame image are mapped onto the front frame image to obtain the front frame GIS coordinates of the front frame image; According to the pixel coordinates in the rear frame image and the GIS coordinates in the front frame, the UAV flight state is corrected in all directions and angles to obtain UAV parameters; wherein the UAV parameters include: mount pitch angle, mount yaw angle, vertical field of view angle, horizontal field of view angle, UAV longitude and UAV latitude; The pipeline position is imaged using the drone parameters and pre-acquired pipeline GIS information to obtain a final pipeline position image.

2. According to the method for correcting the thermal pipeline rendering based on drone images according to claim 1, it is characterized in that: The key points of the front frame image and the back frame image taken by the drone are identified and matched, and the perspective transformation matrix based on the key point pair is determined, including: Using the drone, inspect and photograph the underground thermal pipeline, and select images of any adjacent frames to obtain the previous frame image and the next frame image; Identify and extract key points of the front frame image to obtain key points of the front frame; and identify and extract key points of the rear frame image to obtain key points of the rear frame; By using a preset Brute-Force technology, the key points of the previous frame are matched with the key points of the next frame, and the matched key points are sorted to determine sorted key point pairs; The key point pairs are subjected to matrix transformation to obtain the perspective transformation matrix.

3. According to the method for correcting thermal pipeline rendering based on drone images as described in claim 1, it is characterized in that: According to the perspective transformation matrix, the pixel coordinates selected from the rear frame image are mapped onto the front frame image to obtain the front frame GIS coordinates of the front frame image, specifically including: Based on the resolution of the rear frame image, the rear frame image is divided into nine equal pixel regions, and the corresponding pixel coordinates are determined for the center point of each divided pixel region; wherein the number of the pixel coordinates is 9; According to the perspective transformation matrix, the pixel coordinates in the rear frame image are mapped to the front frame image to determine the mapped pixel coordinates in the front frame image; wherein the number of the mapped pixel coordinates is 9; The GIS coordinates of the mapped pixel coordinates are calculated using the drone status information corresponding to the previous frame image to obtain the previous frame GIS coordinates of the previous frame image.

4. According to the method for correcting thermal pipeline rendering based on drone images according to claim 1, it is characterized in that: According to the pixel coordinates in the rear frame image and the GIS coordinates in the front frame, the UAV flight state is corrected under all-round distance and angle factors to obtain the UAV parameters, which specifically include: Determine the rear frame GIS coordinates of the rear frame image based on the matching correction of the front frame GIS coordinates; and extract the pixel coordinates of the rear frame image; According to the optical center position of the UAV and the GIS coordinates of the rear frame, the distance between each pixel coordinate is calculated to obtain the pixel point spacing; According to the optical center position, the pixel coordinates are subjected to transmission processing related to the ground distance to obtain the ground projection distance of the pixel coordinates; The field of view angle of the UAV is calculated by using a preset trigonometric function and an angle related to the optical center position, and the pixel point spacing and the ground projection distance are obtained; wherein the field of view angle parameters include: the mounting pitch angle, the vertical field of view angle, and the horizontal field of view angle; Based on the longitude and latitude differences corresponding to the pixel coordinates, the longitude and latitude parameters of the drone are calculated; wherein the longitude and latitude parameters include: the mount yaw angle, the longitude of the drone, and the latitude of the drone; The flight state of the UAV is corrected by using the field of view angle parameters and the longitude and latitude parameters, and based on the matching correction between the previous frame GIS coordinates and the subsequent frame GIS coordinates, to obtain the corrected UAV parameters of the UAV.

5. According to the method for correcting the rendering of thermal pipelines based on drone images according to claim 4, it is characterized in that: Based on the longitude and latitude differences corresponding to the pixel coordinates, the longitude and latitude parameters of the drone are calculated, specifically including: Based on the longitude and latitude values ​​of the pixel coordinates at both ends of the edge of the rear frame image, the latitude difference and longitude difference of the pixel coordinates at both ends of the edge are obtained; and according to the latitude difference and the longitude difference, the mounting yaw angle of the UAV is calculated; Calculate the central longitude and latitude difference between the optical center position and the central pixel coordinates, and compare and judge the central longitude and latitude difference with the edge longitude and latitude difference of the pixel coordinates at both ends of the edge; wherein the central pixel coordinates and the optical center position are in a perpendicular relationship; If the edge longitude and latitude difference is greater than the center longitude and latitude difference, the longitude and latitude parameters of the drone are determined as the longitude and latitude of the center pixel coordinates minus the center longitude and latitude difference; If the edge longitude and latitude difference is less than the center longitude and latitude difference, the longitude and latitude parameters of the drone are determined as the longitude and latitude of the center pixel coordinates + the center longitude and latitude difference.

6. According to the method for correcting thermal pipeline rendering based on drone images as described in claim 4, it is characterized in that: The front frame GIS coordinates can correct the coordinates of the drone status information corresponding to the rear frame image.

7. According to the method for correcting the rendering of thermal pipelines based on drone images as described in claim 1, it is characterized in that: Before the pipeline location is imaged by using the drone parameters and the pre-acquired pipeline GIS information to obtain a final pipeline location image, the method further includes: Based on the drone parameters, the four corner points in the current frame image are projected onto the ground to obtain the longitude and latitude of the four corner points; Constructing a straight line function for the longitude and latitude of the four corner points to obtain an edge straight line function based on the four edges of the current frame image; Through a preset sliding window, the pre-acquired pipeline GIS information list is slidably processed to obtain two adjacent thermal pipeline welding points; wherein the pipeline GIS information includes the point information of the thermal pipeline welding points; Based on two adjacent welding points of the thermal pipeline, an actual welding straight line function is constructed.

8. The method for correcting thermal pipeline rendering based on drone images according to claim 7 is characterized in that: The pipeline location is imaged using the drone parameters and the pre-acquired pipeline GIS information to obtain a final pipeline location image, specifically including: Calculating the intersection of the actual welding straight line function and the edge straight line function to obtain two intersection points within the range of the current frame image; If two adjacent welding points of the thermal pipeline are both located in the current frame image, the coordinates corresponding to the welding points of the thermal pipeline are determined as the pipeline position endpoints; If two adjacent welding points of the thermal pipeline are not both located in the current frame image, the coordinates corresponding to the two intersection points are determined as the pipeline position endpoints; According to the pipeline position endpoint, the pipeline position is rendered and drawn by using the camera mounted on the drone to obtain a final pipeline position image.

9. A thermal pipeline rendering correction device based on drone images, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute the thermal pipeline rendering correction method based on drone images according to any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that: The storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions. When the instructions are executed by the terminal, the terminal executes the thermal pipeline rendering correction method based on drone images according to any one of claims 1-8.

Images