Real-time unmanned aerial vehicle image splicing method and system, and storage medium

Abstract

The application discloses a real-time unmanned aerial vehicle image splicing method and system and a storage medium, belongs to the technical field of geographic information, and unmanned aerial vehicles collect image data according to planned routes; the collected image data is transmitted to a remote controller; the image data is spliced in real time through an image processing algorithm built in the remote controller to obtain spliced images; and the spliced images are displayed on a display screen of the remote controller, so that the unmanned aerial vehicles and corresponding remote controllers can splice the images locally in real time under limited resource conditions.

Description

Technical Field

[0001] This invention belongs to the technical field of geographic information, and specifically relates to a real-time UAV image stitching method, system, and storage medium. Background Technology

[0002] Drones are widely used in many fields such as surveying, emergency rescue, agricultural monitoring, and urban planning. They are typically used to capture images of target areas in real time to meet the needs of users in surveying, emergency rescue, agricultural monitoring, and urban planning.

[0003] Due to the limitations of drone flight altitude and camera focal length, a single frame image often cannot cover the entire target area. To obtain large-area, high-resolution panoramic images, it is necessary to stitch together multiple images taken by the drone to provide more comprehensive and accurate basic data, meeting the needs of various fields for large-scale, high-precision images.

[0004] In traditional technology, after a drone completes the acquisition of area images, the raw images need to be transmitted to a backend server for processing. The processing flow is as follows: S1: Data Acquisition: The drone takes a large number of pictures along a preset route; S2: Data return: The original image is transmitted back to the server via wireless transmission methods such as 4G / 5G, wireless Wi-Fi, or wired network; S3: Backend processing: The server uses professional software such as Pix4D and Agisoft Metashape to perform image stitching and 3D reconstruction, and generate stitched maps.

[0005] Based on the above-mentioned existing technology, it mainly has the following drawbacks: (1) Transmission bandwidth bottleneck: High-resolution images have large data volume, with a single image reaching tens of MB, resulting in significant transmission delay over long distances; (2) Insufficient real-time performance: It may take several hours from shooting to generating a map, which cannot meet the needs of emergency rescue, dynamic monitoring and other scenarios; (3) Over-reliance on cloud computing power: Backend servers need to be configured with high-performance GPU / CPU clusters, which is costly.

[0006] (4) Requires communication network support: In some environments with limited resources, the data cannot be returned due to poor communication network.

[0007] Based on the above, there is an urgent need to develop a map generation technology that can perform local stitching of drone images in real time to meet a wider range of application scenarios. Summary of the Invention

[0008] In view of this, in order to solve the above-mentioned problems existing in the prior art, the purpose of the present invention is to provide a real-time drone image stitching method, system and storage medium to achieve the purpose of real-time local stitching of drone images in a limited resource environment using a drone and a corresponding drone remote controller.

[0009] The technical solution adopted in this invention is: a real-time UAV image stitching method, the method comprising: S1: The drone collects image data according to the planned flight path; S2: Transmits each frame of image data acquired to the remote control in real time; S3: The image data is stitched together in real time using the image processing algorithm built into the remote control to obtain the stitched image; S4: Display and dynamically update the stitched image in real time on the remote control's screen.

[0010] Furthermore, the image processing algorithm includes: A1: Extract feature points from the first frame; A2: Extract feature points from the second frame and match them with feature points from the first frame to obtain matching points; A3: Solve for the relative pose of the second frame with respect to the first frame; A4: Calculate the rotation matrix M from camera coordinates to geographic coordinates and simultaneously construct a triangulation network, where the construction of the triangulation network is based on the initial point cloud; A5: Construct a geographic coordinate triangulation W based on the triangulation and rotation matrix M; A6: Generate a preliminary projected image through GPU texture mapping projection.

[0011] Furthermore, the image processing algorithm also includes: B1: Extract feature points of the Nth frame, and execute B2 and B3 simultaneously; B2: Match with frame N-1. If a match is found, proceed to B4. If no match is found, search for a matching frame within the tolerance radius. If no matching frame is found, the process ends. If a matching frame is found, proceed to B4. The tolerance radius search for a matching frame narrows the search range and reduces the computational complexity of frame matching. Spatial constraints replace full traversal, reducing the number of feature matching calculations. Compared to traditional frame matching that traverses all historical frames, the computational power requirement decreases exponentially. B3: Match with the initial point cloud, solve the relative pose relationship between the Nth frame and the (N-1th)th frame, calculate the rotation matrix M' from the current frame coordinates to the geographic coordinates and execute B4 simultaneously; B4: Update the triangular network based on the new dot cloud; B5: Construct a geographic coordinate triangulation W' based on the updated triangulation and rotation matrix M', and search for the best matching frame of the triangle within the tolerance radius; B6: Generates projected images via GPU texture mapping; Where N≥3, the projected image is updated in real time by cyclically executing B1-B6, where 1 frame is superimposed each time.

[0012] Furthermore, the method for calculating the rotation matrix M from camera coordinates to geographic coordinates is as follows: A4011: Calculate the camera center coordinates based on the camera coordinate system; A4012: Reads the geographic coordinates of the camera center based on the geographic coordinate system; A4013: Calculate the rotation matrix M from camera coordinates to geographic coordinates.

[0013] Furthermore, the method used to construct the triangular network is as follows: A4021: Triangulation is performed based on the relative pose relationship between the second frame and the first frame; A4022: Calculate camera coordinates; A4023: Performs coarse screening on triangulated 3D point clouds, filters out invalid data in advance, avoids redundant calculations, and significantly reduces the data scale of subsequent processing. A4024: Performs local beam adjustment optimization. Local beam adjustment sacrifices global optimum for local accuracy, which greatly reduces computational complexity. A4025: Fine-tuning the 3D point cloud and obtaining the initial point cloud; A4026: Constructing a triangular network based on the initial point cloud.

[0014] Furthermore, the method used to calculate the rotation matrix M' from the current frame coordinates to the geographic coordinates is as follows: B301: Calculate the camera center coordinates of the current frame based on the camera coordinate system; B302: Reads the geographic coordinates of the camera center based on the geographic coordinate system; B303: Calculate the rotation matrix M' from the camera coordinates to the geographic coordinates of the current frame.

[0015] Furthermore, the method for updating the triangular network is as follows: B401: Triangulation is performed based on the relative pose relationship between the Nth frame and the (N-1th)th frame; B402: Calculate camera coordinates; B403: Coarse screening of triangulated 3D point clouds; B404: Perform local beam adjustment optimization. Local beam adjustment sacrifices global optimum for local accuracy, which greatly reduces computational complexity. B405: Refine the 3D point cloud and obtain a new point cloud; B406: Local bundle adjustment optimization and triangulation update based on new point cloud.

[0016] Furthermore, the GPU texture mapping includes planar projection and orthographic projection. GPU texture mapping transforms serial pixel operations into parallel processing, leveraging the inherent graphics acceleration capabilities of the GPU. On the one hand, it can avoid the computing power bottleneck of CPU serial processing; on the other hand, the hardware driver is highly optimized, eliminating the need for complex algorithm development.

[0017] The present invention also provides a real-time UAV image stitching system, the system comprising: a UAV, a remote controller establishing a communication connection with the UAV, wherein the main control chip of the remote controller is communicatively connected to a memory, the memory has a built-in application program, and the main control chip executes the real-time UAV image stitching method as described above by calling the application program.

[0018] The present invention also provides a storage medium storing program instructions, wherein a computer reads the program instructions and executes the real-time UAV image stitching method as described above.

[0019] The beneficial effects of this invention are as follows: It can meet the needs of real-time image stitching in environments with limited resources. The entire image stitching process does not require data to be transmitted to the backend server or other ground processing equipment. Instead, the real-time image stitching is completed by the built-in processor of the remote controller. After taking off and shooting, users can directly generate a stitched image from the drone's footage on the remote controller and view it in real time. This reduces reliance on external resources and provides a better user experience.

[0020] At the data level, invalid data is reduced in advance by coarsely screening point clouds, thus reducing the amount of computation at the source; at the search level, the matching range is narrowed by tolerance radius search, reducing the number of feature calculations; at the hardware level, parallel computing is leveraged by GPU texture mapping to avoid CPU bottlenecks; at the optimization level, the scale of variables is compressed by local beam adjustment to avoid high global complexity calculations, thereby maximizing the reduction of computational complexity while meeting the accuracy requirements of UAV image stitching. Attached Figure Description

[0021] Figure 1 This is a logic flowchart of the real-time UAV image stitching method provided by the present invention. Detailed Implementation

[0022] The embodiments of this application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar modules or modules having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. Rather, the embodiments of this application include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0023] Example 1 This addresses situations where limited network communication resources prevent the transmission of large amounts of data, thus creating obstacles to data interaction with devices possessing high computing power. Alternatively, it addresses situations where limited computing resources and data processing hardware resources prevent the use of high-performance computing devices, making image stitching impossible.

[0024] like Figure 1 As shown in this embodiment, a real-time UAV image stitching method is specifically disclosed, which enables real-time UAV image stitching in a resource-constrained environment using a UAV and its corresponding remote controller. The image stitching method includes: S1: The drone collects image data according to the planned route. The image collection is achieved by the camera on the drone body.

[0025] S2: Transmit each frame of image data acquired in real time to the remote controller. The transmission of image data is based on the existing transmission channel between the remote controller and the drone. Generally, 2.4GHz wireless communication and 5.8GHz wireless communication are used. This embodiment is more suitable for the use of 5.8GHz wireless communication between the drone and the remote controller, which has a fast transmission speed and good anti-interference ability.

[0026] S3: Obtain a stitched image by real-time stitching of the image data using the image processing algorithm built into the remote control; the image processing algorithm includes: A1: Extract feature points from the first frame. Feature points are unique, easily identifiable, and matchable points in the image, such as corner points and edge points. Using specific feature extraction algorithms, such as SIFT, SURF, and ORB, representative points are extracted from the first frame image as feature points.

[0027] A2: Similarly, feature extraction algorithms are used to extract feature points from the second frame image and match them with feature points from the first frame to obtain matching points. In this embodiment, vectors describing the features around the feature points are used to find highly similar point pairs between the feature points of the two frames as matching points through distance metrics (such as Euclidean distance).

[0028] A3: Solve for the relative pose relationship between the second frame and the first frame. The relative pose relationship refers to the rotation and translation relationship between the two frames. In this embodiment, the relative pose is recovered from the essential matrix, which contains the rotation and translation information between the two frames. By decomposing the essential matrix, the camera's rotation matrix M and translation vector can be obtained. t This allows the relative pose relationship between the two cameras to be recovered.

[0029] A4: Calculate the rotation matrix M from camera coordinates to geographic coordinates and simultaneously construct the triangulation network. Specifically: On the one hand, the method for calculating the rotation matrix M from camera coordinates to geographic coordinates is as follows: A4011: Calculate the camera center coordinates based on the camera coordinate system. The camera center can be understood as the camera optical center, and thus obtain the position coordinates of the camera center in the camera coordinate system. A4012: Based on the geographic coordinate system, the geographic coordinates of the camera center can be read. Through external devices, such as GPS positioning modules and prior geographic information, the coordinates of the camera optical center in the geographic coordinate system can be directly obtained. A4013: Describe how the camera coordinates' x / y / z axes are rotated using a rotation matrix M to align with the geographic coordinate system. If the vectors in the camera coordinate system are... The vector in the geographic coordinate system is ;but: (1) Based on the above formula (1), the rotation matrix M from camera coordinates to geographic coordinates can be calculated.

[0030] On the other hand, triangulation involves connecting points in the initial point cloud into a triangular mesh to represent the surface shape in three-dimensional space, such as using the Delaunay triangulation algorithm. In this embodiment, the method used to construct the triangulation is as follows: A4021: Triangulation is performed based on the relative pose relationship between the second frame and the first frame. Triangulation utilizes the projection relationship between the two frames and calculates the 3D point cloud corresponding to the matching point based on the pixel coordinates of the matching point in the two frames, the camera's intrinsic parameters, and the relative pose relationship. A4022: Calculate camera coordinates to obtain the coordinates of the 3D point cloud in the current camera coordinate system (understood as: local coordinates); A4023: Coarse screening of triangulated 3D point clouds. The core purpose of coarse screening is to filter out invalid noise points, and camera coordinates are the key basis for determining whether the 3D point cloud is valid. The coarse screening method is as follows: A40231: Camera coordinates provide the basis for filtering the spatial range and determine the "reasonable three-dimensional point spatial range" based on the camera coordinates; A40232: Eliminating "outliers" caused by matching errors. The accuracy of triangulation depends on the matching quality of image feature points. If there are incorrect matches (such as mismatching points from different scenes as corresponding points), "abnormal 3D points" that deviate from the real scene will be generated. The spatial location of these outliers usually does not match the "reasonable shooting range" corresponding to the camera coordinates (such as being too far or too close to the camera). By setting a spatial threshold based on the camera coordinates, these outliers can be quickly and coarsely filtered out.

[0031] A4024: Perform local bundle adjustment optimization. Local bundle adjustment is based on the idea of ​​bundle adjustment and optimizes the position of the 3D point cloud and the pose of the camera within a local range. A4025: Fine-screening of 3D point clouds to obtain initial point clouds. Based on coarse screening and local bundle adjustment, further screening is performed to select more accurate and reliable 3D point clouds as initial point clouds. A4026: Constructing a triangular mesh based on an initial point cloud, connecting the points in the initial point cloud into a triangular mesh to represent the surface shape in three-dimensional space, and constructing the triangular mesh using the Delaunay triangulation algorithm.

[0032] A5: Construct a geographic coordinate triangulation W based on the triangulation and rotation matrix; transform the triangulation to the geographic coordinate triangulation W based on the rotation matrix M, which is to transform the camera coordinate system-based triangulation to the geographic coordinate system, so that it has a real geographic spatial location.

[0033] A6: A preliminary projected image is generated through GPU texture mapping projection. GPU texture mapping utilizes the parallel computing power of the graphics processing unit (GPU) to map the texture information of the image, such as color and texture details, onto a triangular mesh, giving the 3D model a realistic appearance. The GPU texture mapping includes planar projection and orthographic projection. Planar projection projects the 3D triangular mesh onto a plane; orthographic projection is a special type of perpendicular projection that can obtain an effect similar to an orthographic image, facilitating subsequent image stitching and display.

[0034] The above section describes the image stitching method based on the first and second frames. Building upon this, the image stitching method for the subsequent third, fourth, ... N-1 and Nth frames is performed using the following method: B1: Extract feature points from the Nth frame. Use a feature extraction algorithm to extract feature points from each subsequent frame image to prepare for matching with existing 3D point clouds or previous frames. Simultaneously execute B2 and B3. B2: Match with frame N-1. The matching method is to match the feature points of frame N with the projection features of the 3D point cloud in frame N-1 by determining whether there are enough correspondences to perform the matching. If a match is found, proceed to B4; If no match is found, a matching frame is searched within the tolerance radius. If no matching frame is found, the current workflow cannot continue and the process ends. If a matching frame is found, it means that there is a part that can be spliced ​​between the Nth frame and its previous frame, and the process proceeds to B4. The tolerance radius is a set range within which possible matching frames are searched. This is to expand the search range and try to find frames that overlap with subsequent frames when a match fails. Searching for matching frames within the tolerance radius compresses the search range from "all frames" to "local small-range frames" and makes the matching index lightweight, avoiding complex feature calculations.

[0035] B3: Match with the initial point cloud, solve the relative pose relationship between the Nth frame and the (N-1)th frame, and calculate the rotation and translation parameters of the camera in the Nth frame relative to the camera in the (N-1)th frame. That is: calculate the rotation matrix M' from the current frame coordinates to the geographic coordinates and execute B4 simultaneously; the method used to calculate the rotation matrix M' from the current frame coordinates to the geographic coordinates is as follows: B301: Calculate the camera center coordinates of the current frame based on the camera coordinate system, that is: calculate the position coordinates of the camera optical center in the camera coordinate system of the current frame; B302: Based on the geographic coordinate system, the geographic coordinates of the camera center can be read. Similarly, through external devices, such as GPS positioning modules and prior geographic information, the coordinates of the camera optical center in the geographic coordinate system can be directly obtained. B303: Calculate the rotation matrix M' from the camera coordinates to the geographic coordinates of the current frame. The rotation matrix M' describes how the x / y / z axes of the camera coordinates are rotated to align with the geographic coordinate system. It is obtained by calculating the above formula (1).

[0036] B4: Update the triangular network based on the new point cloud; wherein, the method used to update the triangular network is as follows: B401: Triangulation is performed based on the relative pose relationship between the Nth frame and the (N-1th)th frame. Triangulation is performed by using the projection relationship between the two frames and calculating the 3D point cloud corresponding to the matching point based on the pixel coordinates of the matching point in the two frames, the camera's intrinsic parameters, and the relative pose relationship. B402: Calculate camera coordinates, which is the same as "A4022" above, and will not be repeated here; B403: Coarse screening of triangulated 3D point clouds, the principle is the same as "A4023" above, and will not be repeated here; B404: Performs local beam adjustment optimization, which is the same as the principle of "A4024" above, and will not be described again; B405: Refines the 3D point cloud and obtains a new point cloud. The principle is the same as "A4025" above, so it will not be described again. B406: Local beam adjustment optimization is performed based on the new point cloud. The local beam adjustment optimizes the 3D point cloud after the addition of the new point cloud, reduces the reprojection error, and improves the accuracy of the 3D point cloud and camera pose. Based on this, the triangular mesh is readjusted and updated according to the newly added point cloud so that it can more accurately represent the surface of three-dimensional space.

[0037] B5: Construct a geographic coordinate triangulation W' based on the updated triangulation and rotation matrix M' to transform the updated triangulation to the actual geographic coordinate system, ensuring the correctness of its geographic spatial location, and searching for the best matching frame of the triangle within the tolerance radius; Searching for the best-matching frame for the triangle within the tolerance radius is to find the next frame to be processed, ensuring that the subsequent frame has sufficient overlap with the currently processed point cloud or frame to guarantee the continuity of image stitching. This narrows the search range and reduces the computational complexity of frame matching. The method for searching for the best-matching frame for the triangle is as follows: B501: Setting the tolerance radius Typically, the centroid or vertex of the target triangle is used as the center of a circle, and the radius value is set based on the drone's flight altitude, camera field of view (FOV), and frame overlap rate (for example, when the flight altitude is 100 meters and the FOV is 60°, the radius is set to 10-20 meters) to ensure that frames within the radius can cover the triangle. B502: Filtering candidate frames Retrieve the camera geographic coordinates of all stored image frames, and filter out all frames whose camera geographic coordinates fall within the tolerance radius as candidate frames; B503: Calculate the matching degree between the candidate frame and the target triangle, where the matching degree indexes are: feature point overlap rate, texture similarity and projection error; B504: Select the best matching frame All candidate frames are weighted and scored according to matching metrics (e.g., feature point overlap rate of 40%, texture similarity of 40%, projection error of 20%). The frame with the highest score is selected as the best matching frame. The best matching frame contains the scene texture that best matches the geographic coordinate triangulation W'.

[0038] B6: Based on the best matching frame and the geographic coordinate triangulation W', a projected image is generated by GPU texture mapping, which includes planar projection and orthographic projection; wherein, N≥3, and B1-B6 are executed in a loop to update the projected image in real time by superimposing 1 frame at a time.

[0039] S4: Display and dynamically update the stitched image in real time on the remote control's screen.

[0040] The above explanation indicates that the first frame, the second frame, ..., the (N-1)th frame and the Nth frame refer to the first frame image, the second frame image, ..., the (N-1)th frame image and the Nth frame image, respectively.

[0041] Example 2 This embodiment also provides a real-time UAV image stitching system, which includes: a UAV, a remote controller that establishes a communication connection with the UAV, the main control chip of the remote controller being communicatively connected to a memory, the memory containing an application program, and the main control chip executing the real-time UAV image stitching method as described in Embodiment 1 above by calling the application program.

[0042] When the system is running, the drone and the remote controller are connected via a wireless local area network. It does not need to connect to external 4G, 5G or other external networks. Data collection and processing are all completed locally, without the need for a third device, such as a computer or server, which greatly expands the scope of its application scenarios.

[0043] Example 3 This embodiment also provides a storage medium storing program instructions. After reading the program instructions, the computer executes the real-time UAV image stitching method as described in Embodiment 1 above.

[0044] It should be noted that any process or method description in the flowchart or otherwise described herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including in substantially the same manner or in the reverse order of the functions involved, as should be understood by those skilled in the art to which the embodiments of this application pertain.

[0045] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0046] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0047] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0048] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0049] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A real-time UAV image stitching method, characterized in that, The method includes: S1: The drone collects image data according to the planned flight path; S2: Transmits each frame of image data acquired to the remote control in real time; S3: The image data is stitched together in real time using the image processing algorithm built into the remote control to obtain the stitched image; S4: Display and dynamically update the real-time stitched image on the remote control's screen; The image processing algorithm includes: A1: Extract feature points from the first frame; A2: Extract feature points from the second frame and match them with feature points from the first frame to obtain matching points; A3: Solve for the relative pose of the second frame with respect to the first frame; A4: Calculate the rotation matrix M from camera coordinates to geographic coordinates and simultaneously construct a triangulation network, where the construction of the triangulation network is based on the initial point cloud; A5: Construct a geographic coordinate triangulation W based on the triangulation and rotation matrix M; A6: Generate a preliminary projected image through GPU texture mapping projection; The image processing algorithm further includes: B1: Extract feature points of the Nth frame, and execute B2 and B3 simultaneously; B2: Match with frame N-1. If a match is found, proceed to B4. If no match is found, search for a matching frame within the tolerance radius. If no matching frame is found, the process ends. If a matching frame is found, proceed to B4. B3: Match with the initial point cloud, solve the relative pose relationship between the Nth frame and the (N-1th)th frame, calculate the rotation matrix M' from the current frame coordinates to the geographic coordinates and execute B4 simultaneously; B4: Update the triangular network based on the new dot cloud; B5: Construct a geographic coordinate triangulation W' based on the updated triangulation and rotation matrix M', and search for the best matching frame of the triangle within the tolerance radius; B6: Generates projected images via GPU texture mapping; Where N≥3, the projected image is updated in real time by cyclically executing B1-B6, where 1 frame is superimposed each time.

2. The real-time UAV image stitching method according to claim 1, characterized in that, The method for calculating the rotation matrix M from camera coordinates to geographic coordinates is as follows: A4011: Calculate the camera center coordinates based on the camera coordinate system; A4012: Reads the geographic coordinates of the camera center based on the geographic coordinate system; A4013: Calculate the rotation matrix M from camera coordinates to geographic coordinates.

3. The real-time UAV image stitching method according to claim 1, characterized in that, The method used to construct the triangular network is as follows: A4021: Triangulation is performed based on the relative pose relationship between the second frame and the first frame; A4022: Calculate camera coordinates; A4023: Coarse screening of triangulated 3D point clouds; A4024: Perform local beam adjustment optimization; A4025: Fine-tuning the 3D point cloud and obtaining the initial point cloud; A4026: Constructing a triangular network based on the initial point cloud.

4. The real-time UAV image stitching method according to claim 1, characterized in that, The method used to calculate the rotation matrix M' from the current frame coordinates to the geographic coordinates is as follows: B301: Calculate the camera center coordinates of the current frame based on the camera coordinate system; B302: Reads the geographic coordinates of the camera center based on the geographic coordinate system; B303: Calculate the rotation matrix M' from the camera coordinates to the geographic coordinates of the current frame.

5. The real-time UAV image stitching method according to claim 1, characterized in that, The method used to update the triangular network is as follows: B401: Triangulation is performed based on the relative pose relationship between the Nth frame and the (N-1th)th frame; B402: Calculate camera coordinates; B403: Coarse screening of triangulated 3D point clouds; B404: Perform local beam adjustment optimization; B405: Refine the 3D point cloud and obtain a new point cloud; B406: Local bundle adjustment optimization and triangulation update based on new point cloud.

6. The real-time UAV image stitching method according to claim 1, characterized in that, The GPU texture mapping includes planar projection and orthographic projection.

7. A real-time UAV image stitching system, the system comprising: The drone and the remote controller that establishes a communication connection with the drone, wherein the main control chip of the remote controller is connected to a memory, the memory has a built-in application program, and the main control chip executes the real-time drone image stitching method as described in any one of claims 1-6 by calling the application program.

8. A storage medium storing program instructions, wherein a computer reads the program instructions and executes the real-time UAV image stitching method as described in any one of claims 1-6.

Images