An unmanned aerial vehicle (UAV) thermal infrared image stitching method and system based on track keeping

Abstract

The application discloses a kind of based on track keeping's unmanned aerial vehicle thermal infrared image splicing method and system, it is related to image splicing technical field, including steps: constructing image set;Obtain the matching feature point of any pair of to-be-spliced unmanned aerial vehicle's thermal infrared image respectively, and the re-projection error of matching feature point is generated alignment item by the similar transformation matrix of to-be-spliced thermal infrared image;The actual distance between the center of reference image and the thermal infrared image of any unmanned aerial vehicle is calculated, and the track trajectory protection item is obtained between the center point of similar transformation matrix and reference image by pixel distance;Alignment item and track trajectory protection item are weighted and summed to construct global optimization objective function, and global optimization objective function is converted into linear least square model with similar transformation constraint to solve, and generate spliced thermal infrared panoramic image.The application minimizes the cumulative error of splicing process, while ensuring the integrity of global structure.

Description

Technical Field

[0001] This invention relates to the field of image stitching technology, and in particular to a method and system for stitching UAV thermal infrared images based on track keeping. Background Technology

[0002] In recent years, with the continuous improvement of agricultural automation, drone technology has been widely used in agricultural monitoring, becoming a key tool in modern agriculture. Compared with traditional ground surveying and satellite remote sensing, drones can flexibly acquire high-resolution images of large areas of farmland, enabling farmers and researchers to monitor crop health and other key farmland information in real time, providing strong technical support for the further development of precision agriculture.

[0003] Compared to traditional visible light images, thermal infrared images can sense differences in the thermal radiation energy emitted by ground objects, a characteristic that makes them crucial in fields such as agricultural monitoring. By using drones equipped with thermal infrared remote sensing, it is possible to effectively capture thermal radiation information emitted by crops, thereby reflecting temperature changes in the crop canopy. This allows for wide-ranging applications in agriculture and forestry, including soil moisture balance assessment and water stress detection, and crop lodging identification, promoting the development of precision agriculture. Furthermore, thermal infrared remote sensing technology is also widely used in the detailed study of urban thermal environments and real-time ground semantic segmentation. By stitching together multiple images, large-scale panoramic images can be generated, providing researchers with a complete view of farmland and facilitating more comprehensive crop condition assessments.

[0004] Compared to visible light images, thermal infrared images face more challenges in the stitching process. For example, the effects of wind and temperature drift, as well as low geometric resolution and poor imaging quality, often lead to large errors in image feature point extraction and matching, thus affecting the stitching effect of the final panoramic image. In recent years, researchers have proposed various improved methods to address the challenges of large-scale thermal infrared image stitching. Although these methods have made some progress in improving stitching accuracy or efficiency, they still have limitations. Some existing technologies rely on mesh deformation, such as OP-GSP, which uses POS information during UAV flight and uses the estimated overlap of adjacent images as the weight of local matching pairs to optimize the local alignment term of GSP to improve the stitching effect. However, this significantly increases computational complexity and processing time, making it difficult to meet real-time requirements. Furthermore, some fast global stitching schemes, such as MegaStitch, model image stitching as a global minimization of reprojection error and constrain the transformation space to a similarity transformation, which has an advantage in stitching efficiency. However, they often suffer from serious impacts on the global result and the integrity of the stitched structure due to local image alignment failures. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the prior art by providing a method and system for stitching thermal infrared images of unmanned aerial vehicles (UAVs) based on track keeping, so as to solve the problems in the prior art.

[0006] The present invention specifically provides the following technical solution:

[0007] A method for stitching UAV thermal infrared images based on track maintenance, comprising:

[0008] Collect multiple thermal infrared images of the drone and construct an image set;

[0009] Obtain the feature points that match any one of the thermal infrared images to be stitched in the image set, and obtain the similarity transformation matrix between each thermal infrared image and the reference coordinate system. Generate an alignment term by using the reprojection error of the similarity transformation matrix of the thermal infrared images to be stitched and the matching feature points.

[0010] Calculate the actual distance between the center of any thermal infrared image and the center of the reference image, and convert the actual distance to obtain the pixel distance between the center of any thermal infrared image and the center of the reference image in the x and y directions. Generate a flight path protection item based on the pixel distance.

[0011] A global optimization objective function is constructed by weighting and superimposing the alignment term and the flight path protection term. The global optimization objective function is then transformed into a linear least squares model with similarity transformation constraints for solution, generating a stitched thermal infrared panoramic image.

[0012] Preferably, the step of generating an alignment term by using the similarity transformation matrix of the thermal infrared images to be stitched and the reprojection error of the matched feature points specifically involves:

[0013] Based on any feature point matched in the thermal infrared images to be stitched, the difference between the similarity transformation matrix of the thermal infrared images to be stitched and the product of their respective feature points is taken, and the absolute value is squared to generate the alignment term of the thermal infrared images to be stitched.

[0014] Preferably, the step of generating the flight path protection item based on the pixel distance specifically involves:

[0015] Obtain the similarity transformation matrix of the reference image;

[0016] The similarity transformation matrix of the thermal infrared image to be stitched is subtracted from the similarity transformation matrix of the reference image. The difference is then multiplied by the coordinates of the center pixel of the thermal infrared image to be stitched on the original image. The product is then subtracted from the pixel distance, and the absolute value is squared to generate the flight path protection item.

[0017] Preferably, the step of transforming the global optimization objective function into a linear least squares model with similarity transformation constraints for solution, and generating a stitched thermal infrared panoramic image, specifically involves:

[0018] The similarity transformation matrix parameters of each thermal infrared image to be stitched are arranged into a vector in order, and all matching pairs and flight path constraints are linearized into sparse matrix form; where a matching pair is a thermal infrared image to be stitched that is matched with it, and the flight path constraint is a trajectory constraint performed through the flight path trajectory protection term.

[0019] The sparse matrix form is solved by a linear least squares solver with similarity transformation structure constraints to obtain the optimal vector. The similarity transformation matrix is ​​then restored using the optimal vector, and the restored similarity transformation matrix is ​​used to generate a stitched thermal infrared panoramic image.

[0020] This invention provides a UAV thermal infrared image stitching system based on track keeping, comprising:

[0021] The acquisition module is used to acquire multiple thermal infrared images of the UAV and construct an image set;

[0022] The alignment term generation module is used to obtain the feature points that match any one of the thermal infrared images to be stitched in the image set, and to obtain the similarity transformation matrix between each thermal infrared image and the reference coordinate system. The alignment term is generated by the reprojection error between the similarity transformation matrix of the thermal infrared images to be stitched and the matching feature points.

[0023] The protection item generation module is used to calculate the actual distance between the center of any thermal infrared image and the center of the reference image, and convert the actual distance to obtain the pixel distance between the center of any thermal infrared image and the center of the reference image in the x and y directions, and generate the flight path protection item through the pixel distance;

[0024] The stitching module is used to construct a global optimization objective function by weighting and superimposing the alignment term and the flight path protection term, and to solve the global optimization objective function by transforming it into a linear least squares model with similarity transformation constraints, thereby generating a stitched thermal infrared panoramic image.

[0025] The present invention provides a computer device, including a memory and a processor. The memory stores a program, and when the program is executed by the processor, the processor performs the steps of the above-described method for stitching UAV thermal infrared images based on track keeping.

[0026] The present invention provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the above-described method for stitching UAV thermal infrared images based on track keeping.

[0027] Compared with the prior art, the present invention has the following significant advantages:

[0028] This invention generates an alignment term by using the similarity transformation matrix of the thermal infrared images to be stitched and the reprojection error of the matched feature points. Simultaneously, it calculates the actual distance between the center of any UAV's thermal infrared image and the center of a reference image, converting this actual distance into pixel distance to accurately maintain their relative position in physical space. By obtaining the pixel distance between the center of any thermal infrared image and the center of the reference image in the x and y directions, a flight path protection term is generated using this pixel distance. Furthermore, the alignment term and the flight path protection term are combined to construct a global optimization objective function, which is then solved. This significantly accelerates the iterative optimization speed while minimizing the cumulative error in the stitching process and ensuring the integrity of the global structure, thus achieving the generation of high-quality stitched panoramic images. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the spatial layout in an embodiment of the present invention; wherein Figure 1 (a) represents the spatial layout structure that needs to be constrained during the splicing process. Figure 1 (b) connects the sampling points to the central sampling point during the splicing process;

[0030] Figure 2 This refers to the location of the main research area in the embodiments of the present invention; wherein Figure 2 (a) is the dataset of jujube trees in Xinjiang. Figure 2 (b) is a dataset of farmland in Inner Mongolia;

[0031] Figure 3 These are thermal infrared images showing significant exposure differences in embodiments of the present invention; wherein... Figure 3 (a) is a brighter thermal infrared image. Figure 3 (b) is a darker thermal infrared image;

[0032] Figure 4 This is a comparison chart showing the stitching effects of different stitching methods on 558 thermal infrared images of a UAV collected at a flight altitude of 120m in 2018, as described in this embodiment of the invention; wherein... Figure 4 (a) is a Pix4D stitched image. Figure 4 (b) is a PhotoScan stitched image. Figure 4 (c) is a stitched image created using the AutoStitch stitching algorithm. Figure 4 (d) is a mosaic image created using MegaStitch. Figure 4 (e) is a stitched image created using OP-GSP. Figure 4 (f) is a splicing diagram assembled by the method of the present invention;

[0033] Figure 5This is a comparison image showing the stitching effects of different stitching methods on 563 thermal infrared images of a UAV collected at a flight altitude of 120m in 2019, as described in this embodiment of the invention; wherein... Figure 5 (a) is a stitched image created using PTGui. Figure 5 (b) is a stitched image created using AutoStitch. Figure 5 (c) is a mosaic image created using MegaStitch. Figure 5 (d) is a stitched image created using OP-GSP. Figure 5 (e) is a splicing diagram assembled by the method of the present invention;

[0034] Figure 6 This is a comparison image showing the stitching effects of different stitching methods used in this embodiment of the invention on 774 thermal infrared images of a UAV collected at a flight altitude of 20m in 2024; wherein... Figure 6 (a) is a stitched image created using MegaStitch. Figure 6 (b) is a splicing diagram assembled by the method of the present invention;

[0035] Figure 7 These are the RMSE values ​​of the evaluation index for different years in the embodiments of the present invention; wherein Figure 7 (a) represents the RMSE value for the 2018 evaluation indicator. Figure 7 (b) represents the RMSE value for the 2019 evaluation indicator. Figure 7 (c) represents the RMSE value for 2024;

[0036] Figure 8 This is an IDD comparison performed on data captured at different altitudes in an embodiment of the present invention; wherein Figure 8 (a) was taken at a height of 120m in 2018. Figure 8 (b) was taken at a height of 120m in 2019. Figure 8 (c) was taken at a height of 20m in 2024;

[0037] Figure 9 This document compares the stitching times of MegaStitch, OP-GSP, and the method of this invention on 45 sets of thermal infrared datasets in this embodiment of the invention; wherein... Figure 9 (a) is the splicing of MegaStitch. Figure 9 (b) is the splicing of OP-GSP. Figure 9 (c) represents the splicing of the method of the present invention;

[0038] Figure 10 This is a visualization analysis diagram of 519 UAV remote sensing images of farmland taken at a height of 120m in 2019, taken in this embodiment of the invention, under flight path maintenance constraints; wherein, Figure 10(a) is a recorded GPS route map. Figure 10 (b) shows the MegaStitch alignment result. Figure 10 (c) represents the alignment result of the method of the present invention;

[0039] Figure 11 This is a visualization analysis diagram of 774 UAV farmland remote sensing images taken at a height of 20m in 2024, taken in this embodiment of the invention, under flight path maintenance constraints; wherein, Figure 11 (a) is the recorded GPS route. Figure 11 (b) shows the MegaStitch alignment result. Figure 11 (c) represents the alignment result of the method of the present invention;

[0040] Figure 12 This is a comparison chart of the FTE results of the method of the present invention and MegaStitch in this embodiment of the invention; wherein, Figure 12 (a) is a comparison chart of the results of the method of this invention and MegaStitch in 2018 on the FTE index. Figure 12 (b) is a comparison chart of the results of the method of this invention and MegaStitch in 2019 on the FTE index. Figure 12 (c) is a comparison chart of the results of the method of this invention and MegaStitch in 2024 on the FTE index;

[0041] Figure 13 This is an overall flowchart of a UAV thermal infrared image stitching method based on track keeping according to the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0043] Traditional image stitching techniques mainly include basic processes such as feature extraction, matching, transform estimation, and image fusion. Early methods, such as AutoStitch proposed by Brown and Lowe, assumed pure camera rotation and used hand-designed local features in conjunction with RANSAC to complete the stitching by estimating only a single global homography matrix.

[0044] As research has deepened, scholars have increasingly focused on image stitching with parallax. DHW (Demonstration of Hierarchical Image Transformation) was proposed, combining two homography transformations to approximate complex real-world transformation models, but it remains insufficient in high parallax scenarios. SVA (Simplified Image Transformation) was proposed, first using global affine transformation to preserve the overall image structure, then compensating for deviations in the local transformation of each grid, significantly improving local alignment accuracy. APAP (Advanced Image Transformation Approach) was proposed, utilizing "Moving DLT" to calculate local homography for each grid, achieving efficient spatial deformation, but it is prone to causing more severe distortion in high parallax scenarios.

[0045] To address this issue, the GSP method is proposed, which performs scale and rotation prediction for each image and then optimizes the mesh by combining APAP vertex alignment results with local and global similarity constraints, balancing alignment accuracy and natural visual effects. Furthermore, the researchers optimized the stitching process by introducing geometric structure constraints. By using projection invariant features to match coplanar local sub-regions, both the local and global structure of wide parallax images are preserved, while artifacts and distortion are reduced. A structure-preserving image stitching algorithm is proposed, which further enhances the geometric consistency of the stitching by preserving the extracted straight and curved structures.

[0046] In the field of UAV remote sensing image stitching, early research mainly focused on improving visual effects through image post-processing. One approach proposed a bisecting line stitching algorithm based on local geometric features of the image, generating visually seamless remote sensing panoramic images. By utilizing graph cut energy minimization, the optimal stitching line for multi-view images was successfully detected, enabling multi-image stitching of agricultural remote sensing images. However, these methods only alleviated misalignment at the visual level and did not substantially improve geometric alignment deviations.

[0047] As research progressed, researchers began to focus on how to utilize auxiliary information collected by drones to improve stitching results. One approach proposed using IMU data collected during drone flight to estimate and correct each stage of the stitching process, thereby improving stitching quality. Another approach proposed a spatial arrangement structure preservation algorithm based on triangular similarity transformation, utilizing geographic coordinate information collected by drones, effectively maintaining the geometric layout of farmland images.

[0048] Considering the real-time nature of large-scale data, researchers have proposed several acceleration strategies. Distributed feature extraction and parallel computing techniques have significantly improved the real-time video stitching effect. A fast anchor point matching strategy driven by position and pose information has been proposed, achieving high-speed stitching by selecting a small number of key anchor points. An adaptive stitching algorithm has been constructed using geographic tags and local deformation techniques, improving both stitching speed and visual quality. MegaStitch has been proposed, using linear least squares with similarity transformation constraints to transform stitching into globally minimizing reprojection error, enabling fast stitching of large-scale UAV image sets.

[0049] However, most of the aforementioned methods are designed for visible light or limited stitching scenarios, and research on stitching for thermal infrared images remains relatively scarce. This paper proposes a novel thermal infrared image stitching method based on three-stage homography matrix correction and global error optimization. This method calculates the image overlap rate using latitude and longitude data collected during flight, using this as an alignment weight and introducing spatial position constraints. The method achieves millimeter-level high-precision stitching and positioning.

[0050] However, thermal infrared remote sensing often suffers from increased feature extraction and matching errors due to factors such as wind speed, temperature drift, and low geometric resolution, which in turn affects the stitching quality. Therefore, how to ensure both high accuracy and real-time performance in large-scale thermal infrared remote sensing image stitching remains a key problem that urgently needs to be solved in this field.

[0051] How to significantly improve stitching speed while maintaining a certain stitching quality has become a key technical bottleneck that urgently needs to be overcome in the current thermal infrared remote sensing image processing of UAVs.

[0052] To address these challenges, this invention proposes a fast stitching method for UAV thermal infrared images based on feature alignment and track preservation under similarity transformation constraints. Building upon MegaStitch's similarity transformation assumption, the method utilizes the UAV's GPS flight path prior to convert geographic coordinate information into pixel translation estimates, constructing a flight path protection term (FTP). This imposes additional constraints on the translation components of each thermal infrared image to accurately maintain its relative position in physical space. Finally, this invention combines reprojection errors and the flight path protection term (FTP) into a globally solved linear least squares problem with similarity transformation constraints. This minimizes the cumulative error in the stitching process while ensuring the integrity of the global structure, achieving the generation of high-quality panoramic images. Experimental results on real thermal infrared datasets demonstrate that this method outperforms several state-of-the-art stitching methods in both objective and subjective evaluations.

[0053] The contributions of this work include the following:

[0054] This invention designs a flight path protection item based on geographic coordinates. By utilizing the flight path and GPS information during the flight of the UAV, it effectively constrains the displacement of the image in physical space, ensuring that the stitching result faithfully reflects the real flight path as a whole and maintains the spatial position invariance of the sequence to be stitched.

[0055] This invention proposes a stitching method FTP-Stitch based on joint optimization of feature alignment and flight path protection. This method constrains the transformation space to a similarity transformation and provides a good initial solution by introducing a flight path protection term, thereby significantly reducing parameter complexity and achieving fast and robust global alignment.

[0056] Extensive experiments on a dataset of thermal infrared images collected under various crop types, flight altitudes, and environmental conditions demonstrate that the proposed method improves overall stitching efficiency by approximately 80% compared to representative thermal infrared stitching methods while maintaining high-quality stitching.

[0057] The MegaStitch splicing method based on similarity transformation alignment and its limitations are explained below.

[0058] MegaStitch is a fast image stitching method designed for large-scale image stitching. Its core idea is to model image stitching as a global minimization of reprojection error with similarity transformation constraints. It uses a linear least squares optimization strategy to globally align all images, effectively avoiding the error accumulation problem that is prone to occur in traditional incremental methods.

[0059] Specifically, let the image The transformation matrix from the coordinate system in the image to the coordinate system of the reference image is: The authors propose limiting the transformations between images to scaling, rotation, and translation, i.e., employing a similarity transformation model. Specifically, the transformation matrix... It can be described as:

[0060] (1);

[0061] Furthermore, assuming This represents the r rows of the matrix, where the homogeneous coordinates of any interior feature point in image Ii are... For images and The corresponding interior point pair Indicated. Ideally, a point pass The result of mapping to the reference coordinate system should be consistent with the point pass The mapped results are close, that is:

[0062] (2);

[0063] This provides a constraint for each row transformation of each corresponding interior point pair, resulting in the following two equations:

[0064] (3);

[0065] Under this constraint, the transformation matrix is ​​reduced from eight degrees of freedom to four, making the linear least squares solution process more efficient. By solving the above system of equations, the absolute transformation matrix of all images can be solved in one go. This allows each image to be registered to a unified reference coordinate system. Thanks to global linear solution and simplified parameter count, MegaStitch offers a significant speed advantage when processing large-scale image data, effectively suppressing overall drift and local misalignment, and demonstrating good stability and robustness.

[0066] However, rapid decision-making is crucial for agricultural applications, especially in scenarios requiring timely feedback, such as precision irrigation and disease control. In practical applications, particularly when dealing with low-texture, high-noise, or significantly thermally drifted thermal infrared images from drones, the traditional feature point matching method relied upon by MegaStitch is prone to matching failures, thus affecting the accuracy of global optimization and leading to obvious deformation or structural distortion in the overall stitching result. On the other hand, to ensure stitching speed, MegaStitch only aligns the images without fusion, resulting in noticeable seams in the final panoramic image.

[0067] like Figure 13 As shown, this embodiment of a UAV thermal infrared image stitching method based on track keeping includes the following steps:

[0068] Step S1: Collect multiple thermal infrared images of the UAV and construct an image set.

[0069] Step S2: Obtain the feature points that match any one of the thermal infrared images to be stitched in the image set, and obtain the similarity transformation matrix between each thermal infrared image and the reference coordinate system. Generate an alignment term by using the reprojection error of the similarity transformation matrix of the thermal infrared images to be stitched and the matching feature points.

[0070] Building upon MegaStitch's similarity transformation feature alignment, a proposed flight path preservation term is added, thus constructing a fast stitching model based on joint optimization of feature alignment and flight path preservation. Given an image set... ,in For reference, each thermal infrared image The similarity transformation between the reference coordinate system and the reference coordinate system is achieved by a matrix. The final optimization object is the transformation matrix of each image. Considering that agricultural drones typically maintain a fixed flight altitude and quasi-orthophoto imaging mode, the constrained transformation space is a similarity transformation, that is, only scaling, rotation and translation are considered. The specific transformation matrix can be described by equation (1).

[0071] Next, for any one of the thermal infrared images to be stitched together in the image set... and , and For each of the matched feature points, the alignment term is defined using the reprojection error. This is achieved by subtracting the product of the similarity transformation matrix of the thermal infrared images to be stitched from the product of each feature point, taking the absolute value, and then squaring it. In other words, the alignment term is generated using the reprojection error between the similarity transformation matrix of the thermal infrared images to be stitched and the matched feature points. The specific expression is as follows:

[0072] (8);

[0073] in, For alignment items, and Each is an arbitrary pair of thermal infrared images to be stitched together. and The matched feature points, Thermal infrared images to be stitched The similarity transformation matrix, Thermal infrared images to be stitched The similarity transformation matrix.

[0074] Step S3: Calculate the actual distance between the center of any thermal infrared image and the center of the reference image, and convert the actual distance to obtain the pixel distance between the center of any thermal infrared image and the center of the reference image in the x and y directions. Generate a flight path protection item based on the pixel distance.

[0075] The GPS system onboard the UAV can accurately record the flight trajectory and the geographic coordinates of each image frame, providing a reliable spatial prior for the stitching process. In large-scale image stitching, it can effectively suppress accumulated errors and protect the actual spatial relationships during the UAV's flight. Based on this, a flight path protection method based on geographic coordinates is proposed, which uses the geographic coordinate information during the UAV's flight to maintain the relative positional relationships of the images in the actual physical space.

[0076] For large-scale drone-captured scenes, the capture location of each image is treated as a sampling point, and all sampling points are mapped onto a unified reference canvas. Figure 1 (a) illustrates the spatial layout structure that needs to be constrained during the stitching process. Black dots represent the actual sampling locations of all images acquired by the drone, blue dots represent reference image locations, and red arrows show the actual flight path of the drone. Specifically, the geometric center of the shooting area is first determined based on latitude and longitude information, and the corresponding image is used as a reference. Then, the remaining sampling points are sequentially connected to the reference points, and their geographical distances are used to constrain the center position of the distorted image. This uses the drone's GPS flight path as a global spatial layout constraint, ensuring that the stitching result better reflects the actual flight path.

[0077] To further improve alignment accuracy, a route trajectory protection item based on geographic coordinates was introduced. For example... Figure 1 As shown in (b), any sampling point With the central sampling point Connected, their horizontal and vertical pixel distances in the stitched result It can be represented using latitude and longitude. The specific steps are as follows:

[0078] First, the actual distance between the center of any image i and the center of the reference image is calculated using the Haversine formula, specifically the actual distance in the x and y directions. , They are given by the following formulas respectively:

[0079] (4);

[0080] (5);

[0081] in , , , Let be the latitude and longitude of image i and the reference image, respectively, and R be the Earth's radius. By combining the UAV's flight altitude h, focal length f, and pixel size s, the actual distance is further converted into pixel distance. Thus, the pixel distance between any image i and the reference image in the x and y directions can be calculated:

[0082] (6);

[0083] Finally, let Let be the coordinates of the center pixel of image i on the original image. For each image in the set of images to be stitched (excluding the reference image), in order to ensure the spatial layout structure of the image sequence represented by the sampling points as much as possible, it is encouraged that the pixel distance between the center point of each image and the reference image after homography matrix transformation is close to the result calculated by GPS, so as to obtain the final flight trajectory preservation term. This involves obtaining the similarity transformation matrix of the reference image; subtracting the similarity transformation matrix of the thermal infrared image to be stitched from the similarity transformation matrix of the reference image; multiplying the difference by the coordinates of the center pixel of the thermal infrared image to be stitched in the original image; and subtracting the product from the pixel distance, taking the absolute value, and squaring it to generate the flight path protection item. The specific expression is as follows:

[0084] (7);

[0085] in, For route trajectory protection items, The similarity transformation matrix of the reference image, Let be the coordinates of the center pixel of image i in the original image. The pixel distance between any image and a reference image in the x and y directions.

[0086] Step S4: Construct a global optimization objective function by weighting and superimposing the alignment term and the flight path protection term, and then solve the global optimization objective function by transforming it into a linear least squares model with similarity transformation constraints to generate a stitched thermal infrared panoramic image.

[0087] The global optimization objective function is constructed by weighting the alignment term and the flight path protection term, and its specific expression is as follows:

[0088] (9);

[0089] in, Let G be the set of generated matching pairs, representing the global optimization objective function. and These represent the weighting coefficients of the reprojection error term and the flight path protection term, respectively, in all experiments. , .

[0090] Arrange the parameters of each similarity transformation matrix into a vector in order, and reconstruct the similarity transformation matrix using the optimal vector, specifically as follows:

[0091] Through parameters The quadratic function represents the alignment term. With route trajectory protection items The alignment term and the flight path protection term in the constructed global objective optimization function are expressed as quadratic functions of similarity transformation parameters, reducing equation (9) to a large-scale linear least squares problem with similarity constraints.

[0092] Each thermal infrared image to be stitched Similarity transformation matrix The parameters are arranged in order as a vector. The matching pairs and flight path constraints are linearized into a sparse matrix form; where each matching pair is a thermal infrared image to be stitched together, and the flight path constraint is a trajectory constraint performed through a flight path trajectory protection term; specifically represented as follows:

[0093] (10);

[0094] in, The overall constraint matrix is ​​formed by concatenating the Jacobian matrices corresponding to the feature point matching term and the track preservation term. Let be the target value vector, representing the observation residual consisting of the expected matching error and the trajectory error.

[0095] The optimal vector is obtained by solving the sparse matrix form using a linear least squares solver with similarity transformation structural constraints. And through the optimal vector The similarity transformation matrix is ​​restored, and the stitched thermal infrared panoramic image is generated using the restored similarity transformation matrix.

[0096] This joint optimization method, while maintaining MegaStitch's original advantage of fast global alignment, further suppresses accumulated errors and improves structural consistency by introducing a geographic flight path preservation term and similarity transformation constraints. This ensures that the stitched image has both high-precision geometric alignment and maintains geographic spatial consistency. It is suitable for large-scale, low-texture thermal infrared UAV image stitching tasks.

[0097] Experimental verification:

[0098] The UAV thermal infrared remote sensing dataset covers various crop types, meteorological conditions, data collection periods, and flight altitudes, mainly including... Figure 2 The presentation includes two parts: the Inner Mongolia farmland dataset and the Xinjiang jujube tree dataset.

[0099] The Inner Mongolia farmland dataset used was publicly released by Cui et al. in their 2021 study. This dataset was collected in the western part of the Hetao Irrigation District, specifically the Jiefangzha Irrigation Area of ​​the Shahaoqu Irrigation District in Inner Mongolia Autonomous Region, China. The geographic coordinates range from 40°52′ to 41°00′ N, 107°05′ to 107°10′ E, with an average elevation of 1050 m. This region has a mid-latitude continental monsoon climate, with an average annual temperature of 6.7°C and average annual precipitation of 150–250 mm. Major crops include wheat, maize, and sunflower.

[0100] Data covered multiple growth stages of farmland crops (seedling stage, budding stage, and maturity stage) between 2018 and 2019. Data was collected in the morning, at noon, in the afternoon, and under three weather conditions: sunny, cloudy, and light rain, to assess the impact of different time-of-day light on the mosaic effect.

[0101] The study area comprised four farmland plots (Fields 1-4), each ranging in size from 150,000 to 250,000 m². The salinity of the four plots showed a gradually increasing trend, with Field 1 having the lowest salinity and Field 4 the highest. Due to the characteristics of thermal infrared images, the colors and textures of the images within each region exhibited high similarity, posing a challenge to image stitching. Furthermore, thermal infrared images are prone to temperature drift, leading to significant exposure differences between adjacent images (e.g., below). Figure 3As shown in the figure, this further significantly increases the difficulty of image stitching.

[0102] The data acquisition equipment was a DJI Matrice M600 multi-rotor drone equipped with a Zenmuse XT thermal infrared camera system with a resolution of 640×512. The drone's flight altitude was set at 120m. Thermal infrared remote sensing images of each study area were acquired according to a pre-planned flight path, with a forward overlap rate of 80% and a lateral overlap rate of 70% to ensure comprehensive image coverage and sufficient redundancy, supporting subsequent image stitching.

[0103] Xinjiang jujube tree dataset:

[0104] The Xinjiang jujube tree dataset was collected in Ruoqiang County, Bayingolin Mongol Autonomous Prefecture, Xinjiang Uygur Autonomous Region, China. The geographical coordinates range from 39°00′ to 39°15′ north latitude and 88°05′ to 88°20′ east longitude, with an average altitude of approximately 900 meters. This region belongs to an arid desert climate zone, with annual precipitation below 50 mm, high evaporation, and extreme diurnal temperature variations.

[0105] This dataset was collected using a DJI Matrice M300 drone equipped with a Zenmuse H20T thermal infrared camera system. To obtain more details of the jujube orchard structure, the drone's flight altitude was reduced to 20m. Compared to the farmland dataset, the jujube orchard dataset presented additional challenges in image stitching: the lower flight altitude and the growth height of the fruit trees introduced a more severe parallax effect. Simultaneously, the irregularity of the canopy structure and the repetitiveness of textures reduced the stability of feature extraction and matching, significantly increasing the difficulty of image alignment and stitching. Specific data collection details are shown in Table 1.

[0106] Table 1. Environmental conditions and key imaging parameters of UAV remote sensing image data of farmland

[0107]

[0108] Experimental environment and model selection:

[0109] In this experiment, all tests were conducted on a computing server equipped with an Intel(R) Xeon(R) Silver 4210 CPU @ 2.20GHz (40 cores), four GeForce RTX 2080 Ti graphics cards, 128GB of RAM, and 4TB of SSD storage, and running the Ubuntu 16.04.7 LTS operating system.

[0110] The proposed stitching method was implemented in a Python 3.8 environment. In the feature extraction stage, the SURF+RANSAC algorithm was employed to ensure fast and robust feature detection and matching even under low-texture conditions in thermal infrared images. In the algorithm solution stage, the lsq_linear algorithm from the SciPy library was used to solve the linear least squares problem, efficiently obtaining the global transformation matrix for each image. Finally, in the image fusion stage, the stitching boundary was located based on the optimal seam detection algorithm, and the Poisson fusion technique was used to smooth the seam areas, significantly improving the naturalness and visual consistency of the final stitched image. To efficiently process large-scale UAV image datasets, the entire process was accelerated using a parallel computing framework, ensuring strong real-time processing capabilities while maintaining stitching quality.

[0111] Compared with representative methods:

[0112] The proposed stitching method, along with general remote sensing stitching software Pix4D and PhotoScan, and representative image stitching algorithms AutoStitch, OP-GSP, and MegaStitch, were used to evaluate the stitching quality on 45 different sets of UAV remote sensing image data from three aspects: subjective evaluation, objective evaluation, and stitching time.

[0113] Subjective evaluation:

[0114] Subjective evaluation is primarily based on direct human visual perception, making a comprehensive judgment on the stitching result. Evaluation indicators mainly include: whether the image alignment is accurate, whether the stitching result is natural, and whether the overall structure is well preserved, etc.

[0115] Figure 4 This presentation showcases the stitched results of thermal infrared images of a drone taken at a flight altitude of 120 meters in 2018. The results demonstrate the superior quality of the stitched images compared to mainstream commercial drone stitching software Pix4D and PhotoScan. Figure 4 (a) and Figure 4 (b) Due to the low texture characteristics of thermal infrared images, it is difficult to extract effective features, resulting in severe distortion and inability to generate a complete panoramic image. The classic stitching algorithm AutoStitch ( Figure 4 (c) also shows obvious deformation. MegaStitch ( Figure 4 (d) In large-scale image stitching, cumulative errors lead to trapezoidal deformation, and without fusion, the seams are obvious, and the structural integrity is compromised. As the most advanced thermal infrared image stitching algorithm currently available, OP-GSP ( Figure 4 Overall, (e) achieved good splicing results, but some alignment failures and distortions occurred at the straight lines of the field ridges.

[0116] In contrast, the method ( Figure 4 (f) introduces flight path preservation while minimizing global reprojection error, effectively accelerating the stitching speed while maintaining the overall flight path of the UAV, suppressing cumulative errors, and resulting in the best performance in geometric alignment, structural consistency, and visual naturalness. Furthermore, compared to the fusion method used in OP-GSP, this method utilizes stitching lines to effectively preserve temperature information in thermal infrared images. For example, Figure 4 The obvious changes in light and dark that appear on the right side of (f) correspond to the low temperature information in the shadow area formed under sunlight, which is also well preserved.

[0117] then Figure 5 The stitching results of thermal infrared images of a drone taken at a flight altitude of 120m in 2019 were verified. The results show that the commercial stitching software Pix4D failed to stitch this dataset; PTGui ( Figure 5 (a) performs reasonably well on this set of thermal infrared images with relatively obvious features, but there are still local distortions and missing parts of key images; AutoStitch ( Figure 5 (b) performs poorly in processing large-scale images, exhibiting significant distortion; MegaStitch ( Figure 5 (c) is similar to the results from 2018, exhibiting cumulative errors and showing obvious seam marks in the splicing results; OP-GSP ( Figure 5 While (d) achieves a good overall splicing effect, deformation still occurs in localized areas, failing to maintain the overall structure of the ridges. In contrast, the method proposed in this invention ( Figure 5 While ensuring high alignment accuracy, (e) effectively overcomes the above problems, and the splicing result performs best in terms of overall structure, detail preservation and visual naturalness.

[0118] Finally, to verify the algorithm's ability to handle flight altitude and parallax challenges, the flight altitude was lowered to 20 meters to collect data on jujube trees in Xinjiang. Lower flight altitude means more detail, but the complex canopy features also increase the difficulty of matching, and parallax between the ground and the fruit trees is unavoidable. Under these challenges, Figure 6This paper presents the stitched results of thermal infrared images of jujube trees in Xinjiang, taken in 2024 at a flight altitude of 20 meters. Mainstream commercial software such as Pix4D and PhotoScan faced significant challenges, resulting in stitching failures; even the most advanced thermal infrared stitching algorithm, OP-GSP, failed to complete the stitching successfully. Although MegaStitch obtained stitched results, due to the lack of flight path constraints, it failed to maintain the overall structure during the large-scale and complex stitching process, exhibiting significant errors in local alignment. In contrast, the algorithm proposed in this invention effectively maintained the overall structure of the acquired area and achieved excellent alignment for both the jujube trees and the ground plane.

[0119] The results show that the FTP-Stitch method proposed in this invention is superior to the comparison method in terms of alignment accuracy, local alignment and overall naturalness. In particular, when processing images with large temperature drift and exposure differences, it can effectively eliminate ghosting and obvious distortion, thereby generating panoramic images with better visual effects.

[0120] Objective evaluation:

[0121] Since most commercial software (such as Pix4D) is not open source, it is impossible to objectively evaluate them. Therefore, the evaluation in this invention only compares representative stitching methods. This invention uses root mean square error (RMSE) and image distortion degree (IDD) as objective evaluation metrics to measure the performance of different stitching methods.

[0122] To quantitatively evaluate the alignment accuracy of stitched panoramic images, this invention uses the widely used metric Root Mean Square Error (RMSE). RMSE calculates the root mean square of the Euclidean distance between feature points in the transformed images, providing a quantitative measure of the deviation between feature points. This deviation reflects the difference in distance between feature points and their true locations in the transformed image, thus reflecting the quality of image alignment. The calculation formula is typically expressed as:

[0123] (12);

[0124] Where G is the set of image matching pairs. , These are image matching pairs in G. The feature points are denoted by f, which represents the image transformation model of the thermal infrared images to be stitched, and N is the number of matching feature points. A lower RMSE value indicates higher image alignment accuracy. In the experiments of this invention, RMSE was used as an objective evaluation metric to compare the differences in alignment performance between different stitching methods.

[0125] To evaluate the alignment accuracy of the proposed method, the RMSE was calculated on data collected in 2018 and 2019 at a flight altitude of 120m, and in 2024 at a flight altitude of 20m. Figure 8 (a) and Figure 8 (b) shows the comparison of the proposed methods in 2018 and 2019 with representative concatenation algorithms in terms of RMSE metric, while Figure 8 (c) shows the evaluation results at a flight altitude of 20m in 2024. It is worth noting that due to the failure of OPGSP and MegaStitch to splice on some datasets, a valid RMSE could not be calculated, so the corresponding results are presented in a blank form.

[0126] from Figure 7 (a) and Figure 7 Results (b) show that at a flight altitude of 120m, the method of this invention significantly outperforms MegaStitch in terms of RMSE, demonstrating higher alignment accuracy. However, compared to OP-GSP, the RMSE results of this invention are generally better than OP-GSP, although OP-GSP has a slight advantage in some datasets. Nevertheless, OP-GSP experienced stitching failures in multiple datasets, while the algorithm of this invention ran stably, further demonstrating the robustness of the algorithm. Furthermore, since the algorithm of this invention focuses on fast stitching, it can approach the performance of OP-GSP in terms of RMSE while ensuring high efficiency, further verifying the effectiveness of the method. At a relatively challenging flight altitude of 20m, OP-GSP experienced stitching failures in all cases, and the method of this invention significantly outperformed MegaStitch in terms of RMSE, also demonstrating the stitching performance of the algorithm in complex environments.

[0127] In addition to the RMSE evaluation metric, considering that agricultural drones typically perform quasi-orthophoto imaging at a fixed flight altitude, the stitching process mainly involves similarity transformations (scaling, rotation, and translation) without causing drastic projection distortion, this invention utilizes Image Distortion Degree (IDD) as an objective indicator to evaluate the naturalness of the stitching result. IDD is primarily based on the uniformity of the area distribution on the reference plane after image transformation, and the naturalness of the stitching result is measured by calculating the root mean square of the area difference after image deformation. Specifically, the evaluation metric for IDD is defined as follows:

[0128] (13);

[0129] in This represents the area of ​​the i-th deformed image, while This represents the average area of ​​all transformed images. A more uniform area distribution indicates a more natural stitching result; therefore, the smaller the IDD value, the better the naturalness of the image.

[0130] the following Figure 8 The IDD (Integrated Data Distributed) comparisons are presented for data taken at a height of 120m in 2018 and 2019, and at a height of 20m in 2024. Since MegaStitch failed to stitch data on some datasets, a valid RMSE could not be calculated, and therefore the corresponding results are presented as missing values. It can be seen that the method of this invention achieves significantly lower IDD than MegaStitch, regardless of whether the data is stitched from high-altitude or low-altitude remote sensing data. This demonstrates that the algorithm of this invention maintains better overall naturalness and yields better stitching results.

[0131] Comparison of splicing efficiency:

[0132] In addition to subjective and objective evaluation of stitching quality, stitching efficiency is also a key dimension for measuring algorithm performance. Figure 9 The paper compares the stitching times of MegaStitch, OP-GSP, and the method of this invention on 45 thermal infrared datasets. Experimental results show that, under the same hardware environment, the method of this invention significantly outperforms OP-GSP in stitching speed, reducing the overall stitching time by an average of approximately 80%, greatly shortening the entire stitching process, and achieving real-time processing capabilities. Compared to MegaStitch without image fusion, the method of this invention has a slightly longer overall processing time. However, it is precisely because of the integration of post-processing such as optimal seam detection and Poisson fusion that superior stitching quality is achieved with a slightly increased time cost.

[0133] Flight path tracking capability analysis:

[0134] The effectiveness of this method is evaluated from two aspects: first, by visually comparing the flight trajectories of the UAVs before and after stitching; and second, by proposing a new route-keeping evaluation index to quantify the degree of deviation of the stitching result from the GPS route.

[0135] To verify the effectiveness of flight path maintenance constraints in improving stitching quality, this invention superimposes the registration center (red dot) of the stitched image onto the alignment result and compares it with the original GPS trajectory of the UAV flight. Figure 10 Taking a dataset of farmland photographed at a height of 120 m in Inner Mongolia as an example, Figure 10 (a) is the GPS trajectory recorded during the actual flight of the UAV. Figure 10(b) shows the MegaStitch alignment result without the addition of the flight path preservation constraint. Its registration center exhibits a significant shift and distortion compared to the original flight path, resulting in overall tilting and misalignment of the image set. Figure 10 (c) shows the alignment result of the method of the present invention. The registration center of each image is consistent with the original flight path, the field ridge lines are straight and smooth, and the overall structure is complete and visually natural.

[0136] Similarly, Figure 11 The comparison results of a 20m high jujube tree dataset from Xinjiang are presented, and the verification yielded consistent conclusions. Due to the large parallax and sparse texture in this scene, local misalignment is more likely to occur during the stitching process. However, after introducing a flight path protection term, the registration center of each image is consistent with the original GPS flight path, and the image alignment effect remains stable, indicating that the method has good adaptability and robustness even in complex environments.

[0137] The two comparisons above intuitively demonstrate that introducing flight path protection constraints can effectively suppress cumulative drift under various environmental conditions, significantly improving the geospatial consistency and visual naturalness of the stitching results.

[0138] To quantitatively assess flight path keeping capability, this invention proposes a flight path keeping evaluation index, Flight Trajectory Error (FTE), which measures the degree of deviation between the stitched result and the original GPS flight path. Its calculation method is as follows:

[0139] (14);

[0140] in, This represents the Euclidean distance of the i-th stitched image relative to the reference image, calculated using GPS. This represents the Euclidean distance of the i-th stitched image relative to the reference image in the stitched result. The root mean square error between them is used to measure the deviation of the stitched result from the original flight path. A lower flight path error indicates a higher degree of alignment between the stitched image and the actual UAV flight trajectory, thus proving that the stitched result can better preserve the original flight path in space.

[0141] Figure 12This paper presents a comparison of the FTE (Flight Tolerance) results of the method of this invention and MegaStitch. Experimental results show that MegaStitch, without the flight path protection constraint, produces a larger FTE value in the stitched image, indicating that the stitched result deviates from the actual flight trajectory of the UAV under the condition of large-scale reprojection error accumulation. However, after introducing the flight path protection term, this error is significantly reduced, and the stitched result is more accurate in geospatial terms. Further analysis shows that the flight path protection constraint proposed in this invention provides additional global geometric information, enabling the stitching process to not only rely on local matching features but also maintain the spatial consistency of the image over a large area, thereby reducing the impact of accumulated errors.

[0142] Subjective and objective evaluations conducted on thermal infrared datasets covering different crop types, flight altitudes (120m, 20m), and environmental conditions show that the method of this invention not only exhibits high geometric alignment accuracy and naturalness in stitching quality, effectively preserving [the original crop], but also significantly outperforms existing advanced methods in stitching efficiency, demonstrating its practical application potential in large-scale agricultural monitoring.

[0143] This invention proposes a UAV thermal infrared image stitching system based on track keeping, comprising: an acquisition module, an alignment item generation module, a protection item generation module, and a stitching module.

[0144] The system comprises the following modules: an acquisition module for acquiring multiple thermal infrared images from the UAV and constructing an image set; an alignment term generation module for obtaining the matching feature points of any thermal infrared image to be stitched from the image set and obtaining the similarity transformation matrix between each thermal infrared image and the reference coordinate system, generating an alignment term based on the reprojection error between the similarity transformation matrix of the thermal infrared images to be stitched and the matching feature points; a protection term generation module for calculating the actual distance between the center of any thermal infrared image and the center of the reference image, converting the actual distance to obtain the pixel distance between the center of any thermal infrared image and the center of the reference image in the x and y directions, and generating a flight path protection term based on the pixel distance; and a stitching module for constructing a global optimization objective function by weighted summation of the alignment term and the flight path protection term, transforming the global optimization objective function into a linear least squares model with similarity transformation constraints for solution, and generating a stitched thermal infrared panoramic image.

[0145] The present invention also provides a computer device, including a memory and a processor. The memory stores a program, and when the program is executed by the processor, the processor performs the steps of a method for stitching thermal infrared images of a UAV based on track keeping.

[0146] According to the disclosed embodiments, the computer device can communicate with one or more external devices (e.g., keyboard, pointing device, Bluetooth communication, etc.) or with any device that enables the computing device to communicate with one or more other computing devices (e.g., router, demodulator, etc.).

[0147] The above description, in conjunction with specific preferred embodiments, provides a more detailed explanation of the present invention. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for stitching UAV thermal infrared images based on track-keeping, characterized in that, include: Collect multiple thermal infrared images of the drone and construct an image set; Obtain the feature points that match any one of the thermal infrared images to be stitched in the image set, and obtain the similarity transformation matrix between each thermal infrared image and the reference coordinate system. Generate an alignment term by using the reprojection error of the similarity transformation matrix of the thermal infrared images to be stitched and the matching feature points. Calculate the actual distance between the center of any thermal infrared image and the center of the reference image, and convert the actual distance to obtain the pixel distance between the center of any thermal infrared image and the center of the reference image in the x and y directions. Generate a flight path protection item based on the pixel distance. A global optimization objective function is constructed by weighting and superimposing the alignment term and the flight path protection term, and then the global optimization objective function is transformed into a linear least squares model with similarity transformation constraints for solution, generating a stitched thermal infrared panoramic image. The flight path protection item is generated based on the pixel distance, specifically as follows: Obtain the similarity transformation matrix of the reference image; The similarity transformation matrix of the thermal infrared image to be stitched is subtracted from the similarity transformation matrix of the reference image. The difference is then multiplied by the coordinates of the center pixel of the thermal infrared image to be stitched on the original image. The product is then subtracted from the pixel distance, and the absolute value is squared to generate the flight path protection item. The process of transforming the global optimization objective function into a linear least squares model with similarity transformation constraints for solution, and generating a stitched thermal infrared panoramic image, specifically involves: The similarity transformation matrix parameters of each thermal infrared image to be stitched are arranged into a vector in order, and all matching pairs and flight path constraints are linearized into sparse matrix form; where a matching pair is a thermal infrared image to be stitched that is matched with it, and the flight path constraint is a trajectory constraint performed through the flight path trajectory protection term. The sparse matrix form is solved by a linear least squares solver with similarity transformation structure constraints to obtain the optimal vector. The similarity transformation matrix is ​​then restored using the optimal vector, and the restored similarity transformation matrix is ​​used to generate a stitched thermal infrared panoramic image.

2. The method for stitching UAV thermal infrared images based on track keeping as described in claim 1, characterized in that, The process of generating an alignment term by using the similarity transformation matrix of the thermal infrared images to be stitched and the reprojection error of the matched feature points is as follows: Based on any feature point matched in the thermal infrared images to be stitched, the difference between the similarity transformation matrix of the thermal infrared images to be stitched and the product of their respective feature points is taken, and the absolute value is squared to generate the alignment term of the thermal infrared images to be stitched.

3. A UAV thermal infrared image stitching system based on track keeping, characterized in that, include: The acquisition module is used to acquire multiple thermal infrared images of the UAV and construct an image set; The alignment term generation module is used to obtain the feature points that match any one of the thermal infrared images to be stitched in the image set, and to obtain the similarity transformation matrix between each thermal infrared image and the reference coordinate system. The alignment term is generated by the reprojection error between the similarity transformation matrix of the thermal infrared images to be stitched and the matching feature points. The protection item generation module is used to calculate the actual distance between the center of any thermal infrared image and the center of the reference image, and convert the actual distance to obtain the pixel distance between the center of any thermal infrared image and the center of the reference image in the x and y directions, and generate the flight path protection item through the pixel distance; The stitching module is used to construct a global optimization objective function by weighting and superimposing the alignment term and the flight path protection term, and to solve the global optimization objective function by transforming it into a linear least squares model with similarity transformation constraints, thereby generating a stitched thermal infrared panoramic image. The flight path protection item is generated based on the pixel distance, specifically as follows: Obtain the similarity transformation matrix of the reference image; The similarity transformation matrix of the thermal infrared image to be stitched is subtracted from the similarity transformation matrix of the reference image. The difference is then multiplied by the coordinates of the center pixel of the thermal infrared image to be stitched on the original image. The product is then subtracted from the pixel distance, and the absolute value is squared to generate the flight path protection item. The process of transforming the global optimization objective function into a linear least squares model with similarity transformation constraints for solution, and generating a stitched thermal infrared panoramic image, specifically involves: The similarity transformation matrix parameters of each thermal infrared image to be stitched are arranged into a vector in order, and all matching pairs and flight path constraints are linearized into sparse matrix form; where a matching pair is a thermal infrared image to be stitched that is matched with it, and the flight path constraint is a trajectory constraint performed through the flight path trajectory protection term; The sparse matrix form is solved by a linear least squares solver with similarity transformation structure constraints to obtain the optimal vector. The similarity transformation matrix is ​​then restored using the optimal vector, and the restored similarity transformation matrix is ​​used to generate a stitched thermal infrared panoramic image.

4. A computer device, characterized in that, The device includes a memory and a processor, wherein the memory stores a program that, when executed by the processor, causes the processor to perform the steps of a UAV thermal infrared image stitching method based on track keeping as described in any one of claims 1 to 2.

5. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the UAV thermal infrared image stitching method based on track keeping as described in any one of claims 1 to 2.

Images