A deep learning-based unmanned aerial vehicle inspection image registration alignment method

Abstract

The application discloses a kind of unmanned aerial vehicle inspection image registration alignment methods based on deep learning, belong to image processing technical field, including video acquisition and segmented processing, semantic-spatial alignment and synchronous subsequence selection, content enhancement and alignment frame pair extraction, multi-source key point extraction and fusion, cross-video high-quality feature matching, geometric transformation estimation and registration, solve in cross visual angle and non-synchronous image sequence, high-precision frame alignment and matching technical problem, the application provides a kind of semantic-spatial joint image sequence alignment method, realize the high robustness alignment of cross-sequence image frame, make matching more tend to time adjacent frame pair, to effectively inhibit the semantic interference of cross time drift, enhance the physical rationality of matching path, significantly improve the robust alignment ability under the asymmetric sampling, speed variation and visual angle deviation in real flight process.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method for registration and alignment of UAV inspection images based on deep learning. Background Technology

[0002] With the widespread application of drone imagery in surveying, agriculture, security and disaster monitoring, high-precision matching and registration of video images from different perspectives (such as binocular cameras) or different time sequences has become a major technical challenge in the field of visual processing.

[0003] Current mainstream methods mainly rely on image registration frameworks based on local features (such as SIFT and ORB), or introduce deep learning methods such as SuperPoint and LightGlue to achieve more robust feature matching.

[0004] However, existing technologies mostly focus on the spatial consistency between single image pairs. When faced with image sequence matching tasks with time delays, viewpoint shifts, or content differences, they lack effective modeling of inter-frame temporal information, leading to increased matching errors or alignment failures.

[0005] Currently, traditional technologies neglect the decisive impact of "temporal consistency constraints," such as inter-frame acquisition time and motion trends, on registration accuracy. Existing solutions struggle to effectively distinguish between frame pairs that are "semantically similar but temporally misaligned" and those that are "temporally consistent but semantically weak," leading to registration mismatches and reduced overall system stability. Summary of the Invention

[0006] The purpose of this invention is to provide a deep learning-based UAV inspection image registration and alignment method, which solves the technical problem of high-precision frame alignment and matching in cross-viewpoint and asynchronous image sequences.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A deep learning-based method for image registration and alignment during UAV inspections includes the following steps:

[0009] Step 1: Simultaneously acquire video sequences and attitude information using dual cameras, and use timestamps and track points to perform frame sampling and segmentation on the video sequences to construct a preliminary set of video pairs;

[0010] Step 2: Retrieve video frame pairs from the video pair set, extract the semantic features and geometric matching errors of the video frames, construct a semantic + spatiotemporal joint scoring matrix, and use dynamic programming to perform frame-level alignment to obtain content-synchronized video frame pairs;

[0011] Step 3: Retrieve video frame pairs and enhance the aligned video pairs;

[0012] Step 4: For the enhanced video, extract key points and descriptors using the SuperPoint algorithm and SIFT algorithm respectively, integrate multi-source features through a weighted fusion function, and output a fused descriptor set;

[0013] Step 5: Construct a cross-video attention matching model using the LightGlue algorithm, formulate strategies for bidirectional consistency, adaptive confidence, and geometric consistency, and select high-quality matching point pairs;

[0014] Step 6: For high-quality matching point pairs, evaluate the matching quality and spatial distribution, select the corresponding transformation model, optimize the evaluation results using the MAGSAC++ method, and complete the geometric registration and alignment of the video.

[0015] Preferably, step 2 includes the following steps:

[0016] Step 2-1: Retrieve video frame pairs from the video pair set;

[0017] Step 2-2: Extract global semantic feature vectors for video frame pairs and perform preliminary geometric matching error processing;

[0018] When processing semantic feature vectors, semantic features are extracted by convolutional neural networks;

[0019] When performing geometric matching error processing, the geometric error is calculated from the key point matching reprojection error based on the LightGlue algorithm;

[0020] Steps 2-3: Based on semantic similarity, temporal decay weight, and geometric consistency index, construct a joint scoring matrix of semantic and spatiotemporal factors, and calculate the joint score for each pair of video frames:

[0021] ;

[0022] in, For joint scoring, Let be the semantic similarity score between the i-th frame of video A and the j-th frame of video B, where video A and video B are a pair of video frames; This represents the initial geometric consistency score between two frames. This represents the time consistency weighting function; , The weighting parameters represent the relationship between semantic and geometric scores;

[0023] ;

[0024] in, These represent the timestamps of video frame i and video frame j, respectively. This represents the time tolerance parameter; The larger the value, the closer the time;

[0025] Steps 2-4: Based on the results of the joint scoring matrix, use a dynamic programming algorithm to search for ascending paths, obtain the aligned subsequence with the highest joint score, and output the semantically and spatiotemporally aligned video frame pairs, i.e., content-synchronized video frame pairs.

[0026] Preferably, step 4 includes the following steps:

[0027] Step 4-1: For video frame pairs with synchronized content, extract key points and corresponding descriptors using the SuperPoint algorithm and SIFT algorithm respectively, denoted as SuperPoint descriptor and SIFT descriptor;

[0028] Step 4-2: Perform vector normalization on the extracted SuperPoint descriptor and SIFT descriptor respectively;

[0029] Step 4-3: Construct a weighted fusion function to fuse the normalized SuperPoint descriptor, SIFT descriptor, and keypoint spatial encoding into a feature descriptor and output a set of fused feature descriptors.

[0030] Preferably, step 5 includes the following steps:

[0031] Step 5-1: Retrieve the fusion descriptor set and use the Transformer-based LightGlue model to calculate the matching similarity matrix between key points;

[0032] Step 5-2: Based on the principle of bidirectional maximum matching, select key point pairs that are the best match for each other;

[0033] Step 5-3: Based on the statistical characteristics of the matching scores, adaptively set the confidence threshold and remove pairs that are below the confidence threshold;

[0034] Step 5-4: Construct a predicted geometric transformation model, further filter matching pairs that meet the geometric consistency requirements, and construct and output a set of quality matching point pairs.

[0035] Preferably, step 6 includes the following steps:

[0036] Step 6-1: For high-quality matching point pairs, evaluate the number of matching point pairs and the uniformity of spatial distribution, and select the transformation estimation method according to the preset threshold;

[0037] Step 6-2: Use the MAGSAC++ algorithm to eliminate false matches and determine the set of geometrically consistent interior points;

[0038] Step 6-3: Estimate the viewpoint transformation matrix on the set of inliers using a robust optimization method;

[0039] Step 6-4: Apply the estimated transformation matrix to perform geometric transformation on the video frame to complete the pixel-level registration and alignment of the video frame.

[0040] This invention presents a deep learning-based UAV inspection image registration and alignment method, which solves the technical problem of high-precision frame alignment and matching in cross-viewpoint and asynchronous image sequences. This invention provides a semantic-temporal joint image sequence alignment method. By constructing a joint scoring matrix including semantic similarity, geometric consistency score, and temporal consistency weight function, it achieves highly robust alignment of cross-sequence image frames. The temporal consistency weight function automatically adjusts the frame pair score based on the inter-frame index difference or timestamp difference, making the matching more inclined towards temporally adjacent frame pairs. This effectively suppresses semantic interference from cross-temporal drift, enhances the physical rationality of the matching path, and significantly improves the robust alignment capability under asymmetric sampling, velocity changes, and viewpoint deviations during real flight. This invention also considers the joint semantic, geometric, and temporal factors, which can accurately identify image pairs in the image sequence that are semantically similar, temporally synchronized, and geometrically consistent, significantly improving the accuracy of subsequent feature matching and image registration. Attached Figure Description

[0041] Figure 1 This is the main flowchart of the present invention;

[0042] Figure 2 This is a flowchart of step 2 of the present invention;

[0043] Figure 3 This is a flowchart of step 4 of the present invention;

[0044] Figure 4 This is a flowchart of step 5 of the present invention.

[0045] Figure 5 This is a flowchart of step 6 of the present invention. Detailed Implementation

[0046] Depend on Figures 1-5 The image registration and alignment method for UAV inspection based on deep learning is shown in this embodiment. The video data collected by the dual cameras on the UAV is transmitted to an FPGA controller. The FPGA controller sends the video data to the backend server for processing through wireless communication modules such as Wi-Fi.

[0047] The process includes the following steps:

[0048] Step 1: Simultaneously acquire video sequences and attitude information using dual cameras, and use timestamps and track points to perform frame sampling and segmentation on the video sequences to construct a preliminary set of video pairs;

[0049] The video sequence captured by the dual cameras consists of two visible light camera video streams, one on the left and one on the right, represented as follows: Attitude information refers to the attitude and position data output by the IMU / GPS / SLAM devices on the UAV, represented as... ;

[0050] In this embodiment, the two video streams are timed using a unified global clock t, and each frame in the video sequence is accompanied by a timestamp, denoted as t. , where A and B represent video data from two cameras respectively.

[0051] When performing timestamp alignment, the main focus is on aligning the time base of frames from two video data streams to eliminate asynchronous jitter at the acquisition end. Specifically, for each frame index k, the closest peer frame index is found, calculated using the following formula:

[0052] ;

[0053] in, These represent the timestamps of the frames captured by the two cameras; j is the peer frame index synchronized with k.

[0054] When performing frame sampling on a video sequence, either timed sampling or keyframe detection methods are used. If timed sampling is used, frames are extracted at fixed time intervals, such as one frame per second in this embodiment. If keyframe detection is used, the frame extraction operation is triggered only when there are drastic changes in posture or visual appearance. The specific triggering conditions are as follows:

[0055] or ;

[0056] in, The quaternion represents the body's attitude. For image matrix, Denotes the Frobenius norm; All are preset thresholds.

[0057] When segmenting the route, the video is divided into several independent segments according to the preset inspection points or task stages to facilitate subsequent batch processing. In this embodiment, GPS track points are used to mark the segment boundaries.

[0058] Video pairs are represented as , Indicates each paragraph, The number of video frame pairs extracted and successfully matched in segment s.

[0059] Step 2: Retrieve video frame pairs from the video pair set, extract the semantic features and geometric matching errors of the video frames, construct a semantic + spatiotemporal joint scoring matrix, and use dynamic programming to perform frame-level alignment to obtain content-synchronized video frame pairs;

[0060] Step 2 includes the following steps:

[0061] Step 2-1: Retrieve video frame pairs from the video pair set;

[0062] Step 2-2: Extract global semantic feature vectors for video frame pairs and perform preliminary geometric matching error processing;

[0063] When processing semantic feature vectors, semantic features are extracted by convolutional neural networks;

[0064] When performing geometric matching error processing, the geometric error is calculated from the key point matching reprojection error based on the LightGlue algorithm;

[0065] For each pair of original image frames Semantic features are extracted and preliminary geometric consistency is estimated, respectively.

[0066] In this embodiment, semantic feature extraction specifically involves using a lightweight convolutional neural network (such as MobileNet or EfficientNet) to extract frame-level global semantic feature vectors.

[0067] Semantic similarity calculation specifically involves using cosine similarity to measure the similarity between semantic features of two frames, resulting in a semantic similarity matrix.

[0068] Specifically, geometric error estimation involves using the LightGlue algorithm to perform preliminary feature matching on two frames of images and calculating the keypoint reprojection error (such as root mean square error RMSE) as a reflection of the geometric consistency score.

[0069] Steps 2-3: Based on semantic similarity, temporal decay weight, and geometric consistency index, construct a joint scoring matrix of semantic and spatiotemporal factors, and calculate the joint score for each pair of video frames:

[0070] ;

[0071] in, For joint scoring, Let be the semantic similarity score between the i-th frame of video A and the j-th frame of video B, where video A and video B are a pair of video frames; This represents the initial geometric consistency score between two frames. This represents the time consistency weighting function; , The weighting parameters represent the relationship between semantic and geometric scores;

[0072] ;

[0073] in, These represent the timestamps of video frame i and video frame j, respectively. This represents the time tolerance parameter; The larger the value, the closer the time;

[0074] In this embodiment, the time consistency weight function It can effectively solve the problem of potentially mispairing visually “similar” frames by relying solely on image semantic or geometric matching, even if they should not appear together in time. It solves the frame-level registration problem of multi-source UAV images under complex conditions such as semantic similarity, geometric offset, and temporal asynchrony.

[0075] Steps 2-4: Based on the results of the joint scoring matrix, use a dynamic programming algorithm to search for ascending paths, obtain the aligned subsequence with the highest joint score, and output the semantically and spatiotemporally aligned video frame pairs, i.e., content-synchronized video frame pairs.

[0076] In this embodiment, the dynamic programming algorithm can employ either Weighted Dynamic Time Warping (DTW) or Soft-DTW. First, the dynamic programming algorithm searches for an ascending path from the top left to the bottom right in an M×N scoring matrix, maximizing the sum of the joint scores along that path. The formula for the dynamic programming algorithm is as follows:

[0077] ;

[0078] Where Path is the path.

[0079] Then, a sliding window operation is applied along the path to calculate the local average score, and the highest-scoring continuous subsequence is selected to output the final set of aligned frame pairs.

[0080] The formula for calculating the local average score is as follows:

[0081] ;

[0082] Where L represents the length of the selected subsequence.

[0083] The set of aligned frame pairs is represented as , This represents the final aligned frame pair.

[0084] Step 3: Retrieve video frame pairs and enhance the aligned video pairs;

[0085] In this embodiment, for Frame pair enhancement processing can include brightness adjustment, edge refinement, and texture enhancement operations, such as Retinex-based brightness normalization, U-Net / ResUNet models for detail restoration, and GuidedFilter or HistogramMatching for color and style consistency.

[0086] The final processed set of image frame pairs is represented as follows: .

[0087] Step 4: For the enhanced video, extract key points and descriptors using the SuperPoint algorithm and SIFT algorithm respectively, integrate multi-source features through a weighted fusion function, and output a fused descriptor set;

[0088] Step 4 includes the following steps:

[0089] Step 4-1: For video frame pairs with synchronized content, extract key points and corresponding descriptors using the SuperPoint algorithm and SIFT algorithm respectively, denoted as SuperPoint descriptor and SIFT descriptor;

[0090] against Video frame pairs The following two algorithms are used to extract key points and their corresponding descriptors:

[0091] Algorithm 1: Use the SuperPoint algorithm to extract dense keypoints and their descriptors driven by deep learning, denoted as the SuperPoint feature descriptor set.

[0092] Algorithm 2: Use the SIFT algorithm to extract scale-invariant feature points and their traditional descriptors, denoted as the SIFT feature descriptor set.

[0093] Step 4-2: Perform vector normalization on the extracted SuperPoint descriptor and SIFT descriptor respectively;

[0094] In this embodiment, to eliminate the inconsistency in scale and norm of different descriptors, the SuperPoint feature descriptor set and the SIFT feature descriptor set need to be vector normalized to make them distributed in a unified space. Specifically, the L2 normalization method or the unit spherical projection method can be used.

[0095] Step 4-3: Construct a weighted fusion function to fuse the normalized SuperPoint descriptor, SIFT descriptor, and keypoint spatial encoding into a feature descriptor and output a set of fused feature descriptors.

[0096] In this embodiment, the weighted fusion function The specific formula is as follows:

[0097] ;

[0098] in, All are weights. Spatial encoding of the two-dimensional coordinates of key points. This is the set of normalized SuperPoint feature descriptors. This is the set of normalized SIFT feature descriptors.

[0099] In this embodiment, the fusion parameters Adjustments should be made based on the actual scenario. The specific method is as follows:

[0100] If the image texture is weak, increase the SuperPoint weight;

[0101] If the geometric distortion is strong, increase the SIFT weights;

[0102] If the viewpoint space structure needs to be considered during matching, then spatial coding weights are introduced.

[0103] The final set of fused feature descriptors is represented as follows: ,in Let represent the fusion feature of video frame A in the k-th pair of video frames. This represents the fusion feature of video frame B in the k-th pair of video frames.

[0104] Step 5: Construct a cross-video attention matching model using the LightGlue algorithm, formulate strategies for bidirectional consistency, adaptive confidence, and geometric consistency, and select high-quality matching point pairs;

[0105] Step 5 includes the following steps:

[0106] Step 5-1: Retrieve the fusion descriptor set and use the Transformer-based LightGlue model to calculate the matching similarity matrix between key points;

[0107] Retrieve Then, it is fed into the Transformer-based LightGlue matching model.

[0108] The LightGlue matching model can calculate the similarity matrix between keypoint descriptors through a cross-image attention mechanism and output potential matching pairs and their matching confidence scores. The output results in the family are represented as follows:

[0109] ;

[0110] in, This represents the i-th key point in image A; The j-th key point of image B; , which represents the confidence score of the corresponding match, is output by the LightGlue matching model.

[0111] Step 5-2: Based on the principle of bidirectional maximum matching, select key point pairs that are the best match for each other; thereby effectively removing duplicate pairings and low-confidence interference items.

[0112] In this embodiment, to ensure the uniqueness and stability of the matching, the initial matching set is... Implement a bidirectional maximum consistency matching strategy, retaining " yes The best match, and, yes The matching results of "best match" are denoted as the consistency matching set. .

[0113] Step 5-3: Based on the statistical characteristics of the matching scores, adaptively set the confidence threshold and remove pairs that are below the confidence threshold;

[0114] In this embodiment, the matching score Perform distribution statistics, set an adaptive threshold Ts, and remove low-confidence pairs, with the following specific conditions:

[0115] ;

[0116] In this embodiment, Ts can be set as a certain multiple of the mean plus standard deviation of the matching scores.

[0117] Step 5-4: Construct a predicted geometric transformation model, further filter matching pairs that meet the geometric consistency requirements, and construct and output a set of quality matching point pairs.

[0118] In this embodiment, in the adaptively filtered set of point pairs Next, geometric consistency verification is performed to further eliminate false matches caused by local noise or background repetition regions. This embodiment summarizes one of the following strategies:

[0119] Preliminary estimation of the affine / homography transformation model;

[0120] Calculate the reprojection error for each pair of matching points;

[0121] Matching pairs whose reprojection error exceeds a certain threshold are discarded.

[0122] The final set that remains is denoted as the final high-quality matching point set:

[0123] .

[0124] Step 6: For high-quality matching point pairs, evaluate the matching quality and spatial distribution, select the corresponding transformation model, optimize the evaluation results using the MAGSAC++ method, and complete the geometric registration and alignment of the video.

[0125] When performing step 6, a high-quality set of matching point pairs is obtained. Then, global transform estimation and optimization are performed to complete pixel-level alignment between the two frames, and the final geometric registration result is output. The specific steps include the following:

[0126] Step 6-1: For high-quality matching point pairs, evaluate the number of matching point pairs and the uniformity of spatial distribution, and select the transformation estimation method according to the preset threshold;

[0127] For matching point set Conduct quality assessments, including:

[0128] Quantitative assessment: Number of matching point pairs Does it meet the estimation requirements, i.e., is it greater than 10?

[0129] Uniformity assessment: Divide the image into N×N regions and count the number of points in each region: If the points are concentrated in a few regions (at least 75% of the regions contain ≥1 matching point), it indicates local redundancy and difficulty in fitting stable transformations.

[0130] Select the appropriate transformation model type based on the evaluation results:

[0131] If the viewpoint is planar, choose homography transformation; if the change is more local, use only affine transformation.

[0132] Step 6-2: Use the MAGSAC++ algorithm to eliminate false matches and determine the set of geometrically consistent interior points;

[0133] Based on the results of the transformation model selected in step 6-1, robust parameter estimation is performed using the MAGSAC++ algorithm, specifically including:

[0134] Random sampling is performed on the matching point pairs;

[0135] Iteratively fit the transformation matrix and calculate the reprojection error;

[0136] The cost function is automatically optimized based on statistical inference of the error distribution.

[0137] Output the set of points within the largest uniform set and the final transformation matrix H or A.

[0138] The MAGSAC++ algorithm used in this embodiment can adaptively adjust the fault tolerance parameters, effectively handle data containing a large amount of noise or incorrect matching, and has better estimation accuracy than traditional techniques.

[0139] Step 6-3: Estimate the viewpoint transformation matrix on the set of inliers using a robust optimization method;

[0140] Based on the set of interior points, the transformation matrix is ​​refined using the least squares method or a nonlinear optimization method, outputting the final viewpoint transformation matrix, as follows:

[0141] , ;

[0142] in, These are the homogeneous coordinates of the target image and the original image, respectively.

[0143] For affine models: , .

[0144] Least squares or nonlinear optimization methods are existing technologies, so they will not be described in detail.

[0145] Step 6-4: Apply the estimated transformation matrix to perform geometric transformation on the video frame to complete the pixel-level registration and alignment of the video frame.

[0146] In this embodiment, a transformation matrix H or A is specifically applied to the original video frame image. Perform geometric transformations to generate and Aligned image :

[0147] ;

[0148] in, This represents affine / homography geometric mapping and interpolation operations.

[0149] This invention presents a deep learning-based UAV inspection image registration and alignment method, which solves the technical problem of high-precision frame alignment and matching in cross-viewpoint and asynchronous image sequences. This invention provides a semantic-temporal joint image sequence alignment method. By constructing a joint scoring matrix including semantic similarity, geometric consistency score, and temporal consistency weight function, it achieves highly robust alignment of cross-sequence image frames. The temporal consistency weight function automatically adjusts the frame pair score based on the inter-frame index difference or timestamp difference, making the matching more inclined towards temporally adjacent frame pairs. This effectively suppresses semantic interference from cross-temporal drift, enhances the physical rationality of the matching path, and significantly improves the robust alignment capability under asymmetric sampling, velocity changes, and viewpoint deviations during real flight. This invention also considers the joint semantic, geometric, and temporal factors, which can accurately identify image pairs in the image sequence that are semantically similar, temporally synchronized, and geometrically consistent, significantly improving the accuracy of subsequent feature matching and image registration.

Claims

1. A deep learning-based image registration and alignment method for UAV inspection, characterized in that: Includes the following steps: Step 1: Simultaneously acquire video sequences and attitude information using dual cameras, and use timestamps and track points to perform frame sampling and segmentation on the video sequences to construct a preliminary set of video pairs; Step 2: Retrieve video frame pairs from the video pair set, extract the semantic features and geometric matching errors of the video frames, construct a semantic + spatiotemporal joint scoring matrix, and use dynamic programming to perform frame-level alignment to obtain content-synchronized video frame pairs; Specifically, the steps include the following: Step 2-1: Retrieve video frame pairs from the video pair set; Step 2-2: Extract global semantic feature vectors for video frame pairs and perform preliminary geometric matching error processing; When processing semantic feature vectors, semantic features are extracted by convolutional neural networks; When performing geometric matching error processing, the geometric error is calculated from the key point matching reprojection error based on the LightGlue algorithm; Steps 2-3: Based on semantic similarity, temporal decay weight, and geometric consistency index, construct a joint scoring matrix of semantic and spatiotemporal factors, and calculate the joint score for each pair of video frames: ； in, For joint scoring, Let be the semantic similarity score between the i-th frame of video A and the j-th frame of video B, where video A and video B are a pair of video frames; This represents the initial geometric consistency score between two frames. This represents the time consistency weighting function; , The weighting parameters represent the relationship between semantic and geometric scores; ； in, These represent the timestamps of video frame i and video frame j, respectively. This represents the time tolerance parameter; The larger the value, the closer the time; Steps 2-4: Based on the results of the joint scoring matrix, use dynamic programming algorithm to search for ascending paths, obtain the aligned subsequence with the highest joint score, and output the semantically and spatiotemporally aligned video frame pairs, i.e., content-synchronized video frame pairs. Step 3: Retrieve video frame pairs and enhance the aligned video pairs; Step 4: For the enhanced video, extract key points and descriptors using the SuperPoint algorithm and SIFT algorithm respectively, integrate multi-source features through a weighted fusion function, and output a fused descriptor set; Step 5: Construct a cross-video attention matching model using the LightGlue algorithm, formulate strategies for bidirectional consistency, adaptive confidence, and geometric consistency, and select high-quality matching point pairs; Step 6: For high-quality matching point pairs, evaluate the matching quality and spatial distribution, select the corresponding transformation model, optimize the evaluation results using the MAGSAC++ method, and complete the geometric registration and alignment of the video.

2. The image registration and alignment method for UAV inspection based on deep learning as described in claim 1, characterized in that: Step 4 includes the following steps: Step 4-1: For video frame pairs with synchronized content, extract key points and corresponding descriptors using the SuperPoint algorithm and SIFT algorithm respectively, denoted as SuperPoint descriptor and SIFT descriptor; Step 4-2: Perform vector normalization on the extracted SuperPoint descriptor and SIFT descriptor respectively; Step 4-3: Construct a weighted fusion function to fuse the normalized SuperPoint descriptor, SIFT descriptor, and keypoint spatial encoding into a feature descriptor and output a set of fused feature descriptors.

3. The image registration and alignment method for UAV inspection based on deep learning as described in claim 1, characterized in that: Step 5 includes the following steps: Step 5-1: Retrieve the fusion descriptor set and use the Transformer-based LightGlue model to calculate the matching similarity matrix between key points; Step 5-2: Based on the principle of bidirectional maximum matching, select key point pairs that are the best match for each other; Step 5-3: Based on the statistical characteristics of the matching scores, adaptively set the confidence threshold and remove pairs that are below the confidence threshold; Step 5-4: Construct a predicted geometric transformation model, further filter matching pairs that meet the geometric consistency requirements, and construct and output a set of quality matching point pairs.

4. The image registration and alignment method for UAV inspection based on deep learning as described in claim 1, characterized in that: Step 6 includes the following steps: Step 6-1: For high-quality matching point pairs, evaluate the number of matching point pairs and the uniformity of spatial distribution, and select the transformation estimation method according to the preset threshold; Step 6-2: Use the MAGSAC++ algorithm to eliminate false matches and determine the set of geometrically consistent interior points; Step 6-3: Estimate the viewpoint transformation matrix on the set of interior points using a robust optimization method; Step 6-4: Apply the estimated transformation matrix to perform geometric transformation on the video frame to complete the pixel-level registration and alignment of the video frame.

Images