A UAV image stitching method based on vectorized feature selection

Abstract

This invention discloses a UAV image stitching method based on vectorized feature selection, comprising: extracting feature points from each image to obtain feature point matching pairs; constructing matching vectorized feature pairs from the matched feature point pairs; randomly selecting vectorized feature pairs and calculating the parameters of a first similarity transformation matrix; counting the number of vectorized feature pairs that conform to the first similarity transformation matrix, and iterating to obtain the set with the largest number of vectorized feature pairs; recovering feature point pairs from the vectorized feature pairs in the set with the largest number of vectorized feature pairs; randomly selecting feature point pairs and calculating the parameters of a second similarity transformation matrix; counting the number of feature point pairs that conform to the second similarity transformation matrix to obtain the final transformation matrix; obtaining the relative positions of the images based on the calculated transformation matrix; and obtaining the final stitched image through an image fusion step. This invention can effectively improve the accuracy of feature selection and obtain better stitching results, meeting the needs of practical industrial applications.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for stitching UAV images based on vectorized feature selection. Background Technology

[0002] Unmanned aerial vehicle (UAV) remote sensing technology, due to its advantages such as strong weather adaptability, low acquisition cost, flexible takeoff and landing, small size, low risk, simple operation, low flight altitude, short working cycle, high timeliness, and low maintenance cost, can quickly acquire high-scale, high-definition images in the low and medium airspace, effectively overcoming the limitations of traditional aerial photography and satellite remote sensing, and is gradually becoming a remote sensing technology with broad application potential. Despite the many advantages of UAV aerial photography, its image acquisition is still limited by factors such as flight altitude, onboard camera angle, and focal length, resulting in a single UAV image only showing local information of the observed area. To comprehensively cover the research area and achieve global monitoring, the method of combining multiple single-view local sub-images into a single wide-view, high-resolution panoramic image using image stitching technology is widely used in the field of UAV remote sensing. However, due to their light weight and small size, UAVs are more sensitive to changes in airflow, leading to increased instability in image acquisition, resulting in problems such as inconsistent image scale, large angular deviations, and degraded image quality. Using these adversely affected images for stitching may lead to varying degrees of error in the stitching algorithm, such as inaccurate feature point extraction, decreased feature matching accuracy, omissions, and incorrect matches. Traditional feature selection methods typically calculate transformation error based on homography transformation, which is established based on the positional difference of each matched feature point pair. This positional difference is determined by eight parameters. Therefore, homography-based feature selection methods require calculating all eight parameters simultaneously, which is computationally time-consuming, and due to the potentially large errors in the matched point pairs, accuracy is not guaranteed.

[0003] Therefore, designing a UAV image stitching method that improves feature selection accuracy and stitching quality is of urgent practical need and important research value. Summary of the Invention

[0004] In order to overcome the above-mentioned shortcomings and deficiencies of the prior art, the purpose of this invention is to provide a method for UAV image stitching based on vectorized feature selection.

[0005] This invention can improve the accuracy of feature pair screening and obtain better stitching results while ensuring image accuracy, thus meeting the needs of practical industrial applications.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A method for stitching UAV images based on vectorized feature selection, comprising:

[0008] Read multiple drone images to be stitched together and extract feature points from each image;

[0009] Coarse matching is performed on the feature points in each image to obtain feature point matching pairs;

[0010] Construct the matched feature point pairs into matched vectorized feature pairs;

[0011] Randomly select vectorized feature pairs from the matched vectorized feature pairs, calculate the parameters of the first similarity transformation matrix; count the number of vectorized feature pairs in the remaining vectorized feature pairs that conform to the first similarity transformation matrix;

[0012] Repeat the previous step until the set number of iterations is reached, and obtain the set with the largest number of vectorized feature pairs that conform to the first similarity transformation matrix;

[0013] The vectorized feature pairs in the set with the most vectorized feature pairs are recovered to obtain feature point pairs. Feature point pairs are randomly selected, the parameters of the second similarity transformation matrix are calculated, and the number of feature point pairs that conform to the second similarity transformation matrix is ​​counted among the remaining feature point pairs.

[0014] Repeat the previous step until the set number of iterations is reached. Then, obtain the set with the most feature point pairs that match the second similarity transformation matrix, and output the second similarity transformation matrix corresponding to this set as the final transformation matrix.

[0015] Based on the calculated final transformation matrix, the relative positions of the images are obtained, and the final stitched image is obtained through the image fusion step.

[0016] Furthermore, the step of constructing the matched feature point pairs into matched vectorized feature pairs specifically involves:

[0017] First, the matching feature points are sorted according to their distance from the origin. Then, adjacent feature points within the image are connected to form vectorized features within the image. Next, the corresponding vectorized features in another matching image are found in sequence to form a set of m matching vectorized feature pairs.

[0018] Furthermore, vectorized feature pairs are randomly selected from the matched vectorized feature pairs, and the parameters of the first similarity transformation matrix are calculated, specifically as follows:

[0019] Randomly select j pairs of vectorized features from the set of m pairs of matching vectorized features, and use these j pairs of vectorized features to obtain a first similarity transformation matrix S containing only rotation and scaling parameters;

[0020] Vectorize the matched features to v i With v jAfter transforming to a homogeneous coordinate system, let the relationship between the two satisfy the first similarity transformation matrix S, that is:

[0021]

[0022] Where θ is the rotation parameter and s is the scale parameter.

[0023] Furthermore, the number of vectorized feature pairs that conform to the first similarity transformation matrix among the remaining vectorized feature pairs is counted, specifically as follows:

[0024] A pair of matched vectorized feature pairs v i and v j In the middle, one of the vectors v i The new coordinates v after transformation are obtained through the first similarity transformation matrix S. i ′ Calculate v i ′ With v j The difference between d v If d v Less than the threshold ε v If the pair of matching vectors satisfies S, then the feature points corresponding to this pair of matching vectors are stored in set A. f If it is greater than the threshold v v If a pair of matching vectors does not satisfy S, the feature points corresponding to that pair are considered exterior points and removed. Finally, set A is calculated. f The number of interior points f.

[0025] Furthermore, by randomly selecting feature point pairs, the parameters of the second similarity transformation matrix are calculated, specifically as follows:

[0026] Randomly select k pairs of feature points from the vectorized feature pair set A, and use these k pairs of feature points to obtain a second similarity transformation matrix H with 6 parameters;

[0027] Match the feature point pairs p i With p j After transforming to a homogeneous coordinate system, let the relationship between the two satisfy the second similarity transformation matrix H, that is:

[0028]

[0029] Where s represents the scale parameter, θ represents the rotation parameter, and tx and ty are the displacements along the X and Y axes, respectively.

[0030] Furthermore, the number of feature point pairs that conform to the second similarity transformation matrix among the remaining feature point pairs is specifically as follows:

[0031] A pair of matched feature points (x i ,y i ) and (xj ,y j In ), one of the points (x) i ,y i The transformation is performed to obtain the new coordinate point (x). i ′ ,y i ′ ), calculate (x i ′ ,y i ′ ) and (x j ,y j The difference between ) d p If d p Less than the threshold ε p If a pair of matching points satisfies H, it is considered an interior point, and this pair of matching points is stored in the interior point set B. g If H is greater than the threshold ε; p If the pair of matching points does not satisfy H, then it is considered that the pair does not satisfy H, and the pair of matching points is regarded as an exterior point and the exterior point is removed. The interior point set B is then calculated. g The number of interior points n.

[0032] Furthermore, the threshold is 0.1.

[0033] Furthermore, k represents at least three pairs.

[0034] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the UAV image stitching method.

[0035] A computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the UAV image stitching method.

[0036] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0037] This invention proposes a novel feature selection algorithm based on vector characteristics. It replaces homography transformation with similarity transformation and divides the feature selection stage into two phases based on vectorized features to separate scale, rotation, and translation parameters. This reduces computational complexity and further improves the accuracy of feature matching, resulting in well-matched feature pairs after selection. Based on the interior point pairs, optimal transformation matrix parameters are calculated, leading to better stitching results. Compared to traditional image stitching methods, the algorithm significantly improves the selection and stitching performance using vectorized features. Attached Figure Description

[0038] Figure 1This is a schematic diagram of the UAV image stitching method based on vectorized feature selection according to the present invention;

[0039] Figure 2(a) shows the splicing effect of the splicing algorithm based on the traditional feature selection algorithm; Figure 2(b) shows the splicing effect of the splicing method described in this embodiment.

[0040] Figure 3 This is a flowchart of the process of the present invention. Detailed Implementation

[0041] The present invention will be further described in detail below with reference to the embodiments, but the implementation of the present invention is not limited thereto.

[0042] Example

[0043] like Figures 1-3 As shown, a UAV image stitching method based on vectorized feature selection includes:

[0044] S1 reads multiple drone images to be stitched together and extracts feature points from each image.

[0045] S2 performs coarse matching on feature points in each image to obtain feature matching pairs;

[0046] S3 constructs matched vectorized feature pairs from the matched feature point pairs. The specific steps are as follows:

[0047] Let V be the set of coarsely matched feature pairs between images I1 and I2. V contains a total of m pairs of matching points, among which there may be mismatched points.

[0048] The transformation process of UAV images can essentially be divided into two consecutive stages: (1) the re-plane stage, where the camera imaging plane of each sub-image needs to be adjusted to the panoramic plane, that is, the sub-image is transformed to the panoramic plane through a transformation model; (2) the translation stage, where the relative positions of the sub-images that are already on the same plane only need to be adjusted appropriately to make the images effectively overlap and align. By processing the re-plane stage and the translation stage separately, that is, by performing rotation and scaling in the re-plane stage first, and then performing translation in the translation stage, this method will have a more significant computational speed advantage than the traditional transformation method.

[0049] Considering that the similarity of vectors is characterized only by their direction and length and is independent of translation, this method proposes the concept of vectorized features. This allows us to first sort the matching feature points according to their distance from the origin (the top left corner of the image), then connect adjacent feature points within the image to form vectorized features within the image, and then find the corresponding vectorized features in another matching image in sequence. This forms a set of m matching vectorized feature pairs, denoted as set Q.

[0050] In two matched images i and j, two adjacent pairs of interior points are [x...]. i y i ] T and [x j y j ] T 、[x i+1 y i+1 ] T and [x j+1 y j+1 ] T The process of constructing the vectorized features of the match from these two elements is as follows:

[0051] v i =[x i -x i+1 y i -y i+1 ] T

[0052] v j =[x j -x j+1 y j -y j+1 ] T

[0053] Among them, v i With v j For a pair of matched vectorized features.

[0054] In this way, the constructed vectorized features are relatively simple and have high computational efficiency. After constructing the vectorized features, feature selection can naturally be divided into two stages, thereby gaining advantages in computational efficiency and accuracy.

[0055] S4 randomly selects vectorized feature pairs from the matched vectorized feature pairs and calculates the parameters of the first similarity transformation matrix.

[0056] The specific steps are as follows: When a drone performs a shooting mission in the air, its position, orientation, and attitude frequently change. This causes changes in related quantities such as altitude, displacement, pitch angle, roll angle, and yaw angle. Therefore, drone imaging is usually related to the camera's displacement, scaling, and rotation angles. These quantities correspond to the scale, translation, and rotation parameters of the drone image, which happen to be the constituent parameters of the similarity transformation. Therefore, the feature selection algorithm is based on the similarity transformation and directly uses the drone image to estimate the scale, rotation, and translation parameters.

[0057] From the set Q containing m pairs of matched vectorized features, randomly select j (j≥3) pairs of vectorized features, and use these j pairs of vectorized features to obtain a corresponding first similarity transformation matrix S containing only rotation and scaling parameters (i.e., 4 parameters).

[0058] Vectorize the matched features to v i With v j After transforming to a homogeneous coordinate system, let the relationship between the two satisfy the first similarity transformation matrix S, that is:

[0059]

[0060] The first similarity transformation matrix S can be calculated by inputting the coordinates of the selected vectorized feature pairs. Note that the first similarity transformation matrix at this point does not contain the translation parameter t. x With t y θ is the rotation parameter, and s is the scale parameter.

[0061] S5 counts the number of vectorized feature pairs that match the first similarity transformation matrix obtained in the previous step among the remaining vectorized feature pairs. Specifically, it uses the similarity transformation matrix S to transform the remaining (mj) vectorized features to find the correct matching pairs. The specific steps are as follows:

[0062] A pair of matched vectorized feature pairs v i and v j In the middle, one of the vectors v i The new coordinates v after transformation are obtained through the similarity transformation matrix S. i ′ Calculate v i ′ With v j The difference between d v If d v Less than the threshold ε v If the pair of matching vectors satisfies S, then the feature points corresponding to this pair of matching vectors are stored in set A. f If it is greater than the threshold ε v If a pair of matching vectors does not satisfy S, then the feature points corresponding to that pair are considered exterior points and are removed. Calculate set A. f The number of interior points f.

[0063] It should be noted that this step essentially involves comparing the number of interior points obtained from multiple iterations of the first similarity transformation matrix with different parameters. The matrix with the most interior points is taken as the parameter result of the final first similarity transformation matrix. At the same time, the feature pairs corresponding to this matrix are the correct feature pairs initially screened out, while the other feature pairs are determined to be invalid. This completes the first screening.

[0064] S6 repeats the first two steps until the set number of iterations is reached, thereby selecting the set of vectorized feature pairs with the most interior points. The specific steps are as follows: repeat the first two steps within the set number of iterations t, and obtain the set A with the largest f value by comparison, which is the set with the most vectorized feature pairs that conform to the first similarity transformation matrix.

[0065] S7 recovers feature point pairs from the vectorized feature pair set in the previous step, randomly selects feature point pairs from them, and calculates the parameters of the second similarity transformation matrix. The specific steps are as follows: randomly select k (k≥3) pairs of feature points from set A, and use these k pairs of feature points to obtain a corresponding second similarity transformation matrix H with 6 parameters.

[0066] Match the feature point pairs p i With p j After transforming to a homogeneous coordinate system, let the relationship between the two satisfy the similarity transformation matrix H, that is:

[0067]

[0068] The similarity transformation matrix H can be obtained by inputting the coordinates of the selected vectorized feature pairs. S represents the first similarity transformation matrix, θ represents the rotation parameter, and tx and ty are the displacements along the X and Y axes, respectively.

[0069] S8 counts the number of feature point pairs that conform to the second similarity transformation matrix among the remaining feature point pairs. The specific steps are: transform the remaining (qk) feature points using the similarity transformation matrix H to find the correct matching pairs. Specifically, for a matching feature point pair (x... i ,y i ) and (x j ,y j In ), one of the points (x) i ,y i The transformation is performed to obtain the new coordinate point (x). i ′ ,y i ′ ), calculate (x i ′ ,y i ′ ) and (x j ,y j The difference between ) d p If d p Less than the threshold ε p If a pair of matching points satisfies H, it is considered an interior point, and this pair of matching points is stored in the interior point set B. g If H is greater than the threshold ε; p If the pair of matching points does not satisfy H, then it is considered not to satisfy H, and this pair of matching points is considered an outside point and is discarded. Calculate the interior point set B.g The number of interior points n.

[0070] S9 repeats the first two steps until the set number of iterations is reached, thereby selecting the set of feature point pairs with the most interior points, and outputting the second similarity transformation matrix corresponding to the set as the final transformation matrix. The specific steps are: after comparing all interior point sets B g Given the corresponding number of interior points n, obtain the set with the largest n value, and use it as the final interior point set B. Then, obtain the similarity transformation matrix H corresponding to B. n H n This is the optimal transformation model between image I1 and image I2.

[0071] S10 obtains the relative position of the image based on the calculated image transformation matrix, and then obtains the final stitched image through the image fusion step.

[0072] The image fusion step uses the direct averaging method, which extracts the gray values ​​of pixels in the overlapping areas of each image, adds the gray values ​​of each image at that point and divides them equally, and uses the resulting average gray value as the new pixel gray value of that point after fusion, thereby completing the fusion.

[0073] In this embodiment, the threshold is generally set to the range of 0.05 to 0.25, with a preferred value of 0.1 for good filtering effect.

[0074] This invention proposes a UAV image stitching method based on vectorized feature selection. Considering that traditional feature selection algorithms still suffer from many mismatched pairs, a new feature selection algorithm is constructed based on the characteristics of vectors. The homography transformation is replaced with a similarity transformation, and the feature selection stage is divided into two phases based on vectorized features to separate scale, rotation, and translation parameters. This reduces computational complexity and further improves the accuracy of feature matching, resulting in finely matched feature interior point pairs. Based on these interior point pairs, better transformation matrix parameters are calculated, leading to improved stitching results.

[0075] Figure 2(a) shows the stitching effect of the stitching algorithm based on the traditional feature selection algorithm; Figure 2(b) shows the stitching effect of the stitching method described in this embodiment. The accuracy of feature purification using the stitching method based on the traditional feature selection algorithm is not high, and the stitching quality is poor, which cannot meet the needs of actual industrial applications. However, the stitching effect obtained by the vectorized feature selection method proposed in this embodiment is good and the accuracy is high, indicating that this embodiment is more in line with the actual application needs than the existing algorithms.

[0076] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for stitching UAV images based on vectorized feature selection, characterized in that, include: Read multiple drone images to be stitched together and extract feature points from each image; Coarse matching is performed on the feature points in each image to obtain matched feature point pairs; To construct a set of matching vectorized feature pairs, the matching feature points are first sorted according to their distance from the origin. Then, adjacent feature points within the image are connected to form vectorized features within the image. Finally, the corresponding vectorized features in another matching image are found in order to form a set of m matching vectorized feature pairs. Randomly select vectorized feature pairs from the matched vectorized feature pairs, calculate the parameters of the first similarity transformation matrix; count the number of vectorized feature pairs in the remaining vectorized feature pairs that conform to the first similarity transformation matrix; Specifically: Randomly select j pairs of vectorized features from the set of m pairs of matching vectorized features, and use these j pairs of vectorized features to obtain a first similarity transformation matrix S containing only rotation and scaling parameters; Vectorize the matched features to v i With v j After transforming to a homogeneous coordinate system, let the relationship between the two satisfy the first similarity transformation matrix S, that is: Where θ is the rotation parameter and s is the scale parameter; Repeat the previous step until the set number of iterations is reached, and obtain the set with the largest number of vectorized feature pairs that conform to the first similarity transformation matrix; The vectorized feature pairs in the set with the most vectorized feature pairs are recovered to obtain feature point pairs. Feature point pairs are randomly selected, the parameters of the second similarity transformation matrix are calculated, and the number of feature point pairs that conform to the second similarity transformation matrix is ​​counted among the remaining feature point pairs. Specifically: Randomly select k pairs of feature points from the vectorized feature pair set A, and use these k pairs of feature points to obtain a second similarity transformation matrix H with 6 parameters; Match the feature point pairs p i With p j After transforming to a homogeneous coordinate system, let the relationship between the two satisfy the second similarity transformation matrix H, that is: Where s represents the scale parameter, θ represents the rotation parameter, and tx and ty are the displacements along the X and Y axes, respectively. Repeat the previous step until the set number of iterations is reached, and obtain the set with the most feature point pairs that conform to the second similarity transformation matrix. At the same time, output the second similarity transformation matrix corresponding to the set as the final transformation matrix. Based on the calculated final transformation matrix, obtain the relative position of the image, and obtain the final stitched image through the image fusion step.

2. The UAV image stitching method according to claim 1, characterized in that, The process of constructing matched feature point pairs into matched vectorized feature pairs specifically involves: First, the matching feature points are sorted according to their distance from the origin. Then, adjacent feature points within the image are connected to form vectorized features within the image. Next, the corresponding vectorized features in another matching image are found in sequence to form a set of m matching vectorized feature pairs.

3. The UAV image stitching method according to claim 1, characterized in that, Count the number of vectorized feature pairs that conform to the first similarity transformation matrix among the remaining vectorized feature pairs, specifically: A pair of matched vectorized feature pairs v i and v j In the middle, one of the vectors v i The new coordinates v after transformation are obtained through the first similarity transformation matrix S. i ′ Calculate v i ′ With v j The difference between d v If d v Less than the threshold ε v If the pair of matching vectors satisfies S, then the feature points corresponding to this pair of matching vectors are stored in set A. f If it is greater than the threshold ε v If a pair of matching vectors does not satisfy S, the feature points corresponding to that pair are considered exterior points and removed. Finally, set A is calculated. f The number of interior points f.

4. The UAV image stitching method according to claim 1, characterized in that, The number of feature point pairs that conform to the second similarity transformation matrix among the statistically remaining feature point pairs is specifically: A pair of matched feature points (x i ,y i ) and (x j ,y j In ), one of the points (x) i ,y i The transformation is performed to obtain the new coordinate point (x) i ′ ,y i ′ ), calculate (x i ′ ,y i ′ ) and (x j ,y j The difference between ) d p If d p Less than the threshold ε p If a pair of matching points satisfies H, it is considered an interior point, and this pair of matching points is stored in the interior point set B. g In the middle; if d p Greater than the threshold ε p If the pair of matching points does not satisfy H, then it is considered that the pair does not satisfy H, and the pair of matching points is regarded as an exterior point and the exterior point is removed. The interior point set B is then calculated. g The number of interior points n.

5. The UAV image stitching method according to claim 4, characterized in that, The threshold is 0.

1.

6. The UAV image stitching method according to claim 1, characterized in that, k must be at least three pairs.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the UAV image stitching method as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the UAV image stitching method as described in any one of claims 1 to 6.

Images