Adaptive panoramic image seamless splicing method

Abstract

The invention belongs to the technical field of image processing, and discloses an adaptive panoramic image seamless splicing method. The method comprises the following steps: S1, obtaining an image, and preprocessing the image to form a to-be-spliced image set; s2, randomly selecting an image as a selected image P1, and forming a to-be-selected image set S by unselected images; s3, randomly selecting one image from the to-be-selected image set S as a reference image, and recording the reference image as P2; and S4, respectively extracting key feature points with scale invariance from the selected image P1 and the reference image P2. And S5, matching the key feature points of the two images, and performing purification and optimization. And S6, removing mismatching points after matching succeeds, and fusing and splicing the two successfully matched images together. And repeating the steps S3 to S6 to complete splicing of all the pictures. According to the invention, splicing of any multiple images can be realized, splicing gaps are avoided when the images are fused, and seamless connection of the images is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an adaptive panoramic image seamless splicing method. Background Art

[0002] Creating wide-angle panoramic images is essential for virtual tours and building virtual scenes. However, conventional cameras, limited by their wide-angle lens, can only capture partial panoramic images. Therefore, multiple images must be stitched together to create a complete panoramic view. This process often involves capturing real-world footage with conventional cameras, then stitching together multiple images to create a panoramic image, ultimately creating a virtual environment that resembles the real world. However, due to objective factors such as lighting and shooting angle, the stitched image may not conform to reality.

[0003] Existing technology in this field has two drawbacks. First, existing mosaic software must manually import multiple images, requiring extensive manual intervention during the stitching process. This inevitably places high demands on operators and increases the cost of panoramic image construction. Furthermore, existing technology suffers from differences in lighting and shooting angles, making it difficult to match the different images. The stitched image may also have significant differences on both sides of the stitched area, resulting in gaps. Alternatively, overlapping areas may be shadowed or blurred, resulting in poor fusion at the stitched area. Summary of the Invention

[0004] The purpose of the present invention is to provide an adaptive panoramic image seamless stitching method to solve the technical problem that the existing technology is difficult to achieve multi-image stitching while ensuring the matching stitching effect between images and avoiding stitching gaps and shadows at the stitching points.

[0005] The method for seamlessly stitching adaptive panoramic images comprises the following steps.

[0006] S1. After obtaining images and preprocessing them, the obtained images to be stitched are formed into an image set to be stitched. An access array is also set and initialized. The elements in the access array are tags corresponding to the images.

[0007] S2. Randomly select an image as the selected image P1, and the other unselected images form the image set S to be selected; the selected image is marked as flag[1]=1, indicating that the image has been matched.

[0008] S3. Randomly select another image from the image set S as the reference image, denoted as P2.

[0009] S4. Extract key feature points with scale invariance from the selected image P1 and the reference image P2 respectively.

[0010] S5: Match the key feature points of the two images, find the matching points, and purify and optimize the matching points.

[0011] S6. After the two images are matched, if there are matching feature point pairs, the incorrect matching points are removed, and the two successfully matched images are fused and spliced ​​together. The resulting fused image is used as the new selected image P1, and flag[2]=1 is marked, indicating that the reference image P2 has been successfully matched. If the pairing is not successful, the reference image P2 is put back into the image set S to be selected, and is still marked with flag[2]=0, indicating that the reference image P2 fails to match the selected image P1 and cannot be spliced ​​together, and then the process returns to step S3.

[0012] According to the extraction order, the images extracted from the beginning to the end are recorded as P3, P4, P5, ..., P n , the corresponding tags use flag[3], flag[4], flag[5],…, flag[n]; loop through steps S3 to S6 until the image set S is empty or there is no matching image.

[0013] Preferably, in step S6, a weighted average method is used to fuse images to avoid stitching gaps, which includes the following steps.

[0014] S601, the selected image P1 and the reference image P2 both contain overlapping areas in the image stitching. Based on the coordinate values ​​of the key feature points extracted, the width L of the stitching area is calculated; the distances from the matching point to the left and right boundaries of the overlapping area are d1 and d2 respectively, then L=d1+d2.

[0015] S602: Calculate the pixel value of the overlapping area using the following formula to complete the fusion of the two images in the overlapping area:

[0016]

[0017] Among them, f1(x,y) and f2(x,y) represent the pixel values ​​of the corresponding coordinates (x,y) of the two images respectively. and are the weights corresponding to f1(x,y) and f2(x,y).

[0018] Preferably, in step S4, applying the Gaussian difference pyramid and scale-invariant feature conversion algorithm includes the following specific steps.

[0019] S401: Detect extreme points in scale space. This step uses Gaussian difference kernels of different scales and image convolution to construct a scale space. Gaussian pyramids are constructed through Gaussian blurring and downsampling. Extreme points are searched in the DOG scale space to extract key feature points with scale invariance.

[0020] S402: Accurately locate the key feature points. A preliminary set of extreme points is obtained through the previous step. The positions of the key feature points are corrected using a fitted three-dimensional quadratic function. Low-contrast extreme points are removed using sub-pixel interpolation. Unstable edge response points in the extreme points are eliminated using a 2×2 Hessian matrix, thereby obtaining the final key feature points and their positions.

[0021] S403: Determine the directions of the key feature points, describe these key feature points using a unified description method, and use a gradient histogram to count the main directions of the feature points.

[0022] S404: Generate SIFT feature descriptions of key feature points. After obtaining the main direction of the local feature points in the previous step, the coordinate system is rotated to the main direction to make the corresponding key features rotationally invariant. The descriptor is then normalized to remove the effects of illumination and obtain the final feature point descriptor.

[0023] Preferably, in step S4, the gradient value and direction of the image at the pixel point (x, y) are obtained by the following formulas:

[0024]

[0025] θ(x, y) = tan -1 ((L(x,y+1)-L(x,y-1)) / (L(x+1,y)-L(x-1,y)))

[0026] Where L(x,y) represents the pixel intensity of the image at the pixel point (x,y), m(x,y) represents the gradient value at the pixel point (x,y), and θ(x,y) represents the gradient direction at the pixel point (x,y). This step uses the gradient histogram to calculate the direction, and the peak value in the gradient histogram is used as the main direction of the feature point.

[0027] Preferably, step S5 includes the following specific steps.

[0028] S501: A K-nearest neighbor algorithm is used to perform a rough matching of key feature points. First, the k nearest neighbors of the input key feature point in the feature space are found to achieve sample classification. The Euclidean distance method and the cosine method are used to measure the nearest neighbors. Subsequently, a K-nearest neighbor algorithm based on a KD Tree is used in this step to perform a rough matching of key feature points.

[0029] S502: Use the RANSAC algorithm to purify the results of key feature point matching. The results of key feature point matching include N data points, forming a data set P. The vast majority of the N data points are included in a parametric model and can be described by the parametric model. The number of such data points included in the parametric model is n, where n < N. The parameters of this model are calculated using the n data points, thereby classifying the data points into inliers and outliers, with the inliers serving as the purified output.

[0030] Preferably, the step S501 constructs a binary tree using KD Tree, and the specific construction steps are as follows:.

[0031] ① Determine the partitioning dimension: Calculate the data variance of n data points in two dimensions on a two-dimensional plane, and select the dimension with the largest variance as the partitioning dimension.

[0032] ②Calculate the node value Node-data: Sort the data according to the division dimension determined in step ①, and select the median value of the sorted data result as the node value.

[0033] ③ Determine the left and right subspaces: Divide the entire space into two based on the node value a to determine the left and right subspaces.

[0034] ④ Repeat steps ① to ③ for the left and right subspaces until each space contains only one data point, thus establishing a binary tree of the two-dimensional space.

[0035] Preferably, the step S501 uses an optimal node priority search method to search for an index point closest to the node to be found, specifically including:

[0036] ① Starting from the root node of the binary tree, generate a search path according to the nearest neighbor search method, calculate the distance between each node in the path and the node to be searched, and push the sibling nodes of the node into the priority sequence.

[0037] ② Extract the node with the highest priority in the priority sequence, take this node as the "root node", and repeat step ①.

[0038] Preferably, the specific steps of step S502 include.

[0039] ① Randomly select n data points from the data set P as the sample subset M.

[0040] ② Calculate the parameter model S used to describe most of the data points in M ​​through n data points in the sample subset M.

[0041] ③ Calculate the distance between the remaining Nn data points in the data set P except the extracted data points and the parameter model S; then set a threshold and record the number of data points whose distance between the data points and the parameter model S is within the threshold range.

[0042] ④ Repeat steps ① to ③. Among the multiple parameter models S formed, the model with the maximum value of n is the desired model; these n data points in the data set P are the inliers, and the remaining Nn data points are the outliers.

[0043] Preferably, in the step S1, before feature point extraction, the captured image needs to be projected onto an image plane through calculation, and then the image plane image coordinates need to be converted into an image coordinate system to complete image preprocessing.

[0044] The present invention has the following advantages: the present invention can achieve the splicing of any number of images, and can use the weighted averaging method to fuse the images in each step of splicing two images to avoid splicing gaps and achieve a seamless connection between the two images. The present invention also reduces the influence of brightness by constructing a Gaussian difference (DOG) pyramid. Searching for extreme points in the DOG scale space can ensure the scale invariance of feature points. Key feature points with scale invariance are conducive to accurate feature point matching. The present invention can also achieve precise positioning of key feature points and remove the influence of lighting through SIFT feature description.

[0045] The KD Tree-based K-nearest neighbor algorithm of this invention effectively locates the index point closest to the node to be searched, reducing the number of queries and the time required to find the best matching point. It also uses an optimal node-first search method to improve search efficiency and avoid the need to calculate distances and compare sibling nodes during the backtracking process, which increases the algorithmic complexity. Furthermore, this method eliminates mismatched points through a random sample consensus algorithm, achieving accurate matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The present invention is a flow chart of a stitching process after selecting images in an adaptive panoramic image seamless stitching method.

[0047] Figure 2 This is a comparison diagram between an input image sequence and an output panoramic image effect diagram in an embodiment of image stitching using the present invention. DETAILED DESCRIPTION

[0048] The specific implementation methods of the present invention will be further explained in detail below through the description of embodiments with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0049] like Figures 1 to 2 As shown, the present invention provides an adaptive panoramic image seamless stitching method, which includes the following steps.

[0050] S1. After obtaining images and preprocessing them, the obtained images to be stitched are formed into an image set to be stitched.

[0051] In this step, multiple original images are obtained and preprocessed to obtain images to be stitched. These images include multiple consecutive images with overlapping areas. These images are grouped into a set of images to be stitched, P, where P = [n], where n is the number of images to be stitched. This step also sets and initializes an access array, flag[n], with flag[n] = {0}. The elements in the access array are flags corresponding to the images.

[0052] In step S1, before feature point extraction, the captured image needs to be projected onto the image plane through calculation, and then the image plane image coordinates are converted into the image coordinate system to complete the image preprocessing. The specific method includes: first, camera calibration is performed to obtain the camera's intrinsic parameter matrix, and a world coordinate system is established in the real scene to describe the real position of the object. The coordinates of the point in the image are detected and the camera pose relative to the world coordinate system is calculated using the known camera intrinsic parameter matrix, the image coordinates of the corresponding point, and the real coordinates of the corresponding point in the world coordinate system. The camera pose includes a rotation matrix and a translation vector. For any point in the image, it is converted from the camera coordinate system to a plane coordinate system through the camera model and projection matrix established by the camera pose. This conversion process involves complex mathematical operations, including coordinate transformation and projection calculation, and ultimately maps the point in three-dimensional space to a two-dimensional plane.

[0053] S2. Randomly select an image as the selected image P1, and the other unselected images form the image set S to be selected; the selected image is marked with flag[1]=1, indicating that the image has been matched (the selected image P must be matched).

[0054] S3. Randomly select another image from the image set S as the reference image, denoted as P2.

[0055] S4. Extract key feature points with scale invariance from the selected image P1 and the reference image P2 respectively.

[0056] In step S4, feature points are extracted from the images to be stitched, and the Difference of Gaussian (DOG) pyramid and Scale Invariant Feature Transform (SIFT) algorithms are applied. This step includes the following specific steps:

[0057] S401: Detect extreme points in scale space. This step uses Gaussian difference kernels of different scales and convolution with the image to construct a scale space. A Gaussian pyramid is constructed through Gaussian blurring and downsampling. A Difference of Gaussian (DOG) pyramid is then constructed based on the Gaussian pyramid to reduce the effects of brightness. Extreme points are searched in the DOG scale space to ensure scale invariance of feature points. This process extracts key feature points (i.e., detected extreme points) that are scale invariant, facilitating accurate feature point matching.

[0058] S402: Accurately locate the key feature points. A preliminary set of extreme points is obtained through the previous step. The extreme points in this set are discontinuous and discrete in space. Since there is an error between the detected extreme point positions and the true extreme point positions, in order to make the positions and scales of the key feature points more accurate, this step uses a fitted three-dimensional quadratic function to correct the positions of the key feature points. At the same time, this step can also use sub-pixel interpolation to remove low-contrast extreme points; and use a 2×2 Hessian matrix to eliminate unstable edge response points in the extreme points, thereby obtaining the final key feature points and their positions.

[0059] S403: Determine the direction of key feature points. After extracting key feature points from the scale space, a unified description method is used to describe these key feature points. The descriptors generated are rotationally invariant in terms of rotation, deformation, noise, etc. This step uses the gradient histogram to count the main directions of the feature points. The gradient value and direction of the image at the pixel point (x, y) are obtained by the following formulas:

[0060]

[0061] θ(x, y) = tan -1 ((L(x,y+1)-L(x,y-1)) / (L(x+1,y)-L(x-1,y))

[0062] Where L(x,y) represents the pixel intensity at pixel (x,y), m(x,y) represents the gradient value at pixel (x,y), and θ(x,y) represents the gradient direction at pixel (x,y). This step uses the gradient histogram to calculate the direction, and the peak value in the gradient histogram is used as the main direction of the feature point. The main direction is the direction of the feature point in the gradient.

[0063] S404: Generate SIFT feature descriptions of key feature points. After obtaining the main direction of the local feature points in the previous step, the coordinate system is rotated to that main direction to make the corresponding key features rotationally invariant. The descriptors are then normalized to remove the effects of illumination, resulting in the final feature point descriptors. This completes the detection of key feature points using the SIFT algorithm.

[0064] S5: Match the key feature points of the two images (the selected image P1 and the reference image P2), find the matching points, and purify and optimize the matching points.

[0065] In this step, the key feature points of the two images are matched. A pair of key feature points, or matching points, are obtained upon successful matching. These matching points are then purified and optimized. This step includes the following specific steps.

[0066] S501: Use the K-nearest neighbor algorithm to perform rough matching of key feature points.

[0067] First, we find the k nearest neighbor points of the input key feature point in the feature space to achieve sample classification. The Euclidean distance method and the cosine method are used to measure the nearest neighbor points. The Euclidean distance is calculated as follows:

[0068]

[0069] Among them, x and y refer to the two points to be calculated in the selected image P1 and the reference image P2. i ,y i are the values ​​of point x and point y in the i-th dimension, n is the number of dimensions of the space, d(x,y) refers to the Euclidean distance between point x and point y; and the two n-dimensional vectors a(x 11 ,x 12 ,…,x 1n ) and b(x 21 ,x 22 ,…,x 2n ) can also be calculated using the following formula:

[0070]

[0071] Among them, x 1k , x 2k Represents the value of the kth dimension in n-dimensional vectors a and b, d 12 Represents the Euclidean distance between n-dimensional vectors a and b.

[0072] The calculation formula of the clamped cosine method is as follows:

[0073]

[0074] Among them, the value range of the angle cosine cosθ is [-1,1], x 1k , x 2k They represent the value of the kth dimension in the n-dimensional vectors a and b respectively. The size of the cosine of the angle represents the difference between the two vectors a and b. The larger the cosine of the angle, the smaller the angle between the two vectors and the smaller the sample difference.

[0075] Next, in this step, a K-D Tree-based K-nearest neighbor algorithm is used to perform a rough match of key feature points. K-DTree, short for k-dimensional tree, is a tree-like data structure that stores instance points in a multidimensional space for fast retrieval. This step uses the KD Tree to construct a binary tree. The specific construction steps are as follows.

[0076] ① Determine the partitioning dimension: Calculate the data variance of n data points in two dimensions on a two-dimensional plane and select the dimension with the larger variance as the partitioning dimension. If the variance in the x-dimension is larger, use the x-dimension as the partitioning dimension; if the variance in the y-dimension is larger, use the y-dimension as the partitioning dimension.

[0077] ② Calculate the node value: Sort the data by the partition dimension determined in step 1 and select the median value of the sorted data as the node value. In step 1, if the x dimension is the partition dimension, then here we sort the n data points by size along the x dimension and select the median value a of the sorted data as the node value.

[0078] ③ Determine the left and right subspaces: Split the entire space into two parts based on the node value a to determine the left and right subspaces. Draw a line x = a perpendicular to the x-axis through the node value a, dividing the entire space into two parts. The part with x ≤ a is the left subspace, and the part with x > a is the right subspace.

[0079] ④ Repeat steps ① to ③ for the left and right subspaces until each space contains only one data point. This creates a binary tree in the two-dimensional space; the resulting KD tree is what is generated.

[0080] The K-nearest neighbor algorithm based on the KD Tree effectively locates the index point closest to the node being searched, reducing the number of queries and the time it takes to find the best match. However, the backtracking process requires calculating distances and comparing sibling nodes (nodes with the same parent), which increases the algorithm's complexity. Excessive dimensions increase the number of comparisons and calculations, making it even less efficient than the exhaustive search method.

[0081] To address the computationally expensive problem of finding the nearest point in a KD tree, this step uses the Best B in First (BBF) search method. This method establishes a priority sequence during the search, sorting the nodes from smallest to largest based on their Euclidean distance to the node being searched. Each backtracking step compares the nodes in the priority sequence. The process is as follows:

[0082] ① Starting from the root node of the binary tree, generate a search path according to the nearest neighbor search method, calculate the distance between each node in the path and the node to be searched, and push the sibling nodes of the node into the priority sequence.

[0083] ② Extract the node with the highest priority in the priority sequence (that is, the node with the shortest Euclidean distance to the node to be searched), take this node as the "root node", and repeat step ①.

[0084] After the initial matching process using the KD Tree on a feature point set, the matched feature point pairs may not always be correct. This is because lighting, equipment, and angle can cause significant similarities in the grayscale and geometric structure of pixels in the image, leading to mismatches. Directly using these mismatched points to calculate the parameter model will result in inaccurate registration, affecting subsequent fusion and stitching. Therefore, after initially obtaining matching points, it is necessary to remove these mismatched points and perform precise matching.

[0085] S502: Use the RANSAC algorithm (Random Sample Consensus Algorithm) to purify the results of key feature point matching.

[0086] The results of key feature point matching include N data points, forming a data set P. The vast majority of these N data points are included in and can be described by a parametric model. The number of data points included in this parametric model is n, where n < N. The parameters of this model are calculated from these n data points, thereby classifying the data points into inliers and outliers. The inliers are used as the purified output. The specific steps are as follows.

[0087] ① Randomly select n data points from the data set P as the sample subset M.

[0088] ② Calculate the parameter model S used to describe most of the data points in M ​​through n data points in the sample subset M.

[0089] ③ Calculate the distance between the remaining Nn data points in the data set P except the extracted data points and the parameter model S; then set a threshold and record the number of data points whose distance between the data points and the parameter model S is within the threshold range.

[0090] ④ Repeat steps ① through ③. Among the multiple parameter models S, the model with the maximum value of n is the desired model. These n data points in the dataset P are the inliers, and the remaining Nn data points are the outliers.

[0091] S6. After the two images are matched, if there are matching feature point pairs, remove the mismatched points and fuse the two successfully matched images together. The fused image is used as the new selected image P1, and flag[2]=1 is marked, indicating that the reference image P2 has been successfully matched. If there is no successful pairing, the reference image P2 is put back into the image set S to be selected and is still marked with flag[2]=0, indicating that the reference image P2 fails to match the selected image P1 and cannot be spliced. Then return to step S3. According to the extraction order, the images extracted from the first to the last are recorded as P3, P4, P5, ..., P n , the corresponding flags are flag[3], flag[4], flag[5],…, flag[n].

[0092] This step uses the weighted average method to fuse images to avoid stitching gaps and achieve seamless connection between two images. It includes the following steps.

[0093] In step S601, both the selected image P1 and the reference image P2 contain overlapping regions for image stitching. The width L of the stitching region is calculated based on the coordinate values ​​of the matching points. The distances from the matching points to the left and right borders of the overlapping region are d1 and d2, respectively, where L = d1 + d2. If the matching point is on the right side of the image, the right border of the image is the right border of the overlapping region; if the matching point is on the left side of the image, the left border of the image is the left border of the overlapping region.

[0094] S602: Calculate the pixel value of the overlapping area using the following formula to complete the fusion of the two images in the overlapping area:

[0095]

[0096] Among them, f1(x,y) and f2(x,y) represent the pixel values ​​of the corresponding coordinates (x,y) of the two images (selected image P1 and reference image P2), respectively. and The weights corresponding to f1(x,y) and f2(x,y) are calculated. Compared with existing technologies, the image of the overlapping area after stitching is reduced in size and is less likely to cause defects such as shadows.

[0097] Steps S3 to S6 are executed in a loop until the image set S is empty or there is no matching image.

[0098] The present invention is described above by way of example in conjunction with the accompanying drawings. It is obvious that the specific implementation of the present invention is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the inventive concept and technical solution of the present invention, or the inventive concept and technical solution are directly applied to other occasions without improvement, they are all within the scope of protection of the present invention.

Claims

1. An adaptive panoramic image seamless stitching method, characterized by: The following steps are involved: S1, obtaining images to be stitched after preprocessing to form an image set to be stitched, and also setting and initializing an access array, wherein the elements in the access array are tags corresponding to the images; S2, randomly select an image as the selected image P1, and the other unselected images form the image set S to be selected; The selected image is marked as flag[1]=1, indicating that the image has been matched; S3, randomly select another image from the image set S as the reference image, denoted as P2; S4, extracting key feature points with scale invariance from the selected image P1 and the reference image P2 respectively; S5: Match the key feature points of the two images, find the matching points, and purify and optimize the matching points; S6. If there are matching feature point pairs after the two images are matched, remove the mismatched points, fuse the two successfully matched images together, and use the fused image as the new selected image P1. Meanwhile, flag[2]=1 is marked, indicating that the reference image P2 has been successfully matched. If the pairing is not successful, the reference image P2 is put back into the image set S to be selected and is still marked as flag[2]=0, indicating that the reference image P2 fails to match the selected image P1 and the two cannot be spliced, and then the process returns to step S3; According to the extraction order, the images extracted from the beginning to the end are recorded as P3, P4, P5, ..., P n , the corresponding tags use flag[3], flag[4], flag[5], …, flag[n]; loop through steps S3 to S6 until the image set S is empty or there is no matching image.

2. The method for seamlessly stitching panoramic images according to claim 1, characterized in that: In step S6, the weighted average method is used to fuse the images to avoid stitching gaps, including the following steps: S601, the selected image P1 and the reference image P2 both contain overlapping areas in image stitching, and the width L of the stitching area is calculated based on the coordinate values ​​of the key feature points extracted; the distances from the matching point to the left and right boundaries of the overlapping area are d1 and d2 respectively, then L=d1+d2; S602: Calculate the pixel value of the overlapping area by the following formula to complete the fusion of the two images in the overlapping area: Among them, f1(x,y) and f2(x,y) represent the pixel values ​​of the corresponding coordinates (x,y) of the two images respectively. and is the weight corresponding to f1(x,y) and f2(x,y).

3. The method for seamlessly stitching panoramic images according to claim 1, characterized in that: In step S4, Gaussian difference pyramid and scale-invariant feature conversion algorithm are applied, including the following specific steps: S401: Detect extreme points in scale space. This step uses Gaussian difference kernels of different scales and image convolution to construct a scale space, constructs a Gaussian pyramid through Gaussian blur and downsampling, searches for extreme points in the DOG scale space, and extracts key feature points with scale invariance. S402: Accurately locate the key feature points. A preliminary extreme point set is obtained through the previous step. The positions of the key feature points are corrected by fitting a three-dimensional quadratic function. The low-contrast extreme points are removed by using a sub-pixel interpolation method. The unstable edge response points in the extreme points are eliminated by using a 2×2 Hessian matrix, thereby obtaining the final key feature points and their positions. S403: determining the directions of the key feature points, describing the key feature points using a unified description method, and using a gradient histogram to count the main directions of the feature points; S404: Generate SIFT feature description of key feature points. After obtaining the main direction of the local feature points in the previous step, the coordinate system is rotated to the main direction so that the corresponding key features have rotation invariance; the descriptor is then normalized to remove the influence of illumination and obtain the final feature point descriptor.

4. The method for seamlessly stitching panoramic images according to claim 3, characterized in that: In step S4, the gradient value and direction of the image at the pixel point (x, y) are obtained by the following formulas: θ(x,y)=tan -1 ((L(x,y+1)-L(x,y-1)) / (L(x+1,y)-L(x-1,y)) Among them, L(x,y) represents the pixel intensity of the image at the pixel point (x,y), m(x,y) represents the gradient value at the pixel point (x,y), and θ(x,y) represents the gradient direction at the pixel point (x,y). This step uses the gradient histogram to count the direction and takes the peak value in the gradient histogram as the main direction of the feature point.

5. The method for seamlessly stitching panoramic images according to claim 1, characterized in that: The step S5 comprises the following specific steps: S501: Use K nearest neighbor algorithm to complete the rough matching of key feature points. First, find the k nearest neighbor points of the input key feature point in the feature space to achieve sample classification. The Euclidean distance method and the cosine method are used to measure the nearest neighbor points. Then, the K nearest neighbor algorithm based on KD Tree is used in this step to perform rough matching of key feature points; S502: Use the RANSAC algorithm to purify the result of key feature point matching. The result of key feature point matching includes N data points, forming a data set P. Most of the N data points are included in a parameter model and can be described by the parameter model. The number of such data points included in the parameter model is n, n < N. The parameters of the model are obtained through n data points, so that the data points are divided into internal points and external points, and the internal points are used as the purified output.

6. The method for seamlessly stitching an adaptive panoramic image according to claim 5, characterized in that: The step S501 uses KD Tree to construct a binary tree, and the specific construction steps are as follows: ① Determine the partitioning dimension: Calculate the data variance of n data points in two dimensions on a two-dimensional plane, and select the dimension with the largest variance as the partitioning dimension; ② Calculate the node value Node-data: Sort the data according to the division dimension determined in step ①, and select the median value in the sorted data result as the node value; ③ Determine the left and right subspaces: Divide the entire space into two based on the node value a to determine the left and right subspaces; ④ Repeat steps ① to ③ for the left and right subspaces until each space contains only one data point, thus establishing a binary tree of the two-dimensional space.

7. The method for seamlessly stitching an adaptive panoramic image according to claim 5, characterized in that: The step S501 uses the optimal node priority search method to search for the index point closest to the node to be found, specifically including: ① Starting from the root node of the binary tree, generate a search path using the nearest neighbor search method, calculate the distance between each node on the path and the node to be searched, and push the node's sibling nodes into the priority sequence; ② Extract the node with the highest priority in the priority sequence, take this node as the "root node", and repeat step ①.

8. The method for seamlessly stitching an adaptive panoramic image according to claim 5, characterized in that: The specific steps of step S502 include: ① Randomly select n data points from the data set P as the sample subset M; ② Calculate the parameter model S used to describe most of the data points in M ​​through n data points in the sample subset M; ③ Calculate the distance between the remaining Nn data points in the data set P except the extracted data points and the parameter model S; then set a threshold and record the number of data points whose distance between the data points and the parameter model S is within the threshold range; ④ Repeat steps ① to ③ in a loop. Among the multiple parameter models S formed, the model corresponding to the maximum value of n is the desired model; the n data points in the data set P are the inliers, and the remaining Nn data points are the outliers.

9. The method for seamlessly stitching an adaptive panoramic image according to claim 1, characterized in that: In the step S1, before extracting feature points, the captured image needs to be projected onto an image plane through calculation, and then the image plane image coordinates need to be converted into an image coordinate system to complete image preprocessing.

Images