An image stitching method and system based on a heterogeneous multi-mode panoramic stereoscopic imaging system

Abstract

The present application discloses an image stitching method and system for a heterogeneous multi-modal panoramic stereoscopic imaging system. The method includes: inputting original images to be stitched, searching for and matching feature points of the images to be stitched through the SIFT algorithm; calculating a homography matrix based on the searched and matched feature points; adding position weights to the homography matrix to estimate a local homography matrix; obtaining a global similarity transformation by using the RANSAC algorithm and the homography matrix; obtaining a globally optimal homography matrix according to the global similarity transformation and the local homography matrix; and performing translational transformation on the original images to be stitched based on the globally optimal homography matrix to complete image stitching. The present application solves the problems that there will be obvious misalignment and ghosting in the overlapping area of the stitched images, and serious distortion in the non-overlapping area. A panoramic image stitching method for a heterogeneous multi-modal panoramic imaging system is proposed.

Description

Technical Field

[0001] This application belongs to the technical field of panoramic image stitching in images, and specifically relates to a panoramic image stitching method and system based on a heterogeneous multi-modal panoramic stereoscopic imaging system. Background Art

[0002] In today's rapidly developing information age, people can easily obtain a vast amount of various information, especially visual image and video information. With the wide popularity of multimedia devices, such as cameras, surveillance cameras, mobile phones and other devices with video acquisition and display functions, people can conveniently obtain video information from these devices. However, since the scene range captured by a single device is relatively narrow, for example, the viewing angle of a standard lens is only about 50 degrees, which is much lower than the viewing angle of humans, it cannot meet the actual needs. In order to obtain a wider and higher-resolution scene, it is necessary to stitch the video images captured by multiple acquisition devices to expand the viewing range.

[0003] When there is a large parallax in the images to be stitched, obvious misalignment and ghosting will appear in the overlapping area of the stitched image, and serious distortion will also occur in the non-overlapping area. To solve this problem, the present invention introduces local homography transformation linearization and global similarity transformation. After feature point matching, the RANSAC algorithm is used to screen out inliers to more accurately calculate the homography matrix. According to the pixel point positions in the non-overlapping area, the corresponding similarity transformation for each point is calculated to effectively alleviate the distortion problem in the non-overlapping area. To evaluate the effectiveness of the algorithm, this study constructs both subjective and objective evaluation index systems. Compared with traditional image stitching algorithms, the algorithm proposed in this paper can effectively solve the problems of ghosting, misalignment in the overlapping area and distortion in the non-overlapping area, thus significantly improving the stitching effect. Summary of the Invention

[0004] This application aims to solve the deficiencies of the prior art and proposes a panoramic image stitching method for a heterogeneous multi-modal panoramic imaging system, including:

[0005] S1. Obtain the original images to be stitched, and search for and match feature points in the images to be stitched through the SIFT algorithm;

[0006] S2. Calculate the homography matrix based on the searched and matched feature points;

[0007] S3. Add position weights to the homography matrix to estimate the local homography matrix;

[0008] S4. Use the RANSAC algorithm and the homography matrix to obtain the global similarity transformation;

[0009] S5. Obtain the global optimal homography matrix according to the global similarity transformation and the local homography matrix;

[0010] S6. Perform translational transformation on the original image to be stitched based on the globally optimal homography matrix to complete image stitching.

[0011] Optionally, in S1, searching for and matching feature points in the stitched image includes: calculating feature points through the SIFT algorithm and searching for and matching feature points of adjacent images.

[0012] Optionally, the process of searching for and matching feature points includes:

[0013] Let the images and have matching points and in the overlapping part;

[0014] In homogeneous coordinates and , they are represented by the homography matrix as:

[0015] ;

[0016] where represents the homogeneous coordinates of the feature matching points of image ; represents the homogeneous coordinates of the corresponding feature matching points of image ; represents the homography matrix;

[0017] Expand the homography matrix and perform cross - multiplication on both sides to obtain the cross - multiplied homography matrix, represented as:

[0018]

[0019] The set of N matching points of adjacent images can be represented as and ;

[0020] where represents all N matching feature points of image , represents all N matching feature points of image , , and respectively represent each row of the homography matrix ; and represent the abscissa and ordinate of the feature point ; and represent the feature point The abscissa and ordinate of

[0021] Optionally, in S2, the process of obtaining the homography matrix includes:

[0022] Calculating different weights of pixel points according to the feature point positions to obtain a homography transformation matrix; the homography matrix is expressed as:

[0023]

[0024] Where represents The first row of the matrix; represents The second row of the matrix; and at the same time ensure that Ensure that the homography matrix has only 8 degrees of freedom; represents the DLT algebraic error.

[0025] Optionally, the process of estimating the local homography matrix includes:

[0026] Adding position weights to the homography matrix to estimate the local homography matrix. The local homography matrix at position is:

[0027] ,

[0028] Where represents The first row of the matrix; represents The second row of the matrix;

[0029] Weight matrix ; The weights are generated using a Gaussian function, and the formula is expressed as:

[0030] ;

[0031] represents the dimensional minimization of algebraic error; represents the coordinate point at the weight; represents the position of the weight to be obtained; represents The set of feature points around; represents and The pixel difference of; represents the image variance; To prevent numerical problems with the weights.

[0032] Optionally, the process of obtaining the global similarity transformation using the RANSAC algorithm and the homography matrix includes:

[0033] Using the RANSAC algorithm with a threshold to remove outliers and obtain valid feature matching points;

[0034] Using the RANSAC algorithm with a threshold to find the homography transformation of the plane of the preset inliers among the valid feature matching points and remove the inliers;

[0035] Repeating the process of removing inliers until the number of inliers is less than ;

[0036] Calculating the homography matrix based on the remaining inliers, comparing the rotation angles of the homography transformations, and selecting the smallest rotation angle as the global similarity transformation.

[0037] Optionally, the process of obtaining the global optimal homography matrix according to the global similarity transformation and the local homography matrix includes:

[0038] Performing linear weighting on the local similarity transformation and the global similarity transformation to obtain the global optimal homography matrix. The formula is:

[0039] ;

[0040] where is the local homography transformation matrix, S represents the global similarity transformation matrix, and are weighting coefficients, and ; is the local homography transformation matrix.

[0041] Optionally, the calculation method for performing translation transformation on the original image to be stitched includes:

[0042] ;

[0043] where is the local homography transformation matrix; represents the global similarity transformation matrix.

[0044] This application also provides an image stitching system based on a heterogeneous multi-modal panoramic stereo imaging system, including: a feature point acquisition module, a homography matrix calculation module, a local homography matrix estimation module, a global similarity transformation module, a global optimal homography matrix acquisition module, and a stitching module;

[0045] The feature point acquisition module is used to input the original image to be stitched and search for and match feature points in the image to be stitched through the SIFT algorithm;

[0046] The homography matrix calculation module is used to calculate the homography matrix according to the searched and matched feature points;

[0047] The local homography matrix estimation module is used to estimate the local homography matrix by adding position weights to the homography matrix;

[0048] The global similarity transformation module is used to obtain the global similarity transformation by using the RANSAC algorithm and the homography matrix;

[0049] The global optimal homography matrix acquisition module is used to obtain the global optimal homography matrix according to the global similarity transformation and the local homography matrix;

[0050] The splicing module is used to perform translational exchange on the original image to be stitched based on the global optimal homography matrix to complete image stitching.

[0051] Compared with the prior art, the beneficial effects of this application are:

[0052] (1) It solves the problems that there will be obvious misalignment and ghosting in the overlapping area of the stitched image, and serious distortion in the non-overlapping area.

[0053] (2) A panoramic image stitching method for heterogeneous multi-modal panoramic imaging systems is proposed. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of this application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0055] Figure 1 It is a method step diagram of a panoramic image stitching method for a heterogeneous multi-modal panoramic imaging system according to an embodiment of this application;

[0056] Figure 2 It is the original image to be stitched input in a panoramic image stitching method and system for a heterogeneous multi-modal panoramic imaging system according to an embodiment of this application;

[0057] Figure 3 It is the process image of removing outliers by the RANSAC algorithm in a panoramic image stitching method and system for a heterogeneous multi-modal panoramic imaging system according to an embodiment of this application;

[0058] Figure 4 It is the algorithm output result image in a panoramic image stitching method and system for a heterogeneous multi-modal panoramic imaging system according to an embodiment of this application;

[0059] Figure 5 A heterogeneous multi-modal panoramic stereoscopic imaging system for a panoramic image stitching method and system of a heterogeneous multi-modal panoramic imaging system according to an embodiment of the present application;

[0060] Figure 6 A visible light image stitching effect diagram of a system in the image stitching effect of a maritime heterogeneous multi-modal panoramic stereoscopic imaging system for a panoramic image stitching method and system of a heterogeneous multi-modal panoramic imaging system according to an embodiment of the present application;

[0061] Figure 7 An infrared image stitching effect diagram of a system in the image stitching effect of a maritime heterogeneous multi-modal panoramic stereoscopic imaging system for a panoramic image stitching method and system of a heterogeneous multi-modal panoramic imaging system according to an embodiment of the present application. Detailed implementation manners

[0062] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0063] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0064] Embodiment 1

[0065] In this embodiment, as Figure 1 shown, a panoramic image stitching method and system for a heterogeneous multi-modal panoramic imaging system specifically includes:

[0066] S1. Input the original images to be stitched, and search for and match feature points in the images to be stitched through the SIFT algorithm. Among them, Scale-Invariant Feature Transform (SIFT) is an image vision algorithm that can extract stable feature points at different scales and rotation angles, and has strong robustness to factors such as light and noise. SIFT has been widely used in fields such as salient object recognition, image stitching, and image detection. The core principle of the SIFT algorithm is to search for feature points at different scales of the image and estimate the directions of these feature points. The feature matching of the SIFT algorithm has strong anti-interference ability and can maintain stability and robustness regardless of how the picture undergoes translation, illumination, or rotation changes. The concept of scale invariance refers to the image representation of the same object at different scales. This algorithm uses a Gaussian kernel function to transform the initial images to be stitched to obtain images at different scales, and then extracts their feature points at each scale. Searching for and matching feature points in the stitched images includes: calculating feature points through the SIFT algorithm and searching for and matching the feature points of adjacent images.

[0067] The process of searching for and matching feature points includes: assuming that the images and The matching points in the overlapping part are and , in homogeneous coordinates and , can be represented by the homography matrix as:

[0068] (1)

[0069] where, represents the homogeneous coordinates of the feature matching points of image ; represents the homogeneous coordinates of the corresponding feature matching points of image ; represents the homography matrix;

[0070] Expanded into the following form:

[0071] (2)

[0072] Cross-multiplying both sides of formula (2) can obtain:

[0073] (3)

[0074] The set of N matching points of adjacent images can be expressed as and . AutoStitching only calculates one global homography matrix for adjacent images. The set of N matching points of adjacent images can be expressed as and ; wherein, represents all N matching feature points of the image ; represents all N matching feature points of the image ; , and respectively represent each row of the homography matrix ; and represent the abscissa and ordinate of the feature point ; and represent the abscissa and ordinate of the feature point ;

[0075] S2. Calculate the homography matrix according to the searched and matched feature points; wherein, the process of obtaining the homography matrix includes:

[0076] However, if adjacent images are not simply rotated or translated, ghosting and misalignment will inevitably occur in the overlapping area. To alleviate this problem, different weights of pixel points are calculated according to the positions of the feature points to obtain the homography transformation matrix; the homography transformation matrix is expressed as:

[0077] (4)

[0078] wherein, represents the first row of the matrix; represents the second row of the matrix; and at the same time ensure that the homography matrix has only 8 degrees of freedom; represents the DLT algebraic error. At the same time, add the constraint of ;

[0079] S3. Add position weights to the homography transformation matrix to estimate the local homography matrix; the process of estimating the local homography matrix includes:

[0080] Introduce the moving DLT framework, and estimate the local homography matrix by adding position weights to the homography matrix in formula (4). The local homography transformation at position is:

[0081] (5);

[0082] wherein, represents the first row of the matrix; represents the second row of the matrix;

[0083] can be expressed as , where the weight matrix . The weights are generated using a Gaussian function. The distance is closer, the greater the weight value. The weight formula is:

[0084] (6);

[0085] represents the -dimensional minimized algebraic error; represents the weight at the coordinate point ; represents the position of the weight to be obtained; represents the set of feature points around; represents and the pixel difference; represents the image variance; To prevent numerical problems with the weights.

[0086] For each pixel point in the non-overlapping region, it can be calculated using the linear weighted combination of the local homography transformation in the overlapping region. Therefore, it is very important to select an appropriate offset. A large offset error will cause a relatively serious distortion effect in the non-overlapping region. We improve the APAP algorithm and use the moving DLT method to estimate the homography transformation in the non-overlapping region to alleviate the distortion of the non-overlapping region of the stitched image.

[0087] The homography transformation in the non-overlapping region will produce a distortion effect. The main reason for this effect is that the homography transformation is a one-dimensional projective transformation. However, the transformation of the actual corresponding points is non-linear, and serious perspective distortion will occur during the conversion to 2-D. This distortion can be alleviated by linearizing the transformation.

[0088] The linearization of the homography for any point q near a pixel point p in the image can be obtained by the Taylor series of the homography matrix h(q)

[0089] (7);

[0090] where, is at the point the Jacobian matrix. However, in the 1-D case, it is not simple to calculate the linearization at any point q in the non-overlapping region. Because the boundary between the overlapping region and the non-overlapping region may contain multiple points, it is impossible to determine where to calculate the Jacobian matrix. Therefore, the pixel points on the boundary need to be linearized and the weighted average of the transformation is calculated.

[0091] For a set of R boundary points The linearized weighted combination is

[0092] (8);

[0093] where is a function of, and we use to replace the Gaussian function as the weight. Since the weight we use decays more slowly at the tails compared to the Gaussian distribution, it is more stable. It alleviates the distortion effect to a great extent in the non-overlapping regions.

[0094] S4. Obtain the global similarity transformation using the RANSAC algorithm and the homography matrix; among them, the method of homography linearization greatly reduces the distortion in the non-overlapping regions. Next, the distortion is further reduced by the method of similarity transformation in the non-overlapping regions to make the stitched image more natural.

[0095] If the global similarity transformation approximates the camera motion between the target image and the reference image, then the similarity transformation can compensate for the camera motion. However, using all the matching feature points to find the global similarity transformation may not be the optimal solution, especially in scenarios where the overlapping regions contain different image planes.

[0096] The process of obtaining the global similarity transformation using the RANSAC algorithm and the homography matrix includes: using the RANSAC algorithm with a threshold to remove outliers, as shown in Figure 3 to obtain valid feature matching points; using the RANSAC algorithm with a threshold to find the homography transformation of the plane of the preset inliers among the valid feature matching points and remove the inliers; repeat the process of removing inliers until the number of inliers is less than ; calculate the homography matrix based on the inliers, compare the rotation angles of the homography transformations, and select the smallest rotation angle as the global similarity transformation.

[0097] After calculating the global similarity transformation, use this transformation to adjust the distortion degree of the target image, and finally achieve the effect of alleviating the perspective distortion in the stitched image.

[0098] S5. Obtain the global optimal homography matrix based on the global similarity transformation and the local homography matrix; among them, if only the non-overlapping regions are adjusted, it will cause an unnatural effect at the boundaries between the non-overlapping regions and the overlapping regions.

[0099] The process of obtaining the global optimal homography matrix based on the global similarity transformation and the local homography matrix includes:

[0100] Linearly weight the local similarity transformation and the global similarity transformation to obtain the global optimal homography matrix. The formula is:

[0101] (9)

[0102] Wherein, is a local homography transformation matrix, S is a global similarity transformation, and are weighting coefficients, and ; is a local homography transformation matrix.

[0103] S6. Perform a translation transformation on the original image to be stitched based on the globally optimal homography matrix to complete image stitching.

[0104] The calculation method for performing a translation transformation on the original image to be stitched includes:

[0105] (10).

[0106] In this embodiment, is a local homography transformation matrix; represents the global similarity transformation matrix.

[0107] Embodiment 2

[0108] This application further includes an image stitching system based on a heterogeneous multi-modal panoramic stereoscopic imaging system, including: a feature point acquisition module, a homography matrix calculation module, a local homography matrix estimation module, a global similarity transformation module, a globally optimal homography matrix acquisition module, and a stitching module;

[0109] The feature point acquisition module is configured to input the original image to be stitched and search for and match feature points in the image to be stitched through the SIFT algorithm;

[0110] The homography matrix calculation module is configured to calculate a homography matrix based on the searched and matched feature points;

[0111] The local homography matrix estimation module is configured to estimate a local homography matrix by adding a position weight to the homography matrix;

[0112] The global similarity transformation module is configured to obtain a global similarity transformation by using the RANSAC algorithm and the homography matrix;

[0113] The globally optimal homography matrix acquisition module is configured to obtain a globally optimal homography matrix based on the global similarity transformation and the local homography matrix;

[0114] The stitching module is configured to perform a translation transformation on the original image to be stitched based on the globally optimal homography matrix to complete image stitching.

[0115] A method for image stitching based on a heterogeneous multi-modal panoramic stereo imaging system is proposed by linearizing the local homography transformation into a global similarity transformation in the stitching algorithm to solve the obvious misalignment and ghosting in the overlapping area of the stitched images and the serious distortion in the non-overlapping area. First, the feature points of the images collected by the heterogeneous multi-modal panoramic stereo imaging system are searched and matched, and then the RANSACk algorithm is used to screen out the abnormal feature points, and the globally optimal homography matrix is calculated. Then, the target image is moved to the corresponding position of the reference image through translation transformation to complete the image stitching. Finally, the implementation method of this application is applied to the multi-modal panoramic stereo rendering system, and the stitching effect is verified in the actual ship environment at sea.

[0116] In this embodiment, the windows10 operating system, Intel(R) Core(TM) i5-8300H CPU, and 8G memory are used, and the software is opencv3.4.7. The APAP image stitching algorithm is compared with the stitching algorithm in this paper. First, for the original images, as Figure 2 shown, the feature points are searched and matched, and the RANSAC algorithm is used to screen the inliers. In Image 3, the red points represent the outliers, and the green points represent the successfully matched feature points. Then, through the screening of the feature points, the accuracy of the global similarity transformation is improved. Finally, compared with the APAP image stitching algorithm, perspective distortion occurs in the overlapping area of the APAP image, and perspective distortion also exists in the non-overlapping area of the image. The output effect of the algorithm in this application is as Figure 4 shown, and the ghosting and misalignment effects in the overlapping area are significantly smaller than those of the APAP algorithm, and the distortion degree in the non-overlapping area is also greatly alleviated.

[0117] The algorithm is verified using a self-developed heterogeneous multi-modal panoramic stereo imaging system. The system uses the Linux operating system, Hisilicon Hi3559AV100 CPU, dual-core ARM Cortex A73@1.8GHz + dual-core ARM Cortex A53@1.2GHz + ARM Cortex A53@1.2GHz; dual-core ARM Mali G71@900MHz, 8GB of memory, and is equipped with 12 4K cameras to ensure that the horizontal field of view reaches 360° and the vertical field of view reaches 30°. The system internally loads the opencv3.4.7 external library. This system supports visible light and infrared light image acquisition, and has an imaging resolution ability better than 3m within 2km, and can be applied to environments such as night and thick fog to ensure high reliability. The system structure is as Figure 5 shown. This method is experimentally verified in this system, and the experimental results are as Figure 6 - 7 shown.

[0118] The embodiments described above are only descriptions of the preferred embodiments of the present application, and do not limit the scope of the present application. Without departing from the design spirit of the present application, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present application shall fall within the protection scope determined by the claims of the present application.

Claims

1. An image stitching method for a heterogeneous multi-modal panoramic stereoscopic imaging system, characterized in that, It includes the following steps: S1. Obtain the original image to be stitched, and search for and match feature points of the image to be stitched through the SIFT algorithm; S2. Calculate the homography matrix based on the searched and matched feature points; S3. Add position weights to the homography matrix to estimate the local homography matrix; S4. Use the RANSAC algorithm and the homography matrix to obtain the global similarity transformation; S5. Obtain the global optimal homography matrix according to the global similarity transformation and the local homography matrix; S6. Perform translation transformation on the original image to be stitched based on the global optimal homography matrix to complete image stitching; In S1, the search for and matching of feature points of the stitched image includes: calculating feature points through the SIFT algorithm, and searching for and matching the feature points of adjacent images; The process of searching for and matching feature points includes: Let the images and The matching points in the overlapping part are and ; In homogeneous coordinates and , it is represented by the homography matrix as follows: ； Among them, represents the homogeneous coordinates of the feature matching points of the image ; represents the homogeneous coordinates of the corresponding feature matching points of the image ; represents the homography matrix Expand the homography matrix and perform cross-multiplication on both sides to obtain the cross-multiplied homography matrix, expressed as: The set of N matching points of adjacent images can be expressed as and ; Among them, represents all N matching feature points of the image , represents all N matching feature points of the image , , and respectively represent each row of the homography matrix ; and represent the abscissa and ordinate of the feature point ; and represent the abscissa and ordinate of the feature point ; In S2, the process of obtaining the homography matrix includes: Calculating different weights of pixel points according to the feature point positions to obtain the homography transformation matrix; the homography matrix is expressed as: Among them, represents the first row of the matrix; represents the second row of the matrix; and at the same time ensure that the homography matrix has only 8 degrees of freedom; represents the DLT algebraic error; The process of estimating the local homography matrix includes: A position weight is added to the homography matrix to estimate a local homography matrix. The local homography matrix at position is as follows: , where represents the first row of the matrix Representative The second row of the matrix Weight matrix ; The weights are generated using a Gaussian function, expressed by the formula represents the d - dimensional minimized algebraic error; represents the weight at the coordinate point; represents the position of the weight to be obtained; represents the set of feature points around; represents the pixel difference between and represents the image variance; To prevent numerical problems with the weights.

2. The method for image stitching of the heterogeneous multi-modal panoramic stereoscopic imaging system according to claim 1, wherein The process of using the RANSAC algorithm and the homography matrix to obtain the global similarity transformation includes: Using the RANSAC algorithm with a threshold to remove outliers and obtain valid feature matching points; Use the RANSAC algorithm with a threshold to find the homography transformation of the plane of the preset inliers among the effective feature matching points, and remove the inliers; Repeat the process of removing inliers until the number of inliers is less than ; Calculating the homography matrix according to the remaining inliers, comparing the rotation angles of the homography transformations, and selecting the smallest rotation angle as the global similarity transformation.

3. The image stitching method of the heterogeneous multi-modal panoramic stereoscopic imaging system according to claim 2, characterized in that, The process of obtaining the global optimal homography matrix according to the global similarity transformation and the local homography matrix includes: Performing linear weighting on the local similarity transformation and the global similarity transformation to obtain the global optimal homography matrix, and the formula is: ； Among them, is a local homography transformation matrix, S represents a global similarity transformation matrix, and are weighting coefficients, and ; is a local homography transformation matrix.

4. The image stitching method for the heterogeneous multi-modal panoramic stereo imaging system according to claim 3, wherein The calculation method for performing translation transformation on the original image to be stitched includes: ； Among them, is a local homography transformation matrix; represents a global similarity transformation matrix.

5. An image stitching system for a heterogeneous multi-modal panoramic stereo imaging system, characterized in that, It includes: A feature point acquisition module, a homography matrix calculation module, a local homography matrix estimation module, a global similarity transformation module, a global optimal homography matrix acquisition module, and a stitching module; The feature point acquisition module is used to input the original image to be stitched, and search for and match feature points of the image to be stitched through the SIFT algorithm; The homography matrix calculation module is used to calculate the homography matrix based on the searched and matched feature points; The local homography matrix estimation module is used to add position weights to the homography matrix to estimate the local homography matrix; The global similarity transformation module is used to use the RANSAC algorithm and the homography matrix to obtain the global similarity transformation; The global optimal homography matrix acquisition module is used to obtain the global optimal homography matrix according to the global similarity transformation and the local homography matrix; The stitching module is used to perform translation transformation on the original image to be stitched based on the global optimal homography matrix to complete image stitching; In the feature point acquisition module, the search for and matching of feature points of the stitched image includes: calculating feature points through the SIFT algorithm, and searching for and matching the feature points of adjacent images; The process of searching for and matching feature points includes: Suppose the image and The matching points in the overlapping part are and ; In homogeneous coordinates and , it is represented by the homography matrix as follows: ； Among them, represents the homogeneous coordinates of the feature matching points of the image ; represents the homogeneous coordinates of the corresponding feature matching points of the image ; represents the homography matrix. Expand the homography matrix and perform cross - multiplication on both sides to obtain the homography matrix after cross - multiplication, which is expressed as: The set of N matching points of adjacent images can be expressed as and ; Among them, represents all N matched feature points of image ; , and represent each row of the homography matrix and represent the abscissa and ordinate of the feature point ; represent the abscissa and ordinate of the feature point In the homography matrix calculation module, the process of obtaining the homography matrix includes: Calculating different weights of pixel points according to the feature point positions to obtain the homography transformation matrix; the homography matrix is expressed as: Among them, represents the first row of the matrix; represents the second row of the matrix; and at the same time ensure that the homography matrix has only 8 degrees of freedom; represents the DLT algebraic error; The process of estimating the local homography matrix includes: Add position weights to the homography matrix to estimate the local homography matrix. The local homography matrix at position is as follows: ， Among them, represents the first row of the matrix representative the second row of the matrix Weight matrix ; The weights are generated using a Gaussian function and are expressed by the formula represent the dimensional minimization of algebraic error; represent the weight at the coordinate point; represent the position of the weight to be obtained; represent the set of feature points around; represent the pixel difference between and represent the image variance; To prevent numerical problems with the weights.

Images