Fast multi-view fish-eye image and video stitching method independent of camera device parameters
By preprocessing and registering fisheye images using a grid transformation-based method, and combining it with a video keyframe parameter sharing algorithm, the problems of device parameter dependence and low computational efficiency in existing technologies are solved, enabling fast and accurate fisheye image and video stitching, which is suitable for multiple devices and complex scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU INST OF BIOMEDICAL ENG & TECH CHINESE ACADEMY OF SCI
- Filing Date
- 2022-06-22
- Publication Date
- 2026-04-28
AI Technical Summary
Existing technologies cannot achieve fast fisheye image and video stitching independent of camera device parameters. Furthermore, existing methods have poor stitching results in complex scenes, low computational efficiency, and are difficult to meet the requirements of real-time and multi-device compatibility.
A mesh transformation-based method is used to preprocess and register fisheye images. The degrees of freedom of mesh transformation are constrained by feature alignment, local structure preservation, and boundary regularization terms. Dynamic video stitching is then performed in conjunction with a video keyframe parameter sharing algorithm.
It enables fast multi-eye image and video stitching independent of camera device parameters, improving stitching accuracy and computational efficiency. It is highly adaptable, suitable for various natural scenes, and meets the needs of high-quality panoramic images and videos.
Smart Images

Figure CN115222591B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and computer graphics, and in particular to a fast multi-view fisheye image and video stitching method that is independent of camera device parameters. Background Technology
[0002] Fisheye cameras have multiple lenses and can simultaneously capture multiple fisheye images from multiple angles, which can be combined to cover a full field of view. However, the stitching algorithms and software provided by camera vendors are usually offline and tied to undisclosed camera hardware settings, failing to meet the requirements of real-time and multi-device compatibility applications. Therefore, it is essential to research a fast fisheye image and video stitching method that is independent of camera device parameters.
[0003] Due to the severe content distortion and irregular boundaries of fisheye images, improving the alignment accuracy and computational efficiency of image stitching has always been a research hotspot both domestically and internationally. In recent years, deep learning has been widely applied in computer vision, with some studies designing convolutional neural networks to stitch images. While deep learning-based methods can reduce the incompleteness of manually designed features, they suffer from two main problems: first, poor adaptability, as the types of images to be stitched are limited by the dataset, making it difficult for deep learning-based methods to stitch multiple images in complex scenes; second, low computational efficiency, requiring significant computational resources and demanding high-performance hardware. Some researchers have used non-deep learning methods to decompose a single global registration model into multiple local registration models, including grid alignment and superpixel segmentation methods. While these methods exhibit good robustness and maintain good stitching results even in complex scenes, existing algorithms based on non-deep learning are not suitable for stitching distorted fisheye images, and the stitched panoramic images are prone to overlapping artifacts and irregular image boundaries. Fisheye video is a sequence of fisheye images showing changes in content over time. In the task of video stitching, existing algorithms focus on reducing content jitter during the stitching process, but cannot effectively balance content quality and computational efficiency.
[0004] Therefore, it is now necessary to improve existing technologies to provide a reliable solution. Summary of the Invention
[0005] The technical problem to be solved by the present invention is to provide a fast multi-eye image and video stitching method that is independent of camera device parameters, in order to address the shortcomings of the prior art.
[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a fast multi-view fisheye image and video stitching method independent of camera device parameters, comprising the following steps:
[0007] S1. Preprocess the fisheye image captured by the fisheye camera, the preprocessing including image correction and inter-lens parallax optimization;
[0008] S2. The preprocessed image is registered using a grid transformation-based method;
[0009] S3, Using feature alignment terms Local structural retention items and boundary regularization items The image constraint terms are composed of Constrain the degrees of freedom of the mesh transformation to optimize image registration;
[0010] S4. Perform dynamic video stitching to obtain panoramic images and videos.
[0011] Preferably, the image correction method in step S1 specifically includes the following steps:
[0012] S1-1-1 Extracting the effective region of a fisheye image based on the scan line method;
[0013] S1-1-2, Map the planar coordinates P of the effective region to spherical coordinates S;
[0014] S1-1-3. Project the spherical coordinates S onto the orthogonal plane coordinates O to obtain the corrected fisheye image.
[0015] The inter-lens parallax optimization method in step S1 specifically includes the following steps:
[0016] S1-2-1. Calculate the minimum permissible parallax distance for the camera. :
[0017] ;
[0018] in, This represents the minimum inscribed radius of the camera. , Indicates the field of view of the camera lens;
[0019] S1-2-2. Crop the image horizontally to make the horizontal field of view... Reduced to 180°, thus obtaining the distance It is infinitely large;
[0020] S1-2-3. Divide the first fisheye image into two equal parts along the horizontal direction, and prepare to stitch it with the second and last fisheye images respectively. Stitch the remaining fisheye images in the original order.
[0021] Preferably, step S2 specifically includes:
[0022] S2-1. Extract feature points from the fisheye image processed in step S2 based on ORB or SIFT feature point detection algorithms;
[0023] S2-2, Use the RANSAC algorithm to establish the feature point matching relationship between adjacent fisheye images and remove erroneous feature points;
[0024] S2-3. Divide the fisheye image into multiple rectangular sub-images;
[0025] S2-4. Calculate the local homography matrix of the initial mesh vertex W based on the direct linear transformation algorithm;
[0026] S2-5. Calculate the mapping position V of the mesh vertices and register the image.
[0027] Preferably, in step S3, the mapping position V of the mesh vertices obtained after optimizing the image registration is calculated by a sparse linear solver, and the calculation formula is as follows:
[0028] .
[0029] Preferably, in which, The calculation method is as follows:
[0030] (1) Traversal and Image Images with overlapping areas ;
[0031] (2) Calculate the image based on the position of the feature points in step S2. Matching grid ;
[0032] (3) Calculate the matching image according to the APAP algorithm Medium grid vertices Weighting factors Among them, the weighting factor determines the position of the grid vertices after registration;
[0033] (4) Calculate according to the following formula :
[0034]
[0035] Where I represents the set of all fisheye images, i and j represent the indices of the fisheye images, and k represents any matching grid in the set.
[0036] Preferably, in which, The calculation method is as follows:
[0037] (1) Calculate the initial image The set of grid edges in and the set of grid edges after image registration ;
[0038] (2) According to The position of the grid edge is used to calculate the weight of each grid edge. Among them, the closer to the image center, the higher the weight, and the closer to the image boundary, the lower the weight;
[0039] (3) Calculate the similarity transformation matrix of the grid based on the spatial positional relationship of each grid edge in the grid. ;
[0040] (4) Calculate according to the following formula :
[0041]
[0042] Where e represents any mesh edge in E before registration, f represents any mesh edge in F after registration, represents e and f with a matching relationship, and k represents any matching mesh edge in .
[0043] Preferably, in which, The calculation method is as follows:
[0044] (1) Extract all vertices at the registered grid boundary to ensure that each edge of the panoramic image is intact. The top of the vertex Both are the midpoints of their respective sides Align vertically or horizontally;
[0045] Where c = 1, 2, 3, 4, Including the left side right side , upper side bottom side ;
[0046] To ensure the continuity of panoramic image content, and The corresponding grid vertices need to be aligned in the horizontal direction;
[0047] (2) Calculate according to the following formula :
[0048]
[0049] in, Indicates the left side The top of the vertex, Indicates the right side The vertex on top.
[0050] Preferably, step S4 specifically includes:
[0051] S4-1, Calculate the time-series smoothing term ;
[0052] S4-1-1, Based on the number of feature points in two adjacent video frames Calculate weighting factors ; t represents the video frame with index t in the video;
[0053] S4-1-2, Calculate according to the following formula :
[0054]
[0055] Where I represents the set of all fisheye images, i represents the index of the fisheye image, and W... i Image I i All grid vertices in the w k W i Any grid vertex;
[0056] S4-2. Determine whether video frames can share splicing parameters. The specific steps are as follows:
[0057] S4-2-1 Calculate the average gradient of the current video frame ;
[0058] S4-2-2, Calculate the camera's intrinsic parameters based on the camera calibration algorithm. ;
[0059] S4-2-3. Generate the camera's eigenvalue matrix based on the feature points detected in adjacent video frames. ;
[0060] S4-2-4 Calculate the camera's extrinsic parameter matrix using the SfM 3D reconstruction algorithm
[0061]
[0062] Among them, the function This represents the intrinsic image decomposition and camera displacement. By calculating the translation matrix The F-norm is used to obtain the translation matrix. ;
[0063] Where R represents the camera rotation matrix;
[0064] if If L is less than the preset threshold, it means that the camera displacement is small and the content difference between video frames is not significant. In this case, the stitching parameters of the previous frame are reused to stitch the current video frame.
[0065] S4-3. Complete dynamic video stitching to obtain panoramic images and videos.
[0066] The present invention also provides a storage medium having a computer program stored thereon, which, when executed, is used to implement the method described above.
[0067] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described above.
[0068] The beneficial effects of this invention are:
[0069] This invention provides a fast multi-eye fisheye image and video stitching method that is independent of camera device parameters. It has low computational complexity, can quickly capture scene information around the camera, and can effectively improve the accuracy, adaptability and computational efficiency of image and video stitching. It is also beneficial to combine it with computer tasks such as computer vision and computer graphics.
[0070] This invention employs a fisheye image registration algorithm based on grid transformation and a video keyframe parameter sharing algorithm. The method of this invention has a wide range of applications and strong robustness. It can still achieve high-precision panoramic stitching quality in different types of natural scenes and can be applied to relevant computer applications with high requirements for panoramic image and video quality. Attached Figure Description
[0071] Figure 1 This is a flowchart of the camera device parameter-independent fast multi-eye fisheye image and video stitching method of the present invention. Detailed Implementation
[0072] The present invention will be further described in detail below with reference to embodiments, so that those skilled in the art can implement it based on the description.
[0073] It should be understood that terms such as “having,” “comprising,” and “including” as used herein do not exclude the presence or addition of one or more other elements or combinations thereof. Example 1
[0074] Reference Figure 1 This embodiment presents a fast multi-view fisheye image and video stitching method independent of camera device parameters. The overall idea is as follows:
[0075] To reduce the difficulty of fisheye image stitching, fisheye images captured by fisheye cameras first need to undergo image preprocessing, including fisheye image correction and inter-lens parallax optimization. Then, a grid transformation-based method is used to register multiple preprocessed images. To improve the registration quality, this invention innovatively designs three energy functions to constrain the degrees of freedom of the grid transformation. Finally, the fisheye image stitching method is extended to video stitching tasks, sharing stitching parameters between different video frames to quickly obtain dynamic scene information.
[0076] The method provided by this invention specifically includes the following steps:
[0077] S1. Preprocess the fisheye image captured by the fisheye camera. The preprocessing includes image correction and inter-lens parallax optimization.
[0078] S1-1, Image Correction:
[0079] While gaining a wider field of view, fisheye images produce severe barrel distortion, which is highly detrimental to image stitching between adjacent images. Fisheye image correction helps eliminate barrel distortion and reduces the difficulty of subsequent image stitching. The specific steps are as follows:
[0080] S1-1-1 Extracting the effective region of a fisheye image based on the scan line method;
[0081] S1-1-2, Map the planar coordinates P of the effective region to spherical coordinates S;
[0082] S1-1-3. Project the spherical coordinates S onto the orthogonal plane coordinates O to obtain the corrected fisheye image.
[0083] When using the same fisheye camera, the correction parameters are identical. Therefore, the fisheye image correction algorithm only needs to be pre-calculated once, and the correction parameters are stored locally for subsequent image stitching.
[0084] S1-2, Optimization of inter-lens parallax:
[0085] The closer the object is to the camera, the greater the parallax in the images captured by adjacent lenses. When the object is extremely close to the camera, it may only be captured by one lens. In this case, image stitching may fail and produce severe artifacts. To avoid this, this invention designs a clever method for optimizing inter-lens parallax, the specific steps of which are as follows:
[0086] S1-2-1. Calculate the minimum permissible parallax distance for the camera. :
[0087] ;
[0088] in, This represents the minimum inscribed radius of the camera. , Indicates the field of view of the camera lens;
[0089] S1-2-2, Crop the image horizontally to increase the field of view in the horizontal direction. Reduced to 180°, thus obtaining the distance It is infinite (d=+∞);
[0090] S1-2-3. Divide the first fisheye image into two equal parts along the horizontal direction, and prepare to stitch it with the second and last fisheye images respectively. Stitch the remaining fisheye images in the original order without any other processing; thus, the parallax optimization between lenses is achieved.
[0091] S2. The preprocessed image is registered using a grid transformation-based method.
[0092] For preprocessed fisheye images, registration is required. Using a single global homography matrix to register adjacent images cannot guarantee correct alignment of every pixel. Calculating a local homography matrix for each pixel leads to excessive computational complexity, failing to meet the demands of real-time applications. To balance accuracy and efficiency, this invention employs a grid-based transformation method, uniformly dividing the image into multiple rectangular sub-regions. Each sub-region uses a separately designed homography matrix for image registration. The specific steps are as follows:
[0093] S2-1. Extract feature points from the fisheye image processed in step S2 based on ORB or SIFT feature point detection algorithms;
[0094] S2-2, Use the RANSAC algorithm to establish the feature point matching relationship between adjacent fisheye images and remove erroneous feature points;
[0095] S2-3. Divide the fisheye image into multiple rectangular sub-images;
[0096] S2-4. Calculate the local homography matrix of the initial mesh vertex W based on the direct linear transformation algorithm;
[0097] S2-5. Calculate the mapping position V of the mesh vertices and register the image.
[0098] S3, Using feature alignment terms Local structural retention items and boundary regularization items The image constraint terms are composed of Constrain the degrees of freedom of the mesh transformation to optimize image registration.
[0099] Because image content is often complex and images captured in different environments vary significantly, making images within a grid relatively independent inevitably leads to severe distortion and deformation of the image structure. To address this issue, this invention innovatively designs three constraint terms to control the degrees of freedom of grid transformation, thereby optimizing image registration quality; image constraint terms... By feature alignment terms Local structural retention items and boundary regularization items The final mesh vertex mapping positions V, obtained after optimizing image registration, are calculated by a sparse linear solver, using the following formula:
[0100] .
[0101] S3-1, Calculation
[0102] in, Its function is to align the grids in the overlapping areas between adjacent images, which helps improve the accuracy of registration and eliminate artifacts in the stitched image. The specific steps are as follows:
[0103] (1) Traversal and Image Images with overlapping areas ;
[0104] (2) Calculate the image based on the position of the feature points in step S2. Matching grid ;
[0105] (3) Calculate the matching image according to the APAP algorithm Medium grid vertices Weighting factors Among them, the weighting factor determines the position of the grid vertices after registration;
[0106] (4) Calculate according to the following formula :
[0107]
[0108] Where I represents the set of all fisheye images, i and j represent the indices of the fisheye images, and k represents any matching grid in the set.
[0109] S3-2, Calculation
[0110] Since the fisheye image correction in step 1 can only approximately eliminate barrel distortion, only adding This can lead to inconsistent transformation amplitudes for each grid, resulting in severe image distortion. To avoid distortion during panoramic stitching, this invention adds... To constrain the degrees of freedom of the mesh transformation, ensuring that the registered mesh retains as much of its original geometry as possible, the specific steps are as follows:
[0111] (1) Calculate the initial image The set of grid edges in and the set of grid edges after image registration ;
[0112] (2) According to The position of the grid edge is used to calculate the weight of each grid edge. Among them, the closer to the image center, the higher the weight, and the closer to the image boundary, the lower the weight;
[0113] (3) Calculate the similarity transformation matrix of the grid based on the spatial positional relationship of each grid edge in the grid. ;
[0114] (4) Calculate according to the following formula :
[0115]
[0116] Where e represents any mesh edge in E before registration, f represents any mesh edge in F after registration, represents e and f with a matching relationship, and k represents any matching mesh edge in .
[0117] S3-3, Calculation
[0118] Existing stitching algorithms can only guarantee the integrity of image content, but not the regularity of image boundaries or the continuity of content. When panoramic images are mapped to three-dimensional space, visual discontinuities in content can severely impact the user's immersion. To address these issues, this invention adds... To optimize the position of image boundaries, the specific steps are as follows:
[0119] (1) Extract all vertices at the registered grid boundary to ensure that each edge of the panoramic image is intact. The top of the vertex Both are the midpoints of their respective sides Align vertically or horizontally;
[0120] Where c = 1, 2, 3, 4, Including the left side right side , upper side bottom side ;
[0121] To ensure the continuity of panoramic image content, and The corresponding grid vertices need to be aligned in the horizontal direction;
[0122] (2) Calculate according to the following formula :
[0123]
[0124] in, Indicates the left side The top of the vertex, Indicates the right side The vertex on top.
[0125] S4. Perform dynamic video stitching to obtain panoramic images and videos.
[0126] If the above multi-eye image stitching algorithm is simply extended to video stitching tasks, two main problems will arise. First, the independent stitching of video frames will cause jitter in the video content; second, step 4 of the algorithm must be repeated for each frame, resulting in high computational complexity. To solve these problems, the method provided by this invention is as follows:
[0127] S4-1, Calculate the time-series smoothing term ;
[0128] Step S3 proposes three image constraint terms to improve image stitching quality in static scenes. Furthermore, this invention extends these image constraint terms to video constraint terms. To maintain the continuity of content between video frames, the grid of each image needs to be in the same position as much as possible. Therefore, this invention designs... This is to prevent video jitter while ensuring that the resolution of each panoramic image is consistent. The specific calculation steps are as follows:
[0129] S4-1-1, Based on the number of feature points in two adjacent video frames Calculate weighting factors ; t represents the video frame with index t in the video;
[0130] S4-1-2. Ensure that the grid vertices of two adjacent video frames do not change as much as possible, and calculate according to the following formula. :
[0131]
[0132] Where I represents the set of all fisheye images, i represents the index of the fisheye image, and W... i Image I i All grid vertices in the w kW i Any grid vertex;
[0133] S4-2. Determine whether splicing parameters can be shared between video frames:
[0134] Since the relative positions of all camera lenses are fixed, theoretically, all video frames can share the same set of stitching parameters. However, because sparse feature point detection in step S2 cannot provide a correspondence between all image pixels, the image feature relationships established for each frame are different. Sharing the same set of stitching parameters across all video frames would cause serious cumulative errors. To balance video stitching quality and computational efficiency, stitching parameters are shared within a certain frame range, within the allowable range of stitching errors. The specific steps are as follows:
[0135] S4-2-1 Calculate the average gradient of the current video frame ;
[0136] S4-2-2, Calculate the camera's intrinsic parameters based on the camera calibration algorithm. ;
[0137] S4-2-3. Generate the camera's eigenvalue matrix based on the feature points detected in adjacent video frames. ;
[0138] S4-2-4 Calculate the camera's extrinsic parameter matrix using the SfM 3D reconstruction algorithm
[0139]
[0140] Among them, the function This represents the intrinsic image decomposition and camera displacement. By calculating the translation matrix The F-norm is used to obtain the translation matrix. ;
[0141] Where R represents the camera rotation matrix;
[0142] if If L is less than the preset threshold, it means that the camera displacement is small and the content difference between video frames is not significant. In this case, the stitching parameters of the previous frame are reused to stitch the current video frame.
[0143] S4-3. Complete dynamic video stitching to obtain panoramic images and videos. Example 2
[0144] This embodiment provides a storage medium on which a computer program is stored, which, when executed, is used to implement the method of Embodiment 1. Example 3
[0145] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method of Embodiment 1.
[0146] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details.
Claims
1. A fast multi-view fisheye image and video stitching method independent of camera device parameters, characterized in that, Includes the following steps: S1. Preprocess the fisheye image captured by the fisheye camera, the preprocessing including image correction and inter-lens parallax optimization; S2. The preprocessed image is registered using a grid transformation-based method; S3, Using feature alignment terms Local structural retention items and boundary regularization items Composition of image constraint terms Constrain the degrees of freedom of the mesh transformation to optimize image registration; S4. Perform dynamic video stitching to obtain panoramic images and videos; In step S3, the mapping positions V of the mesh vertices obtained after optimizing the image registration are calculated by the sparse linear solver, and the calculation formula is as follows: ; in, The calculation method is as follows: (1) Traversal and Image Images with overlapping regions N represents the function for determining whether there are overlapping regions; (2) Calculate the image based on the position of the feature points in step S2. Matching grid ; (3) Calculate the matching image according to the APAP algorithm Medium grid vertices Weighting factors Among them, the weighting factor determines the position of the grid vertices after registration; (4) Calculate according to the following formula : Where I represents the set of all fisheye images, i and j represent the indices of the fisheye images, and k represents... Any matching grid in the grid; in, The calculation method is as follows: (1) Calculate the initial image The set of grid edges in and the set of grid edges after image registration ; (2) According to The position of the grid edge is used to calculate the weight of each grid edge. Among them, the closer to the image center, the higher the weight, and the closer to the image boundary, the lower the weight; (3) Calculate the similarity transformation matrix of the grid based on the spatial positional relationship of each grid edge in the grid. ; (4) Calculate according to the following formula : Where e represents any mesh edge in E before registration, and f represents any mesh edge in F after registration. Let e and f represent the matching relationship, and k represent... Any matching grid edge in the grid; in, The calculation method is as follows: (1) Extract all vertices at the registered grid boundary to ensure that each edge of the panoramic image is intact. The top of the vertex Both are the midpoints of their respective sides Align vertically or horizontally; Where c = 1, 2, 3, 4, Including the left side right side , upper side bottom side ; To ensure the continuity of panoramic image content, and The corresponding grid vertices need to be aligned in the horizontal direction; (2) Calculate according to the following formula : in, Indicates the left side The top of the vertex, Indicates the right side The vertex on top.
2. The fast multi-view fisheye image and video stitching method independent of camera device parameters according to claim 1, characterized in that, The image correction method in step S1 specifically includes the following steps: S1-1-1 Extracting the effective region of a fisheye image based on the scan line method; S1-1-2, Map the planar coordinates P of the effective region to spherical coordinates S; S1-1-3. Project the spherical coordinates S onto the orthogonal plane coordinates O to obtain the corrected fisheye image; The inter-lens parallax optimization method in step S1 specifically includes the following steps: S1-2-1. Calculate the minimum permissible parallax distance for the camera. : ; in, This represents the minimum inscribed radius of the camera. , Indicates the field of view of the camera lens; S1-2-2, Crop the image horizontally to increase the field of view in the horizontal direction. Reduced to 180°, thus obtaining the distance It is infinitely large; S1-2-3. Divide the first fisheye image into two equal parts along the horizontal direction, and prepare to stitch it with the second and last fisheye images respectively. Stitch the remaining fisheye images in the original order.
3. The fast multi-view fisheye image and video stitching method independent of camera device parameters according to claim 2, characterized in that, Step S2 specifically includes: S2-1. Extract feature points from the fisheye image processed in step S2 based on ORB or SIFT feature point detection algorithms; S2-2, Use the RANSAC algorithm to establish the feature point matching relationship between adjacent fisheye images and remove erroneous feature points; S2-3. Divide the fisheye image into multiple rectangular sub-images; S2-4. Calculate the local homography matrix of the initial mesh vertex W based on the direct linear transformation algorithm; S2-5. Calculate the mapping position V of the mesh vertices and register the image.
4. The fast multi-view fisheye image and video stitching method independent of camera device parameters according to claim 3, characterized in that, Step S4 specifically includes: S4-1, Calculate the time-series smoothing term ; S4-1-1, Based on the number of feature points in two adjacent video frames Calculate weighting factors ; t represents the video frame with index t in the video; S4-1-2, Calculate according to the following formula : Where I represents the set of all fisheye images, i represents the index of the fisheye image, and W... i Image I i All grid vertices in the w k W i Any grid vertex; S4-2. Determine whether video frames can share splicing parameters. The specific steps are as follows: S4-2-1 Calculate the average gradient of the current video frame ; S4-2-2, Calculate the camera's intrinsic parameters based on the camera calibration algorithm. ; S4-2-3. Generate the camera's eigenvalue matrix based on the feature points detected in adjacent video frames. ; S4-2-4 Calculate the camera's extrinsic parameter matrix using the SfM 3D reconstruction algorithm Among them, the function This represents intrinsic image decomposition and camera displacement. By calculating the camera translation matrix The F-norm is used to obtain the translation matrix. ; Where R represents the camera rotation matrix; if If L is less than the preset threshold, it means that the camera displacement is small and the content difference between video frames is not significant. In this case, the stitching parameters of the previous frame are reused to stitch the current video frame. S4-3. Complete dynamic video stitching to obtain panoramic images and videos.
5. A storage medium having a computer program stored thereon, characterized in that, When executed, this program is used to implement the method as described in any one of claims 1-4.
6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-4.
Citation Information
Patent Citations
Panoramic video rapid splicing method and system
CN110782394A
Fisheye image splicing method and device
CN111461963A