Single-camera panorama splicing system and method

Abstract

The invention relates to the technical field of computer vision and image processing, and particularly provides a single-camera panorama splicing system and method, and the method comprises the steps: employing a 12 * 9 checkerboard to shoot a plurality of images at different angles and different distances, and obtaining an internal reference matrix and a distortion coefficient of a camera; adjusting the camera to a position where the optical center coincides with the rotating shaft by adopting an optical center alignment method; driving the camera to rotate, and shooting a plurality of to-be-spliced images covering 360 degrees; and splicing the to-be-spliced images into a complete image by adopting image distortion removal, spherical coordinate system conversion and gradient fuzzy fusion methods. The problems of high hardware cost, chromatic aberration, parallax, low splicing automation degree and the like existing in a multi-camera shooting scheme in the prior art are solved, and low-cost, high-precision and full-automatic seamless panorama splicing is realized through optical center accurate alignment, spherical coordinate system conversion and brightness equalization algorithms.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and image processing technology, and in particular to a single-camera panoramic image stitching system and method, which is applicable to panoramic image acquisition, virtual reality, three-dimensional reconstruction and other fields. Background Art

[0002] Existing panoramic image stitching technologies primarily utilize multi-camera stitching and feature extraction-based stitching. Multi-camera stitching utilizes multiple cameras to simultaneously capture images from different angles for stitching. However, the optical centers of multiple cameras are difficult to align, resulting in parallax between the captured images and distortion of the stitched area. Furthermore, these technologies present high hardware costs, difficulty eliminating color and parallax differences between cameras, and complex installation and calibration. For example, the multi-camera panoramic device disclosed in patent CN118646833A requires precise synchronization of multiple camera parameters, significantly increasing system complexity and costs.

[0003] Feature extraction-based stitching schemes stitch image data together through feature extraction, feature matching, and machine learning. However, in low-texture scenes, feature extraction can lead to matching failures or errors if feature points are too few or unevenly distributed. They are also susceptible to interference from dynamic objects, resulting in mismatches. If the camera is not rotated about the optical center, parallax exists between the images, and direct matching can distort the stitched area. When stitching multiple images, feature matching errors accumulate frame by frame, ultimately causing the entire panorama to bend or misaligned. Therefore, this scheme suffers from image distortion after stitching and low algorithm robustness, making it unsuitable for scenarios requiring high precision. For example, in the paper "Simulation of Stitching and Matching Contour Features of Animated Images in Complex Scenes," traditional methods mistakenly identify adjacent rows of crops as matching points in the furrows of wheat fields, causing misalignment and distortion in the stitched image. Even after improvements were made and the matching accuracy increased to 92%, stitching failures could still occur.

[0004] In addition, existing stitching solutions do not consider optical center alignment, which leads to large parallax that is difficult to eliminate, resulting in distortion in the stitching area. Summary of the Invention

[0005] In view of this, the present invention proposes a single-camera panoramic image stitching system and method to solve the problems of high hardware cost, multi-camera chromatic aberration and parallax, and low degree of stitching automation in the existing technology. Through precise optical center alignment, spherical coordinate system conversion and brightness balance algorithm, low-cost, high-precision, fully automatic seamless panoramic image stitching is achieved.

[0006] On one hand, to achieve the above-mentioned object, the present invention proposes a single-camera panoramic image stitching system, characterized by comprising:

[0007] Hardware module: single camera, stepper motor;

[0008] Software modules: camera calibration module, dedistortion processing module, spherical coordinate system conversion module, and stitching and fusion module;

[0009] Controlling the single camera to rotate and shoot by the stepping motor to obtain the image to be stitched;

[0010] Obtaining the intrinsic parameter matrix and distortion coefficient of the single camera through the camera calibration module;

[0011] The images to be spliced are spliced into a complete image by using the dedistortion processing module, the spherical coordinate system conversion module and the splicing and fusion module respectively using image dedistortion, spherical coordinate system conversion and gradient fuzzy fusion methods.

[0012] Furthermore, the hardware module also includes a connector connecting the single camera and the stepper motor, and the optical center of the camera is aligned with the rotation axis of the stepper motor through the connector.

[0013] Furthermore, the software module also includes an image acquisition control module, which controls the motor to drive the camera to rotate 360 degrees and controls the camera to capture an image every time it rotates 120 degrees.

[0014] Furthermore, the stitching and fusion module applies transparency gradient and Gaussian blur to the stitching boundary area of the dedistorted image to obtain the complete image.

[0015] On the other hand, to achieve the above-mentioned object, the present invention proposes a single-camera panoramic image stitching method, which is characterized by comprising:

[0016] Use a 12*9 chessboard to capture several images at different angles and distances to obtain the camera's intrinsic parameter matrix and distortion coefficients.

[0017] Use the optical center alignment method to adjust the camera to the position where the optical center coincides with the rotation axis;

[0018] Drive the camera to rotate and take several images to be stitched covering 360°;

[0019] The images to be spliced are spliced into a complete image by using image dedistortion, spherical coordinate system conversion and gradient fuzzy fusion methods.

[0020] Furthermore, the process of adjusting the camera to a position where the optical center coincides with the rotation axis using the optical center alignment method includes:

[0021] Place two wooden boards in the shooting environment. The foreground board is used as a block, and the background board is used for measurement. Adjust the camera installation position by capturing images before and after rotation until the parallax change is below the allowable range or there is no parallax. In this case, the camera is considered to be rotating around the optical center.

[0022] Furthermore, in the process of adjusting the camera installation position by taking images before and after rotation, the error before and after rotation is no more than 5 mm.

[0023] Furthermore, the camera is driven to rotate by an angle of 120°, and an image is captured every time it rotates by 120°.

[0024] Furthermore, the process of stitching the images to be stitched into a complete image by using image dedistortion, spherical coordinate system conversion, and gradient fuzzy fusion methods includes:

[0025] Calculating the average brightness of the images to be stitched and adjusting the brightness to be consistent;

[0026] Performing dedistortion processing on the image to be stitched based on the intrinsic parameter matrix and the distortion coefficient to obtain a dedistorted image;

[0027] Mapping the dedistorted image to a spherical coordinate system to eliminate perspective distortion;

[0028] Transparency gradient and Gaussian blur are applied to the stitching boundary area of the dedistorted image to obtain the complete image.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention adopts a collaborative control architecture of a single camera and an optical center alignment motor to effectively reduce hardware costs. Through hardware-level optical center alignment, it ensures that only a pure rotational relationship exists between images, avoiding multi-camera geometric errors. In an embodiment of the present invention, images are directly aligned through rotation angles in a spherical coordinate system, avoiding errors that rely on feature point matching, and the stitching success rate can reach 100%. In addition, the present invention adopts dynamic brightness equalization and gradient blur fusion algorithms to improve the color consistency of the panoramic image. The stitching process of the present invention is based on a preset physical rotation angle (120° step size), without cumulative errors, ensuring accurate geometric relationships. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0032] Figure 1 Schematic diagram of the flow of a single-camera panoramic image stitching method in an embodiment of the present invention;

[0033] Figure 2 Schematic diagram of camera calibration in an embodiment of the present invention;

[0034] Figure 3Schematic diagram of image stitching in an embodiment of the present invention, wherein (a) is a panoramic stitching image without color leveling and seam removal; (b) is a panoramic stitching image after color leveling and seam removal;

[0035] Figure 4 Schematic diagram of optical center measurement in an embodiment of the present invention;

[0036] Figure 5 Schematic diagram of the structure of a single-camera panoramic image stitching system in an embodiment of the present invention;

[0037] Figure 6 2 is a hardware structure diagram of a single-camera panoramic image stitching system in an embodiment of the present invention;

[0038] Figure numbers: 1-RK3588 mainboard; 2-network camera; 3-motor driver board; 4-four-phase stepper motor. DETAILED DESCRIPTION

[0039] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, unless there is a conflict, the embodiments of the present disclosure and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] This embodiment proposes a single-camera panoramic image stitching method, such as Figure 1 As shown, the following steps are included:

[0041] 1. Camera calibration: Figure 2 As shown, a 12*9 chessboard is used to capture 50-60 images at different angles and distances for calibration to obtain the intrinsic parameter matrix K and distortion coefficient D for subsequent dedistortion.

[0042] (1) Camera model and intrinsic parameter matrix K

[0043] The camera imaging follows the pinhole imaging model, which transforms the three-dimensional world point P = [X, Y, Z] T Projection to the two-dimensional pixel coordinate system P = [u, v] T , the model formula is:

[0044]

[0045] Where: s is the scale factor; K is the internal parameter matrix, the form is:

[0046]

[0047] f x , f y : focal length in x / y direction (in pixels);

[0048] c x , c x : coordinates of the principal point (the intersection of the optical axis and the imaging plane);

[0049] r: Interaxial tilt coefficient (usually 0).

[0050] (2) Distortion model and distortion coefficient D

[0051] Actual lenses have radial distortion and tangential distortion, which need to be corrected by a polynomial model. The distortion formula is:

[0052]

[0053] in:

[0054] x=(Xc x ) / f x ,y=(Yc y ) / f y is the normalized coordinate;

[0055] r 2 =x 2 +y 2 ;

[0056] D = [k1, k2, p1, p2, k3] is the distortion coefficient (usually k3 can be ignored);

[0057] 2. Optical center alignment:

[0058] a. Optical center measurement: Place two wooden boards in the environment, the foreground board is used for blocking, and the background board is used for measurement, such as Figure 4 Take two images rotated 60° to verify the parallax change.

[0059] b. The distance from the camera to the foreground is X, and the distance to the background is Y. When the X:Y ratio is 1:5, the error before and after rotation shall not exceed 5mm.

[0060] c. Adjust the camera installation position to ensure that the optical center coincides with the motor rotation axis to eliminate parallax. Observe whether the static scene image changes only due to rotation of the image plane without parallax changes. If the measured parallax change is small or no parallax is present, the camera is assumed to be rotating around the optical center.

[0061] 3. Automatic image acquisition: The motor drives the camera to rotate 120° to capture an image, and a total of three fisheye images covering 360° are collected, such as Figure 3 (a)

[0062] 4. Image processing and stitching:

[0063] Brightness balance: calculate the average brightness of three images and dynamically adjust the local brightness to be consistent;

[0064] Dedistortion: Correct the image based on K and D;

[0065] Spherical coordinate system conversion: Map the dedistorted image to the spherical coordinate system to eliminate perspective distortion;

[0066] Gradient blur fusion: Apply transparency gradient and Gaussian blur to the stitching boundary area to eliminate the seams and obtain Figure 3 (b) shows the stitched image.

[0067] Based on the above method, this embodiment further proposes a single-camera panoramic image stitching system, such as Figure 5-6 As shown, the system consists of:

[0068] Hardware modules: single camera (wide-angle fisheye lens), RK3588 motherboard (core control unit), stepper motor (rotation axis aligned with the camera's optical center). It's understandable that the single-camera solution reduces costs by over 60% compared to a traditional four-camera system.

[0069] Software modules: camera calibration, image acquisition control, dedistortion processing, spherical coordinate system conversion, and stitching and fusion algorithm.

[0070] The specific implementation process is as follows:

[0071] 1. Hardware Setup: Install the fisheye camera on the stepper motor's rotating shaft through connectors, ensuring that the optical center and the shaft center coincide.

[0072] 2. Calibration process: Use a 12*9 checkerboard grid to capture 50-60 images at different angles and distances, and obtain the intrinsic parameter matrix K and distortion coefficient D through pinhole imaging model calibration;

[0073] 3. Image acquisition: Control the motor to rotate in 120° steps, triggering the camera to capture three original images;

[0074] 4. Software processing: Calculate the average brightness of the three images and dynamically adjust the local brightness to make the average brightness of the three images consistent; call the OpenCV dedistortion function to map the dedistorted image to a spherical coordinate system to eliminate perspective distortion; apply transparency gradient and Gaussian blur to the stitching boundary area to eliminate seams.

[0075] It can be understood that this embodiment avoids the problem of traditional feature extraction being difficult in low-texture scenes through the optical center alignment solution, and the stitching success rate is increased to 100%; the optical center alignment and spherical coordinate system conversion make the stitching error less than 0.5 pixels; and no manual intervention is required from acquisition to dedistortion to stitching, and the processing efficiency is increased by 40%; in addition, this embodiment uses a gradient blur algorithm to make the transition of the seam area natural, and the PSNR (peak signal-to-noise ratio) reaches more than 35dB.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A single-camera panoramic image stitching system, characterized in that: include: Hardware module: single camera, stepper motor; Software modules: camera calibration module, dedistortion processing module, spherical coordinate system conversion module, and stitching and fusion module; Controlling the single camera to rotate and shoot by the stepping motor to obtain the image to be stitched; Obtaining the intrinsic parameter matrix and distortion coefficient of the single camera through the camera calibration module; The images to be spliced are spliced into a complete image by using the dedistortion processing module, the spherical coordinate system conversion module and the splicing and fusion module respectively using image dedistortion, spherical coordinate system conversion and gradient fuzzy fusion methods.

2. The single-camera panoramic image stitching system according to claim 1, characterized in that: The hardware module further includes a connector connecting the single camera and the stepping motor, and the optical center of the camera is aligned with the rotation axis of the stepping motor through the connector.

3. The single-camera panoramic image stitching system according to claim 1, characterized in that: The software module also includes an image acquisition control module, which controls the motor to drive the camera to rotate 360 degrees and controls the camera to capture an image every time it rotates 120 degrees.

4. The single-camera panoramic image stitching system according to claim 1, characterized in that: The stitching and fusion module applies transparency gradient and Gaussian blur to the stitching boundary area of the dedistorted image to obtain the complete image.

5. A single-camera panoramic image stitching method, characterized in that: include: Use a 12*9 chessboard to capture several images at different angles and distances to obtain the camera's intrinsic parameter matrix and distortion coefficients. Use the optical center alignment method to adjust the camera to the position where the optical center coincides with the rotation axis; Drive the camera to rotate and take several images to be stitched covering 360°; The images to be spliced are spliced into a complete image by using image dedistortion, spherical coordinate system conversion and gradient fuzzy fusion methods.

6. The single-camera panoramic image stitching method according to claim 5, characterized in that: The process of adjusting the camera to a position where the optical center coincides with the rotation axis using the optical center alignment method includes: Place two wooden boards in the shooting environment. The foreground board is used as a block, and the background board is used for measurement. Adjust the camera installation position by capturing images before and after rotation until the parallax change is below the allowable range or there is no parallax. In this case, the camera is considered to be rotating around the optical center.

7. The single-camera panoramic image stitching method according to claim 6, characterized in that: In the process of adjusting the camera installation position by taking images before and after rotation, the rotation error is no more than 5mm.

8. The single-camera panoramic image stitching method according to claim 5, characterized in that: The camera is driven to rotate by an angle of 120°, and an image is captured every 120° of rotation.

9. The single-camera panoramic image stitching method according to claim 5, characterized in that: The process of stitching the images to be stitched into a complete image by using image dedistortion, spherical coordinate system conversion, and gradient fuzzy fusion methods includes: Calculating the average brightness of the images to be stitched and adjusting the brightness to be consistent; Performing dedistortion processing on the image to be stitched based on the intrinsic parameter matrix and the distortion coefficient to obtain a dedistorted image; Mapping the dedistorted image to a spherical coordinate system to eliminate perspective distortion; Transparency gradient and Gaussian blur are applied to the stitching boundary area of the dedistorted image to obtain the complete image.

Images