Weak light enhancement and color temperature correction panoramic image splicing system and method

Through the combination of multi-threaded control and image processing modules, panoramic image stitching under low light conditions is achieved, the problem of inconsistent image brightness and color temperature is solved, image quality and stitching effect are improved, and it is suitable for multi-camera field of view stitching scenes.

CN118967434AActive Publication Date: 2024-11-15SHENYANG FIRE RES INST OF MEM

Patent Information

Application Number
CN202410979901.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-11-15
Estimated Expiration
2044-07-22

AI Technical Summary

Technical Problem

Under low light conditions, the panoramic image quality of multi-camera field of view stitching is affected, inconsistent image brightness and mismatch in color temperature lead to low contrast and information loss, affecting the stitching effect.

Method used

Using multi-threaded control module, video reading and image correction module, parameter initialization module, image projection module, optimal suture search module and image fusion module, through low-light image enhancement, color temperature correction, camera internal reference calibration, key point detection and Multi-band fusion achieves clear and natural stitching of images.

Benefits of technology

Obtaining clear and natural panoramic images under low light conditions improves image processing speed and quality, ensuring that the stitched panoramic images have no obvious seams, and enhancing the visual experience and practical application.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967434A_ABST
    Figure CN118967434A_ABST
Patent Text Reader

Abstract

The invention provides a panoramic image splicing system and method for weak light enhancement and color temperature correction. The system mainly comprises a multi-thread control module which is used for integrally controlling the execution sequence of each module; the video reading and image correction module is responsible for reading four paths of camera images and carrying out image correction operation; the parameter initialization module is used for processing the images, carrying out key point detection and matching, calculating a rotation matrix among the images and estimating camera parameters; the image projection module is used for projecting each image to a cylindrical surface according to the rotation matrix and camera parameters; the optimal suture line searching module is used for calculating the optimal suture line of each overlapping region; the image fusion module is used for fusing the images according to the optimal suture line and a multi-band fusion method to obtain a panoramic image; and the panoramic image display module is responsible for displaying the panoramic image. According to the system, efficient and accurate panoramic image stitching can be realized, the problems of weak light and color temperature are effectively solved, and the image impression is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and more specifically relates to a panoramic image stitching system and method for weak light enhancement and color temperature correction. Background Art

[0002] In recent years, with the development of high-tech, robotics has made rapid progress. It is not only widely used in military defense construction, medical treatment and industrial development, but also gradually enters daily life. As an important member of intelligent robots, the biggest advantage of mobile robots is that they can move freely and have corresponding intelligent response capabilities. Common examples include industrial AGVs, drones, humanoid robots and sweeping robots. However, in some application scenarios, the field of view of a single camera is obviously not enough, and the field of view of multiple cameras is chaotic. Therefore, a technology is needed to stitch the fields of view of multiple cameras into a wider field of view.

[0003] Panoramic image stitching technology can obtain panoramic images of the surrounding environment, breaking through the limitations of a single camera's viewing angle, ensuring that both high resolution and a large field of view are met. Panoramic vision is widely used in many fields. For example, the panoramic vision system on private cars can monitor the surrounding environment to ensure driving safety; panoramic video surveillance in large supermarkets can cover all areas; and panoramic vision systems on mobile robots can be used in special environments such as post-fire investigations. These applications obtain comprehensive environmental information through panoramic vision, which can be used for three-dimensional reconstruction, security monitoring, and environmental detection.

[0004] In some special environments, such as the narrow collapsed space formed at the fire scene, the wire shaft, the elevator shaft, etc., the dim light and narrow space often lead to poor image quality, low brightness, inconsistent color temperature, low contrast and information loss. The images captured by each camera have different brightness and color temperature, which not only affects the subjective visual experience, but also limits the effect of panoramic image stitching. For these special scenes, how to use image processing technology to improve the visibility and color consistency of images under low light conditions, so that the stitched panoramic images are clearer and more natural, has become an urgent problem to be solved. Summary of the invention

[0005] The present invention provides a method for stitching panoramic videos under different color temperatures under low light conditions, which solves the problems of image correction, image registration, image projection and image fusion respectively by different functional modules. Experiments have shown that this method can obtain clear and natural panoramic images under the condition of inconsistent color temperature in low light, and has strong versatility and extensibility.

[0006] In order to achieve the above object, the present invention is implemented by adopting the following technical scheme: the stitching system includes a multi-thread control module, a video reading and image correction module, a parameter initialization module, an image projection module, an optimal stitching line search module, an image fusion module,

[0007] Panoramic image display module;

[0008] The multi-thread control module is responsible for overall control of the execution order of each module;

[0009] The video reading and image correction module is responsible for reading the images from four cameras and performing image correction operations;

[0010] The parameter initialization module reads the processed images, performs key point detection and matching, and calculates the rotation matrix between images and estimates the camera parameters;

[0011] The image projection module projects each image onto a cylindrical surface according to the rotation matrix and camera parameters;

[0012] The best seam line finding module calculates the best seam line for each overlapping area;

[0013] The image fusion module fuses the images according to the optimal stitching line and multi-band fusion method to obtain a panoramic image, which is displayed by the panoramic image display module.

[0014] In one solution, the multi-thread control module loads the initial parameters required by each module, loads the required functions of the open source computer vision library OpenCV, and establishes a thread pool;

[0015] Add modules that require repeated operations, such as video reading and image correction module, image projection module, optimal seam line search module, image fusion module, and panoramic image display module, to the thread pool;

[0016] Then open all threads in the thread pool to enable all modules to operate in parallel;

[0017] The multi-threaded control module is also responsible for regulating and connecting various modules, such as saving the parameters calculated by the parameter initialization module and providing them to the image projection module. When each camera has one image saved by the program, the control module notifies the image projection module to start working.

[0018] In one scenario, the video reads:

[0019] Firstly, low-light image enhancement is performed on each input image. The corresponding illumination weight matrix is ​​obtained according to the input image to guide the adaptive initialization of the illumination component. Then, under the constraint of structured illumination, the initial illumination component is optimized and smoothed by an equivalent objective function, and further nonlinear illumination adjustment is performed on it. Finally, the low-light image enhancement is achieved by combining the Retinex theory.

[0020] Then, we randomly select an image as the reference image, calculate the average color temperature of the reference image, and apply it to other images to ensure that the color temperature of all images is consistent.

[0021] The image correction uses MATLAB's auxiliary toolbox cameraCalibrator to perform camera intrinsic parameter calibration and distortion parameter calculation;

[0022] First, use Opencv to write code to use buttons to take pictures of the checkerboard calibration plate at different angles, and take 20 to 30 pictures;

[0023] Then import the image into MATLAB's cameraCalibrator toolbox, calculate the camera's distortion parameters and intrinsic parameter matrix through the cameraCalibrator toolbox, pass the camera intrinsic parameter matrix and distortion parameters calculated by the cameraCalibrator toolbox to OpenCV's remap() function to obtain the dedistorted image.

[0024] 10. A panoramic image stitching system with weak light enhancement and color temperature correction according to claim 1, characterized in that: the parameter initialization module calculates the rotation matrix between the corrected images and estimates the focal length of the camera to provide parameters for the subsequent image projection module;

[0025] First, detect the key points of each image, use OpenCV's SURF detector to detect the key point locations and assign feature descriptors;

[0026] The detected image is divided into 4 parts, and only the key points of the leftmost and rightmost quarter areas are detected;

[0027] After obtaining the position and feature descriptors of the key points, the key points of each image are matched through the 2NN algorithm to obtain a set of roughly registered matching point pairs.

[0028] The RANSAC algorithm is used to obtain the internal points of the set of matching point pairs. The homography matrix is ​​obtained based on these internal points using the least squares method. The rotation matrix between images is obtained based on the homography matrix, thereby obtaining the external parameters of the camera.

[0029] In one embodiment, the image projection module uses the following image projection model: Assume that P (X, Y, Z) is a point in the world coordinate system. According to the arc length formula, the horizontal coordinate of the projection point p (u, v) of point P on the cylinder is: The vertical axis is:

[0030] In one embodiment, the optimal suture line finding module adopts an improved optimal suture line algorithm based on dynamic programming; the improvement is as follows:

[0031] The intensity value difference of the energy function in the optimal seam line algorithm based on dynamic programming is changed to be obtained from the HSI color space. The object edges in the overlapping area are extracted so that the seam line can avoid the object edges to a certain extent. The SSIM index of the pixels in the overlapping area is calculated and added to the energy function, expanding the search range when dynamic programming is looking for the next node.

[0032] In one solution, the image fusion module uses a multi-band fusion method to fuse images to obtain a more natural panoramic image, and then stores the stitched panoramic image in a container.

[0033] In one solution, the panoramic image display module opens a separate thread to continuously read the container storing the panoramic image. Whenever a picture is read, the picture is displayed through the imshow() function of OpenCV and the picture is erased from the container to prevent memory overflow.

[0034] In another aspect, a panoramic image stitching method with weak light enhancement and color temperature correction is provided, and the panoramic image stitching method is applicable to the panoramic image stitching system;

[0035] S1. Run the video reading and image correction module to read the original images from the four cameras.

[0036] Perform low-light enhancement and color temperature correction operations on low-light and color temperature inconsistent images, dedistort the images according to the camera parameters calculated by MATLAB's cameraCalibrator toolbox, and save the processed four-channel images into a vector container for use by other modules;

[0037] S2, running parameter initialization module, using OpenCV's SURF detector to detect the position of key points of each adjacent image and assign feature descriptors;

[0038] Use the next nearest neighbor algorithm + RANSAC algorithm to match the key points between images, and use the least squares method to obtain the homography matrix based on these key points. Based on the homography matrix, the rotation matrix between images is obtained, thereby obtaining the external parameters of the camera;

[0039] S3, running the image projection module, selecting an image as a reference image according to the rotation matrix and cylindrical projection model between the images, and projecting the other images onto the same cylinder consistent with the reference image;

[0040] S4, running the best stitching line search module and the image fusion module, converting each projected image into the HIS color space, calculating the energy matrix of the overlapping area of ​​each adjacent image based on the new energy function, finding the best stitching line, and using the multi-band fusion method to fuse the images to eliminate the obvious stitching marks around the stitching seams;

[0041] S5. Run the panoramic image display module to display the final panoramic image.

[0042] Beneficial effects of the present invention:

[0043] The present invention can effectively realize the parallel operation of each module through a multi-threaded control module, thereby improving the image processing speed. At the same time, in view of the problems of weak light and inconsistent color temperature, the clarity and continuity of the image are guaranteed through weak light enhancement and color temperature correction. In addition, the rotation matrix between each image and the estimated camera parameters are calculated through the parameter initialization module, so as to accurately realize the projection of the image on the cylinder; the combination of the optimal seam line search module and the image fusion module makes the panoramic image have no obvious seam lines, thereby improving the overall aesthetics of the image. Moreover, when processing large data panoramic images, the panoramic image display module can prevent memory overflow and improve the stability of the panoramic image display. Therefore, the present invention can realize high-quality, real-time panoramic image stitching and display, and has high practical value and practical prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a system block diagram of the present invention;

[0045] Figure 2 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0046] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Typical embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in the present invention. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0047] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0048] like Figure 1 As shown, the present invention provides a panoramic image stitching system with weak light enhancement and color temperature correction. The camera type selected is a USB industrial camera with a viewing angle of 130 degrees. The camera can take pictures with a resolution of 1280*720. Four cameras of the same type are arranged in a circle at 90 degrees. The system reads the image data of four cameras through the serial port based on the OpenCV library function.

[0049] The stitching system includes a multi-threaded control module, a video reading and image correction module, a parameter initialization module, an image projection module, an optimal stitching line search module, an image fusion module, and a panoramic image display module; the relationship between the modules is as follows: the multi-threaded control module is responsible for overall control of the execution order of the modules. The video reading and image correction module is responsible for reading the images of the four cameras and performing image correction operations. The parameter initialization module reads the processed images, performs key point detection and matching, and calculates the rotation matrix between the images and estimates the camera parameters. The image projection module projects each image onto a cylindrical surface according to the rotation matrix and camera parameters. The optimal stitching line search module calculates the optimal stitching line of each overlapping area, and the image fusion module fuses the images according to the optimal stitching line and the multi-band fusion method to obtain a panoramic image, which is displayed by the panoramic image display module.

[0050] Multi-threaded control module: In order to speed up the operation, the present invention uses multi-threading to run the panoramic image stitching modules. The main idea is to establish a thread pool, add modules that need to be repeatedly operated, such as video reading and image correction module, image projection module, optimal stitching line search module, image fusion module and panoramic image display module, to the thread pool, and then open all threads in the thread pool to enable all modules to operate in parallel. In addition, the control module is also responsible for regulating and connecting the modules, such as saving the parameters calculated by the parameter initialization module and providing them to the image projection module. For example, when each camera is saved by the program, the control module notifies the image projection module to start working.

[0051] Video reading and image correction module: The system establishes four camera threads to continuously read the camera's pictures. First, each input image is enhanced in low light, and the corresponding illumination weight matrix is ​​obtained according to the input image, aiming to guide the adaptive initialization of the illumination component, and then under the constraint of structured illumination, the initial illumination component is optimized and smoothed by the equivalent objective function, and further nonlinear illumination adjustment is performed on it, and finally the enhancement of low light images is achieved by combining Retinex theory. After that, an image is randomly selected as a reference image, and the average color temperature of the reference image is calculated and applied to other images to ensure that the color temperature of all images is consistent. Since the lenses of the cameras used in the present invention are all ultra-wide-angle cameras, there are problems of radial distortion and tangential distortion, and generally speaking, the influence of tangential distortion is small. The pictures taken with the camera have obvious barrel distortion, and the pictures just taken need to be corrected. The present invention uses MATLAB's auxiliary toolbox cameraCalibrator to perform camera internal parameter calibration and distortion parameter calculation. First, Opencv is used to write code for using buttons to shoot pictures of the checkerboard calibration plate at different angles, and about 20 to 30 pictures are taken. Then import the image into MATLAB's cameraCalibrator toolbox, calculate the camera's distortion parameters and intrinsic parameter matrix through the cameraCalibrator toolbox, pass the camera intrinsic parameter matrix and distortion parameters calculated by the cameraCalibrator toolbox to OpenCV's remap() function to obtain the dedistorted image.

[0052] The parameter initialization module mainly calculates the rotation matrix between each corrected image and estimates the focal length of the camera, and provides parameters for the subsequent image projection module. First, the key points of each picture are detected. The present invention uses the SURF detector of OpenCV to detect the key point position and assign feature descriptors. Since only a small part of the area between the images overlaps, it is not necessary to detect the key points in the whole image. The present invention divides the image into 4 parts and only detects the key points of the leftmost and rightmost quarter areas. After obtaining the position and feature descriptors of the key points, the key points between the images are matched by the 2NN, i.e., the next nearest neighbor algorithm, after obtaining the key points of each image, a set of matching point pairs of coarse registration is obtained, and then the inliers of the matching point pair set are obtained by the RANSAC algorithm, and the homography matrix is ​​obtained by the least squares method according to these inliers, and the rotation matrix between the images is obtained according to the homography matrix, so as to obtain the external parameters of the camera.

[0053] Image projection module: The present invention arranges four USB industrial cameras at 90 degrees. Since the images taken by two cameras have large visual differences, an image projection algorithm is required to project each image onto the same plane. The image projection model used in the present invention is as follows: Let P (X, Y, Z) be a point in the world coordinate system. According to the arc length formula, the horizontal coordinate of the projection point p (u, v) of point P on the cylinder is: The vertical axis is:

[0054] Optimal seam line search module: The fusion results of the traditional optimal seam line algorithm based on dynamic programming will have the problem of object disappearance and dislocation. This is because the adjacent lens parallax is large, resulting in different object positions. The present invention has improved the traditional method. The main idea of ​​the improved method is: the intensity value difference in the energy function is changed to be obtained from the HSI color space, and the edge of the object in the overlapping area is extracted so that the seam line can avoid the edge of the object to a certain extent. The SSIM index of the pixel points in the overlapping area is calculated and added to the energy function, expanding the search range when the dynamic programming is looking for the next node. Compared with the traditional algorithm, the improved optimal seam image fusion algorithm of the present invention is more natural and greatly reduces the phenomenon of object dislocation and disappearance.

[0055] Image fusion module: Directly stitching the images together through masks will result in obvious stitching seams. The multi-band fusion method is used to fuse the images to obtain a more natural panoramic image. The stitched panoramic image is then stored in a container.

[0056] Panoramic image display module: Open a separate thread to continuously read the container that stores panoramic images. Whenever a picture is read, it is displayed through the imshow() function of OpenCV and the picture is erased from the container to prevent memory overflow.

[0057] like Figure 2 As shown, a panoramic image stitching method with weak light enhancement and color temperature correction is shown.

[0058] S1, run the video reading and image correction module to read the original images from the four cameras, such as Figure 2 As shown. Perform low-light enhancement and color temperature correction operations on low-light and color temperature inconsistent images. Perform dedistortion operations on images based on camera parameters calculated by MATLAB's cameraCalibrator toolbox. Save the processed four-channel images into a vector container for use by other modules.

[0059] S2. Run the parameter initialization module. Use OpenCV's SURF detector to detect the key point positions of each adjacent image and assign feature descriptors. Use the next nearest neighbor algorithm + RANSAC algorithm to match the key points between images, and use the least squares method to obtain the homography matrix based on these key points. Based on the homography matrix, the rotation matrix between images is obtained, thereby obtaining the camera's external parameters.

[0060] S3, running the image projection module. According to the rotation matrix and cylindrical projection model between the images, one image is selected as the reference image, and the other images are projected onto the same cylinder consistent with the reference image.

[0061] S4. Run the best stitching line search module and image fusion module. Convert each projected image to the HIS color space, calculate the energy matrix of the overlapping area of ​​each adjacent image based on the new energy function, find the best stitching line, and use the multi-band fusion method to fuse the images to eliminate the obvious stitching marks around the stitching seams.

[0062] S5. Run the panoramic image display module to display the final panoramic image.

[0063] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0064] It should be understood that the detailed description of the technical solutions of the present invention by means of the preferred embodiments is illustrative rather than restrictive. A person skilled in the art may modify the technical solutions described in the embodiments, or replace some of the technical features by equivalents, based on reading the specification of the present invention; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A panoramic image stitching system with low-light enhancement and color temperature correction, characterized by: The stitching system includes a multi-thread control module, a video reading and image correction module, a parameter initialization module, an image projection module, an optimal stitching line search module, an image fusion module, and a panoramic image display module; The multi-thread control module is responsible for overall control of the execution order of each module; The video reading and image correction module is responsible for reading the images from four cameras and performing image correction operations; The parameter initialization module reads the processed images, performs key point detection and matching, and calculates the rotation matrix between images and estimates the camera parameters; The image projection module projects each image onto a cylindrical surface according to the rotation matrix and camera parameters; The best seam line finding module calculates the best seam line for each overlapping area; The image fusion module fuses the images according to the optimal stitching line and multi-band fusion method to obtain a panoramic image, which is displayed by the panoramic image display module.

2. The panoramic image stitching system with weak light enhancement and color temperature correction according to claim 1, characterized in that: The multi-thread control module loads the initial parameters of each module, loads the functions of the open source computer vision library OpenCV, and establishes a thread pool; Add modules that need to be repeatedly operated, such as video reading and image correction module, image projection module, optimal seam line search module, image fusion module, and panoramic image display module, to the thread pool; Then open all threads in the thread pool to enable all modules to operate in parallel; The multi-threaded control module is also responsible for regulating and connecting various modules, such as saving the parameters calculated by the parameter initialization module and providing them to the image projection module. When each camera has one image saved by the program, the control module notifies the image projection module to start working.

3. The panoramic image stitching system with weak light enhancement and color temperature correction according to claim 1, characterized in that: The video reading method first performs low-light image enhancement on each input image, obtains the corresponding illumination weight matrix according to the input image, and aims to guide the adaptive initialization of the illumination component. Then, under the constraint of structured illumination, the initial illumination component is optimized and smoothed by an equivalent objective function, and further nonlinear illumination adjustment is performed on it, and finally the low-light image enhancement is realized in combination with the Retinex theory. Then, we randomly select an image as the reference image, calculate the average color temperature of the reference image, and apply it to other images to ensure that the color temperature of all images is consistent. The image correction uses MATLAB's auxiliary toolbox cameraCalibrator to perform camera intrinsic parameter calibration and distortion parameter calculation; First, use Opencv to write code to use buttons to take pictures of the checkerboard calibration plate at different angles, and take 20 to 30 pictures; Then import the image into MATLAB's cameraCalibrator toolbox, calculate the camera's distortion parameters and intrinsic parameter matrix through the cameraCalibrator toolbox, pass the camera intrinsic parameter matrix and distortion parameters calculated by the cameraCalibrator toolbox to OpenCV's remap() function to obtain the dedistorted image.

4. The panoramic image stitching system with weak light enhancement and color temperature correction according to claim 1, characterized in that: The parameter initialization module calculates the rotation matrix between the corrected images and estimates the focal length of the camera to provide parameters for the subsequent image projection module; First, detect the key points of each image, use OpenCV's SURF detector to detect the key point locations and assign feature descriptors; The detected image is divided into 4 parts, and only the key points of the leftmost and rightmost quarter areas are detected; After obtaining the position and feature descriptors of the key points, the key points of each image are matched through the 2NN algorithm to obtain a set of roughly registered matching point pairs. The RANSAC algorithm is used to obtain the internal points of the set of matching point pairs. The homography matrix is ​​obtained based on these internal points using the least squares method. The rotation matrix between images is obtained based on the homography matrix, thereby obtaining the external parameters of the camera.

5. The panoramic image stitching system with weak light enhancement and color temperature correction according to claim 1, characterized in that: The image projection module uses the following image projection model: Assume that P (X, Y, Z) is a point in the world coordinate system. According to the arc length formula, the horizontal coordinate of the projection point p (u, v) of point P on the cylinder is: The vertical axis is:

6. The panoramic image stitching system with weak light enhancement and color temperature correction according to claim 1, characterized in that: The optimal suture line finding module adopts an improved optimal suture line algorithm based on dynamic programming; the improvements are as follows: The intensity value difference of the energy function in the optimal seam line algorithm based on dynamic programming is changed to be obtained from the HSI color space. By extracting the object edges in the overlapping area, the SSIM index of the pixels in the overlapping area is calculated and added to the energy function, thereby expanding the search range when dynamic programming is looking for the next node.

7. The panoramic image stitching system with weak light enhancement and color temperature correction according to claim 1, characterized in that: The image fusion module uses a multi-band fusion method to fuse images to obtain a more natural panoramic image, and then stores the spliced ​​panoramic image in a container.

8. The panoramic image stitching system with weak light enhancement and color temperature correction according to claim 1, characterized in that: The panoramic image display module opens a separate thread to continuously read the container storing the panoramic image. Whenever a picture is read, the picture is displayed through the imshow() function of OpenCV and the picture is erased from the container to prevent memory overflow.

9. A panoramic image stitching method for low-light enhancement and color temperature correction, the panoramic image stitching method being applicable to the panoramic image stitching system according to any one of claims 1 to 8, characterized in that: S1. Run the video reading and image correction module to read the original images from the four cameras. Perform low-light enhancement and color temperature correction operations on low-light and color temperature inconsistent images, dedistort the images according to the camera parameters calculated by MATLAB's cameraCalibrator toolbox, and save the processed four-channel images into a vector container for use by other modules; S2, running parameter initialization module, using OpenCV's SURF detector to detect the position of key points of each adjacent image and assign feature descriptors; Use the next nearest neighbor algorithm + RANSAC algorithm to match the key points between images, and use the least squares method to obtain the homography matrix based on these key points. Based on the homography matrix, the rotation matrix between images is obtained, thereby obtaining the external parameters of the camera; S3, running the image projection module, selecting an image as a reference image according to the rotation matrix and cylindrical projection model between the images, and projecting the other images onto the same cylinder consistent with the reference image; S4, running the best stitching line search module and the image fusion module, converting each projected image into the HIS color space, calculating the energy matrix of the overlapping area of ​​each adjacent image based on the new energy function, finding the best stitching line, and using the multi-band fusion method to fuse the images to eliminate the obvious stitching marks around the stitching seams; S5. Run the panoramic image display module to display the final panoramic image.

Citation Information

Patent Citations

  • Multichannel real-time video splicing processing system

    CN103856727A

  • Panoramic video mosaicing system and method based on camera automatic calibration and color correction

    CN105931186A

  • Panoramic image stitching method

    CN108510445A

  • Panoramic image splicing method based on optimal suture line

    CN112991176A

  • Video fusion algorithm based on dynamic optimal suture line and improved fade-in and fade-out method

    CN113221665A

Cited By

  • Panoramic image generation method and device based on multiple threads, and chip

    CN119383453A

  • Multithread-based panoramic image generation method, device and chip

    CN119383453B

  • Multi-thread-based panoramic image generation method and apparatus, and chip

    WO2026130006A1