Splicing algorithm dense feature point matching management tool based on GIMP
By using a dense feature point detection algorithm and GIMP plugin in multi-camera panoramic video stitching, the problem of insufficient feature point detection in small overlapping areas of traditional algorithms is solved, achieving more comprehensive feature point matching and visual confirmation, and improving the stitching success rate.
Patent Information
- Application Number
- CN202511850936.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-10
- Publication Date
- 2026-03-20
AI Technical Summary
In multi-camera panoramic video stitching, traditional feature point detection algorithms cannot effectively detect feature points in finite overlapping areas, leading to stitching failure.
A dense feature point detection algorithm is used to set a grid on the overlapping area image, and feature points are detected for each grid block. The feature point data is then visualized and saved using the GIMP plugin.
It improves the coverage and reliability of feature point detection, reduces false detections, and achieves more efficient stitching results.
Smart Images
Figure CN121707822A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical fields of panoramic stitching digital image processing and Linux application programming, specifically to the application scenario of feature point matching and management in overlapping areas during multi-camera panoramic video stitching. It is applied to video / image stitching software tools to generate better stitching parameters and achieve better stitching results. Background Technology
[0002] Video / image stitching algorithms typically require calculating feature point matching between camera images to compute the homography matrix and estimating camera parameters to establish correspondences between the original images. Then, by selecting an image plane (e.g., a first image plane or an estimated intermediate plane), a projection plane is determined, and the transformation matrix between the image to be registered and the projection plane is calculated. Finally, by blending corresponding pixels in overlapping areas and preserving pixels in non-overlapping areas, the aligned images are fused onto a single projection plane, generating the corresponding stitching projection parameters.
[0003] Therefore, many image stitching applications suffer from small overlapping areas and parallax, making traditional stitching algorithms based on SIFT or ORB feature point detection methods unusable. To address this issue, this invention introduces a dense feature point detection algorithm and develops a GIMP-based dense feature point matching management tool for stitching algorithms. The main objective is to densely extract feature points within a limited overlapping image region. By setting a grid on the overlapping image and performing dense feature point detection calculations on each grid block, a more comprehensive feature description is obtained. This avoids the limitations of traditional SIFT and ORB feature point detection algorithms, which only detect features at local extrema, thus improving the coverage of feature point detection.
[0004] Since the relevant stitching parameter generation tool is developed in a Linux environment, the feature points detected by the dense feature point detection algorithm need to be stored in the stitching parameter database file. In order to facilitate the use of the stitching parameter generation tool, this invention is developed in C code using the GIMP plugin. GIMP is the GNU ImageManipulation Program, which is a powerful image processing tool under the Linux system. Summary of the Invention
[0005] The purpose of this invention is to provide a dense feature point matching management tool based on the GIMP stitching algorithm. It extracts feature points densely in a limited overlapping area of images. By setting a grid on the overlapping area image, dense feature point detection calculation is performed on each grid block, thereby obtaining a more comprehensive feature description. This avoids the problem that traditional feature point detection algorithms such as SIFT and ORB only detect features at local extreme points, thus improving the coverage of feature point detection.
[0006] The objective of this invention is achieved through the following technical solution:
[0007] A dense feature point matching management tool based on the GIMP stitching algorithm includes a GIMP dense feature point matching plugin, which loads the dense feature point detection and matching algorithm function into the run() function. The tool is characterized by the following steps in the dense feature point detection and matching algorithm:
[0008] Step 1: Construct Gaussian pyramids for the selected matching images at multiple scales, each scale corresponding to a different resolution;
[0009] Step 2: Obtain the specified matching region of the matching image, and set the grid at different resolutions of the matching region. If no grid is specified, set the grid globally. Within each grid region, calculate the gradient intensity and direction of the image, and perform histogram statistics based on the gradient direction to form an oriented gradient histogram.
[0010] Step 3: Find local maxima and minima on each directional gradient histogram as candidate key points;
[0011] Step 4: For each detected keypoint, calculate the gradient histogram within a small window around it to generate a descriptor vector containing the shape and texture information of the region.
[0012] Preferably, in step three, nonmaximum suppression is also performed to remove edge responses, and the quality of key points is confirmed by Hessian matrix detection.
[0013] Preferably, a dense feature point matching management tool based on the GIMP stitching algorithm completes the stitching in a visual manner, with the following steps:
[0014] The first step is to allow users to import a stitching database file containing the images to be stitched, image information, and image positional relationships in the interface;
[0015] The second step is to provide the user with an image pair in the interface where feature points between two images need to be calculated;
[0016] The third step is to allow users to specify the matching area for feature point calculation in the interface, or the default is to calculate the entire map area.
[0017] The fourth step is to display the calculated feature points on the interface, allowing the user to choose between "recalculate" or "incremental calculation";
[0018] The fifth step involves rendering the calculated feature points prominently on each pair of images. It is necessary to manually check whether the points are correct. If the detection is successful, the calculation results will be saved and written to the stitching database.
[0019] The sixth step is to stitch the images together using a stitching database containing feature point data and display the results on the interface.
[0020] The beneficial effects of this invention are as follows:
[0021] 1. A dense feature point matching management tool based on GIMP stitching algorithm solves the problem that traditional feature point detection algorithms cannot detect feature points 100% of the time in certain stitching application scenarios due to the small overlap area between images, thus causing stitching failure.
[0022] 2. This invention enables a visual operation for dense feature point detection. The feature points detected by the algorithm can be manually confirmed for acceptance, increasing the reliability of feature point matching and improving the efficiency of successful splicing.
[0023] 3. This invention realizes the functions of feature point detection, reading, saving and displaying of the stitching parameter database in GIMP, achieves seamless connection with the stitching parameter generation tool, and optimizes the operation process of the stitching tool. Attached Figure Description
[0024] Figure 1 This is an example of importing and stitching an image database in this invention.
[0025] Figure 2 This is an example of the interface for selecting image pairs to be matched in this invention.
[0026] Figure 3 This is an example of an image pair to be matched in this invention.
[0027] Figure 4 This is an example of calculating dense feature points in a specified region in this invention.
[0028] Figure 5 This is an example of "recalculation" or "incremental calculation" in this invention.
[0029] Figure 6 This is an example of the panoramic stitching result based on the detected dense feature points in this invention. Detailed Implementation
[0030] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.
[0031] To address the uncertainty in feature point detection in low-overlap regions, the stitching and fusion algorithm is first improved: Camera intrinsic and extrinsic parameters are pre-calibrated, and these calibration parameters are used to initially establish relationships between cameras, thus determining the overlap areas between each pair of cameras. Then, a dense feature point algorithm is used for feature point extraction and matching in horizontal / vertical / arbitrary directional overlap areas. This algorithm uses a grid within the overlap area and calculates dense feature points for each grid block, avoiding the limitations of traditional algorithms like SIFT which only detect features at local extrema. This improves the coverage of feature point detection and solves the uncertainty caused by small overlap areas. Of course, the accuracy of the detected feature points will be relatively lower. Manual verification can confirm the reliability and matching of automatically detected feature points. If reliable, the algorithm accepts and saves the points; otherwise, it discards them and recalculates. This approach retains the deterministic advantages of dense feature point detection while avoiding the possibility of false detections.
[0032] To address the aforementioned stitching and fusion algorithm process, this embodiment provides a dense feature point matching management tool based on GIMP, including a GIMP dense feature point matching plugin. The GIMP dense feature point matching plugin is constructed according to the GIMP developer manual, and the dense feature point detection and matching algorithm function is loaded into the run() function. The steps of the dense feature point detection and matching algorithm are as follows:
[0033] Step 1: Scale-space extremum detection. First, Gaussian pyramids are constructed on the selected matching images at multiple scales, each corresponding to a different resolution. This ensures that feature points can be detected at different scaling levels.
[0034] Step 2: Gradient calculation. Obtain the specified matching region of the matching image. Set the grid at different resolutions for the matching region (if not specified, set the grid globally). Within each grid region, calculate the gradient intensity and direction of the image. Then, perform histogram statistics based on the gradient direction to form an oriented gradient histogram.
[0035] Step 3: Key point localization. Local maxima and minima are found on the gradient histogram of each orientation. These points are candidate key points. To ensure stability, nonmaxima suppression is performed to remove edge responses, and the quality of key points is confirmed by Hessian matrix detection.
[0036] Step 4: Feature Descriptor Generation. For each detected keypoint, a gradient histogram is calculated within a small window surrounding it, generating a descriptor vector. This descriptor vector contains shape and texture information of the region, helping to distinguish different feature points. This descriptor vector possesses scale invariance, rotation invariance, and brightness invariance properties. Scale invariance is achieved by detecting feature points at different scales; rotation invariance is achieved by calculating gradient histograms in various directions; and brightness invariance is because the descriptor is insensitive to local brightness changes.
[0037] The GIMP-based dense feature point matching management tool for stitching algorithms allows users to import stitched image databases, select image pairs to be matched, select matching regions, detect dense feature points, and save feature point data in a visual manner, as detailed below.
[0038] Step 1: Import the stitched image database: Open GIMP software and enable the Autostitch dense feature point management tool plugin, such as... Figure 1 As shown.
[0039] Under "File", click the "Open Stitching Database" button, and then select the stitching database file (*.db format) generated by the stitching tool, which contains external parameter information such as the images to be stitched, image information, and image positional relationships.
[0040] The second step is to select the image pair to be matched: After opening the stitching database, click "Match" to enter the feature point interface, such as... Figure 2 As shown, the area marked by the red box at the bottom allows for image selection.
[0041] Select the image pair from which you want to calculate feature points between two images. You can select any pair of images for calculation. After selection, the selected images will be displayed side-by-side in the GIMP window, such as... Figure 3 As shown. If images with matching feature points already exist in the database, the tool will display "Matched Images," which are images that match feature points in the database.
[0042] Third step: Select the matching region: Figure 3 Of the four buttons in the lower right corner of the interface, clicking the red button will calculate the matching area of the selected image. By default, the calculation is performed across the entire image.
[0043] Alternatively, you can specify a region to calculate dense feature points. In the GIMP toolbar, select the "Path" tool, and then manually mark a closed polygonal region on the left side of the image. Subsequent dense feature point calculations will only be performed within the selected region. Figure 4 As shown in the red box at the top left.
[0044] Step 4, Dense Feature Point Detection: After confirming the dense feature point detection area, click... Figure 4 The second button in the bottom right corner starts the dense feature point detection algorithm.
[0045] Feature points between two images can be "recalculated" or "incrementally calculated." When the option is "recalculated," the tool will prompt for confirmation. You can select the "incremental calculation" option to perform incremental calculation.
[0046] Step 5: Save feature point data: The feature points calculated by the dense feature point matching management tool will be rendered as "highlighted red dots" on the pairs of images. You need to manually check if they are correct. If there are no problems with the detection, you can choose to accept the detection result; otherwise, you can discard the detection result. If accepted, the calculation result will be saved and written to the stitching database.
[0047] Step 6: Use a stitching database containing feature point data to stitch the images together, the result of which is shown below. Figure 6 As shown, the detected feature points can achieve good stitching results.
[0048] The stitching scenarios to which this invention is applied have the following characteristics: the overlap area between camera images is small, which results in traditional feature point detection algorithms detecting too few matching feature points or failing to detect any feature points, thus failing to accurately calculate the transformation between images, leading to stitching errors.
[0049] Therefore, to address the difficulties brought about by such splicing scenarios, this invention designs a dense feature point matching management tool based on the GIMP splicing algorithm to provide a targeted solution:
[0050] 1) The dense feature point algorithm solves the problem of reliable feature point extraction.
[0051] The dense feature point detection algorithm avoids the limitation of traditional algorithms such as SIFT feature point detection, which only detect features at local extrema, thus improving the coverage of feature point detection and solving the detection uncertainty problem caused by small overlapping areas. This allows feature points to be extracted even in low-overlap areas in the application scenarios of this invention. While the dense feature point detection algorithm increases the algorithm's complexity, the increased complexity and computation time are not a concern because feature point matching and concatenation parameter generation in the concatenation algorithm are offline steps without real-time requirements.
[0052] 2) Visual confirmation solves the problem of false detection.
[0053] Dense feature point detection algorithms, by setting up a grid within the overlapping area and calculating dense feature points for each grid block, have a relatively lower accuracy rate and are prone to false detections. By displaying the algorithm-detected feature points as colored dots in GIMP and then manually verifying them, we can confirm whether the automatically detected feature points are a match and reliable. If reliable, they are accepted and saved; if unreliable, they can be discarded and recalculated. This approach retains the deterministic detection advantage of dense feature point detection algorithms while avoiding the possibility of false detections.
[0054] It is understood that those skilled in the art can make equivalent substitutions or modifications to the technical solution and inventive concept of the present invention, and all such substitutions or modifications should fall within the protection scope of the appended claims.
Claims
1. A dense feature point matching management tool based on GIMP stitching algorithm, comprising a GIMP dense feature point matching plugin, wherein the dense feature point detection and matching algorithm function is loaded into the run() function, characterized in that... The steps of the dense feature point detection and matching algorithm are as follows: Step 1: Construct Gaussian pyramids for the selected matching images at multiple scales, each scale corresponding to a different resolution; Step 2: Obtain the specified matching region of the matching image, and set the grid at different resolutions of the matching region. If no grid is specified, set the grid globally. Within each grid region, calculate the gradient intensity and direction of the image, and perform histogram statistics based on the gradient direction to form an oriented gradient histogram. Step 3: Find local maxima and minima on each directional gradient histogram as candidate key points; Step 4: For each detected keypoint, calculate the gradient histogram within a small window around it to generate a descriptor vector containing the shape and texture information of the region.
2. The dense feature point matching management tool based on GIMP stitching algorithm according to claim 1, characterized in that... In step three, nonmaximum suppression is performed to remove edge responses, and the quality of key points is confirmed by Hessian matrix detection.
3. The dense feature point matching management tool based on GIMP stitching algorithm according to claim 1, characterized in that... To complete the stitching visually, follow these steps: The first step is to allow users to import a stitching database file containing the images to be stitched, image information, and image positional relationships in the interface; The second step is to provide the user with an image pair in the interface where feature points between two images need to be calculated; The third step is to allow users to specify the matching area for feature point calculation in the interface, or the default is to calculate the entire map area. The fourth step is to display the calculated feature points on the interface, allowing the user to choose between "recalculate" or "incremental calculation"; The fifth step involves rendering the calculated feature points prominently on each pair of images. It is necessary to manually check whether the points are correct. If the detection is successful, the calculation results will be saved and written to the stitching database. The sixth step is to stitch the images together using a stitching database containing feature point data and display the results on the interface.