A lightweight real-time image stitching method and system based on fixed overlapping area constraint

Abstract

The present application relates to the field of image processing and computer vision, and more particularly to a lightweight real-time image stitching method and system based on fixed overlap area constraint, comprising the following steps: step S1, synchronously scanning and collecting fixed overlap ratio sequence images through a time sequence control unit; step S2, determining the overlap ROI according to the overlap ratio, extracting features and matching only in the overlap ROI area after downsampling, and estimating the affine transformation parameters containing 6 degrees of freedom by using RANSAC; step S3, determining the stitching seam, performing multi-band fusion only in the preset band-shaped area on both sides of the seam by using low-bit width integer data, and directly copying pixels in the non-overlapping area; and step S4, outputting the stitched panoramic image. The present application greatly reduces the operation cost, adapts to the edge end scene, realizes a processing performance close to 1fps, the geometric registration error is stably less than 2-3 pixels, and the stitching seam is smooth without ghosting distortion.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a lightweight real-time image stitching method and system based on fixed overlapping region constraints. Background Technology

[0002] Image stitching technology aims to accurately register and fuse multiple input images with overlapping areas to generate panoramic images with wide field of view, high resolution, and visual continuity. With the development of modern high-throughput imaging and computer vision technologies, this technology has been widely applied in typical scenarios such as remote sensing satellite ground simulation verification, industrial visual inspection, UAV aerial image stitching, and panoramic monitoring, all geared towards sweeping imaging or sequential image acquisition. Especially in high-speed image stitching engineering applications such as wide-swath imaging ground verification systems, the system not only requires the panoramic image to completely preserve the topological structure and key details of the original scene, but also imposes extremely stringent requirements on the sub-second real-time processing performance of massive high-resolution image sequences.

[0003] Currently, common image stitching techniques are mainly divided into traditional methods based on handcrafted features and end-to-end learnable frameworks based on deep learning. Traditional feature stitching methods typically follow a classic three-stage processing flow: feature detection and matching, geometric transformation estimation, and image fusion. The core of this approach uses SIFT or ORB algorithms to extract features and RANSAC algorithms to solve for the global homography transformation matrix. Finally, a multi-band fusion method using a global Gaussian pyramid and Laplacian pyramid is employed to achieve seamless fusion of overlapping regions. Deep learning-based methods, on the other hand, automatically learn feature representations by constructing deep networks specifically for homography matrix estimation or by using the Transformer self-attention mechanism to model long-range dependencies. Furthermore, existing multi-camera stitching methods often rely on external hardware registration schemes based on calibration boards, calculating the transformation matrix between multiple cameras offline by pre-capturing images of the calibration board.

[0004] The following shortcomings were found when implementing the existing technology: 1. Existing technologies do not have a spatial reduction and logical cropping processing structure that utilizes the geometric prior of the fixed overlapping area of ​​adjacent frames in the imaging system. Therefore, when processing images, the algorithm is usually regarded as a general global stitching problem. It needs to blindly perform feature detection, full-image traversal matching, and dense global image pyramid fusion reconstruction over a very large area of ​​the entire full-resolution image, resulting in huge redundant floating-point operations and dense spatial computation overhead. 2. Existing technologies suffer from severe inference latency and memory bandwidth consumption due to their extremely high overall computational complexity, making them unsuitable for deployment on embedded or edge computing platforms with limited computing power. When processing multi-frame high-resolution image stitching, the time consumption is often as high as tens of seconds, which is completely unable to achieve the industrial real-time processing index of more than 1fps.

[0005] Therefore, we propose a lightweight real-time image stitching method and system based on fixed overlapping region constraints to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a lightweight real-time image stitching method and system based on fixed overlapping region constraints, which solves the problems of high computational complexity, poor real-time performance and difficulty in edge processing in the general stitching technology.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A lightweight real-time image stitching method based on fixed overlap region constraints includes the following steps: Step S1: Acquire a sequence of images with a fixed overlap ratio. Control the dual cameras to synchronously scan and acquire images according to a fixed time sequence through the timing control unit, and acquire N sequence images with adjacent overlap relationships, so that the overlap ratio between adjacent frames in the sequence images is locked and consistent within a preset error range. Step S2: Perform ROI feature detection and matching based on fixed overlapping region constraints: S2.1: Based on the system's preset overlap ratio parameters, determine the overlapping region of interest (ROI) between adjacent images before image acquisition; S2.2: Feature detection is restricted to the downsampled image, and image feature points are extracted only within the overlapping ROI region of the downsampled image; S2.3: Only within the overlapping ROI region, perform preliminary pairing of feature points and remove false matches to obtain valid matching point pairs; S2.4: A robust estimation algorithm is used to calculate the effective matching point pairs and estimate the affine transformation model parameters between adjacent images. The affine transformation model parameters include six degrees of freedom: scaling, rotation, shearing, and translation. Step S3: Perform local image fusion based on fixed overlap bands: S3.1: Determine the stitching seam position between adjacent images based on the affine transformation model parameters calculated in step S2; S3.2: Local multi-band fusion, multi-band fusion calculation is performed only in the strip area of ​​each preset pixel width on both sides of the splicing seam, using low bit width integer data type instead of floating point operation; S3.3: Direct copying of non-overlapping regions: no fusion calculation is performed on image regions outside the strip region; pixel copying is performed directly. Step S4: Output the stitched panoramic image. The adjacent images that have undergone local fusion are stitched together according to their spatial positions to output a large-format panoramic image.

[0008] Preferably, in step S2.2 of step S2, the downsampling ratio of the downsampled image is 1 / 4 or 1 / 8.

[0009] Preferably, in step S2.2 of step S2, the image feature point extraction specifically involves: The ORB algorithm is used for feature detection. First, the FAST algorithm is used to quickly detect corner features in the image, and the non-maximum suppression algorithm is used to remove redundant corners. Then, the main direction of the feature points is calculated based on the gray centroid of the pixels in the neighborhood of the corner points. Finally, the rotated BRIEF descriptor is used to encode the feature points to generate binary feature vectors.

[0010] Preferably, in step S2.3 of step S2, the specific method for preliminary feature point pairing and mismatch removal is as follows: Within the overlapping ROI region, preliminary pairing of feature points is achieved based on BFMatcher. By traversing all feature vectors of the image to be matched, the Hamming distance between two feature vectors is calculated, and the feature point pair with the smallest distance is taken as the preliminary matching result. Subsequently, the KNN matching strategy and the Love ratio test criterion are used to eliminate false matches: for each feature point, K nearest candidate matching points are selected, and the distance ratio between the first candidate matching point and the second candidate matching point is calculated. If the ratio is less than a set threshold, it is determined to be a valid matching point pair; otherwise, it is considered a false match and is eliminated.

[0011] Preferably, in step S2.4 of step S2, the RANSAC algorithm is used to perform robust estimation on the effective matching point pairs to calculate the affine transformation model parameters.

[0012] Preferably, in step S3.2 of step S3, the preset pixel width of the strip region with the preset pixel width is 256 to 1024 pixels.

[0013] 7. A lightweight real-time image stitching method based on fixed overlapping region constraints according to claim 6, characterized in that, in step S3 to step S3.2, the low-bit-width integer data type is specifically uint16 integer.

[0014] Preferably, in step S3.2 of step S3, the specific method for multi-band fusion is as follows: Gaussian pyramid decomposition is performed on the registered adjacent images to obtain low-frequency component images at different scales; the corresponding Laplacian pyramid is obtained through Gaussian pyramid difference to obtain the high-frequency detail components of the image; a weighted average fusion strategy is used for the low-frequency components and a gradient weighted fusion strategy is used for the high-frequency components; finally, the final fused image is obtained through Laplacian pyramid reconstruction; the Gaussian pyramid and the Laplacian pyramid have two pyramid layers.

[0015] Preferably, after obtaining the sequence of images in step S1, the method further includes: optional step S5 color correction: color correction is achieved by using a histogram matching algorithm, constructing a histogram matching lookup table in the overlapping ROI region, and performing color correction on the entire image based on the lookup table; specifically, the grayscale histograms of the reference image and the image to be corrected are calculated respectively, the cumulative distribution function of the two histograms is determined, and the pixel grayscale values ​​of the image to be corrected are mapped to the grayscale space of the reference image through a mapping function.

[0016] A lightweight real-time image stitching system based on fixed overlap region constraints, comprising the lightweight real-time image stitching method based on fixed overlap region constraints as described in claim 1, including: The image acquisition module is used to control the dual cameras to synchronously scan and acquire N images with adjacent overlapping relationships through the timing control unit, so that the proportion of the overlapping area between adjacent frames is locked and consistent within a preset error range. The ROI feature matching module is connected to the image acquisition module. It is used to determine the overlapping region of interest (ROI) in the preset overlapping area according to the system's preset overlap ratio parameters, and to restrict feature detection to the downsampled image. Feature detection and matching are only performed in the overlapping ROI area of ​​the downsampled image, and the parameters of the affine transformation model with a total of 6 degrees of freedom, including scaling, rotation, shearing and translation, are calculated. The local fusion module, connected to the ROI feature matching module, is used to determine the stitching seam position between adjacent images based on the calculated affine transformation model parameters. It performs multi-band fusion using low-bit-width integer data type only within a strip area of ​​a preset pixel width on both sides of the stitching seam. Image areas outside the strip area are directly copied pixel by pixel. The image output module, connected to the local fusion module, is used to spatially stitch the locally fused images and output a wide-angle panoramic image.

[0017] The present invention has at least the following beneficial effects: Synergy between physical temporal control and reduction of the computational domain of degraded ROIs: General image stitching algorithms must assume arbitrary displacement and overlap between input image pairs, thus feature extraction and fusion mask calculation must traverse the entire image. This invention precisely controls the temporal consistency of dual-camera sweep imaging, locking the overlap zone in consistency at the physical source. This hardware prior directly supports and endows the software algorithm with the legitimacy and robustness of "performing downsampled feature detection only within statically preset overlapping ROIs." Since the hardware guarantees the determinism of the ROI position, the algorithm does not need to perform high-computational global full-scale spatial feature extraction in each frame. It can directly capture sufficient high-quality registration features within a very small local degradation domain, thereby compressing the total time limit of feature detection and matching to less than 0.53 seconds, fundamentally removing the biggest computational bottleneck of traditional algorithms.

[0018] The present invention also has the following beneficial effects: Synergy between 6-DOF affine transformation and uint16 lightweight local multi-band fusion: This invention effectively solves the pain point by replacing the general 8-DOF homography transformation model with a 6-DOF affine transformation model. Its parameter space is low, and convergence in RANSAC estimation requires very few iterations (transform estimation time is only 0.005 seconds). More importantly, it ensures that the mapping matrix has extremely high geometric rigidity and spatial linearity within the sweep overlap band. It is precisely because of this strong geometric linear constraint provided by the 6-DOF affine transformation that the rounding error of subsequent multi-band fusion is not nonlinearly amplified under the lightweight condition of replacing single-precision floating-point pixel operations with uint16 integers. The deep coupling between linear mapping and integer multi-band fusion perfectly ensures a smooth transition at the splicing seam, eliminating any ghosting or misalignment and controlling the geometric error within 2-3 pixels. Meanwhile, the shrinking of local strip regions (such as 512 pixels on each side of the seam) reduces the fusion computation to a fraction of that of full-image fusion, achieving a leapfrog combination of high-precision geometric fidelity and ultra-high real-time performance (close to 1fps). Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 The process flow diagram corresponding to the hardware prior and physical source control stage in the embodiments of the present invention; Figure 2 A detailed flowchart of the space constraint and lightweight feature registration stage in the embodiment of the present invention; Figure 3 The corresponding data parallel flow topology diagram in the embodiment of the present invention is as follows: local strip-shaped non-floating-point fusion and direct copying of non-overlapping areas. Figure 4 The corresponding logic block diagram for the final output of the panoramic image and adaptive color correction in the embodiments of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1: System Overall Architecture and Hardware Implementation Environment This embodiment provides a lightweight real-time image stitching system based on fixed overlap region constraints. This system is primarily deployed in high-speed image stitching engineering applications for swiveling imaging, sequential image acquisition, or wide-swath imaging ground verification systems.

[0023] At the hardware coordination level, the system includes a microcontroller unit, a dual-camera scanning imaging mechanism, an image transmission bus, and an edge computing unit (such as a vision processing board integrating an embedded GPU / CPU). Specifically, the system described in this embodiment includes: Image acquisition module: Used to generate high-precision trigger pulses through the internal timer of the microcontroller unit, precisely controlling the dual-camera scanning imaging mechanism to perform synchronous scanning and image acquisition at fixed time intervals (e.g., 650ms). This ensures, at the physical hardware level, that the overlap ratio between adjacent frames in the acquired N image sequences is locked within a preset error range, maintaining a high degree of geometric topological consistency. ROI Feature Matching Module: Connected to the image acquisition module. This module pre-calculates and determines the overlapping regions of interest (ROIs) between adjacent images before image acquisition, based on the system's preset overlap ratio parameters. Simultaneously, to achieve extreme lightweight design, this module restricts feature detection to images downsampled by a preset ratio, extracting image feature points only within the overlapping ROI regions of the downsampled images and performing preliminary matching and mismatch removal. Finally, a robust estimation algorithm calculates and outputs the affine transformation model parameters, including scaling, rotation, shearing, and translation—a total of six degrees of freedom. Local fusion module: Connected to the ROI feature matching module. This module determines the seam position between adjacent images based on the calculated affine transformation model parameters. Furthermore, to bypass the computational bottleneck of traditional full-image fusion, this module only performs multi-band fusion within a preset pixel-width band on both sides of the seam, using low-bit-width integer data types instead of standard single-precision floating-point operations. For non-overlapping image areas outside the banded region, no fusion transformation is performed; instead, pixels are directly copied in memory. Image output module: Connected to the local fusion module. Used to stitch adjacent images that have undergone local strip fusion and non-overlapping pixel copying according to their spatial topological positions, outputting a visually continuous wide panoramic image; Color correction module (optional): Works in conjunction with the local fusion module and output module. Used in scenarios requiring high color fidelity, it constructs a histogram matching lookup table only within the overlapping ROI region and performs global color mapping correction on the entire image based on the lookup table. This embodiment uses a fixed overlap geometric prior constraint implemented at the hardware level through a microcontroller timer to directly provide deterministic physical boundary conditions for the reduction of the spatial computation domain and the extreme lightweighting of the software algorithm in the image stitching process.

[0024] Detailed workflow of a lightweight real-time image stitching method: To further illustrate the specific implementation principle of the system described in this invention, this invention also provides a lightweight real-time image stitching method based on fixed overlap region constraints. This method relies on the fixed overlap geometric prior constructed by the aforementioned physical system, and specifically includes the following steps: Step S1: Obtain a sequence of images with a fixed overlap ratio: The microcontroller unit controls the dual cameras to synchronously scan and acquire N images with adjacent overlapping relationships by controlling the timer at a fixed time sequence. Due to the high stability of the hardware acquisition timing and the rigid constraints of the scanning trajectory, the proportion of overlapping areas between adjacent frames in the image sequence remains strictly consistent within a preset error range.

[0025] Step S2: Perform ROI feature detection and matching based on fixed overlapping region constraints: This step, by introducing a fixed overlap prior, forcibly confines the spatial computation domain of feature extraction and matching to the overlapping region of interest (ROI), and includes the following sub-steps: S2.1 Determining Overlapping Regions of Interest (ROIs): Based on the overlap ratio parameters preset during the system's design or calibration phase (these parameters are jointly determined by the sweep speed, camera field of view, and trigger timing), the specific pixel boundaries of the overlapping ROIs between adjacent images are directly calculated and pre-determined before image stitching calculation. Since the overlap ratios of adjacent frames are highly consistent, these ROI boundaries remain static throughout the entire time-series acquisition process, eliminating the need for dynamic blind searching in each frame. S2.2 Downsampling ROI Feature Detection: Feature detection is restricted to the downsampled image, with a preferred downsampling ratio of 1 / 4 or 1 / 8. Feature detection operators are only called within the downsampled overlapping ROI region to extract image feature points. In this embodiment, the ORB algorithm is preferably used. This algorithm first uses the FAST algorithm to quickly detect corner features in the image, and then uses the non-maximum suppression (NMS) algorithm to remove redundant and spatially dense corners. Subsequently, the principal direction of the feature point is calculated based on the gray-level centroid of the neighboring pixels of the corner point, achieving rotation invariance. Finally, the feature point is encoded using a BRIEF descriptor after rotation correction for the principal direction, generating a lightweight binary feature vector. S2.3 Local Feature Matching and Mismatch Removal: Similarly, initial pairing of feature points is performed only within the overlapping ROI region. In this embodiment, a brute-force matcher (BFMatcher) traverses the binary feature vectors within the ROI of the image to be matched, calculates the Hamming distance between two feature vectors, and takes the feature point pair with the smallest distance as the initial matching result. Subsequently, a K-nearest neighbor (KNN) matching strategy combined with the Love ratio test criterion is used to perform mismatch removal: for each feature point within the ROI, K nearest candidate matching points are selected, and the Hamming distance ratio between the first and second candidate matching points is calculated. If this ratio is less than a set threshold, it is determined to be a valid matching point pair; otherwise, it is directly regarded as a mismatch and removed, thereby quickly constructing a high-purity matching point set within the locally overlapping region. S2.4 Geometric Transformation Model Parameter Estimation: The Random Sample Consensus (RANSAC) algorithm is used to perform robust iterative estimation of the effective matching point pairs obtained within the ROI. Based on the geometric feature of highly consistent overlapping zones in the sweeping imaging, the parameter space is intentionally narrowed, and the affine transformation model parameters between adjacent images are calculated and estimated. The affine transformation model contains six degrees of freedom: scaling, rotation, shearing, and translation. It has a low parameter space, converges very quickly during RANSAC iteration, and exhibits excellent geometric rigidity under fixed overlap constraints.

[0026] Step S3: Perform local image fusion based on fixed overlap bands: This step cuts through the computationally intensive bottleneck using a strategy of "local strip fusion + direct copying of non-overlapping regions," and includes the following sub-steps: S3.1 Determine the seam location: Based on the 6-DOF affine transformation parameters calculated in step S2, map adjacent images to a unified reference canvas coordinate system to determine the seam location between adjacent images. S3.2 Local Non-Floating-Point Multi-Band Fusion: Multi-band fusion calculation is performed only within a strip-shaped region (i.e., a fixed overlapping band, preferably 256 to 1024 pixels wide) on both sides of the splicing seam. To reduce the floating-point operation pressure on the computing unit, the multi-band fusion calculation in this region adopts low-bit-width integer data type (preferably uint16 integer) instead of standard single-precision floating-point operation. The specific fusion mechanism is as follows: Gaussian pyramid decomposition is performed on the registered adjacent images to obtain low-frequency component images of different scales; the corresponding Laplacian pyramid is obtained by differencing adjacent layers of the Gaussian pyramid to obtain the high-frequency detail components of the image; a spatial weighted average fusion strategy is used for the low-frequency components, and a gradient weighted fusion strategy is used for the high-frequency components to ensure smooth transition and no loss of edge details; finally, the final fused image is obtained by inverse reconstruction through the Laplacian pyramid. To adapt to low-computing-power scenarios at the edge and reduce computational overhead, the number of pyramid layers of the Gaussian pyramid and the Laplacian pyramid is fixed at 2. S3.3 Direct Copy of Non-overlapping Areas: In the image area outside the preset pixel width strip area (the non-overlapping main area that occupies most of the entire image area), no fusion calculation is performed at all, and pixel-level direct copying and splicing are performed directly in the video memory space.

[0027] Step S4: Output the stitched panoramic image: The adjacent images that have undergone local fusion and direct copying of non-overlapping areas in step S3 are topologically combined and stitched together according to their spatial location, ultimately outputting a seamless, high-resolution, wide-format panoramic image.

[0028] Step S5: Color Correction (optional step as a modular component): To eliminate the overall visual color difference caused by slight variations in the optical axis of the dual cameras or ambient lighting, this method includes an optional step S5, color correction, after acquiring the image sequence in step S1. This step is also completely confined to the overlapping ROI region: color correction is achieved through a histogram matching algorithm, specifically by calculating the gray-level histograms of the reference image and the image to be corrected within the overlapping ROI, determining the cumulative distribution function (CDF) of the two histograms, and constructing a histogram matching lookup table (LUT) within the overlapping ROI region; subsequently, global pixel gray-level mapping correction is directly performed on the entire image based on this lookup table, thereby avoiding the overhead of dynamically calculating the histogram across the entire image. Whether to enable this color correction module can be flexibly selected based on the system's real-time engineering requirements.

[0029] Example 2: Real-time stitching of 3-frame image sequence in a wide-swath imaging ground verification system This embodiment presents a typical engineering application scenario of the present invention in a wide-swath remote sensing imaging ground verification system, used for near real-time stitching of high-resolution airport satellite image sequences.

[0030] 1. Hardware deployment and geometric parameters: The wide-swath imaging verification system employs a dual-camera synchronous scanning mechanism. A timer in the timing control unit serves as the hardware trigger source, with a timer interrupt period of 650ms, precisely controlling the synchronous exposure of the two cameras. The cameras capture high-resolution remote sensing images, acquiring three images (Image 1, Image 2, and Image 3) in a single horizontal scan. Due to minimal timing jitter from the microcontroller control, the overlap ratio between Image 1 and Image 2 is physically locked at approximately 69%, and the overlap ratio between Image 2 and Image 3 is physically locked at approximately 50%. The computational core was tested on a desktop-level high-performance computing platform, and the algorithm demonstrates potential for deployment on edge computing platforms (the computational core utilizes an image processing board integrating an embedded microprocessor and computing unit), with all modules deployed throughout the algorithm's lifecycle.

[0031] 2. ROI Extraction and ORB Feature Extraction: Based on a fixed overlap ratio of 69% and 50%, the system directly delineates the static overlapping ROI matrix regions corresponding to images 1-2 and 2-3 in memory. The ROI region image is directly downsampled by 1 / 4 resolution to drastically reduce the feature search space; The ORB algorithm was run within a 1 / 4 downsampled ROI, with the FAST corner detection threshold set to 20. Redundant corners were eliminated using non-maximum suppression (NMS) to maintain the uniformity of corner distribution in space. The principal orientation angle of the feature points was obtained by calculating the gray-level centroid, generating a 256-bit binary BRIEF descriptor. Due to the physical limitation of the spatial computation domain, the total feature detection time was only 0.484 seconds.

[0032] 3. Local feature matching and geometric parameter estimation: The BFMatcher is used to calculate the Hamming distance between binary descriptors within adjacent downsampled ROIs; By introducing KNN (K=2) search, the ratio of the distance between the first nearest neighbor and the distance between the second nearest neighbor is calculated. The Love ratio test criterion is then applied, and the ratio threshold is strictly set to 0.65. Pairs with a ratio greater than 0.65 are directly discarded, resulting in a highly pure set of effective matching points. The total time for feature matching is significantly reduced to 0.046 seconds. The effective matching point set is input into the RANSAC estimator to estimate a 6-DOF affine transformation matrix including scaling, rotation, shearing, and translation. The reprojection error tolerance of the affine model is set to 3.0 pixels. The transformation estimation converges instantaneously in an extremely low parameter space, taking only 0.005 seconds.

[0033] 4. Color correction and multi-band local fusion: In this embodiment, the color correction module in step S5 is activated to overcome the overall color difference caused by slight differences in the image sensors of the two cameras. The R, G, and B channel grayscale histograms of the reference image and the image to be corrected are extracted only within the overlapping ROI, the cumulative distribution function is calculated, and a 256-order histogram matching lookup table (LUT) is built within the ROI. Then, the lookup table is used to directly perform full-image broadcast mapping correction. Based on the affine transformation matrix, images 1, 2, and 3 are projected onto a unified canvas, and a geometric splicing seam is defined in the center of the overlapping area. The seam extends outward by 256 pixels on each side, together forming a local strip-shaped fusion area with a width of 512 pixels (within the preset range of 256~1024 pixels). The 512-pixel-wide strip region data was forcibly converted from standard single-precision floating-point numbers to uint16 integer data type. Within this local region, two layers of Gaussian pyramids and two layers of Laplacian pyramids were constructed. Low-frequency components were fused using a spatial distance-based weighted average operator, while high-frequency components were fused using a gradient operator-based weighted fusion to preserve core topological details in the wide-swath image. After the fusion calculation was completed, inverse reconstruction was performed directly, with the uint16 multi-band fusion stage taking only 0.527 seconds. In the non-overlapping main area outside the 512-pixel strip fusion area, without performing any pyramid operation, the original pixels are directly copied to the corresponding panoramic canvas by calling the operating system's direct memory copy instruction (memcpy). The image transformation and direct copy take about 0.780 seconds.

[0034] 5. Evaluation of splicing performance and accuracy: According to the full-time statistics of the ground verification system, when processing these 3 high-resolution remote sensing images, the total core time of the stitching algorithm was perfectly controlled at 1.854 seconds (including other system I / O overhead, the total time was only 2.664 seconds), and the average time for stitching each pair of images was 1.332 seconds, which perfectly meets the engineering system's requirement of close to 1fps. Meanwhile, due to the strong linear geometric rigidity constraint of the 6-DOF affine transformation suppressing the integer rounding error of uint16, the transition at the splicing seam is extremely smooth, with no obvious splicing traces, no ghosting, no misalignment, highly uniform color tone, and geometric registration error stable at less than 2~3 pixels, completely preserving the spatial topology of the original scene.

[0035] Example 3: Industrial visual inspection stitching on an edge computing platform pursuing ultimate real-time performance: This embodiment presents a third typical implementation of the present invention in an edge computing detection platform (or UAV-borne embedded computing platform) for industrial pipelines that require extreme real-time performance and are subject to resource constraints: 1. Modular trimming configuration: In industrial visual inspection scenarios, since ambient lighting is typically provided by rigid industrial light sources (such as LED strip light sources), adjacent cameras already have highly consistent exposure and tone at the hardware level. To further reduce processing time to the limit, this embodiment fully utilizes the modular design and highly customizable features of the present invention. In the system configuration parameters, the color correction module in step S5 is completely disabled to completely eliminate the overhead of histogram matching and lookup table construction.

[0036] 2. Adjustment of parameters for the extreme lightweight algorithm: Spatial downsampling limit constraint: In step S2.2, in order to adapt to the embedded edge processing board with weaker computing power, the downsampling ratio of the feature detection image is directly increased to 1 / 8 resolution downsampling. This further sparsifies the number of feature points within the overlapping ROI, retaining only the backbone architecture features, and successfully reduces the total feature detection time to less than 0.2 seconds.

[0037] Bandwidth limit narrowing of fusion band: In step S3.2, since the detection targets of industrial production lines are mostly rigid flat plates or regular geometric components, the transition bandwidth requirement for multi-band fusion is low. Therefore, the width of the preset strip area on both sides of the splicing seam is limited to the minimum limit of 256 pixels.

[0038] Minimalist Pyramid Fusion: The data type for fusion calculations remains uint16 integer, and the number of pyramid layers remains fixed at 2. Because the pixel operation area for multi-band fusion is further reduced by half from 512 pixels in Example 2 (to 256 pixels wide), the computation time for local integer multi-band fusion is reduced to approximately 0.25 seconds.

[0039] 3. Implementation Results and Project Efficiency: Under the modular cropping configuration described in this embodiment, when the system runs 3-frame image stitching on an ultra-low power embedded edge computing platform, the total processing time of the algorithm core can theoretically be further compressed to less than 1.1 seconds, close to the real-time processing requirement of 1fps. This fully verifies the strong hardware adaptability, flexibility and high engineering practicality of the present invention on resource-constrained platforms.

[0040] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A lightweight real-time image stitching method based on fixed overlap region constraints, characterized in that, Includes the following steps: Step S1: Acquire a sequence of images with a fixed overlap ratio. Control the dual cameras to synchronously scan and acquire images in a fixed time sequence through the timing control unit, and acquire N sequence images with adjacent overlap relationships, so that the overlap ratio between adjacent frames in the sequence images is locked and consistent within a preset error range. Step S2: Perform ROI feature detection and matching based on fixed overlapping region constraints: S2.1: Based on the system's preset overlap ratio parameters, determine the overlapping region of interest (ROI) between adjacent images before image acquisition; S2.2: Feature detection is restricted to the downsampled image, and image feature points are extracted only within the overlapping ROI region of the downsampled image; S2.3: Only within the overlapping ROI region, perform preliminary pairing of feature points and remove false matches to obtain valid matching point pairs; S2.4: A robust estimation algorithm is used to calculate the effective matching point pairs and estimate the affine transformation model parameters between adjacent images. The affine transformation model parameters include six degrees of freedom: scaling, rotation, shearing, and translation. Step S3: Perform local image fusion based on fixed overlap bands: S3.1: Determine the stitching seam position between adjacent images based on the affine transformation model parameters calculated in step S2; S3.2: Local multi-band fusion, multi-band fusion calculation is performed only in the strip area of ​​each preset pixel width on both sides of the splicing seam, using low bit width integer data type instead of floating point operation; S3.3: Direct copying of non-overlapping regions: no fusion calculation is performed on image regions outside the strip region; pixel copying is performed directly. Step S4: Output the stitched panoramic image. The adjacent images that have undergone local fusion are stitched together according to their spatial positions to output a large-format panoramic image.

2. The lightweight real-time image stitching method based on fixed overlap region constraints according to claim 1, characterized in that, In step S2.2 of step S2, the downsampling ratio of the downsampled image is 1 / 4 or 1 / 8.

3. The lightweight real-time image stitching method based on fixed overlap region constraints according to claim 2, characterized in that, In step S2.2 of step S2, the image feature point extraction specifically involves: The ORB algorithm is used for feature detection. First, the FAST algorithm is used to quickly detect corner features in the image, and the non-maximum suppression algorithm is used to remove redundant corners. Then, the main direction of the feature points is calculated based on the gray centroid of the pixels in the neighborhood of the corner points. Finally, the rotated BRIEF descriptor is used to encode the feature points to generate binary feature vectors.

4. The lightweight real-time image stitching method based on fixed overlap region constraints according to claim 1, characterized in that, In step S2.3 of step S2, the specific methods for preliminary feature point pairing and mismatch removal are as follows: Within the overlapping ROI region, preliminary pairing of feature points is achieved based on BFMatcher. By traversing all feature vectors of the image to be matched, the Hamming distance between two feature vectors is calculated, and the feature point pair with the smallest distance is taken as the preliminary matching result. Subsequently, the KNN matching strategy and the Love ratio test criterion are used to eliminate false matches: for each feature point, K nearest candidate matching points are selected, and the distance ratio between the first candidate matching point and the second candidate matching point is calculated. If the ratio is less than a set threshold, it is determined to be a valid matching point pair; otherwise, it is considered a false match and is eliminated.

5. A lightweight real-time image stitching method based on fixed overlap region constraints according to claim 1, characterized in that, In step S2.4 of step S2, the RANSAC algorithm is used to perform robust estimation on the effective matching point pairs to calculate the affine transformation model parameters.

6. A lightweight real-time image stitching method based on fixed overlap region constraints according to claim 1, characterized in that, In step S3.2 of step S3, the preset pixel width of the strip region with the preset pixel width is 256 to 1024 pixels.

7. A lightweight real-time image stitching method based on fixed overlap region constraints according to claim 6, characterized in that, In step S3.2 of step S3, the low-width integer data type is specifically uint16 integer.

8. A lightweight real-time image stitching method based on fixed overlap region constraints according to claim 7, characterized in that, In step S3.2 of step S3, the specific method for multi-band fusion is as follows: Gaussian pyramid decomposition is performed on the registered adjacent images to obtain low-frequency component images at different scales; the corresponding Laplacian pyramid is obtained through Gaussian pyramid difference to obtain the high-frequency detail components of the image; a weighted average fusion strategy is used for the low-frequency components and a gradient weighted fusion strategy is used for the high-frequency components; finally, the final fused image is obtained through Laplacian pyramid reconstruction; the Gaussian pyramid and the Laplacian pyramid have two pyramid layers.

9. A lightweight real-time image stitching method based on fixed overlap region constraints according to claim 1, characterized in that, After obtaining the sequence of images in step S1, the method further includes: optional step S5 color correction: color correction is achieved by using a histogram matching algorithm, constructing a histogram matching lookup table in the overlapping ROI region, and performing color correction on the entire image based on the lookup table; specifically, the grayscale histograms of the reference image and the image to be corrected are calculated respectively, the cumulative distribution function of the two histograms is determined, and the pixel grayscale values ​​of the image to be corrected are mapped to the grayscale space of the reference image through a mapping function.

10. A lightweight real-time image stitching system based on fixed overlap region constraints, comprising the lightweight real-time image stitching method based on fixed overlap region constraints as described in claim 1, characterized in that, include: The image acquisition module is used to control the dual cameras to synchronously scan and acquire N images with adjacent overlapping relationships through the timing control unit, so that the proportion of the overlapping area between adjacent frames is locked and consistent within a preset error range. The ROI feature matching module is connected to the image acquisition module. It is used to determine the overlapping region of interest (ROI) in the preset overlapping area according to the system's preset overlap ratio parameters, and to restrict feature detection to the downsampled image. Feature detection and matching are only performed in the overlapping ROI area of ​​the downsampled image, and the parameters of the affine transformation model with a total of 6 degrees of freedom, including scaling, rotation, shearing and translation, are calculated. The local fusion module, connected to the ROI feature matching module, is used to determine the stitching seam position between adjacent images based on the calculated affine transformation model parameters. Multi-band fusion is performed only within a strip area of ​​a preset pixel width on both sides of the stitching seam using a low bit-width integer data type. Image areas outside the strip area are directly copied pixel by pixel. The image output module, connected to the local fusion module, is used to spatially stitch the locally fused images and output a wide-angle panoramic image.

Images