A multi-camera-based picture splicing method

Abstract

The application relates to the technical field of computer vision, and discloses a picture splicing method based on multiple cameras. The method comprises the following steps: acquiring multiple image frames collected synchronously; correcting and aligning the images; constructing an initial pixel-level difference cost map; combining a previous frame splicing seam to generate a space-time guide weight map; fusing to obtain a dynamic optimization cost map; performing shortest path search in a limited area to determine a current best splicing seam; and generating a seamless splicing image by weighted fusion according to the current best splicing seam. By introducing a space-time guide mechanism and an edge preservation factor, the calculation complexity is reduced, the splicing continuity and visual quality are guaranteed, and the method is suitable for high-dynamic scenes such as vehicle-mounted surround view and unmanned aerial vehicle aerial photography.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method for stitching images based on multiple cameras. Background Technology

[0002] With the rapid development of applications such as intelligent surveillance, autonomous driving, and panoramic imaging systems, multi-camera image stitching technology has become an important research direction in the field of computer vision. This technology aims to seamlessly fuse image sequences from multiple perspectives in space to generate panoramic images with a wide field of view and high consistency. Its core challenge lies in finding a visually imperceptible stitching seam in the overlapping region to avoid obvious seam artifacts caused by differences in lighting, viewpoint distortion, or moving objects. Traditional methods typically construct cost functions based on image gradients, color consistency, or structural similarity, and use dynamic programming or graph cut algorithms to solve for the optimal path.

[0003] The graph-cut-based image stitching method treats pixels as graph nodes and the differences between adjacent pixels as edge weights, transforming the stitching seam search problem into an energy minimization problem. This method performs well in static scenes and effectively suppresses local discontinuities.

[0004] However, in dynamic scenes containing complex textures, fast-moving targets, or drastic lighting changes, the search space for the energy function expands dramatically, leading to an exponential increase in computational complexity. Conventional CPU or GPU platforms struggle to complete the search for the global optimum within a finite timeframe, often forcing them to employ heuristic approximation strategies, sacrificing stitching quality for real-time performance.

[0005] Existing technologies suffer from the following problems: accurate modeling of pixel relationships requires high-dimensional features and dense connections, significantly increasing the graph size; traditional optimization algorithms are limited by the serial processing bottleneck of classical computing architectures, making it impossible to efficiently explore the solution space. Especially in high frame rate video streams captured simultaneously by multiple cameras, stitching delays can easily cause image stuttering or misalignment, severely impacting user experience. Therefore, there is an urgent need for a novel stitching seam solution mechanism that can break through the limitations of classical computing paradigms and achieve fast global optimization of high-dimensional energy functions. Summary of the Invention

[0006] This invention provides a multi-camera-based image stitching method, aiming to solve the technical problem of high computational complexity and difficulty in real-time optimization for finding the optimal stitching seam in dynamic scenes. This method constructs a spatiotemporally consistent dynamic cost graph and introduces a hierarchical guided path search mechanism, significantly reducing computational resource consumption while ensuring visual continuity of the stitched image, achieving seamless real-time image fusion at high frame rates.

[0007] This invention provides a method for stitching images from multiple cameras, comprising: Acquire a sequence of raw image frames simultaneously captured by at least two adjacent cameras; Each frame of the original image is preprocessed to generate a corrected, aligned image; An initial pixel-level difference cost map is constructed based on the aligned image; Based on the optimal seam position information determined in the previous frame, a spatiotemporal guidance weight map is generated. The initial pixel-level difference cost map is weighted and fused with the spatiotemporal guided weight map to obtain a dynamic optimized cost map. Perform a shortest path search within the restricted region on the dynamic optimization cost graph to determine the optimal stitching seam for the current frame; The adjacent aligned images are weighted and fused according to the optimal stitching seam to generate the final stitched image.

[0008] Preferably, acquiring the sequence of raw image frames simultaneously captured by at least two adjacent cameras includes: Multiple cameras are controlled to complete image acquisition within the same exposure period by hardware synchronous triggering signals. The original image is raw sensor data without geometric transformation.

[0009] Preferably, the preprocessing of each original image frame to generate the corrected aligned image includes: Using pre-calibrated camera intrinsic and extrinsic parameter matrices, perspective transformation and distortion correction are performed on the original image; The transformed image is resampled using a bilinear interpolation algorithm, so that the imaging areas of adjacent cameras have overlapping areas in the world coordinate system.

[0010] Preferably, constructing the initial pixel-level difference cost map based on the aligned image includes: Within the overlapping area, the color space difference value is calculated for the corresponding pixels of two adjacent aligned images, where the color space is the CIELAB color space; The color space difference value is calculated using the Euclidean distance formula, which is to calculate the difference between the three channels L*, a*, and b* of each pixel, square the sum of the squares, and then take the square root. Normalize the difference values ​​of all pixels to an integer range of 0 to 255 to form an initial pixel-level difference cost map.

[0011] Preferably, the step of generating a spatiotemporal guidance weight map by combining the optimal stitching seam position information determined in the previous frame includes: Using the best stitching seam of the previous frame as the center line, construct a strip-shaped guide area within the overlapping area of ​​the current frame; Within the strip-shaped guiding area, each pixel is assigned a guiding weight value, which decreases in a Gaussian distribution as it moves vertically from the center line.

[0012] Preferably, the step of weightedly fusing the initial pixel-level difference cost map with the spatiotemporal guided weight map to obtain a dynamically optimized cost map includes: Perform element-wise addition on the pixel values ​​at the same position in the two images to obtain the fused cost. If the value after fusion is greater than 255, then it is truncated to 255; In the final dynamic optimization cost map, the value of each pixel represents the comprehensive cost of setting the seam at that location.

[0013] Preferably, the step of performing a shortest path search within a restricted region on the dynamic optimization cost graph to determine the optimal stitching seam for the current frame includes: The overlapping regions of the dynamic optimization cost graph are modeled as directed graphs, where each pixel is a node in the graph, and each node has a directed edge with its three adjacent nodes in the row below it. The weight of an edge is equal to the cost of the target node; Starting from any starting node at the top of the overlapping region, use dynamic programming to calculate the minimum cumulative cost path to any ending node at the bottom. The path with the minimum cumulative cost is the optimal stitching seam for the current frame, which consists of a series of continuous pixel coordinates.

[0014] Preferably, the step of weighted fusion of adjacent aligned images based on the optimal stitching seam to generate the final stitched image includes: Using the optimal seam as the dividing line, the left image pixels are completely preserved on the left side of the overlapping area, and the right image pixels are completely preserved on the right side. A transition zone of 5 pixels is extended on both sides of the splicing seam. Within the transition zone, a linear gradient weight is used to perform a weighted average of the pixels in the left and right images. The weight on the left decreases from 1 to 0, and the weight on the right increases from 0 to 1. Non-overlapping areas are directly copied from the corresponding image content, and finally a panoramic image without visible seams is synthesized.

[0015] Preferably, the method further includes an exception handling mechanism: When the motion amplitude between the current frame and the previous frame is detected to be greater than a preset threshold, it is determined to be a violent motion scene; the generation of spatiotemporal guidance weight map using the stitching seam information of the previous frame is suspended, and the minimum difference path search strategy in the whole region is adopted instead. The motion amplitude is quantized by calculating the average magnitude of the optical flow vector in the overlapping region between two frames.

[0016] Preferably, the method maintains a seam history buffer queue during execution, and the queue stores the best seam coordinate sequence of the most recent 5 frames; When generating the spatiotemporal guide weight map, not only the stitching seam of the previous frame is used, but also all stitching seams in the historical queue are weighted and averaged to obtain a more stable guide center line. The weights of each historical frame decrease exponentially in reverse chronological order.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention introduces a spatiotemporal guidance mechanism based on historical splicing seams, which constrains the global optimal search, which originally had to be carried out in the fully overlapping region, to a narrow dynamic guidance zone, thereby greatly reducing the solution space dimension of the shortest path search and reducing computational complexity.

[0018] 2. By integrating color difference cost and spatiotemporal consistency weight, the visual continuity of the seams is ensured in fast-moving scenes, effectively suppressing flickering and abrupt changes.

[0019] 3. The introduction of the edge preservation factor further improves the stitching quality and avoids misalignment artifacts at the object outline. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall technical solution architecture of the present invention; Figure 2 This is a schematic diagram of the core principle framework of the spatiotemporally guided dynamic cost graph construction and hierarchical path search mechanism in this invention; Figure 3 This is a flowchart illustrating the logical process of the original image acquisition and alignment preprocessing stage in this invention. Figure 4 This is a logical flowchart of the initial difference cost map generation and spatiotemporal guided weight map fusion stage in this invention; Figure 5 This is a logical flowchart of the shortest path search and seam weighted fusion stage within the confined area in this invention. Figure 6 This is a schematic diagram of the multi-level interaction relationship and data flow between the multi-camera system and the embedded processing platform in this invention. Detailed Implementation

[0021] Example 1: Reference Figures 1 to 6This invention provides a multi-camera-based image stitching method, aiming to solve the technical problem of high computational complexity and difficulty in real-time optimization of optimal stitching seam search in dynamic scenes. This method constructs a spatiotemporally consistent dynamic cost graph and introduces a hierarchical guided path search mechanism, significantly reducing computational resource consumption while ensuring visual continuity of the stitched image, achieving seamless real-time image fusion at high frame rates.

[0022] The method includes the following steps: S1, acquire the sequence of raw image frames synchronously captured by at least two adjacent cameras; S2, preprocess each frame of the original image to generate the corrected aligned image; S3, Construct an initial pixel-level difference cost map based on the aligned image; S4. Combine the optimal stitching seam position information determined in the previous frame to generate a spatiotemporal guidance weight map. S5, the initial pixel-level difference cost map and the spatiotemporal guiding weight map are weighted and fused to obtain a dynamic optimized cost map; S6, perform a shortest path search within the restricted area on the dynamic optimization cost graph to determine the optimal stitching seam for the current frame; S7. Based on the optimal stitching seam, adjacent aligned images are weighted and fused to generate the final stitched image.

[0023] In step S1, a sequence of raw image frames is acquired synchronously by at least two adjacent cameras. This process uses a hardware synchronization trigger signal to control multiple cameras to complete image acquisition within the same exposure cycle, ensuring that the timestamp deviation of the output images from each camera is less than 1 millisecond. The raw images are the original sensor data without geometric transformation, with a resolution greater than 1920×1080 pixels.

[0024] All cameras share the same master clock source and transmit synchronization trigger signals through a low-voltage differential signal interface to ensure frame-level synchronization accuracy. In a four-camera system configuration, each camera has a horizontal field of view of 92 degrees, with 20% overlap between adjacent cameras, thus forming a continuous panoramic view. The image acquisition module continuously outputs time-aligned raw frame streams, each frame carrying a precise timestamp and camera identifier for subsequent processing modules to identify and schedule.

[0025] In step S2, each frame of the original image is preprocessed to generate a corrected, aligned image. This preprocessing procedure uses pre-calibrated camera intrinsic and extrinsic parameter matrices to perform perspective transformation and distortion correction on the original image. The intrinsic parameter matrix includes focal length, principal point offset, and lens distortion coefficients, while the extrinsic parameter matrix describes the rotation and translation parameters of each camera relative to the world coordinate system.

[0026] The correction process first projects each pixel in the original image to distortion-free coordinates under an ideal pinhole camera model via reverse mapping. Then, a bilinear interpolation algorithm is used to resample the transformed image. The resampled image has a clear spatial correspondence in the world coordinate system, and the imaging areas of adjacent cameras form an overlapping area in the horizontal direction. The width of the overlapping area is not less than 15% of the total image width.

[0027] The correction results serve as the basis for subsequent stitching processing, ensuring that images from different perspectives are strictly aligned geometrically and eliminating misalignment caused by lens distortion or installation deviation.

[0028] In step S3, an initial pixel-level difference cost map is constructed based on the aligned images. The construction process is limited to the overlapping region of two adjacent aligned images. Within the overlapping region, for each pair of spatially corresponding pixels, their color difference value in the CIELAB color space is calculated. The CIELAB color space, due to its perceptual uniformity, more accurately reflects the human eye's sensitivity to color differences. (Color difference value) The distance is calculated using the Euclidean distance formula, which involves taking the differences between the L*, a*, and b* channels of each pixel, squaring them, summing them, and then taking the square root. The expression is as follows: ; and These represent the positions of the left and right images, respectively. The CIELAB value at each location is calculated. The calculated difference values ​​are then normalized to an integer range of 0 to 255, forming an initial pixel-level difference cost map. Normalization uses a linear mapping method, mapping the minimum difference value to 0, the maximum difference value to 255, and scaling the remaining values ​​proportionally.

[0029] Furthermore, before constructing the initial cost map, Canny edge detection is performed on both aligned images to extract significant edge structures. At the detected edge pixel locations, the aforementioned color difference value is multiplied by an amplification factor of 1.5 to form an edge preservation factor, thereby increasing the cost value of the edge region and guiding the stitching seam to avoid the object outline, reducing visual artifacts.

[0030] In step S4, a spatiotemporal guidance weight map is generated by combining the optimal stitching seam position information determined in the previous frame. This process relies on a stitching seam history buffer queue, which stores the optimal stitching seam coordinate sequence of the most recent 5 frames. The optimal stitching seam in each frame consists of a series of consecutive pixel coordinates, representing a path from the top to the bottom of the overlapping area. When generating the guidance weight map for the current frame, not only are the stitching seams from the previous frame used, but also all stitching seams in the history queue are weighted and averaged to obtain a more stable guidance centerline. The weights of each historical frame decay exponentially in reverse chronological order, with a decay coefficient of 0.8.

[0031] Let the first The seam of the frame is The current guiding center line The calculation formula is: ; y is the vertical row index, k is the number of historical frames to be traced, and t represents the sequence number of the currently processed frame. A strip-shaped guide region is constructed within the overlapping area of ​​the current frame, centered on the aforementioned guide center line. The width of this strip-shaped guide region is 30 pixels. Within this strip-shaped guide region, each pixel... guiding weight value Follow it to the center line vertical distance The decay follows a Gaussian distribution (x is the row index in the horizontal direction), with a standard deviation set to 10 pixels, as shown in the following expression: ; For all pixels outside the strip-shaped guide area, the guide weight value is set to the maximum constant value of 255, indicating that these areas are not suitable as candidate locations for splicing seams.

[0032] In step S5, the initial pixel-level difference cost map and the spatiotemporal guided weight map are weighted and fused to obtain a dynamically optimized cost map. The fusion operation performs element-wise addition on the pixel values ​​at the same position in the two maps, that is, for each pixel... ,calculate: ; To dynamically optimize the cost map at the pixel level The pixel value at the location is used. If the cost value after fusion is greater than 255, it is truncated to 255. In the final dynamic optimization cost map, the value of each pixel represents the comprehensive cost of setting a seam at that location. The lower the comprehensive cost, the more suitable the location is as a seam. This dynamic optimization cost map reflects both local color consistency and global spatiotemporal continuity, providing optimization targets for subsequent path search.

[0033] In step S6, a shortest path search is performed within the restricted region on the dynamic optimization cost graph to determine the optimal stitching seam for the current frame. This shortest path search process models the overlapping region of the dynamic optimization cost graph as a directed graph, where each pixel is a node, and each node has directed edges with its three adjacent nodes in the row below it (i.e., directly below, to the lower left, and to the lower right). The weight of each edge is equal to the cost of the target node. Starting from any starting node at the top of the overlapping region, a dynamic programming algorithm is used to calculate the minimum cumulative cost path to any ending node at the bottom.

[0034] During state transitions, a smoothing constraint is applied: transitions are only allowed from nodes with a horizontal offset of less than three pixels from the previous row, thus prohibiting sharp angles greater than 45 degrees in the path and forcing spatial continuity and smoothness of the seams. The dynamic programming recursive formula is as follows: ; Indicates arrival position The minimum cumulative cost, Indicates the location reached. The minimum cumulative cost. Boundary conditions are: After completing the full graph traversal, select the position with the minimum cumulative cost from the bottom row as the endpoint, and reconstruct the complete path by backtracking the pointer chain. This is the optimal stitching seam for the current frame.

[0035] In step S7, adjacent aligned images are weighted and fused according to the optimal stitching seam to generate the final stitched image. Using the optimal stitching seam as the dividing line, pixels from the left image are completely preserved on the left side of the overlapping area, and pixels from the right image are completely preserved on the right side. A transition band of 5 pixels is extended on each side of the stitching seam. Within this transition band, a weighted average of the left and right image pixels is calculated using a linear gradient weighting. Let the stitching seam position be... Then in position At this location, the pixel weight in the left image is... The pixel weights in the right image are .

[0036] The merged pixel value is calculated as follows . For the fused output image at pixel position Pixel value at that location, The left and right source images are located at pixel positions respectively. The pixel values ​​at the non-overlapping areas are directly copied from the corresponding image content, and finally a panoramic image without visible seams is synthesized.

[0037] The method also includes an anomaly handling mechanism. Before processing each frame, the optical flow vector field of the overlapping area between the current frame and the previous frame is calculated, and pixel motion is estimated using the Lucas-Kanade optical flow method. The magnitude of all optical flow vectors is taken and averaged to obtain a motion amplitude index. If this motion amplitude index is greater than a preset threshold of 20 pixels per second, it is determined to be a scene of violent motion.

[0038] In this scenario, the generation of spatiotemporal guidance weight maps using historical stitching seam information is suspended. Instead, the guidance weights for the entire overlapping region are uniformly set to 0, causing the dynamic optimization cost map to degenerate into a pure color difference cost map. Shortest path search is then performed across the entire overlapping region. This strategy ensures that stitching seams can still be selected based on the optimal difference in the current frame during rapid scene transitions or sudden motion, avoiding artifacts caused by incorrect guidance.

[0039] The method is deployed on an embedded image processing platform, which includes a dedicated image signal processor and a neural network acceleration unit. All image preprocessing, cost map calculation, and path search operations are executed in parallel on the image signal processor.

[0040] The image signal processor features a multi-channel DMA controller, enabling simultaneous reading of raw data from multiple cameras. Its built-in hardware acceleration module supports bilinear interpolation, Canny edge detection, and CIELAB color space conversion. Dynamic path planning is optimized through a customized instruction set, leveraging SIMD parallel computing capabilities to accelerate state transitions. The end-to-end latency of the entire stitching process is controlled within 33 milliseconds, supporting real-time processing of 30 frames per second, meeting the low-latency, high-robustness image stitching requirements of dynamic application scenarios such as vehicle surround view and drone aerial photography.

[0041] Example 2: This example is based on the multi-camera image stitching method described above, adapted to the monitoring scenario of rail transit platforms, to achieve panoramic stitching and real-time streaming of the platform area without blind spots, and to cope with dynamic scenarios such as dense crowds, trains entering and leaving the station, and strong light and backlight interference.

[0042] Image acquisition synchronization: Eight 1080P high-definition cameras are deployed in the passenger boarding and alighting area of ​​the platform and along the edge of the tracks, with the overlap rate of adjacent cameras' fields of view set to 25%. Hardware synchronization trigger signals control all cameras to acquire images within the same exposure cycle, with timestamp deviation controlled within 0.5 milliseconds. The raw images are the original sensor data, retaining the full dynamic range to cope with complex lighting conditions such as strong platform light and backlighting in the tunnel.

[0043] Preprocessing and correction: Perspective transformation and distortion correction are performed on the original image using pre-calibrated camera intrinsic and extrinsic parameter matrices. An additional exposure correction step is added, using a GPU-computed shader to analyze the image brightness histogram and adaptively compensate for pixel values ​​in backlit areas. Bilinear interpolation resampling is employed to ensure precise alignment of all camera imaging areas in the world coordinate system, eliminating geometric misalignments caused by platform pillars and platform screen doors.

[0044] Cost Map Construction and Spatiotemporal Guidance: CIELAB color space difference values ​​are calculated in overlapping areas to construct an initial pixel-level difference cost map. For scenarios with rapid movement, such as trains entering stations or crowds, the threshold for determining the average modulus of the optical flow vector is adjusted to 30 pixels per second. When generating the spatiotemporal guidance weight map, the stitching seam history buffer queue stores the most recent 8 frames of data, improving the stability of the stitching seams in dynamic scenes and preventing seam jumps when trains pass by.

[0045] Path search and weighted fusion: Within the constrained area of ​​the dynamically optimized cost map, a dynamic programming algorithm is used to search for the shortest path to determine the optimal stitching seam. An 8-pixel transition band is extended on each side of the stitching seam, and linear gradient weighting is used for fusion to eliminate stitching artifacts in densely populated areas. The fused panoramic image is streamed to the platform monitoring terminal via the RTSP protocol, with latency controlled within 20 milliseconds, meeting the real-time monitoring requirements of train dispatching.

[0046] Example 3: This example is adapted to the smart factory production workshop scenario, realizing panoramic stitching of the production line and the robotic arm operation area, dealing with interference factors such as metal reflection and rapid equipment movement, and supporting intelligent factory monitoring and fault early warning.

[0047] Image acquisition synchronization: Six industrial-grade cameras are deployed on both sides of the production line and above the robotic arm station. These cameras are resistant to electromagnetic interference. Frame synchronization is achieved through hardware-triggered synchronization, increasing the original image resolution to 2K to meet the needs of detailed equipment monitoring. The overlapping area of ​​adjacent cameras covers key areas such as production line seams and the range of motion of the robotic arm.

[0048] Preprocessing Correction: Building upon perspective transformation and distortion correction, color consistency matching correction based on lighting calculations is added. GPU computational shaders are used to analyze lighting differences at different workstations, smoothing pixel values ​​in reflective metal areas to eliminate stitching gaps caused by equipment reflections. After resampling, it is ensured that the pipeline of images from all cameras is straight-line aligned in the world coordinate system.

[0049] Cost Map Construction and Spatiotemporal Guidance: When constructing the initial difference cost map, the amplification factor of the edge preservation factor is adjusted to 2.0 for the edge regions of moving targets such as robotic arms and material carts, guiding the seams to avoid the contours of the moving targets. When generating the spatiotemporal guidance weight map, the guidance weight of fixed trajectory regions is enhanced by combining the motion trajectory data of workshop equipment, improving the spatial continuity of the seams.

[0050] Path search and weighted fusion: A dynamic programming algorithm is used to search for the shortest path, and a horizontal offset threshold of 2 pixels is set for path smoothing constraints to avoid jagged artifacts caused by equipment vibration at the stitching seams. The fused panoramic image is streamed to the factory's MES system via the SPOUT protocol to achieve visualized monitoring of the production process and support intelligent identification of abnormal behaviors.

[0051] Example 4: This example is adapted to large-scale monitoring scenarios in detention centers and cell monitoring scenarios, achieving panoramic stitching of the detention area courtyard and cell corridors without blind spots, balancing large-scale coverage with clear local details, and meeting the high reliability requirements of security monitoring.

[0052] Image acquisition synchronization: A combination of 4 fisheye cameras and 8 bullet cameras is deployed in the prison courtyard. The fisheye cameras are responsible for wide-area coverage, while the bullet cameras supplement local details. One camera is deployed at the entrance of each cell in the cell corridor, and the overlap rate of the field of view of adjacent bullet cameras is set to 20%. The entire system of cameras is synchronized through hardware synchronization triggering, with a timestamp deviation of less than 1 millisecond. The original images have night vision enhancement capabilities.

[0053] Preprocessing Correction: Addressing the significant distortion characteristics of fisheye cameras, a dedicated fisheye distortion correction model is employed in the perspective transformation and distortion correction steps, combined with GPU shaders to accelerate correction calculations. Rotation / translation / scaling transformation adaptations are added to ensure accurate fusion of the fisheye and bullet camera images in the world coordinate system. Resampling eliminates geometric misalignments caused by prison walls and iron fences, guaranteeing a distortion-free panoramic image.

[0054] Cost Map Construction and Spatiotemporal Guidance: When constructing the initial difference cost map, for low-light nighttime scenes in the prison area, the normalization range of the CIELAB color space difference values ​​is adjusted to 0-128 to improve the difference differentiation of low-brightness areas. When generating the spatiotemporal guidance weight map, the splicing seam historical buffer queue stores the most recent 5 frames of data, and exponential decay weights are used to ensure the stability of the splicing seams in scenes where prison personnel move slowly.

[0055] Path search and weighted fusion: Shortest path search is performed on the dynamically optimized cost map for confined areas. Considering the lack of drastic movement in the prison area, the optical flow vector determination threshold is reduced to 10 pixels per second, improving stitching efficiency. A 5-pixel transition band is set on both sides of the stitching seam, and linear gradient weight fusion is used to ensure seamless stitching of areas such as cell corridors and courtyards. The fused panoramic image is streamed to the detention center's security platform via the RTSP protocol, supporting simultaneous viewing on multiple terminals.

[0056] The aforementioned method constrains the global search to a narrow dynamic band through a spatiotemporal guidance mechanism, reducing the computational complexity of the shortest path search. Simultaneously, it integrates color differences, edge preservation, and historical continuity to ensure stitching quality and stability. This scheme achieves high frame rate real-time stitching without sacrificing visual effects, resolving the fundamental contradiction between real-time performance and stitching quality inherent in existing technologies due to excessive computational burden.

[0057] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0058] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for stitching images based on multiple cameras, characterized in that, include: Acquire a sequence of raw image frames simultaneously captured by at least two adjacent cameras; Each frame of the original image is preprocessed to generate a corrected, aligned image; An initial pixel-level difference cost map is constructed based on the aligned image; Based on the optimal seam position information determined in the previous frame, a spatiotemporal guidance weight map is generated. The initial pixel-level difference cost map is weighted and fused with the spatiotemporal guided weight map to obtain a dynamic optimized cost map. Perform a shortest path search within the restricted region on the dynamic optimization cost graph to determine the optimal stitching seam for the current frame; The adjacent aligned images are weighted and fused according to the optimal stitching seam to generate the final stitched image.

2. The image stitching method based on multiple cameras according to claim 1, characterized in that, The acquisition of the raw image frame sequence simultaneously captured by at least two adjacent cameras includes: Multiple cameras are controlled to complete image acquisition within the same exposure period by hardware synchronous triggering signals. The original image is raw sensor data without geometric transformation.

3. The image stitching method based on multiple cameras according to claim 2, characterized in that, The preprocessing of each original image frame to generate the corrected aligned image includes: Using pre-calibrated camera intrinsic and extrinsic parameter matrices, perspective transformation and distortion correction are performed on the original image; The transformed image is resampled using a bilinear interpolation algorithm, so that the imaging areas of adjacent cameras have overlapping areas in the world coordinate system.

4. The image stitching method based on multiple cameras according to claim 3, characterized in that, The construction of the initial pixel-level difference cost map based on the aligned image includes: Within the overlapping area, the color space difference value is calculated for the corresponding pixels of two adjacent aligned images. The color space is CIELAB color space. The color space difference value is calculated using the Euclidean distance formula, which is to calculate the difference between the three channels L*, a*, and b* of each pixel, square the sum of the squares, and then take the square root. Normalize the difference values ​​of all pixels to an integer range of 0 to 255 to form an initial pixel-level difference cost map.

5. The image stitching method based on multiple cameras according to claim 4, characterized in that, The step of generating a spatiotemporal guidance weight map by combining the optimal stitching seam position information determined in the previous frame includes: Using the best stitching seam of the previous frame as the center line, construct a strip-shaped guide area within the overlapping area of ​​the current frame; Within the strip-shaped guiding area, each pixel is assigned a guiding weight value, which decreases in a Gaussian distribution as it moves vertically from the center line.

6. The image stitching method based on multiple cameras according to claim 5, characterized in that, The step of weightedly fusing the initial pixel-level difference cost map with the spatiotemporal guided weight map to obtain a dynamically optimized cost map includes: Perform element-wise addition on the pixel values ​​at the same position in the two images to obtain the fused cost. If the value after fusion is greater than 255, then it is truncated to 255; In the final dynamic optimization cost map, the value of each pixel represents the comprehensive cost of setting the seam at that location.

7. The image stitching method based on multiple cameras according to claim 6, characterized in that, The step of performing a shortest path search within a restricted region on the dynamic optimization cost graph to determine the optimal stitching seam for the current frame includes: The overlapping regions of the dynamic optimization cost graph are modeled as directed graphs, with each pixel being a node in the graph, and each node having a directed edge established with the three adjacent nodes in the row below it. The weight of an edge is equal to the cost of the target node; Starting from any starting node at the top of the overlapping region, use dynamic programming to calculate the minimum cumulative cost path to any ending node at the bottom. The path with the minimum cumulative cost is the optimal stitching seam for the current frame, which consists of a series of continuous pixel coordinates.

8. The image stitching method based on multiple cameras according to claim 7, characterized in that, The step of weighted fusion of adjacent aligned images based on the optimal stitching seam to generate the final stitched image includes: Using the optimal seam as the dividing line, the left image pixels are completely preserved on the left side of the overlapping area, and the right image pixels are completely preserved on the right side. A transition zone of 5 pixels is extended on both sides of the splicing seam. Within the transition zone, a linear gradient weight is used to perform a weighted average of the pixels in the left and right images. The weight on the left decreases from 1 to 0, and the weight on the right increases from 0 to 1. Non-overlapping areas are directly copied from the corresponding image content, and finally a panoramic image without visible seams is synthesized.

9. The image stitching method based on multiple cameras according to claim 8, characterized in that, The method also includes an exception handling mechanism: When the motion amplitude between the current frame and the previous frame is detected to be greater than a preset threshold, it is determined to be a violent motion scene; the generation of spatiotemporal guidance weight map using the stitching seam information of the previous frame is suspended, and the minimum difference path search strategy in the whole region is adopted instead. The motion amplitude is quantized by calculating the average magnitude of the optical flow vector in the overlapping region between two frames.

10. The image stitching method based on multiple cameras according to claim 9, characterized in that, The method maintains a seam history buffer queue during execution, and the queue stores the best seam coordinate sequence of the most recent 5 frames; When generating the spatiotemporal guide weight map, not only the stitching seam of the previous frame is used, but also all stitching seams in the historical queue are weighted and averaged to obtain a more stable guide center line. The weights of each historical frame decrease exponentially in reverse chronological order.

Images