Multi-channel video real-time splicing processing system capable of dynamically adjusting parameters

Abstract

The invention belongs to the technical field of video processing, and particularly relates to a multi-channel video real-time splicing processing system capable of dynamically adjusting parameters, which is based on an embedded processing platform and dynamically adjusts related parameters of a splicing algorithm according to scene features (such as light intensity, field angle, baseline length and the like) of an input video. Comprising field angle correction, exposure compensation and dynamic adjustment of baseline length. The system ensures that high-quality video stitching and fusion are realized in different scenes through weighted fusion of pixel values of video overlapping regions. Aiming at a scene with large illumination variation, the system can automatically adjust a brightness parameter to realize effective exposure compensation; when the field angle changes, the system dynamically adjusts the correction of the field angle to ensure the seamless splicing of the images. Besides, the dynamic adjustment of the length of the base line is helpful for realizing accurate geometric calibration under different camera positions or movement conditions, so that splicing misalignment caused by difference of visual angles of the camera is reduced, and the consistency of spliced images is ensured.

Description

Technical Field

[0001] This invention belongs to the field of video processing technology, specifically to a system for real-time multi-channel video splicing and processing with dynamically adjustable parameters. Based on an embedded processing platform, this system enables real-time splicing and fusion of multiple video channels. It is widely applicable to complex scenarios requiring multi-channel video processing, such as autonomous driving, security monitoring, and mapping. It exhibits exceptional adaptability to dynamic scene changes. Background Art

[0002] With the continuous advancement of video technology, real-time multi-channel video stitching is increasingly being used in fields such as autonomous driving, security monitoring, and surveying and mapping. However, existing technologies often lack the ability to adapt to dynamic environments such as changing lighting or rapidly shifting camera angles. This results in low stitching efficiency, poor real-time performance, and unstable stitching quality, making it difficult to ensure high-quality stitching. Therefore, how to efficiently achieve real-time stitching of multiple channels in diverse environments has become a pressing technical challenge. Summary of the Invention

[0003] 1. Technical issues to be resolved

[0004] The technical problem to be solved by the present invention is: to provide a multi-channel video real-time splicing processing system based on an embedded processing platform, which can dynamically adjust the splicing algorithm parameters according to the changes in the video scene, improve the efficiency and real-time performance of the splicing processing, and optimize the splicing quality, especially when dealing with dynamically changing complex scenes, thereby solving the splicing defects in the existing technology.

[0005] (II) Technical solution

[0006] To solve the above technical problems, the present invention provides a multi-channel video real-time splicing processing system with dynamically adjustable parameters. The system is based on an embedded processing platform and integrates multiple image processing algorithms to achieve efficient multi-channel video splicing and output. The system includes:

[0007] The image download module uses a universal image interface to obtain raw video image data from multiple camera video sources and receives and decodes it within the FPGA. The decoded video image data is transmitted to the embedded processor via a high-speed bus for format and resolution conversion to ensure efficient transmission and consistency of the video image data during subsequent processing.

[0008] An image scene analysis module that analyzes and extracts the scene information of the input video image data before image stitching processing; by analyzing the image illumination intensity change, camera field of view angle, and baseline length parameter, dynamically adjusts the input image brightness, camera field of view angle, focal length, and baseline length, and dynamically adjusts the geometric correction part in the stitching algorithm to ensure the best geometric correction effect before stitching;

[0009] An image stitching and fusion module that adopts multiple algorithms including cylindrical projection and weighted stitching and fusion based on the dynamically adjusted relevant parameters; performs task scheduling and pixel point calculation in an embedded processor to quickly stitch and fuse multi-channel image data to achieve efficient stitching and fusion of images;

[0010] An image uploading module that converts the stitched and fused image data into the resolution format required by the user and uploads it to the FPGA for encoding through a high-speed bus; finally, outputs a high-definition stitched and fused video in real time through a serial interface.

[0011] Among them, the image scene analysis module includes:

[0012] An image illumination intensity analysis unit that detects the change of image illumination intensity, expected and actual brightness values, and brightness sensitivity constant, and dynamically adjusts the input image brightness to ensure the brightness balance of each channel and eliminate obvious edges in the stitching area;

[0013] A field of view angle analysis unit that estimates the dynamic changes of the field of view angle and focal length based on the detection and matching of fast feature points, depth information of calibration points, and perspective projection model; then adjusts the geometric correction model according to the changed field of view angle and focal length to ensure the precise alignment of the image in the coordinate system during the stitching process;

[0014] A baseline length analysis unit that estimates the relative position change and baseline length between cameras by using the fundamental matrix and essential matrix and the matched feature points to ensure that precise geometric calibration has been completed before cylindrical projection of the cameras.

[0015] Among them, the image stitching and fusion module includes:

[0016] A cylindrical projection stitching unit that uses the field of view angle and focal length provided by the image scene analysis module to convert the image from the plane coordinate system to the cylindrical coordinate system through the cylindrical projection formula; this link ensures the precise alignment and fusion of multi-channel videos in the same coordinate system, thereby improving the overall accuracy and quality of stitching and meeting the real-time processing requirements in dynamic scenes;

[0017] An overlapping region weight fusion unit, which, through a weighted stitching fusion algorithm, dynamically determines the weighted processing strategy for pixels in the overlapping region based on the distance and visual features of the edges of multiple video stitched images, and performs task scheduling and stitching fusion in real time. This process effectively eliminates the abruptness at the stitching edges, ensures seamless connection between images, and enhances the overall visual effect. In addition, this method takes into account the influence of illumination changes and moving objects in different scenarios, further optimizing the stitching effect and ensuring high-quality output even in complex environments.

[0018] Among them, the image interface is Cameralink.

[0019] Among them, the high-speed bus is PCIe.

[0020] Among them, the serial interface is a digital component serial interface, Serial Digital Interface, SDI.

[0021] Among them, the image upload module can support multiple output formats and resolutions, and has a real-time encoding function to meet the output requirements in different application scenarios.

[0022] (III) Beneficial effects

[0023] Compared with the prior art, the present invention discloses a multi-channel video real-time stitching processing system, which includes an image download module, an image scene analysis module, an image stitching fusion module, and an image upload module. Through the collaborative work of these modules, real-time stitching, fusion, and output of multi-channel videos are achieved, and the system can dynamically adjust the parameters in the stitching algorithm according to scene changes to ensure the stitching quality in different scenarios. The system supports multiple video formats and resolutions and is applicable to various complex application scenarios.

[0024] In the existing video stitching technology, when dealing with multi-channel high-resolution videos, in the face of rapidly changing illumination and scenes, problems such as poor stitching effect, even misalignment and ghosting often occur. How to dynamically adjust parameters to improve the stitching quality while ensuring the stitching speed has become the core technical breakthrough point of the present invention.

[0025] Compared with the prior art, the multi-channel video real-time stitching processing system of the present invention has the following

[0026] Beneficial effects:

[0027] 1. Improve stitching efficiency and real-time performance: The system can dynamically adjust stitching parameters to adapt to scene changes (such as illumination and camera parameter changes), ensuring real-time performance in high-resolution video processing.

[0028] 2. Enhance stitching quality: By adjusting parameters adaptively, reduce problems such as ghosting and misalignment at the stitching edges, and achieve seamless stitching under different lighting conditions.

[0029] 3. Support multiple formats and resolutions: The system supports the input and output of multiple video formats and resolutions, meeting the requirements of different application scenarios.

[0030] 4. The system is efficient and compact: Suitable for integration on embedded platforms, with low power consumption and efficient image processing, and is widely used in scenarios such as autonomous driving, security monitoring, and surveying and mapping. Brief Description of the Drawings

[0031] Figure 1 It is a schematic diagram of a multi-channel video real-time stitching processing system.

[0032] Figure 2 It is a schematic diagram of the overall framework of a multi-channel video real-time stitching processing system.

[0033] Figure 3 It is a schematic diagram of the display effect of multi-channel video real-time stitching. Detailed Embodiment

[0034] To make the objectives, contents, and advantages of the present invention clearer, the following further describes the detailed embodiments of the present invention in conjunction with the drawings and embodiments.

[0035] To solve the above technical problems, the present invention provides a multi-channel video real-time stitching processing system with dynamically adjustable parameters. The system is based on an embedded processing platform, integrates multiple image processing algorithms, and realizes efficient multi-channel video stitching and output. The system includes:

[0036] An image download module, which uses a general image interface to obtain original video image data from multiple camera video sources, and receives and decodes it within the FPGA; the decoded video image data is transmitted to the embedded processor through a high-speed bus for format and resolution conversion to ensure the efficient transfer and consistency of the video image data during subsequent processing.

[0037] An image scene analysis module, which analyzes and extracts the scene information of the input video image data before image stitching processing; by analyzing the image light intensity change, camera field of view angle, and baseline length parameters, it dynamically adjusts the input image brightness, camera field of view angle, focal length, and baseline length, and dynamically adjusts the geometric correction part in the stitching algorithm to ensure the best geometric correction effect before stitching.

[0038] The image stitching and fusion module, based on dynamically adjusted relevant parameters, adopts multiple algorithms including cylindrical projection and weighted stitching and fusion; performs task scheduling and pixel point calculation in an embedded processor to quickly stitch and fuse multi-channel image data, achieving efficient stitching and fusion of images;

[0039] The image upload module converts the stitched and fused image data into the resolution format required by the user and uploads it to the FPGA for encoding through a high-speed bus; finally, outputs a high-definition stitched and fused video in real time through a serial interface.

[0040] Among them, the image scene analysis module includes:

[0041] The image illumination intensity analysis unit detects the change of image illumination intensity, the expected and actual brightness values, and the brightness sensitivity constant, and dynamically adjusts the brightness of the input image to ensure the brightness balance of each channel and eliminate the obvious edges in the stitching area;

[0042] The field of view angle analysis unit estimates the dynamic changes of the field of view angle and focal length based on the detection and matching of fast feature points, the depth information of calibration points, and the perspective projection model; then adjusts the geometric correction model according to the changing field of view angle and focal length to ensure the precise alignment of the image in the coordinate system during the stitching process;

[0043] The baseline length analysis unit estimates the relative position change and baseline length between cameras by using the fundamental matrix and essential matrix and the matched feature points to ensure that the cameras have completed precise geometric calibration before cylindrical projection.

[0044] Among them, the image stitching and fusion module includes:

[0045] The cylindrical projection stitching unit uses the field of view angle and focal length provided by the image scene analysis module to convert the image from the plane coordinate system to the cylindrical coordinate system through the cylindrical projection formula; this link ensures the precise alignment and fusion of multi-channel videos in the same coordinate system, thereby improving the overall accuracy and quality of stitching and meeting the real-time processing requirements in dynamic scenes;

[0046] The overlapping area weight fusion unit dynamically determines the weighted processing strategy for the pixels in the overlapping area according to the distance and visual features of the stitching edges of multi-channel videos through the weighted stitching and fusion algorithm, and performs task scheduling and stitching and fusion in real time; this process effectively eliminates the abruptness of the stitching edges, ensures the seamless connection between images, and improves the overall visual effect; in addition, this method considers the influence of illumination changes and moving objects in different scenarios, further optimizes the stitching effect, and ensures high-quality output in complex environments.

[0047] Among them, the image interface is Cameralink

[0048] Among them, the high-speed bus is PCIe.

[0049] Among them, the serial interface is a digital component serial interface, Serial Digital Interface, SDI.

[0050] Among them, the image upload module can support multiple output formats and resolutions, and has a real-time encoding function to meet the output requirements in different application scenarios.

[0051] Embodiment 1

[0052] 1. Overall framework and core module design of the system in this embodiment

[0053] The overall framework of the multi-channel video real-time stitching processing system is as Figure 2 shown.

[0054] After the video image data is input through the video image interface, it is directly stored in the memory using a high-speed bus system (such as PCIe QDMA) to achieve efficient data transmission.

[0055] After the FPGA stores the image data in the buffer through the high-speed bus, the embedded processor reads the image data in the buffer through the underlying driver for format conversion or stitching and fusion processing. After the processing is completed, real-time output of the video image is achieved through the high-speed bus system and the serial interface.

[0056] 1.1 Implementation method of the image download module:

[0057] 1.1.1 Connection and data acquisition: This module is connected to multiple cameras through the image interface to acquire the original image data transmitted by the cameras. For example, the Cameralink interface is used, which is a high-speed image transmission interface based on a serial communication protocol and is widely used in the field of high-speed image acquisition and processing. In the present invention, the image data is directly received and decoded inside the FPGA, thereby significantly improving the speed and efficiency of image transmission and processing.

[0058] 1.1.2 Data decoding and information extraction: In the FPGA, the received image data is decoded to extract effective image information to ensure the accuracy and integrity of the data required for subsequent steps.

[0059] 1.1.3 Format conversion and transmission: The decoded image data is transmitted to the embedded processor through the high-speed bus, and format and resolution conversion are performed in the processor to ensure that the image data can be efficiently transmitted and maintain consistency during subsequent processing.

[0060] 1.2 Implementation method of the image scene analysis module:

[0061] The image scene analysis module detects key scene features such as light changes in the video and adjusts the camera perspective by obtaining and analyzing the environmental information of the input video in real time, using the following methods.

[0062] 1.2.1 Light intensity analysis and adjustment method: This method can monitor the changes in light conditions in the scene in real time and dynamically adjust the pixel values of the input image according to the real-time and expected light intensities. When the light intensity fluctuates significantly, the system will automatically adjust the brightness weight, and then change the input pixel values of the multi-channel images to ensure a smooth transition of the brightness in the overlapping area, thereby eliminating obvious stitching traces.

[0063] Light intensity calculation formula:

[0064] The light intensity L can be represented by the average of the brightness values of the image pixels:

[0065]

[0066] Where:

[0067] ① L represents the light intensity.

[0068] ② N is the total number of pixels in the image.

[0069] ③ I(i) is the brightness value of the i-th pixel.

[0070] Dynamically adjust the brightness of the input image: When a significant change in light intensity is detected, the system will dynamically adjust the weight W b , to change the input pixel values of the multi-channel images. This adjustment ensures that the brightness of the image can adapt to environmental changes in a timely manner, thereby achieving a smoother transition in the overlapping area and reducing the possible visual incoherence during the stitching process. The brightness weight adjustment formula can be expressed as follows:

[0071]

[0072] Where:

[0073] ① W b represents the weight coefficient of brightness.

[0074] ② α is a constant used to control the sensitivity of weight adjustment.

[0075] ③ L target is the target brightness value (the brightness that is expected to be achieved during the smooth transition period. Assuming a monitoring system with obvious light changes outdoors during the day, the target brightness set is 128).

[0076] Analysis and Adjustment Method for Camera Field of View Angle and Baseline Length: In video image stitching processing, the field of view angles and baseline lengths of different cameras are crucial for image preprocessing and alignment. To ensure accurate geometric correction of multiple video sources during subsequent cylindrical projection and stitching, the image scene analysis module re-estimates the relative positions (i.e., external parameters) of the cameras by quickly detecting feature points or known markers in the scene. Thus, the system can dynamically adjust the baseline length and field of view angle to ensure more accurate geometric correction during the stitching process, enabling better fusion of multiple videos in cylindrical projection.

[0077] The formula methods involved in camera field of view angle analysis and adjustment are as follows:

[0078] (1) Feature Point Detection and Matching:

[0079] First, detect the set of feature points P in the scene through the feature point detection algorithm ORB A (p A 1 ~p A n (feature points of this set) and P B (p B 1 ~p B n (feature points of this set), which come from the images of camera A and camera B respectively:

[0080]

[0081] Match the set of feature points of camera A with the set of feature points of camera B through the feature matching method BFMatcher to obtain a set of matching point pairs:

[0082] (p A i , p B i ) i = 1, 2,... s

[0083] In the above formula, s is the total number of feature matching points.

[0084] (2) Estimation of Field of View Angle

[0085] The estimation of the field of view angle is completed through the following steps:

[0086] Depth information of calibration points: By using known markers in the scene or through the depth estimation method of stereo vision, the depth values z of some matching points captured by camera A and camera B can be calculated.

[0087] Camera imaging model: Using the perspective projection model, the real-world coordinates (X, Y, Z) corresponding to the points (x, y) on the camera imaging plane are related by the following formula:

[0088]

[0089] where f is the focal length, z is the depth, and x and y are the pixel coordinates in the image.

[0090] (3) Derivation of the field of view angle: When the true size w of the marker in the scene is known (e.g., the width of an object), the field of view angle of the camera can be estimated based on the position of the feature points in the image and the depth information. The formula for calculating the field of view angle θ is:

[0091]

[0092] where w is the width of the object captured by the camera, and z is the distance from the object to the camera. By detecting the feature point pairs and the known depth information, we can estimate the field of view angle of the camera. By obtaining the field of view angle information of the camera in real time, the system can dynamically adjust the focal length f to adapt to the changes in the scene. If the scene analysis function detects that the field of view angle needs to be expanded (such as for a larger range of stitching), the focal length will be correspondingly reduced to increase the field of view angle; vice versa.

[0093] The formula methods involved in the baseline length analysis and adjustment are as follows:

[0094] (1) Fundamental matrix and essential matrix

[0095] The relative position and pose (extrinsic parameters) between cameras can be estimated by calculating the fundamental matrix F or the essential matrix E for the matched feature point pairs. Among them, the essential matrix E can be calculated from the fundamental matrix F and the intrinsic matrices K A and K B as follows:

[0096]

[0097] where F is the fundamental matrix, and K A and K B are the intrinsic matrices of camera A and camera B.

[0098] (2) Camera extrinsic parameter estimation

[0099] Through the essential matrix E, we can further decompose the relative rotation matrix R and translation vector t between camera A and camera B:

[0100] E = [t]×R

[0101] where, [t] xis the skew-symmetric matrix of the translation vector t, R is the rotation matrix between cameras, and t is the translation vector between cameras, i.e., the baseline length. It serves as the basis for adjusting weights during the stitching and fusion process to ensure geometric correction under different camera positions and perspectives.

[0102] (3) Application of Geometric Correction

[0103] Through the above dynamic adjustment of the baseline length t and the field of view angle θ, the geometric correction formula can be applied in combination with the cylindrical projection model. Before geometric correction, ensure the alignment of the camera's perspective and the adjustment of the baseline:

[0104]

[0105] Among them, x' and y' are the image coordinates after perspective alignment and baseline adjustment. During the cylindrical projection process, the dynamically adjusted baseline length and field of view angle are used to precisely adjust the projection model to ensure more accurate stitching of the images of each camera in the same coordinate system.

[0106] (4) Effect of Final Geometric Correction

[0107] Through the dynamic adjustment of these extrinsic parameters, the images can be more precisely aligned during stitching, reducing problems such as misalignment and ghosting during the stitching process and improving the overall stitching effect.

[0108] These formulas and processes detail how to use feature points or known markers to estimate the field of view angle and baseline length of the camera through the image scene analysis module, thereby dynamically adjusting the geometric correction part in the stitching algorithm.

[0109] 1.3 Implementation Modes of the Image Stitching and Fusion Module:

[0110] Once the image points of all cameras have been geometrically corrected and aligned, the next step is to apply these aligned coordinate points to the cylindrical projection formula, that is, to convert the image from the planar coordinates to the cylindrical coordinate system so that multiple videos can be correctly aligned and fused in the same coordinate system. Subsequently, through the weighted stitching and fusion algorithm, the pixels in the overlapping area are weighted to eliminate the abruptness at the stitching edge and achieve a seamless stitching effect of the images.

[0111] Cylindrical Projection Method: Assume that there is a pixel point on the imaging plane of the camera, and its coordinates in the planar coordinate system are (x, y), and the focal length of the camera is f. In the cylindrical projection, the coordinates of this pixel point in the cylindrical coordinate system are (σ, h), where:

[0112]

[0113] σ represents the horizontal position, and h represents the vertical height. Through this conversion, the pixels of multiple images can be mapped into the same cylindrical coordinate system.

[0114] Weighted stitching and fusion method: To eliminate the abruptness at the stitching edge, the weighted average method is used to fuse the pixel values in the overlapping area. Suppose the pixel values of two images in the overlapping area are I1(x, y) and I2(x, y) respectively, and their weights are w1(x, y) and w2(x, y). Then, in the fused image, the value I(x, y) of this pixel is calculated as follows:

[0115]

[0116] Generally, the weights w1(x, y) and w2(x, y) can be determined according to the distance from the stitching edge. The farther away from the stitching edge, the greater the weight, thus ensuring a smooth transition in the stitching area. In the method of the present invention, dynamically adjusted parameters (such as illumination conditions, field of view angle changes, and baseline length) also affect the calculation of the weights. When the system detects a significant change in illumination intensity, it will adjust the weights accordingly to balance the brightness; at the same time, as the field of view angle and baseline length change, the system will update the weights to adapt to the new geometric correction, thereby further optimizing the stitching effect and ensuring seamless fusion.

[0117] For example, assuming the distance from the stitching edge to a certain pixel point is d(x, y), the weights can be calculated using the following formula:

[0118] and w2(x, y) = 1 - w1(x, y)

[0119] In practical applications, assuming d(x, y) is 5, the calculated results are:

[0120]

[0121] If the system detects a significant change in illumination intensity, for example, the indoor light source suddenly increases, the brightness weights may be dynamically adjusted to increase the value of w1(x, y). For example, if d(x, y) becomes 3 at this time, the new weights are:

[0122]

[0123] This dynamic weight adjustment not only improves the smooth transition effect in the stitching area but also ensures that the stitched images maintain a high-quality fusion effect under different illumination and geometric conditions.

[0124] Task scheduling and real-time method for stitching and fusion: In an embedded processor, the task scheduling mechanism ensures that the stitching and fusion processes of each image can be efficiently completed in a short time, thereby significantly improving the real-time performance of the system. By optimizing the scheduling strategy, the embedded processor can quickly process multi-channel image data within the specified time, ensuring the stitching effect while meeting the real-time requirements. Suppose the system requires within ttotal Complete the stitching and fusion of Q images within a certain time, and the stitching and fusion of each image requires time t i (i = 1, 2,..., Q), then the real-time requirement of the system can be expressed as:

[0125]

[0126] By optimizing the scheduling algorithm, it can be ensured that all images are stitched and fused within the specified time, thereby improving the real-time performance of the system.

[0127] 1.4 Implementation method of the image upload module:

[0128] 1.4.1 After the stitched and fused image data is converted in format and resolution, it meets the output requirements set by the user.

[0129] Format and resolution conversion method of image data: In the process of processing the stitched and fused image data, it is first necessary to ensure that the image data meets the output requirements set by the user, including format and resolution conversion. The following steps are involved:

[0130] (1) Format conversion: It is to convert the image data from one color space or encoding format to another. For example, converting from the RGB color space to the YUV color space to meet the subsequent video processing or transmission requirements. The conversion formula is as follows:

[0131] Conversion from RGB to YUV:

[0132] Y = 0.299R + 0.587G + 0.114B

[0133] U = -0.147R - 0.289G + 0.436B

[0134] V = 0.615R - 0.515G - 0.100B

[0135] (2) Resolution conversion: Resolution conversion is to scale the image to the target resolution, such as scaling from the original resolution W in ×H in to the target resolution W out ×H out , the present invention adopts the bilinear interpolation method, and the interpolation formula is as follows: Let the position (r', u') in the original image correspond to the position (r, u) in the scaled image, then the pixel value I(r, u) is obtained by interpolating the adjacent four pixel values:

[0136]

[0137] where, w m，n is the weight calculated according to the distance.

[0138] 1.4.2 Data Transmission and Coding Processing: After format and resolution conversion, the image data is transmitted to the FPGA through a high-speed bus (such as PCIe). The PCIe protocol supports high-bandwidth transmission. The typical bandwidth of a PCIe 3.0 x16 channel can reach 16 GB / s. For PCIe 3.0, the data transmission rate per line is 8 GT / s, and the coding efficiency is 128b / 130b (about 98.46%). In the FPGA, the image data is encoded into a signal format compliant with the serial interface (such as SDI) standard. The common coding format of the SDI standard is the 4:2:2 YUV format under the BT.709 standard. The specific coding steps include: bit-depth conversion, that is, converting the data to the 10-bit or 12-bit precision required for the SDI signal. Frame encapsulation, that is, packaging the image data into the frame structure of the SDI signal according to the SDI standard, including synchronization, data, and CRC check.

[0139] 1.4.3 Finally, after coding processing, the stitched image data is output as a high-definition stitched video in real time through the serial interface. The real-time stitching display effect of multiple videos is as Figure 3 shown (in the figure, it is the stitching of 3 video images).

[0140] In summary, the present invention belongs to the technical field of video processing, and specifically relates to a real-time stitching processing system for multiple videos with dynamically adjustable parameters. Based on an embedded processing platform, this system can dynamically adjust the relevant parameters of the stitching algorithm according to the scene characteristics of the input video (such as light intensity, field of view angle, baseline length, etc.), including field of view angle correction, exposure compensation, and dynamic adjustment of the baseline length. By weighted fusion of the pixel values in the overlapping area of the video, this system ensures high-quality video stitching and fusion in different scenarios. For scenes with large lighting changes, the system can automatically adjust the brightness parameters to achieve effective exposure compensation; when the field of view angle changes, the system dynamically adjusts the field of view angle correction to ensure seamless stitching of the images. In addition, the dynamic adjustment of the baseline length helps to achieve accurate geometric calibration under different camera positions or motion conditions, thereby reducing stitching misalignment caused by differences in camera viewing angles and ensuring the coherence of the stitched images. Therefore, this system is widely applied to application scenarios with multiple video inputs and complex and diverse environmental changes, such as autonomous driving, security monitoring, and surveying and mapping, and can effectively adapt to real-time scene changes and maintain the high quality and consistency of video output.

[0141] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the technical principle of the present invention, several improvements and modifications can still be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A multi-channel video real-time stitching processing system for dynamically adjusting parameters, characterized in that The system includes: An image download module that uses a general image interface to obtain original video image data from multiple camera video sources, and performs reception and decoding within the FPGA; the decoded video image data is transmitted to the embedded processor through a high-speed bus for format and resolution conversion to ensure efficient transmission and consistency of the video image data during subsequent processing; An image scene analysis module that analyzes and extracts the scene information of the input video image data before image stitching processing; by analyzing the image illumination intensity change, camera field of view angle, and baseline length parameters, it dynamically adjusts the input image brightness, camera field of view angle, focal length, and baseline length, and dynamically adjusts the geometric correction part in the stitching algorithm to ensure the best geometric correction effect before stitching; An image stitching and fusion module that adopts multiple algorithms including cylindrical projection and weighted stitching and fusion based on the dynamically adjusted relevant parameters; performs task scheduling and pixel point calculation in the embedded processor to quickly stitch and fuse multiple-channel image data to achieve efficient stitching and fusion of images; An image upload module that converts the stitched and fused image data into the resolution format required by the user and uploads it to the FPGA through a high-speed bus for encoding; finally, it outputs a high-definition stitched and fused video in real time through a serial interface.

2. The multi-channel video real-time splicing and processing system for dynamically adjusting parameters according to claim 1, characterized in that, The image scene analysis module includes: An image illumination intensity analysis unit that dynamically adjusts the input image brightness by detecting the image illumination intensity change, expected and actual brightness values, and brightness sensitivity constant to ensure the brightness balance of each channel and eliminate obvious edges in the stitching area; A field of view angle analysis unit that estimates the dynamic changes of the field of view angle and focal length based on the detection and matching of fast feature points, depth information of calibration points, and perspective projection model; then adjusts the geometric correction model according to the changed field of view angle and focal length to ensure accurate alignment of the image in the coordinate system during stitching; A baseline length analysis unit that estimates the relative position change and baseline length between cameras by using the fundamental matrix and essential matrix and the matched feature points to ensure accurate geometric calibration of the cameras before cylindrical projection.

3. The multi-channel video real-time stitching and processing system for dynamically adjusting parameters according to claim 2, characterized in that, The image stitching and fusion module includes: A cylindrical projection stitching unit that uses the field of view angle and focal length provided by the image scene analysis module to convert the image from the plane coordinate system to the cylindrical coordinate system through the cylindrical projection formula; An overlapping area weight fusion unit that dynamically determines the weighted processing strategy for overlapping area pixels according to the distance and visual features of the edges of multiple-channel video stitching through the weighted stitching and fusion algorithm, and performs task scheduling and stitching and fusion in real time.

4. The multi-channel video real-time stitching processing system for dynamically adjusting parameters as claimed in claim 3, wherein The working process of the cylindrical projection stitching unit ensures the accurate alignment and fusion of multiple-channel videos in the same coordinate system, thereby improving the overall accuracy and quality of stitching and meeting the real-time processing requirements in dynamic scenes.

5. The multi-channel video real-time stitching processing system for dynamically adjusting parameters according to claim 3, wherein, The working process of the overlapping area weight fusion unit effectively eliminates the abruptness at the stitching edge, ensures seamless connection between images, and improves the overall visual effect.

6. The multi-channel video real-time stitching processing system for dynamically adjusting parameters according to claim 3, wherein, The working process of the overlapping area weight fusion unit takes into account the influence of light changes and moving objects in different scenarios, further optimizing the stitching effect and ensuring high-quality output even in complex environments.

7. The multi-channel video real-time stitching processing system for dynamically adjusting parameters according to claim 1, characterized in that, The image interface is Cameralink.

8. The multi-channel video real-time stitching processing system for dynamically adjusting parameters according to claim 1, wherein, The high-speed bus is PCIe.

9. The multi-channel video real-time stitching processing system for dynamically adjusting parameters according to claim 1, characterized in that, The serial interface is a digital component serial interface.

10. The multi-channel video real-time stitching processing system for dynamically adjusting parameters according to claim 1, characterized in that, The image upload module can support multiple output formats and resolutions and has a real-time encoding function to meet the output requirements in different application scenarios.

Images