Video transmission high-definition image intelligent splicing method and system

Abstract

The application relates to the technical field of image splicing, and discloses a video transmission high-definition image intelligent splicing method and system; the system comprises an image preprocessing module, a feature point matching and screening module, an image optimal splicing seam generation module and a video output module; first, a video is acquired and dynamic range equalization is carried out, an adaptive noise reduction method is used for filter processing, and image correction is implemented; second, an adaptive corner point detection algorithm is used for feature point extraction, feature points are matched based on multidimensional space data indexing, a feature point matching pair is obtained, and feature point screening is carried out; then, an overlapping area is searched, a dynamic search algorithm is introduced to generate an image optimal splicing path, and an image optimal splicing seam is obtained; finally, the image is divided according to a dynamic grid, and video output is realized by using priority sorting; the application realizes the purpose of image intelligent splicing by processing and splicing the video image, and the method is accurate and objective.

Description

Technical Field

[0001] This invention relates to the technical field of image stitching, specifically to a method and system for intelligent stitching of high-definition images for video transmission. Background Technology

[0002] With the rapid development of information technology, the demand for the acquisition, transmission, and presentation of high-definition video content has exploded. In a wide range of application scenarios such as security monitoring, live streaming, video conferencing, virtual reality, and drone aerial photography, the field of view and resolution of a single camera device are often insufficient to meet the requirements of panoramic coverage and high-detail display. Therefore, transmitting video streams from multiple cameras and performing real-time or near-real-time image stitching at the receiving end to form seamless high-resolution panoramic video or wide-angle views has become an important technical means to obtain high-quality, wide-field-of-view images. Currently, the widespread adoption of high-speed wireless communication technology and fiber optic networks provides higher bandwidth and lower latency possibilities for the transmission of high-definition video streams. However, how to efficiently, robustly, and intelligently complete the reception, registration, fusion, and stitching of multiple high-definition video streams to form high-quality panoramic images remains a highly challenging technical hurdle.

[0003] Traditional image stitching methods mainly use optical flow-based methods, which utilize the pixel motion information (optical flow) between adjacent frames to perform continuous registration and stitching, taking advantage of the continuity of the video stream; or they use algorithms such as SIFT and SURF to detect and match significant feature points between images, then perform coordinate transformation and image deformation, and finally use specific fusion algorithms (such as linear weighting and multi-band fusion) to eliminate stitching seams.

[0004] Traditional image stitching methods involve huge computational demands when processing high-resolution images. Traditional feature extraction, matching, and global optimization fusion algorithms are too time-consuming and cannot meet the real-time requirements for low-latency stitching output. The stitching process suffers from high latency, poor quality, low intelligence, and a lack of end-to-end optimization. Summary of the Invention

[0005] To address the problems in related technologies, this invention provides a high-definition image intelligent stitching system for video transmission, thereby overcoming the aforementioned technical problems in existing related technologies.

[0006] To solve the aforementioned technical problem, the present invention is achieved through the following technical solution:

[0007] This invention is a high-definition image intelligent stitching system for video transmission, specifically including: an image preprocessing module, a feature point matching and filtering module, an image optimal stitching seam generation module, and a video output module;

[0008] The image preprocessing module is used to acquire the transmitted video to be processed and perform dynamic range equalization, then use an adaptive noise reduction method for filtering, and implement image correction to generate a set of processed images to be stitched together.

[0009] The feature point matching and filtering module uses an adaptive corner detection algorithm to extract feature points from the processed image set to be stitched, obtains feature point matching pairs based on multidimensional spatial data indexing, and then introduces an improved RANSAC algorithm to filter feature points, outputting the processed image set to be stitched.

[0010] The image optimal stitching seam generation module is used to find the overlapping area of ​​the processed set of images to be stitched, and to obtain the optimal image stitching seam based on the minimum pixel difference by introducing a dynamic search algorithm.

[0011] The video output module is used to stitch images according to the best stitching seam, divide images according to a dynamic grid, and use priority sorting to achieve video output.

[0012] Preferably, the step of acquiring the transmitted video to be processed and performing dynamic range equalization includes:

[0013] The transmitted video to be processed is acquired, segmented into frames to obtain an initial set of images to be stitched, and divided into non-overlapping sub-images to be stitched. The pixel values ​​of the sub-images to be stitched are then counted.

[0014] The system sets up overexposed, underexposed, and normal exposure areas, divides the pixel values ​​of the sub-images to be stitched into these areas, outputs adjusted pixel values, and generates a balanced set of images to be stitched.

[0015] Preferably, the filtering process using the adaptive noise reduction method includes:

[0016] Identify the moving and static regions of the set of images to be stitched together;

[0017] For the static region, bilateral filtering is used to generate denoised pixel values, which are then used to replace the pixel values ​​in the static region to obtain a denoised static region; for the moving region, adaptive median filtering is used to denoise, resulting in a set of filtered images to be stitched together.

[0018] Image correction is performed on the filtered set of images to be stitched together to generate a processed set of images to be stitched together.

[0019] Preferably, the feature point extraction includes:

[0020] An adaptive threshold is obtained from the processed set of images to be stitched, followed by preliminary screening and full ring detection to obtain a set of feature pixels.

[0021] The feature pixels in the feature pixel set are selected as the center, and the image size is scaled according to the pyramid hierarchy to form a three-layer pyramid structure. Then, Gaussian difference detection is performed to calculate a 128-dimensional feature vector. The feature vectors of the feature pixel set are extracted in sequence to form feature points, and a feature point set is obtained.

[0022] Preferably, obtaining feature point matching pairs based on multidimensional spatial data indexing includes:

[0023] In the set of feature points, the dimension with the largest variance is selected as the root node of the binary tree. At this point, recursive partitioning begins until the number of feature points in the set of feature points is 0, thus generating a spatial index binary tree.

[0024] Search for the nearest and second nearest neighbor feature points of a feature point, and determine whether to accept a match between the nearest and second nearest neighbor feature points to obtain a feature point matching pair.

[0025] Preferably, the feature point selection using the improved RANSAC algorithm includes:

[0026] Select half of the feature point matching pairs, and choose the maximum and minimum feature point matching pair distances to form distance intervals. Discretize the distance intervals into... j Calculate the candidate thresholds;

[0027] The feature point is filtered by traversing all candidate thresholds. When the distance between the candidate thresholds is greater than 2 pixels, it is judged as a mismatch; otherwise, it is judged as a positive match. The accuracy of the positive matches is calculated in turn, and the candidate threshold corresponding to the highest accuracy is taken as the global threshold. The feature point matching pairs are filtered to generate the final feature point matching pairs, and the processed set of images to be stitched is output.

[0028] Preferably, finding the overlapping region of the processed image set to be stitched includes:

[0029] According to the final feature point matching pairs in the processed image set to be stitched, all images to be stitched are stitched together according to the final feature point matching pairs to obtain the overlapping region.

[0030] Preferably, finding the overlapping region of the processed image set to be stitched includes:

[0031] Calculate the color difference value and the structural difference value in the overlapping area to obtain the total difference value;

[0032] Determine the first row of pixels in the overlapping region, calculate the total difference value of the stitched pixels in the first row, then select the next row of pixels, calculate the total difference value of the three adjacent pixels between the next row and the stitched pixels in the first row, and add them to the total difference value of the stitched pixels in the first row; calculate the total difference value of the stitched pixels in the overlapping region in turn, and establish a difference value function.

[0033] Record the minimum total difference value of each layer in the overlapping area, and connect the pixels corresponding to the minimum total difference value of each layer in the overlapping area to form the image stitching path. Continue until the overlapping area is traversed. Then, mark the pixels on the best stitching path of the image in sequence to obtain the best stitching seam of the image.

[0034] Preferably, finding the overlapping region of the processed image set to be stitched includes:

[0035] The images are stitched together according to the optimal seam, overlapping areas are cropped, and the stitched panoramic image is output; the images are divided according to a grid to obtain a moving grid and a static grid.

[0036] Obtain the focal regions of the moving and static grids, calculate the image proportion of the focal regions, set priorities, and prioritize the transmission of stitched panoramic images with higher priorities.

[0037] The present invention has the following beneficial effects:

[0038] 1. This invention acquires the transmitted video to be processed and performs dynamic range equalization, and processes overexposed / underexposed sub-images in partitions to solve the problem of loss of effective information caused by underexposure of video images. At the same time, by using sub-image partition processing and motion region selective filtering, compared with traditional global filtering algorithms, it reduces single-frame processing latency and resource occupation, improves spatial measurement accuracy, and provides a reliable preprocessing foundation for the automation, high precision, and low latency splicing of large-scale video surveillance systems.

[0039] 2. This invention uses an adaptive corner detection algorithm to extract feature points, matches feature points based on multi-dimensional spatial data indexing, and integrates adaptive corner detection and multi-scale pyramid weighted descriptors to significantly improve the stability of dynamic scene features. Then, an improved RANSAC algorithm is introduced for feature point screening. Compared with traditional algorithms, it first performs primary screening and then full-loop verification, reducing a large amount of invalid computation, improving matching speed and reducing mismatch rate, and can successfully screen out feature point pairs that are mismatched.

[0040] 3. This invention finds overlapping areas, introduces a dynamic search algorithm based on the minimum pixel difference to generate the best image stitching path, and obtains the best image stitching seam. It takes color difference and structural difference into account as constraints, and achieves the path search with the minimum cost by finding the minimum pixel difference. This solves the problems of ghosting and breakage that occur in the overlapping areas of the stitched image, and the quality of the fused image is more natural.

[0041] 4. This invention stitches images together using the optimal seam, divides images according to a dynamic grid, and uses priority sorting based on the moving static grid to achieve video output. It ensures redundant transmission of high-priority grids in dual channels, balances details and bandwidth, and enhances bandwidth compression and key information through dynamic grid division and focus priority transmission. It has a high degree of intelligence and achieves stable video transmission. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, the drawings can be obtained from these drawings without creative effort.

[0043] Figure 1 The present invention provides a flowchart of a high-definition image intelligent splicing system for video transmission;

[0044] Figure 2 This invention provides a flowchart illustrating a method for intelligent stitching of high-definition images in video transmission. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Traditional image stitching methods involve huge computational demands when processing high-resolution images. Traditional feature extraction, matching, and global optimization fusion algorithms are too time-consuming and cannot meet the real-time requirements for low-latency stitching output. The stitching process suffers from high latency, poor quality, low intelligence, and a lack of end-to-end optimization.

[0047] To solve the above technical problems, such as Figure 1As shown in the figure, this invention provides a high-definition image intelligent stitching system for video transmission, specifically including: an image preprocessing module, a feature point matching and filtering module, an optimal stitching seam generation module, and a video output module; the image preprocessing module is used to acquire the transmitted video to be processed and perform dynamic range equalization, then use an adaptive noise reduction method for filtering, and implement image correction to generate a processed set of images to be stitched; the feature point matching and filtering module uses an adaptive corner detection algorithm to extract feature points from the processed set of images to be stitched, obtains feature point matching pairs based on multi-dimensional spatial data indexing, and then introduces an improved RANSAC algorithm for feature point filtering, outputting the processed set of images to be stitched; the optimal stitching seam generation module is used to find the overlapping areas of the processed set of images to be stitched, and introduces a dynamic search algorithm based on the minimum pixel difference to obtain the optimal stitching seam; the video output module is used to stitch images according to the optimal stitching seam, divide images according to a dynamic grid, and use priority sorting to achieve video output.

[0048] A specific implementation example is the intelligent reconstruction and expansion project of the ring expressway, spanning 120 kilometers and containing eight types of heterogeneous monitoring equipment (built in batches from 2010 to 2023). Issues include lighting differences (dark areas in tunnels / high-light ramps), lens distortion (old-style PTZ cameras), and frame rate mismatch. The daily traffic flow is 150,000 vehicles, with peak hours exceeding 5,000 vehicles per hour. The existing cameras have a maximum resolution of 1080P, and 72% are outdated standard-definition equipment. A collaborative "end-edge-cloud" solution is constructed: at the terminal layer, dynamic range equalization and noise reduction address uneven lighting and motion blur; at the edge layer, intelligent feature matching and optimal stitching seams achieve seamless alignment and ghosting elimination across cameras; and at the cloud layer, dynamic grid priority transmission reduces bandwidth pressure and ensures real-time analysis of critical events.

[0049] In the specific implementation process of the above embodiments, the transmitted video to be processed is first acquired and dynamic range equalization is performed. Then, an adaptive noise reduction method is used for filtering and image correction is implemented to generate a set of processed images to be stitched. By using dynamic range equalization to partition and process overexposed / underexposed sub-images, the problem of loss of effective information caused by underexposure of video images is solved. At the same time, sub-image partitioning and motion region selective filtering are used. Compared with traditional global filtering algorithms, this reduces single-frame processing latency and resource consumption, improves spatial measurement accuracy, and provides a reliable preprocessing foundation for automated, high-precision, and low-latency stitching of large-scale video surveillance systems. Secondly, an adaptive corner detection algorithm is used to extract feature points. Feature points are matched based on multi-dimensional spatial data indexing, and an improved RANSAC algorithm is introduced for feature point filtering, outputting a set of processed images to be stitched. This method integrates adaptive corner detection and multi-scale pyramid weighted descriptors, significantly improving the stability of dynamic scene features. The introduction of a spatial index binary tree accelerates matching. Compared with traditional algorithms, the improved algorithm is significantly superior. First, a preliminary screening followed by a full-loop inspection reduces a large amount of unnecessary computation, improves matching speed, and lowers the false match rate, successfully filtering out feature point pairs that match incorrectly. Next, overlapping areas are searched, and a dynamic search algorithm is introduced based on the minimum pixel difference to generate the optimal image stitching path, resulting in the optimal stitching seam. In this process, dynamic programming is used to comprehensively consider color difference and structural differences as constraints, achieving a path search with minimum cost by finding the minimum pixel difference. This solves problems such as ghosting and breaks in overlapping areas of the stitched image, resulting in a more natural image quality after fusion. Finally, image stitching is performed based on the optimal stitching seam. The image is divided according to a dynamic grid, and video output is achieved using priority sorting based on the moving static grid. This method is based on spatial adaptive compression, dynamic scaling of the moving / static grid, and dynamic adaptation of the priority queue to channel quality, ensuring dual-channel redundant transmission of high-priority grids, balancing details and bandwidth. Through dynamic grid division and focus-priority transmission, bandwidth compression and key information are enhanced, resulting in a high degree of intelligence and stable video transmission.

[0050] Furthermore, to better illustrate the technical solutions of the embodiments of the present invention, such as... Figure 2 As shown, a method for intelligent stitching of high-definition images in video transmission is used to describe a high-definition image intelligent stitching system for video transmission. The specific details include the following:

[0051] S1. Acquire the transmitted video to be processed and perform dynamic range equalization to obtain a set of equalized images to be stitched together. Then, use an adaptive noise reduction method for filtering and perform image correction to generate a set of processed images to be stitched together.

[0052] S1 includes the following steps:

[0053] S11. Obtain the transmission video to be processed, and divide the transmission video to be processed into several transmission video images to be processed according to each frame, which are denoted as images to be stitched together, forming an initial set of images to be stitched together; divide the images to be stitched in the initial set of images to be stitched into non-overlapping sub-images to be stitched together, and count the pixel values ​​of the sub-images to be stitched together.

[0054] The system defines overexposed, underexposed, and normal exposure zones. The pixel values ​​of the sub-images to be stitched are divided into these zones. The sub-images to be stitched within the normal exposure zone are preserved. The sub-images to be stitched within the overexposed and underexposed zones are adjusted, and the adjusted pixel values ​​are output. ,in This represents the pixel values ​​of the sub-images to be stitched together. The adjustment factor is 1.5 in overexposed areas and 0.5 in underexposed areas. The adjusted pixel values ​​are used to replace the pixel values ​​of the sub-images to be stitched, resulting in a balanced set of images to be stitched.

[0055] S12. Identify the moving and static regions of the set of images to be stitched together, and combine bilateral filtering and adaptive median filtering to obtain an adaptive noise reduction method. Use the adaptive noise reduction method to filter the moving and static regions respectively to obtain a filtered set of images to be stitched together. The specific steps are as follows:

[0056] S121. Select consecutive frames of images to be stitched from the set of images to be stitched in the balanced image set. For any pixel in the consecutive frames of images to be stitched, calculate the difference between the pixel values ​​of any pixel in the consecutive frames of images to be stitched to obtain the pixel difference of consecutive frames. Set a motion threshold. When the pixel difference of consecutive frames is greater than the motion threshold, mark the moving pixel to form a moving region. Otherwise, obtain a static region.

[0057] S122. For the static region, bilateral filtering is used for denoising. A pixel neighborhood is set in the static region. In the pixel neighborhood, spatial weight is calculated based on the Euclidean distance between pixels. Value range weight is calculated based on the difference of pixel values. The spatial weight and value range weight are multiplied and then weighted averaged to generate denoised pixel values. The denoised pixel values ​​are then replaced with the pixel values ​​in the static region to obtain the denoised static region.

[0058] S123. For the motion region, adaptive median filtering is used for noise reduction. The variance of pixel values ​​in the motion region is calculated, and a variance threshold is set. When the variance of pixel values ​​in the motion region is greater than the variance threshold, the corresponding motion region is marked as a heavily noisy motion region; otherwise, the corresponding motion region is marked as a lightly noisy motion region. A first filtering window is set in the lightly noisy motion region, and the size of the first filtering window is expanded to obtain a second filtering window. A second filtering window is set in the heavily noisy motion region.

[0059] Select the median pixel value of the first filtering window and the second filtering window, and replace the pixel value in the middle position of the first filtering window and the second filtering window to complete the first median filtering; move the filtering windows sequentially until the filtering of the image set to be stitched is completed, and the filtered image set to be stitched is obtained.

[0060] S13. Use the checkerboard calibration method to capture checkerboard images from different angles, detect the corner coordinates of the checkerboard images, establish coordinate mapping relationships, obtain lens distortion parameters, perform radial distortion compensation and tangential distortion compensation on the filtered image set to be stitched, calculate the pixel correction coordinates in reverse, complete image correction, and generate the processed image set to be stitched.

[0061] In this embodiment, the transmitted video to be processed is acquired and subjected to dynamic range equalization. Then, an adaptive noise reduction method is used for filtering and image correction is implemented to generate a set of processed images to be stitched together. By using dynamic range equalization to partition and process overexposed / underexposed sub-images, the problem of loss of effective information caused by underexposed video images is solved. At the same time, sub-image partitioning and motion region selective filtering are used. Compared with traditional global filtering algorithms, this reduces single-frame processing latency and resource consumption, improves spatial measurement accuracy, and provides a reliable preprocessing foundation for automated, high-precision, and low-latency stitching of large-scale video surveillance systems. Specifically, for example, in a 4-channel 1080P traffic monitoring video stream (30fps), one channel has strong light overexposure (road surface reflection), and the other channel has tunnel entrance underexposure. Each frame of video is divided into 32×32 pixel non-overlapping sub-images.

[0062] The adjustment factor is set to 1.5 in overexposed areas to compress highlights while preserving detail in bright areas, and to 0.5 in underexposed areas to non-linearly amplify the grayscale values ​​in dark areas, conforming to the characteristics of human dark vision. Overexposed areas (pixel value > 220): 18% (strong light on the road surface); underexposed areas (pixel value < 35): 15% (tunnel shadows); normal areas (35 ≤ pixel value ≤ 220). The balanced image set restores details in both overexposed and underexposed areas. Three consecutive frames (t-1, t, t+1) are selected, and pixel differences are calculated. Areas with a difference > 15 (e.g., wheels, body edges) are marked as moving areas, while areas with a difference ≤ 15 (e.g., stationary roads, markings) are marked as static areas. Region (road background): Use bilateral filtering (spatial weight 3, range weight 10); Motion region (vehicle): Complex vehicle texture regions (such as license plates) with variance > 100 are considered heavily noisy regions, while smooth vehicle body regions with variance ≤ 100 are considered lightly noisy regions. Lightly noisy: Window size 3×3, using median to replace the center pixel; Heavyly noisy: Window expanded to 5×5 to enhance noise reduction. Use a 10×8 checkerboard calibration board to take 20 images from multiple angles at the monitoring points, detect corner coordinates, establish the mapping relationship between ideal coordinates (no distortion) and actual coordinates, solve for radial distortion coefficients (k1, k2) and tangential distortion coefficients, and then perform pixel correction to eliminate lens barrel distortion.

[0063] S2. Use an adaptive corner detection algorithm to extract feature points from the processed image set to be stitched, match feature points based on multidimensional spatial data index to obtain feature point matching pairs, and then introduce an improved RANSAC algorithm to filter feature points, and output the processed image set to be stitched.

[0064] S2 includes the following steps:

[0065] S21. Select any image to be stitched from the processed set of images to be stitched, and denot it as the image to be stitched. Calculate the average contrast of the image to be stitched to obtain the full image contrast, and use 0.7 times the full image contrast as an adaptive threshold.

[0066] In the image to be stitched, select a pixel and denote it as the key pixel. Calculate the values ​​of four pixels in the circular neighborhood of the key pixel with a radius of 3 at 0°, 90°, 180°, and 270°. If the pixel values ​​of three of the four pixels are less than the adaptive threshold, full-ring detection is not required, and the detection of other pixels in the image to be stitched continues. Otherwise, full-ring detection is activated, and 16 pixels in the circular neighborhood of the key pixel with a radius of 3 are detected. If the pixel values ​​of 12 consecutive pixels in the 16 pixels are greater than or equal to the adaptive threshold, the corresponding pixels are marked as feature pixels. This process is repeated to iterate through the processed set of images to be stitched to obtain the set of feature pixels.

[0067] S22. Select the feature pixel in the feature pixel set as the center and select a pixel neighborhood of size 32×32; divide the processed image set to be stitched into a pyramid structure according to the pyramid level, and scale the image size by 100%, 50% and 25% to form a three-layer pyramid structure, and then perform Gaussian difference detection to calculate the difference between adjacent Gaussian layers to locate the extreme point.

[0068] The extreme point is within the pixel neighborhood. The pixel gradient of the extreme point is calculated, and then a direction histogram is drawn. The peak value of the direction histogram is selected as the main direction to obtain the main direction of the extreme point. The pixel neighborhood is rotated to the main direction of the extreme point, and the 32×32 pixel neighborhood is divided into 16 8×8 pixel sub-neighborhoods. The eight-directional gradient of each pixel sub-neighborhood is counted, and a 128-dimensional feature vector is calculated. The feature vectors of the feature pixel set are extracted in sequence to form feature points, and the feature point set is obtained.

[0069] S23. For the set of feature points, feature points are matched based on multidimensional spatial data indexing to obtain feature point matching pairs. The specific steps are as follows:

[0070] S231. Calculate the dimension with the largest variance in the feature point set and use it as the root node of the binary tree. Then, start recursively partitioning. When the dimension in the feature point set is less than or equal to the dimension of the root node of the binary tree, add the corresponding feature point to the left subtree of the binary tree; otherwise, add it to the right subtree of the binary tree. Continue until the feature point set has 0 feature points, and generate a spatial index binary tree.

[0071] S232. For each feature point in the spatial index binary tree, search for the nearest and second nearest feature points of the feature point, and calculate the distance from the feature point to the nearest and second nearest feature points to obtain the nearest distance and the second nearest distance; when the ratio of the nearest distance to the second nearest distance is less than 0.7, accept the matching of the nearest and second nearest feature points to obtain a feature point matching pair;

[0072] S24. Select half of the feature point matching pairs, mark the feature point matching pairs in the processed image set to be stitched, and then perform homography transformation on the processed image set to be stitched to generate a homography matrix, obtaining the transformed image set to be stitched. The distances between the feature point matching pairs in the processed image set to be stitched and the transformed image set to be stitched are used to select the maximum and minimum feature point matching pair distances to form a distance interval, which is then discretized into... j Calculate the candidate threshold, . i Candidate thresholds ,in This represents the distance between the matching pairs of the largest feature points. The minimum feature point matching pair distance is represented. All candidate thresholds are traversed to filter feature points. When a candidate threshold is greater than 2 pixels away, it is determined to be a mismatch; otherwise, it is determined to be a positive match. The accuracy of positive matches is calculated sequentially, and the candidate threshold corresponding to the highest accuracy is taken as the global threshold. The global threshold is then used to filter feature point matching pairs to generate the final feature point matching pairs, which are marked in the processed image set to be stitched. The processed image set to be stitched is then output.

[0073] In this embodiment, an adaptive corner detection algorithm is used for feature point extraction. Feature points are matched based on multi-dimensional spatial data indexing, and then an improved RANSAC algorithm is introduced for feature point screening, outputting a processed set of images to be stitched. This method integrates adaptive corner detection and multi-scale pyramid weighted descriptors, significantly improving the stability of dynamic scene features. A spatial index binary tree is introduced to accelerate matching. Compared with traditional algorithms, the improved RANSAC screening mechanism performs initial screening followed by full-loop verification, reducing a large amount of invalid computation, improving matching speed and reducing the false matching rate, and can successfully screen out feature point pairs with mismatches. Specifically, for example, the average contrast of a single frame image is calculated to be 85, and the adaptive threshold is 0.7×85≈60. For each pixel, the neighboring pixels in four directions (0°, 90°, 180°, 270°) are quickly screened. Suspected corner points (such as vehicle corners) activate 16-direction loop detection, extracting more than 1,200 stable corner points per frame (compared to only 800+ in the traditional FAST algorithm), and key structures (lane line intersections, etc.). The detection rate of license plate corners is >95%; pyramid hierarchical structure: level L0, scaling ratio 100%, fine-scale features (license plate character corners); level L1, scaling ratio 50%, medium-scale features (vehicle outline corners); level L2, scaling ratio 25%, large-scale features (building edge corners); calculate the principal direction within a 32×32 neighborhood, rotate and divide into 4×4 sub-blocks, calculate the gradient histogram of 8 directions, and generate a 4×4×8=128-dimensional vector; then select the dimension with the largest variance (such as the gradient direction dimension) as... Let Q be the root node, P1 be the nearest neighbor (0.15), and P2 be the second nearest neighbor (0.62). Since d1 / d2 ≈ 0.24 < 0.7, Q and P1 are accepted as a valid match. The initial match pair distance range is: minimum feature point match pair distance 0.1 pixels, maximum feature point match pair distance 12.3 pixels. Twenty candidate thresholds are discretized, and their accuracy is calculated. The 8th candidate threshold of 5.0 has the highest accuracy. Match pairs with accuracy > 5.0 are deleted, and those with strong geometric consistency are retained.

[0074] S3. Find the overlapping area of ​​the processed image set to be stitched, and based on the minimum pixel difference, introduce a dynamic search algorithm to generate the best image stitching path and obtain the best image stitching seam.

[0075] S3 includes the following steps:

[0076] S31. According to the final feature point matching pairs in the processed image set to be stitched, all images to be stitched are stitched together according to the final feature point matching pairs to obtain an overlapping area. For the overlapping area of ​​two images to be stitched, the pixels of the final feature point matching pairs in the overlapping area are recorded as stitched pixels. The absolute value of the pixel value difference of the stitched pixels is calculated to obtain the color difference value.

[0077] Set up horizontal and vertical convolutional kernels with a specification of 3×3. Align the center of the horizontal convolutional kernel with the stitched pixels, calculate the product of the horizontal convolutional kernel and the pixels covered by the horizontal convolutional kernel, and add them together to obtain the horizontal gradient value. Align the center of the vertical convolutional kernel with the stitched pixels, calculate the product of the vertical convolutional kernel and the pixels covered by the vertical convolutional kernel, and add them together to obtain the vertical gradient value. Calculate the structural difference value based on the absolute value of the pixel value difference between the stitched pixels. ,in Indicates the horizontal gradient value. Represents the vertical gradient value. This represents the absolute value of the pixel value difference between the stitched pixels;

[0078] Calculate the total difference value based on the color difference value and the structural difference value. ,in This indicates the color difference value;

[0079] S32. Determine the first row of pixels in the overlapping region, calculate the total difference value of the spliced ​​pixels in the first row, then select the next row of pixels, calculate the three pixels adjacent to the spliced ​​pixels in the next row and the first row, and calculate the total difference value of the three pixels, adding it to the total difference value of the spliced ​​pixels in the first row; calculate the sum of the total difference values ​​of the spliced ​​pixels in the overlapping region in turn, and establish a difference value function.

[0080] Record the minimum total difference value of each layer in the overlapping region, and connect the pixels corresponding to the minimum total difference value of each layer in the overlapping region to form an image stitching path. Continue until the overlapping region is traversed to obtain the minimum difference value function value. At this time, the image stitching path corresponding to the minimum difference value function value is the optimal image stitching path. Mark the pixels on the optimal image stitching path in sequence to obtain the optimal image stitching seam.

[0081] In this embodiment, overlapping regions are found, and a dynamic search algorithm is introduced to generate the optimal image stitching path based on the minimum pixel difference, resulting in the optimal stitching seam. This process uses dynamic programming to comprehensively consider color difference and structural differences as constraints, achieving a path search with minimum cost by finding the minimum pixel difference. This solves problems such as ghosting and breaks in the overlapping regions of the stitched images, resulting in a more natural image quality after fusion. Specifically, for example, corrected images of intersections A and B are selected from the processed set of images to be stitched, with an overlapping area of ​​30%. The absolute difference of the matching pixel values ​​is calculated to be 75 (color difference value). The horizontal gradient convolution kernel is set to [-1, 0, 1; -2, 0, 2; -1, 0, 1], resulting in a horizontal gradient value of 25. The vertical gradient convolution kernel is set to [1, 2, 1; 0, 0, 0; -1]. [1, -2, -1], the vertical gradient value is 120, and the structural difference value is (120×75)^2+(25×75)^2≈8.1×10^7; Starting from the first row of the overlapping area (the first row at the top of the image), the path cost Cost(x, y)=B(x, y)+min{Cost(x-1, y-1), Cost(x-1, y), Cost(x-1, y+1)} is calculated for each pixel P(x, y), where B(x, y) represents the total difference value of pixel P. The total difference value is compared with the total difference value of the three adjacent pixels in the next row one by one. The minimum cumulative cost point is selected as the path node in each row. All nodes are connected to form a splicing seam. The dynamic programming algorithm has a complexity of O(n×m) (where n is the number of rows and m is the number of columns), and has high real-time search efficiency;

[0082] S4. Perform image stitching according to the optimal stitching seam of the image, divide the image according to the dynamic grid, and use priority sorting to achieve video output;

[0083] S4 includes the following steps:

[0084] S41. The images to be stitched in the processed image set are stitched together according to the optimal stitching seam, the overlapping areas are cropped, and the stitched panoramic image is output. The stitched panoramic image is divided into several image grids according to the grid. The optical flow amplitude in the image grid is calculated and an optical flow threshold is set. When the optical flow amplitude is greater than the optical flow threshold, the corresponding grid is recorded as a moving grid and the size of the moving grid is increased; otherwise, the corresponding grid is recorded as a static grid and the size of the static grid is reduced, thus completing the dynamic grid division.

[0085] S42. Obtain the focal regions of the motion grid and static grid, the focal regions include face regions, vehicle regions and text regions, calculate the image proportion of the focal regions, set priorities according to the image proportion of the focal regions and the motion grid and static grid, sort them from high to low, and prioritize the transmission of the stitched panoramic images with high priority to achieve video output;

[0086] In this embodiment, image stitching is performed based on the optimal stitching seam, the image is divided according to a dynamic grid, and video output is achieved using priority sorting based on the moving and static grids. This method is based on spatial adaptive compression, dynamic scaling of moving / static grids, and dynamic adaptation of priority queues to channel quality, ensuring redundant dual-channel transmission of high-priority grids, balancing details and bandwidth. Through dynamic grid division and focus-priority transmission, bandwidth compression and key information are enhanced, resulting in a high degree of intelligence and stable video transmission. Specifically, for example, a panoramic image is divided into 240 grids (128×72 pixels per grid), with an optical flow amplitude > 5 pixels / frame (vehicle movement), and the size is magnified by 150%. For optical flow amplitude ≤ 5 pixels / frame (road background), the size is reduced by 70%. For example, in the area of ​​overtaking vehicles: optical flow amplitude 12 pixels / frame, grid enlarged to 192×108 pixels; in the area of ​​stationary guardrails: optical flow amplitude 0.8 pixels / frame, grid reduced to 90×50 pixels, resulting in a 42% reduction in total data volume. Focus areas are identified and prioritized: high priority (image proportion of focus area > 10%), first batch transmitted, bandwidth allocated 10Mbps; medium priority (image proportion of focus area 5%-10%), second batch transmitted, bandwidth allocated 5Mbps; low priority (image proportion of focus area < 5%), last batch transmitted, bandwidth allocated 1Mbps.

[0087] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic point described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristic points described may be combined in any suitable manner in one or more embodiments or examples.

[0088] The preferred embodiments of the invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A high-definition video transmission intelligent image splicing system, characterized in that, include: The image preprocessing module is used to acquire the transmitted video to be processed and perform dynamic range equalization, then use an adaptive noise reduction method for filtering, and implement image correction to generate a set of processed images to be stitched together. The feature point matching and filtering module uses an adaptive corner detection algorithm to extract feature points from the processed image set to be stitched, obtains feature point matching pairs based on multidimensional spatial data indexing, and then introduces an improved RANSAC algorithm to filter feature points, outputting the processed image set to be stitched. The optimal stitching seam generation module is used to find the overlapping area of ​​the processed set of images to be stitched. Based on the minimum pixel difference, a dynamic search algorithm is introduced to obtain the optimal stitching seam. The video output module is used to stitch images according to the best stitching seam, divide images according to a dynamic grid, and use priority sorting to achieve video output. The process of acquiring the transmitted video to be processed and performing dynamic range equalization includes: The transmitted video to be processed is acquired, segmented into frames to obtain an initial set of images to be stitched, and divided into non-overlapping sub-images to be stitched. The pixel values ​​of the sub-images to be stitched are then counted. Set up overexposed, underexposed, and normal areas, divide the pixel values ​​of the sub-images to be stitched into the overexposed, underexposed, and normal areas, output the adjusted pixel values, and generate a balanced set of images to be stitched. The process of using an adaptive noise reduction method for filtering and implementing image correction includes: Identify the moving and static regions of the set of images to be stitched together; For the static region, bilateral filtering is used to generate denoised pixel values, which are then used to replace the pixel values ​​in the static region to obtain a denoised static region; for the moving region, adaptive median filtering is used to denoise, resulting in a set of filtered images to be stitched together. Image correction is performed on the filtered set of images to be stitched to generate a processed set of images to be stitched. The process of stitching images according to the optimal seam, dividing images according to a dynamic grid, and using priority sorting to achieve video output includes: The images are stitched together according to the optimal stitching seam, overlapping areas are cropped, and the stitched panoramic image is output; the images are divided according to a grid to obtain a moving grid and a static grid. The focal regions of the moving and static grids are obtained, including face regions, vehicle regions, and text regions. The image proportion of the focal regions is calculated. Based on the image proportion of the focal regions and the priority of the moving and static grids, they are sorted from high to low. The stitched panoramic images with high priority are transmitted first to achieve video output.

2. The intelligent video transmission high-definition image stitching system according to claim 1, characterized in that, The feature point extraction includes: An adaptive threshold is obtained from the processed set of images to be stitched, followed by preliminary screening and full ring detection to obtain a set of feature pixels. The feature pixels in the feature pixel set are selected as the center, and the image size is scaled according to the pyramid hierarchy to form a three-layer pyramid structure. Then, Gaussian difference detection is performed to calculate a 128-dimensional feature vector. The feature vectors of the feature pixel set are extracted in sequence to form feature points, and a feature point set is obtained.

3. The intelligent video transmission high-definition image stitching system according to claim 2, characterized in that, The feature point matching pairs obtained based on multidimensional spatial data indexing include: Based on the different dimensions in the feature point set, the dimension with the largest variance is selected as the root node of the binary tree. At this point, recursive partitioning begins until the feature point set contains 0 feature points, generating a spatial index binary tree. Search for the nearest and second nearest neighbor feature points of the feature points, and determine the matching between the accepted feature point and the nearest neighbor feature point to obtain the feature point matching pair.

4. The intelligent video transmission high-definition image stitching system according to claim 3, characterized in that, The improved RANSAC algorithm for feature point selection includes: Select half of the feature point matching pairs, and choose the maximum and minimum feature point matching pair distances to form distance intervals. Discretize the distance intervals into... j Calculate the candidate thresholds; The feature point is filtered by traversing all candidate thresholds. When the distance between the candidate thresholds is greater than 2 pixels, it is judged as a mismatch; otherwise, it is judged as a positive match. The accuracy of the positive matches is calculated in turn, and the candidate threshold corresponding to the highest accuracy is taken as the global threshold. The feature point matching pairs are filtered to generate the final feature point matching pairs, and the processed set of images to be stitched is output.

5. The intelligent video transmission high-definition image stitching system according to claim 4, characterized in that, The process of finding overlapping regions in the processed set of images to be stitched includes: According to the final feature point matching pairs in the processed image set to be stitched, all images to be stitched are stitched together according to the final feature point matching pairs to obtain the overlapping region.

6. The intelligent video transmission high-definition image splicing system according to claim 5, characterized in that, The process of finding the overlapping region of the processed image set to be stitched, based on the minimum pixel difference, introduces a dynamic search algorithm to obtain the optimal stitching seam, including: Calculate the color difference value and the structural difference value in the overlapping area to obtain the total difference value; Determine the first row of pixels in the overlapping region, calculate the total difference value of the stitched pixels in the first row, then select the next row of pixels, calculate the total difference value of the three adjacent pixels between the next row and the stitched pixels in the first row, and add them to the total difference value of the stitched pixels in the first row; calculate the total difference value of the stitched pixels in the overlapping region in turn, and establish a difference value function. Record the minimum total difference value of each row in the overlapping area, and connect the pixels corresponding to the minimum total difference value of each row in the overlapping area to form the image stitching path. Continue until the overlapping area is traversed. Then, mark the pixels on the best stitching path in sequence to obtain the best stitching seam of the image.

7. A method for intelligent stitching of high-definition images in video transmission, characterized in that, The system based on any one of claims 1-6 specifically includes: S1. Acquire the transmitted video to be processed and perform dynamic range equalization to obtain a set of equalized images to be stitched together. Then, use an adaptive noise reduction method for filtering and perform image correction to generate a set of processed images to be stitched together. S2. Use an adaptive corner detection algorithm to extract feature points from the processed image set to be stitched, match feature points based on multidimensional spatial data index to obtain feature point matching pairs, and then introduce an improved RANSAC algorithm to filter feature points, and output the processed image set to be stitched. S3. Find the overlapping area of ​​the processed image set to be stitched, and based on the minimum pixel difference, introduce a dynamic search algorithm to generate the best image stitching path and obtain the best image stitching seam. S4. Perform image stitching according to the optimal stitching seam of the image, divide the image according to the dynamic grid, and use priority sorting to achieve video output.

Images