Video transmission high-definition image intelligent splicing method and system

Abstract

The invention relates to the technical field of image stitching, and discloses a video transmission high-definition image intelligent stitching method and system. The system comprises an image preprocessing module, a feature point matching and screening module, an image optimal splicing seam generation module and a video output module. The method comprises the following steps: firstly, acquiring a video, carrying out dynamic range equalization, carrying out filtering processing by using an adaptive noise reduction method, and carrying out image correction; secondly, extracting feature points by using a self-adaptive corner detection algorithm, matching the feature points based on a multi-dimensional spatial data index to obtain feature point matching pairs, and screening the feature points; searching an overlapping region, and introducing a dynamic search algorithm to generate an optimal image splicing path to obtain an optimal image splicing seam; and finally, dividing the image according to the dynamic grid, and realizing video output by using priority ranking. According to the method, the video images are processed and spliced, the purpose of intelligent image splicing is achieved, and the method is accurate and objective.

Description

Technical Field

[0001] The present invention relates to the technical field of image splicing, and in particular to a method and system for intelligently splicing high-definition images in video transmission. Background Art

[0002] With the rapid development of information technology, the demand for the capture, transmission, and presentation of high-definition video content has exploded. In a wide range of applications, including security surveillance, live broadcasting, video conferencing, virtual reality, and drone aerial photography, the field of view and resolution of a single camera often fail to meet the requirements for panoramic coverage and high-detail presentation. Therefore, transmitting video streams from multiple cameras and stitching them together in real-time or near-real time at the receiving end to create seamless, high-resolution panoramic videos or wide-angle views has become an important technical approach for obtaining high-quality, wide-field images. Currently, the widespread availability of high-speed wireless communication technologies and fiber optic networks offers the potential for higher bandwidth and lower latency for the transmission of high-definition video streams. However, efficiently, robustly, and intelligently receiving, registering, fusing, and stitching multiple HD videos to create high-quality panoramic images remains a significant technical challenge.

[0003] Traditional image stitching methods mainly use optical flow-based methods. Aiming at the continuity of video streams, they use the pixel motion information (optical flow) between adjacent frames for continuous alignment and stitching; or use algorithms such as SIFT and SURF to detect and match significant feature points between images, then perform coordinate transformation and image deformation, and finally use specific fusion algorithms (such as linear weighting and multi-band fusion) to eliminate stitching seams.

[0004] Traditional image stitching methods have a huge amount of computational complexity when processing high-resolution images. Traditional feature extraction, matching, and global optimization fusion algorithms take too long to meet the real-time requirements for low-latency stitching output. The stitching process has high latency, poor quality, low intelligence, and lacks end-to-end optimization. Summary of the Invention

[0005] In response to the problems in the related art, the present invention provides a video transmission high-definition image intelligent splicing system to overcome the technical problems existing in the existing related art.

[0006] To solve the above technical problems, the present invention is achieved through the following technical solutions: The present invention is a high-definition image intelligent splicing system for video transmission, which specifically includes: an image preprocessing module, a feature point matching and screening module, an image optimal splicing seam generation module and a video output module; The image preprocessing module is used to obtain the transmitted video to be processed and perform dynamic range equalization, then use the adaptive noise reduction method to perform filtering processing, and implement image correction to generate a processed set of images to be spliced; The feature point matching and screening module uses an adaptive corner point detection algorithm to extract feature points from the processed image set to be stitched, obtains feature point matching pairs based on multidimensional spatial data index, and then introduces an improved RANSAC algorithm to screen feature points, and outputs the processed image set to be stitched; The optimal image stitching seam generation module is used to find the overlapping area of ​​the processed image set to be stitched, and introduce a dynamic search algorithm based on the minimum pixel difference to obtain the optimal image stitching seam; The video output module is used to perform image splicing according to the optimal image splicing seam, divide the image according to the dynamic grid, and realize video output using priority sorting.

[0007] Preferably, the obtaining of the to-be-processed transmission video and performing dynamic range equalization includes: Obtain the transmitted video to be processed, split it frame by frame, obtain an initial set of images to be stitched, divide it into non-overlapping sub-images to be stitched, and count the pixel values ​​of the sub-images to be stitched; Set overexposed areas, underexposed areas, and normal areas, divide the pixel values ​​of the sub-images to be stitched into the overexposed areas, underexposed areas, and normal areas, output the adjusted pixel values, and generate a balanced set of images to be stitched.

[0008] Preferably, the filtering process using the adaptive noise reduction method includes: Identifying motion regions and static regions of the balanced set of images to be stitched; For the static area, bilateral filtering is used to generate denoised pixel values, which are used to replace the pixel values ​​in the static area to obtain a denoised static area; for the moving area, adaptive median filtering is used to generate denoised pixel values ​​to obtain a set of filtered images to be spliced; Image correction is performed on the filtered image set to be stitched to generate a processed image set to be stitched.

[0009] Preferably, the feature point extraction includes: Obtaining an adaptive threshold of the processed image set to be stitched, and then performing preliminary screening and full-ring detection to obtain a set of characteristic pixel points; A feature pixel point in the feature pixel point set is selected as the center, and the image size is scaled according to the pyramid level to form a three-layer pyramid structure. Gaussian difference detection is then performed to calculate a 128-dimensional feature vector. The feature vectors of the feature pixel point set are extracted in sequence to form feature points, thereby obtaining a feature point set.

[0010] Preferably, obtaining feature point matching pairs based on multidimensional spatial data indexes includes: The dimension with the largest variance in the feature point set is selected as the root node of the binary tree, and recursive partitioning is started at this time until the feature point in the feature point set is 0, thereby generating a spatial index binary tree; The nearest feature point and the second nearest feature point of the feature point are searched, and the matching between the nearest feature point and the second nearest feature point is determined and accepted to obtain a feature point matching pair.

[0011] Preferably, the introducing of the improved RANSAC algorithm to perform feature point screening includes: Select half of the feature point matching pairs, select the maximum feature point matching pair distance and the minimum feature point matching pair distance to form a distance interval, and discretize the distance interval into j candidate thresholds, calculate the candidate thresholds; Traverse all candidate thresholds to screen feature points. When the candidate threshold is greater than the distance between two pixels, it is judged as a false match. Otherwise, it is judged as a positive match. The accuracy of the positive match is calculated in sequence. The candidate threshold corresponding to the highest accuracy is used as the global threshold. The feature point matching pairs are screened to generate the final feature point matching pairs and output the processed image set to be stitched.

[0012] Preferably, the step of searching for an overlapping area of ​​the processed set of images to be stitched comprises: According to the final feature point matching pairs in the processed set of images to be spliced, all the images to be spliced ​​are spliced ​​according to the final feature point matching pairs to obtain overlapping areas.

[0013] Preferably, the step of searching for an overlapping area of ​​the processed set of images to be stitched comprises: Calculate the color difference value and the structure difference value in the overlapping area to obtain the total difference value; Determine the first row of pixels in the overlapping area, calculate the total difference value of the spliced ​​pixels in the first row of pixels, then select the next row of pixels from the first row of pixels, calculate the three pixels adjacent to the spliced ​​pixels in the next row of pixels and the first row of pixels, and calculate the total difference value of the three pixels, and add it to the total difference value of the spliced ​​pixels in the first row of pixels; calculate the cumulative sum of the total difference values ​​of the spliced ​​pixels in the overlapping area in sequence, and establish a difference value function; The minimum total difference value of each layer in the overlapping area is recorded, and the pixel points corresponding to the minimum total difference value of each layer in the overlapping area are connected to form an image stitching path. Until the overlapping area is traversed, the pixel points on the optimal image stitching path are marked in sequence to obtain the optimal image stitching seam.

[0014] Preferably, the step of searching for an overlapping area of ​​the processed set of images to be stitched comprises: Stitch the images according to the optimal stitching seam, crop the overlapping areas, and output the stitched panoramic image; divide the images according to the grid to obtain the moving grid and the static grid; The focus areas of the moving grid and the static grid are obtained, the image proportions of the focus areas are calculated, priorities are set, and the stitched panoramic images with high priorities are transmitted first.

[0015] The present invention has the following beneficial effects: 1. This invention obtains the transmitted video to be processed and performs dynamic range equalization, and partitions the overexposed / underexposed sub-images to solve the problem of effective information loss caused by underexposure of video images. At the same time, it uses sub-image partitioning processing and motion area selective filtering. Compared with traditional global filtering algorithms, it reduces single-frame processing delay and resource occupation, improves spatial measurement accuracy, and provides a reliable preprocessing foundation for the automated, high-precision, and low-latency splicing of large-scale video surveillance systems.

[0016] 2. This invention uses an adaptive corner detection algorithm to extract feature points, matches feature points based on multi-dimensional spatial data indexes, and integrates adaptive corner detection and multi-scale pyramid weighted descriptors to significantly improve the stability of dynamic scene features. It then introduces an improved RANSAC algorithm for feature point screening. Compared with traditional algorithms, it performs primary screening followed by full-ring testing, which reduces a large amount of invalid calculations, improves matching speed, and reduces the mismatching rate, and can successfully screen out feature point pairs with incorrect matching.

[0017] 3. This invention generates the optimal image stitching path based on the minimum pixel difference by finding the overlapping area and introducing a dynamic search algorithm to obtain the optimal image stitching seam. It takes color difference and structural difference into comprehensive consideration as constraints and realizes the minimum cost path search by finding the minimum pixel difference. It solves the problems of ghosting and breakage in the overlapping area of ​​the stitched image, and the quality of the fused image is more natural.

[0018] 4. The invention performs image stitching through the best image stitching seams, divides the image according to the dynamic grid, and uses priority sorting according to the moving static grid to achieve video output, ensuring dual-channel redundant transmission of high-priority grids, balancing details and bandwidth, and enhancing bandwidth compression and key information through dynamic grid division and focus priority transmission. It has a high degree of intelligence and achieves stable video transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, they can also obtain drawings based on these drawings without paying any creative work.

[0020] Figure 1 The present invention provides a flow chart of a system for intelligently splicing high-definition images for video transmission; Figure 2 The present invention provides a flow chart of a method for intelligently splicing high-definition images in video transmission. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0022] Traditional image stitching methods have a huge amount of computational complexity when processing high-resolution images. Traditional feature extraction, matching, and global optimization fusion algorithms take too long to meet the real-time requirements for low-latency stitching output. The stitching process has high latency, poor quality, low intelligence, and lacks end-to-end optimization.

[0023] In order to solve the above technical problems, Figure 1 As shown, an embodiment of the present invention provides a high-definition image intelligent splicing system for video transmission, which specifically includes: an image preprocessing module, a feature point matching and screening module, an image optimal splicing seam generation module and a video output module; the image preprocessing module is used to obtain the transmission video to be processed and perform dynamic range equalization, then use an adaptive noise reduction method to perform filtering processing, and implement image correction to generate a processed set of images to be spliced; the feature point matching and screening module uses an adaptive corner point detection algorithm to extract feature points from the processed set of images to be spliced, obtains feature point matching pairs based on multi-dimensional spatial data index, then introduces an improved RANSAC algorithm to perform feature point screening, and outputs the processed set of images to be spliced; the image optimal splicing seam generation module is used to find the overlapping area of ​​the processed set of images to be spliced, and introduces a dynamic search algorithm based on the minimum pixel difference to obtain the optimal image splicing seam; the video output module is used to perform image splicing according to the optimal image splicing seam, divide the image according to the dynamic grid, and use priority sorting to achieve video output.

[0024] A specific implementation example is that in the intelligent renovation and expansion project of the ring expressway, the entire line is 120 kilometers long and contains 8 types of heterogeneous monitoring equipment (built in batches from 2010 to 2023). There are problems such as lighting differences (dark areas in tunnels / high-light ramps), lens distortion (old-style dome cameras), and frame rate mismatch. The average daily traffic volume is 150,000 vehicles, and during peak hours, it is >5,000 vehicles / hour. The existing maximum resolution of the camera is 1080P, and 72% are old standard-definition equipment; build an "end-edge-cloud" collaborative solution: terminal-layer dynamic range equalization + noise reduction solves uneven lighting and motion blur problems, edge-layer feature intelligent matching + optimal stitching seams to achieve seamless alignment and ghost elimination across cameras, cloud-layer dynamic grid priority transmission to reduce bandwidth pressure and ensure real-time analysis of key events.

[0025] In the specific implementation process of the above embodiment, the video to be processed is first obtained and dynamic range equalization is performed, and then the adaptive noise reduction method is used for filtering and image correction to generate a set of processed images to be spliced; the overexposed / underexposed sub-images are processed by dynamic range equalization partitioning to solve the problem of effective information loss caused by underexposure of video images. At the same time, sub-image partitioning and motion area selective filtering are used. Compared with the traditional global filtering algorithm, the single-frame processing delay and resource occupation are reduced, the spatial measurement accuracy is improved, and a reliable preprocessing basis is provided for the automated, high-precision, and low-latency splicing of large-scale video surveillance systems; secondly, an adaptive corner detection algorithm is used to extract feature points, and feature points are matched based on multi-dimensional spatial data indexes. Then, an improved RANSAC algorithm is introduced to screen feature points and output the processed set of images to be spliced; this method integrates adaptive corner detection and multi-scale pyramid weighted descriptors, significantly improves the stability of dynamic scene features, introduces spatial index binary tree to accelerate matching, and the improved algorithm is compared with the traditional algorithm. , first perform primary screening and then full-loop inspection, reduce a lot of invalid calculations, improve matching speed and reduce mismatching rate, and successfully screen out incorrectly matched feature point pairs; then find the overlapping area, and based on the minimum pixel difference, introduce a dynamic search algorithm to generate the best image stitching path, and obtain the best image stitching seam; in this process, the dynamic programming method is used to comprehensively consider color difference and structural difference as constraints, and the minimum cost path search is achieved by finding the minimum pixel difference, which solves the problems of ghosting and breakage in the overlapping area of ​​the stitched image, and the quality of the fused image is more natural; finally, the image is stitched according to the best image stitching seam, the image is divided according to the dynamic grid, and the video output is realized by priority sorting according to the moving static grid; this method is based on spatial adaptive compression, dynamic scaling of moving / static grids, and dynamic adaptation of priority queues to channel quality, ensuring dual-channel redundant transmission of high-priority grids, balancing details and bandwidth, and enhancing bandwidth compression and key information through dynamic grid division and focus priority transmission. It has a high degree of intelligence and realizes stable video transmission.

[0026] Furthermore, in order to better introduce the technical solutions of the embodiments of the present invention, Figure 2 As shown, a video transmission high-definition image intelligent splicing method is combined with a video transmission high-definition image intelligent splicing system to be described in detail, including the following contents: S1. Obtain the transmitted video to be processed and perform dynamic range equalization to obtain a balanced set of images to be spliced. Then, use an adaptive noise reduction method to perform filtering and image correction to generate a processed set of images to be spliced. Said S1 comprises the following steps: S11. Obtain a transmission video to be processed, segment the transmission video to be processed according to each frame, obtain a plurality of transmission video images to be processed, record them as images to be stitched, and form an initial set of images to be stitched; divide the images to be stitched in the initial set of images to be stitched into non-overlapping sub-images to be stitched, and count the pixel values ​​of the sub-images to be stitched; Set overexposed area, underexposed area and normal area, divide the pixel values ​​of the sub-image to be spliced ​​into the overexposed area, underexposed area and normal area, keep the sub-image to be spliced ​​in the normal area, adjust the sub-image to be spliced ​​in the overexposed area and underexposed area, and output the adjusted pixel values ,in Represents the pixel value of the sub-image to be spliced, represents the adjustment coefficient, which is set to 1.5 in the overexposed area and 0.5 in the underexposed area; the pixel values ​​of the sub-images to be stitched are replaced with the adjusted pixel values ​​to obtain a balanced set of images to be stitched; S12, identifying the motion region and the static region of the balanced set of images to be stitched, combining bilateral filtering and adaptive median filtering to obtain an adaptive noise reduction method, using the adaptive noise reduction method to filter the motion region and the static region respectively to obtain a filtered set of images to be stitched, the specific steps are as follows: S121. Selecting consecutive frames of images to be stitched from the set of balanced images to be stitched, and calculating, for any pixel point in the consecutive frames of images to be stitched, a difference in pixel value between the consecutive frames of images to be stitched and the pixels in the consecutive frames of images to be stitched, to obtain a consecutive frame pixel difference; setting a motion threshold, and when the consecutive frame pixel difference is greater than the motion threshold, marking a moving pixel point to form a motion region; otherwise, obtaining a static region; S122. De-noising the static area using bilateral filtering. A pixel neighborhood is set in the static area. Within the pixel neighborhood, a spatial weight is calculated based on the Euclidean distance between pixels. A range weight is calculated based on the difference between pixel values. The spatial weight and the range weight are multiplied together, and then a weighted average is performed to generate a denoised pixel value. The denoised pixel value is used to replace the pixel value in the static area to obtain a denoised static area. S123, using adaptive median filtering to denoise the motion region, calculating the variance of pixel values ​​in the motion region, setting a variance threshold, and marking the corresponding motion region as a heavy noise motion region when the variance of pixel values ​​in the motion region is greater than the variance threshold; otherwise, marking the corresponding motion region as a light noise motion region; setting a first filtering window in the light noise motion region, and expanding the size of the first filtering window to obtain a second filtering window, and setting a second filtering window in the heavy noise motion region; Select the median of the pixel values ​​of the first filtering window and the second filtering window, replace the pixel value at the middle position of the first filtering window and the second filtering window, and complete the first median filtering; move the filtering windows in sequence until the filtering of the balanced image set to be spliced ​​is completed, and obtain the filtered image set to be spliced; S13, using a checkerboard calibration method to capture checkerboard images at different angles, detecting the coordinates of corner points of the checkerboard images, establishing a coordinate mapping relationship, obtaining lens distortion parameters, performing radial distortion compensation and tangential distortion compensation on the filtered set of images to be stitched, performing inverse calculation to obtain pixel correction coordinates, completing image correction, and generating a processed set of images to be stitched; In this embodiment, a transmitted video to be processed is obtained and subjected to dynamic range equalization. It is then filtered using an adaptive noise reduction method and image correction is performed to generate a set of processed images to be stitched. Dynamic range equalization is used to partition overexposed / underexposed sub-images to address the loss of effective information caused by underexposure in video images. Sub-image partitioning and motion-region selective filtering are used simultaneously to reduce single-frame processing latency and resource usage compared to traditional global filtering algorithms, improving spatial measurement accuracy and providing a reliable preprocessing foundation for automated, high-precision, and low-latency stitching in large-scale video surveillance systems. Specifically, for example, consider four 1080P traffic surveillance video streams (30 fps), one of which is overexposed due to strong light (road reflections) and the other underexposed at the tunnel entrance. Each frame of the video is segmented into non-overlapping sub-images of 32×32 pixels. The adjustment coefficient is 1.5 in the overexposed area to compress the highlight area and retain the layering of the bright area. It is 0.5 in the underexposed area to nonlinearly amplify the grayscale value of the dark area, which is in line with the dark vision characteristics of the human eye. The overexposed area (pixel value > 220) is set to account for 18% (high-light road surface), the underexposed area (pixel value < 35) is set to account for 15% (tunnel shadow), and the normal area (35≤pixel value≤220) is set to restore the details of the overexposed / underexposed area after equalization. Three consecutive frames (t-1, t, t+1) are selected to calculate the pixel difference. The moving area of ​​the vehicle (such as the wheel and the edge of the vehicle body) with a difference of > 15 is marked as the moving area, and the stationary road and road marking with a difference of ≤ 15 is marked as the static area. Region (road background): bilateral filtering (spatial weight 3, range weight 10) is used. Motion region (vehicle): complex vehicle texture areas (such as license plates) with a variance > 100 are considered heavily noisy, while smooth vehicle body areas with a variance ≤ 100 are considered lightly noisy. For lightly noisy images, a 3×3 window size is used, with the median value replacing the center pixel. For heavy noise, the window size is expanded to 5×5 to enhance denoising capabilities. A 10×8 checkerboard calibration plate is used to capture 20 images at multiple angles at the monitoring point. Corner coordinates are detected, and a mapping relationship between ideal coordinates (undistorted) and actual coordinates is established. The radial distortion coefficients (k1, k2) and tangential distortion coefficients are calculated. Pixel correction is then performed to eliminate lens barrel distortion. S2. Extracting feature points from the processed image set to be stitched using an adaptive corner detection algorithm, matching the feature points based on a multidimensional spatial data index to obtain feature point matching pairs, then introducing an improved RANSAC algorithm to screen the feature points, and outputting the processed image set to be stitched; The S2 comprises the following steps: S21, selecting any image to be stitched from the processed set of images to be stitched, recording it as a detected image to be stitched, calculating the average contrast of the detected image to be stitched to obtain a full-image contrast, and using 0.7 times the full-image contrast as an adaptive threshold; A pixel point is selected in the image to be spliced ​​and recorded as a key pixel point. Four pixels in the circular neighborhood with a radius of 3 of the key pixel point in the directions of 0°, 90°, 180° and 270° are calculated. When the pixel values ​​of three of the four pixels are less than the adaptive threshold, full ring detection is not required and the detection of other pixels in the image to be spliced ​​is continued. Otherwise, full ring detection is activated to detect 16 pixels in the circular neighborhood with a radius of 3 of the key pixel point. When the pixel values ​​of 12 consecutive pixels among the 16 pixels are greater than or equal to the adaptive threshold, the corresponding pixels are marked as feature pixels. The processed image set to be spliced ​​is traversed in sequence to obtain a feature pixel set. S22, selecting a characteristic pixel point in the characteristic pixel point set as the center and selecting a pixel neighborhood with a size of 32×32; scaling the image size of the processed image set to be stitched by 100%, 50%, and 25% according to the pyramid level to form a three-layer pyramid structure, and then performing Gaussian difference detection to calculate the difference between adjacent Gaussian layers to locate the extreme value point; The extreme point is within the pixel neighborhood, the pixel gradient of the extreme point is calculated, and then a directional histogram is drawn, and the peak of the directional histogram is selected as the main direction to obtain the main direction of the extreme point; the pixel neighborhood is rotated to the main direction of the extreme point, and the pixel neighborhood of size 32×32 is divided into 16 pixel sub-neighborhoods of size 8×8, and the eight-directional gradient of each pixel sub-neighborhood is counted to calculate a 128-dimensional feature vector, and the feature vectors of the feature pixel point set are sequentially extracted to form feature points to obtain a feature point set; S23. For the feature point set, match the feature points based on the multidimensional spatial data index to obtain a feature point matching pair. The specific steps are as follows: S231, calculating the dimension with the largest variance among different dimensions in the feature point set and using it as the root node of the binary tree. Then, recursive partitioning begins. When the dimension in the feature point set is less than or equal to the dimension of the root node of the binary tree, the corresponding feature point is added to the left subtree of the binary tree. Otherwise, the corresponding feature point is added to the right subtree of the binary tree. This continues until the number of feature points in the feature point set is zero, thereby generating a spatial index binary tree. S232. For each feature point in the spatial index binary tree, search for the nearest feature point and the next nearest feature point of the feature point, and calculate the distance from the feature point to the nearest feature point and the next nearest feature point to obtain the nearest distance and the next nearest distance; when the ratio of the nearest distance to the next nearest distance is less than 0.7, accept the match between the nearest feature point and the next nearest feature point to obtain a feature point matching pair; S24, selecting half of the feature point matching pairs, marking the feature point matching pairs in the processed image set to be stitched, and then performing homography transformation on the processed image set to be stitched to generate a homography matrix to obtain the transformed image set to be stitched; calculating the distances between the feature point matching pairs in the processed image set to be stitched and the transformed image set to be stitched, selecting the maximum feature point matching pair distance and the minimum feature point matching pair distance to form a distance interval, and discretizing the distance interval into j candidate thresholds, calculate the i Candidate thresholds ,in Indicates the maximum feature point matching distance, Represents the minimum feature point matching distance; traverse all candidate thresholds to screen feature points. When the candidate threshold is greater than the distance of 2 pixels, it is determined to be a false match. Otherwise, it is determined to be a positive match. The accuracy of the positive match is calculated in sequence, and the candidate threshold corresponding to the highest accuracy is used as the global threshold. The global threshold is then used to screen the feature point matching pairs, generate the final feature point matching pairs, and mark them in the processed image set to be stitched, and output the processed image set to be stitched; In this embodiment, an adaptive corner detection algorithm is used to extract feature points, feature points are matched based on multi-dimensional spatial data indexes, and an improved RANSAC algorithm is introduced to screen feature points, and a processed set of images to be stitched is output; this method integrates adaptive corner detection and multi-scale pyramid weighted descriptors, significantly improves the stability of dynamic scene features, introduces a spatial index binary tree to accelerate matching, and the improved RANSAC screening mechanism, compared with the traditional algorithm, performs primary screening first and then full-ring inspection, which reduces a large amount of invalid calculations, improves matching speed and reduces the mismatching rate, and can successfully screen out incorrectly matched feature point pairs; specifically, for example, the average contrast of a single frame image is calculated to be 85, the adaptive threshold is 0.7×85≈60, and the neighboring pixels in the four directions of 0°, 90°, 180°, and 270° are detected for each pixel for rapid preliminary screening. Suspected corners (such as vehicle corners) activate 16-directional ring detection, and more than 1,200 stable corners are extracted from a single frame (the traditional FAST algorithm only has 800+), key structures (lane line intersections, The detection rate of license plate corners is >95%; pyramid layering: level L0, scaling ratio 100%, fine-scale features (license plate character corners), level L1, scaling ratio 50%, medium-scale features (vehicle outline corners), level L2, scaling ratio 25%, large-scale features (building edge corners); calculate the main direction in a 32×32 neighborhood, rotate and divide it into 4×4 sub-blocks, calculate the 8-directional gradient histogram, and generate a 4×4×8=128-dimensional vector; then select the dimension with the largest variance (such as the gradient direction dimension) as the For the root node Q, the nearest neighbor P1 has a distance of 0.15, the next nearest neighbor P2 has a distance of 0.62, and d1 / d2≈0.24<0.7. At this time, Q and P1 are accepted as valid matching pairs. The initial matching pair distance range is: the minimum feature point matching pair distance is 0.1 pixels, and the maximum feature point matching pair distance is 12.3 pixels. 20 candidate thresholds are discretized, and the accuracy of the candidate thresholds is calculated. The 8th candidate threshold of 5.0 has the highest accuracy. Matching pairs with accuracy > 5.0 are deleted, and matching pairs with strong geometric consistency are retained. S3, finding the overlapping area of ​​the processed set of images to be stitched, and introducing a dynamic search algorithm to generate an optimal image stitching path based on the minimum pixel difference to obtain the optimal image stitching seam; The S3 includes the following steps: S31, stitching all the images to be stitched together according to the final feature point matching pairs in the processed set of images to be stitched together to obtain an overlapping area, and for the overlapping area of ​​the two images to be stitched together, recording the pixels of the final feature point matching pairs in the overlapping area as stitching pixels, and calculating the absolute value of the pixel value difference of the stitching pixels to obtain a color difference value; Set the horizontal convolution kernel and vertical convolution kernel with a size of 3×3, align the center of the horizontal convolution kernel with the splicing pixel point, calculate the product of the horizontal convolution kernel and the pixel point covered by the horizontal convolution kernel and add them to get the horizontal gradient value; align the center of the vertical convolution kernel with the splicing pixel point, calculate the product of the vertical convolution kernel and the pixel point covered by the vertical convolution kernel and add them to get the vertical gradient value; calculate the structural difference value based on the absolute value of the pixel value difference of the splicing pixel point ,in represents the horizontal gradient value, Indicates the vertical gradient value, Indicates the absolute value of the pixel value difference of the spliced ​​pixels; Calculate the total difference value based on the color difference value and the structure difference value ,in Indicates the color difference value; S32, determining the first row of pixels in the overlapping area, calculating the total difference value of the spliced ​​pixels in the first row of pixels, then selecting the next row of pixels from the first row of pixels, calculating three pixels adjacent to the spliced ​​pixels in the next row of pixels and the first row of pixels, and calculating the total difference value of the three pixels, and adding the total difference value to the total difference value of the spliced ​​pixels in the first row of pixels; sequentially calculating the cumulative sum of the total difference values ​​of the spliced ​​pixels in the overlapping area, and establishing a difference value function; Record the minimum total difference value of each layer in the overlapping area, and connect the pixel points corresponding to the minimum total difference value of each layer in the overlapping area to form an image stitching path. Until the overlapping area is traversed, the minimum difference value function value is obtained. At this time, the image stitching path corresponding to the minimum difference value function value is the optimal image stitching path. Mark the pixel points on the optimal image stitching path in sequence to obtain the optimal image stitching seam. In this embodiment, the overlapping area is found, and based on the minimum pixel difference, a dynamic search algorithm is introduced to generate the optimal image stitching path to obtain the optimal image stitching seam; in this process, the dynamic programming method is used to comprehensively consider the color difference and structural difference as constraints, and the minimum cost path search is achieved by finding the minimum pixel difference, which solves the problems of ghosting and breakage in the overlapping area of ​​the stitched image, and the quality of the fused image is more natural; specifically, for example, the corrected images of intersection A and intersection B are selected from the processed image set to be stitched, and the overlapping area accounts for 30%. The absolute difference of the pixel value of the matching point is calculated to be 75 (color difference value), the horizontal gradient convolution kernel = [-1, 0, 1; -2, 0, 2; -1, 0, 1], and the horizontal gradient value is 25, and the vertical gradient convolution kernel = [1, 2, 1; 0, 0, 0; -1 , -2, -1], the vertical gradient value is 120, and the structural difference value is (120×75)^2+(25×75)^2≈8.1×10^7; starting from the first row of the overlapping area (the first row at the top of the image), the path cost Cost(x, y)=B(x, y)+min{Cost(x-1, y-1), Cost(x-1, y), Cost(x-1, y+1)} is calculated for each pixel point P(x, y), where B(x, y) represents the total difference value of the pixel point P. The total difference value is accumulated and compared with the total difference values ​​of the three adjacent pixels in the next row one by one. The minimum cumulative cost point is selected as the path node in each row, and all nodes are connected to form a splicing seam. The complexity of the dynamic programming algorithm is O(n×m) (where n is the number of rows and m is the number of columns), and the real-time search efficiency is high; S4, stitching the images according to the optimal stitching seam of the images, dividing the images according to a dynamic grid, and implementing video output using priority sorting; The S4 comprises the following steps: S41, stitching the images to be stitched in the processed set of images to be stitched according to an optimal stitching seam, cropping the overlapping areas, and outputting a stitched panoramic image; dividing the stitched panoramic image into grids to obtain a plurality of image grids, calculating the optical flow amplitude in the image grids, and setting an optical flow threshold; when the optical flow amplitude is greater than the optical flow threshold, marking the corresponding grid as a moving grid and enlarging the size of the moving grid; otherwise, marking the corresponding grid as a static grid and reducing the size of the static grid to complete dynamic grid division; S42: Obtaining focus areas of the moving grid and the static grid, the focus areas including face areas, vehicle areas, and text areas, calculating image proportions of the focus areas, setting priorities based on the image proportions of the focus areas and the moving grids and static grids, and sorting them from high to low, preferentially transmitting the stitched panoramic image with a high priority to achieve video output; In this embodiment, image stitching is performed based on the optimal stitching seam of the image, the image is divided according to the dynamic grid, and the video output is achieved using priority sorting based on the moving static grid. This method is based on spatial adaptive compression, dynamic scaling of moving / static grids, and dynamic adaptation of priority queues to channel quality, ensuring dual-channel redundant transmission of high-priority grids, balancing details and bandwidth, and enhancing bandwidth compression and key information through dynamic grid division and focus priority transmission. It is highly intelligent and achieves stable video transmission. Specifically, for example, a panoramic image is divided into 240 grids (each grid is 128×72 pixels), the optical flow amplitude is >5 pixels / frame (vehicle movement), and the size is magnified by 150%. , the optical flow amplitude is ≤5 pixels / frame (road background), and the size is reduced by 70%. For example, in the overtaking vehicle area, the optical flow amplitude is 12 pixels / frame, and the grid is enlarged to 192×108 pixels. In the stationary guardrail area, the optical flow amplitude is 0.8 pixels / frame, and the grid is reduced to 90×50 pixels, which reduces the total data volume by 42%; identify the focus area and set the priority: high priority (the image proportion of the focus area is >10%), the first batch of transmission, the bandwidth allocation is 10Mbps; medium priority (the image proportion of the focus area is 5%-10%), the second batch of transmission, the bandwidth allocation is 5Mbps; low priority (the image proportion of the focus area is <5%), the last batch of transmission, the bandwidth allocation is 1Mbps.

[0027] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0028] The preferred embodiments of the invention disclosed above are intended only to help illustrate the invention. These preferred embodiments do not exhaust all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A video transmission high-definition image intelligent splicing system, characterized in that: include: The image preprocessing module is used to obtain the transmitted video to be processed and perform dynamic range equalization, then use the adaptive noise reduction method to perform filtering processing, and implement image correction to generate a set of processed images to be spliced; The feature point matching and screening module uses an adaptive corner detection algorithm to extract feature points from the processed image set to be stitched, obtains feature point matching pairs based on multi-dimensional spatial data index, and then introduces an improved RANSAC algorithm to screen feature points, outputting the processed image set to be stitched; The optimal image stitching seam generation module is used to find the overlapping area of ​​the processed image set to be stitched, and introduce a dynamic search algorithm based on the minimum pixel difference to obtain the optimal image stitching seam; The video output module is used to stitch images according to the best stitching seams, divide the images according to the dynamic grid, and use priority sorting to achieve video output.

2. The intelligent splicing system for high-definition video transmission according to claim 1, characterized in that: The obtaining of the to-be-processed transmission video and performing dynamic range equalization includes: Obtain the transmitted video to be processed, split it frame by frame, obtain an initial set of images to be stitched, divide it into non-overlapping sub-images to be stitched, and count the pixel values ​​of the sub-images to be stitched; Set overexposed areas, underexposed areas, and normal areas, divide the pixel values ​​of the sub-images to be stitched into the overexposed areas, underexposed areas, and normal areas, output the adjusted pixel values, and generate a balanced set of images to be stitched.

3. The intelligent splicing system for high-definition video transmission according to claim 2, characterized in that: The filtering process using the adaptive noise reduction method includes: Identifying motion regions and static regions of the balanced set of images to be stitched; For the static area, bilateral filtering is used to generate denoised pixel values, which are used to replace the pixel values ​​in the static area to obtain a denoised static area; for the moving area, adaptive median filtering is used to generate denoised pixel values ​​to obtain a set of filtered images to be spliced; Image correction is performed on the filtered image set to be stitched to generate a processed image set to be stitched.

4. The intelligent splicing system for high-definition video transmission according to claim 3, characterized in that: The feature point extraction includes: Obtaining an adaptive threshold of the processed image set to be stitched, and then performing preliminary screening and full-ring detection to obtain a set of characteristic pixel points; A feature pixel point in the feature pixel point set is selected as the center, and the image size is scaled according to the pyramid level to form a three-layer pyramid structure. Gaussian difference detection is then performed to calculate a 128-dimensional feature vector. The feature vectors of the feature pixel point set are extracted in sequence to form feature points, thereby obtaining a feature point set.

5. The intelligent splicing system for high-definition video transmission according to claim 4, characterized in that: Obtaining feature point matching pairs based on multidimensional spatial data indexes includes: The dimension with the largest variance in the feature point set is selected as the root node of the binary tree, and recursive partitioning is started at this time until the feature point in the feature point set is 0, thereby generating a spatial index binary tree; The nearest feature point and the second nearest feature point of the feature point are searched, and the matching between the nearest feature point and the second nearest feature point is determined and accepted to obtain a feature point matching pair.

6. The intelligent splicing system for high-definition video transmission according to claim 5, characterized in that: The improved RANSAC algorithm is introduced to perform feature point screening, including: Select half of the feature point matching pairs, select the maximum feature point matching pair distance and the minimum feature point matching pair distance to form a distance interval, and discretize the distance interval into j candidate thresholds, calculate the candidate thresholds; Traverse all candidate thresholds to screen feature points. When the candidate threshold is greater than the distance between two pixels, it is judged as a false match. Otherwise, it is judged as a positive match. The accuracy of the positive match is calculated in sequence. The candidate threshold corresponding to the highest accuracy is used as the global threshold. The feature point matching pairs are screened to generate the final feature point matching pairs and output the processed image set to be stitched.

7. The intelligent splicing system for high-definition video transmission according to claim 6, characterized in that: The step of searching for an overlapping area of ​​the processed image set to be stitched comprises: According to the final feature point matching pairs in the processed set of images to be spliced, all the images to be spliced ​​are spliced ​​according to the final feature point matching pairs to obtain overlapping areas.

8. The intelligent splicing system for high-definition video transmission according to claim 7, characterized in that: The step of searching for an overlapping area of ​​the processed image set to be stitched comprises: Calculate the color difference value and the structure difference value in the overlapping area to obtain the total difference value; Determine the first row of pixels in the overlapping area, calculate the total difference value of the spliced ​​pixels in the first row of pixels, then select the next row of pixels from the first row of pixels, calculate the three pixels adjacent to the spliced ​​pixels in the next row of pixels and the first row of pixels, and calculate the total difference value of the three pixels, and add it to the total difference value of the spliced ​​pixels in the first row of pixels; calculate the cumulative sum of the total difference values ​​of the spliced ​​pixels in the overlapping area in sequence, and establish a difference value function; The minimum total difference value of each layer in the overlapping area is recorded, and the pixel points corresponding to the minimum total difference value of each layer in the overlapping area are connected to form an image stitching path. Until the overlapping area is traversed, the pixel points on the optimal image stitching path are marked in sequence to obtain the optimal image stitching seam.

9. The intelligent splicing system for high-definition video transmission according to claim 8, characterized in that: The step of searching for an overlapping area of ​​the processed image set to be stitched comprises: Stitch the images according to the optimal stitching seam, crop the overlapping areas, and output the stitched panoramic image; divide the images according to the grid to obtain the moving grid and the static grid; The focus areas of the moving grid and the static grid are obtained, the image proportions of the focus areas are calculated, priorities are set, and the stitched panoramic images with high priorities are transmitted first.

10. A method for intelligent splicing of high-definition images in video transmission, characterized in that: Specifically include: S1. Obtain the transmitted video to be processed and perform dynamic range equalization to obtain a balanced set of images to be spliced. Then, use an adaptive noise reduction method to perform filtering and image correction to generate a processed set of images to be spliced. S2. Extracting feature points from the processed image set to be stitched using an adaptive corner detection algorithm, matching the feature points based on a multidimensional spatial data index to obtain feature point matching pairs, then introducing an improved RANSAC algorithm to screen the feature points, and outputting the processed image set to be stitched; S3, finding the overlapping area of ​​the processed set of images to be stitched, and introducing a dynamic search algorithm to generate an optimal image stitching path based on the minimum pixel difference to obtain the optimal image stitching seam; S4. Perform image stitching according to the optimal stitching seam of the image, divide the image according to a dynamic grid, and use priority sorting to achieve video output.

Images