Method for improving video stitching quality based on improved optimal suture line algorithm

Abstract

The invention discloses a method for improving video stitching quality based on an improved optimal stitching line algorithm, relates to the field of application of vehicle-mounted cameras, the field of panoramic image fusion, the field of reversing parking and the field of unmanned driving, and solves the common problems of image distortion, obvious stitching lines, discontinuous stitching areas and the like in the video stitching process. An improved optimal suture line algorithm is selected, and the method can automatically find the optimal overlapping area and the optimal suture line position, so that the distortion and discontinuity phenomena in the splicing process are reduced. By improving the splicing effect, the method can be widely applied to the fields of panoramic video production, virtual reality and the like, and the user experience is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the fields of vehicle-mounted camera application, panoramic image fusion, reverse parking, and unmanned driving, and specifically, to a method for improving video splicing quality based on an improved optimal stitching line algorithm. Background Art

[0002] In recent years, with the development of digital image processing technology and computer science technology, people's demand for wide-field-of-view videos has become increasingly strong. However, the traditional improvement solutions for expanding the video field of view are not effective, so video stitching technology came into being.

[0003] Traditional video stitching techniques, such as those based on SIFT, SURF or ORB algorithms, show certain limitations when processing complex video data. First, these methods usually perform global feature extraction and matching within the entire video frame range, which not only leads to a huge consumption of computing resources, but also makes it difficult to maintain high efficiency in application scenarios with high real-time requirements. Second, in complex scenes with lighting changes, significant differences in viewing angles or the presence of dynamic objects, the feature matching accuracy of traditional methods is often seriously affected, and the increase in the mismatch rate directly weakens the accuracy of the stitching results. In addition, due to the possible differences in color, brightness and texture between video frames, traditional stitching methods are difficult to achieve visually seamless seams when processing seams, resulting in insufficient continuity and naturalness of the stitched video. In view of the above limitations, academia and industry have begun to devote themselves to the development of video stitching technology based on improved algorithms. The core of this technology is to achieve more efficient, accurate and natural video stitching effects by optimizing key links such as feature detection, matching and seam line determination.

[0004] When stitching images or offline video files, the time requirement is not high and can even be ignored. However, when stitching real-time video streams, the time requirement is extremely strict, because it must be ensured that the current video image can complete the entire stitching process within the frame spacing of the video stream, otherwise the final stitched video will drop frames or even freeze. In addition, with the growing demand for ultra-high-definition videos such as 4k or even 8k, the video acquisition and processing process has become complicated and difficult. Therefore, real-time performance has always been one of the difficulties that need to be overcome in video stitching technology. In addition to the real-time requirement, the biggest difference between video stitching and image stitching is that moving objects in the overlapping area of ​​the video will seriously affect the stitching effect of the video. If the traditional image stitching method is continued to be used to stitch each video frame by frame, the overlapping area of ​​the final stitched image will be blurred and ghosted.

[0005] The present invention proposes a method for improving the quality of video splicing based on an improved optimal seam algorithm. First, an improved feature detection algorithm (such as an improved version of SURF, ORB, etc.) is used to extract feature points from video frames. These algorithms can reduce the amount of calculation and the mismatch rate while ensuring the stability of the feature points, so as to quickly and accurately find potential matching points between video frames. Then, the extracted feature points are matched by an efficient feature matching algorithm (Hamming distance matching, bidirectional matching combined with RANSAC algorithm, GMS feature point matching, etc.) to find the corresponding points between the video frames. These corresponding points will be used for subsequent overlapping area identification. Based on the matched feature point pairs, the overlapping area between the video frames is determined by calculating geometric transformation parameters such as homography matrices. In order to improve the accuracy of the overlapping area, multi-frame fusion or motion estimation-based methods can be used to further optimize the boundaries of the overlapping area. In order to find the optimal seam position, it is necessary to define a seam criterion, which usually considers factors such as color consistency, geometric continuity, and image content. The present invention adopts an algorithm based on graph cut to transform the seam search problem into an energy minimization problem. After determining the seam line criteria, an optimization algorithm such as dynamic programming is used to search for the optimal seam line position in the overlapping area. The dynamic programming algorithm can gradually build a solution while retaining the optimal solution of each step, thereby ensuring that the seam line position finally found is the global optimal. In order to further improve the quality of the seam line, the seam line position initially searched can be post-processed and optimized. The present invention uses morphological operations (such as dilation and corrosion) to smooth the seam line, and uses more refined pixel-level adjustments to eliminate potential artifacts or chromatic aberrations. After determining the optimal overlapping area and seam line position, the image fusion algorithm is used to fuse the video frames in the overlapping area. During the fusion process, it is necessary to ensure that the transition is natural and seamless, and avoid obvious splicing marks. Summary of the invention

[0006] In order to solve the problem of finding the overlapping area and the best stitching line of each two frames of video signals, the present invention designs a model based on an improved algorithm to find the best overlapping area and stitching line position and improve the quality of video splicing, and uses algorithms such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Robust Features) or ORB (Oriented FAST and Rotated BRIEF) to extract feature points from each frame of the video. These feature points are robust to image scaling, rotation and illumination changes. The feature points detected in two adjacent frame images are roughly matched. The similarity is calculated based on the descriptor of the feature point (128-dimensional vector of SIFT), and the matching process is accelerated using data structures such as Kd tree or FLANN (Fast Nearest Neighbor Search Library). The rough matching results are purified using RANSAC (Random Sampling Consensus Algorithm), the mismatched points are removed, and the accurately matched feature point pairs are obtained. Based on the accurately matched feature point pairs, the overlapping area of ​​the two images is determined. Usually, the feature points with the largest distance from the middle of the two images are selected, and the positions of these feature points can determine the outermost edge of the overlapping area. By calculating the percentage of the length of the overlapping area to the total length of the image, the feature point detection range of the subsequent frames is further determined to improve processing efficiency. A corresponding weighted graph is constructed in the overlapping area of ​​the two frames. The nodes in the graph represent the pixels in the overlapping area, and the edges represent the connections between adjacent pixels, and are assigned corresponding weight values. The weights can be calculated based on factors such as color difference, gradient change, and structural similarity. A graph cutting algorithm based on the maximum flow of the network is used to find an optimal stitching line in the weighted graph. This line divides the overlapping area into two parts so that the boundary difference between the two parts is minimized. They find the optimal stitching line by solving the minimum cut problem. After determining the optimal stitching line, the pixels in the overlapping area are fused. A fade-in and fade-out weighted fusion algorithm can be used, but considering the moving objects and exposure differences, different fusion strategies can be used on both sides of the stitching line. For pixels near the stitching line, special processing can be performed to reduce ghosting and ghosting.

[0007] The method can be implemented by the following steps:

[0008] Step 1: Use an improved feature detection algorithm (optimized version such as ORB) to reduce the computational complexity and mismatch rate while ensuring the stability of feature points, and quickly and accurately extract potential matching points in video frames.

[0009] Step 2: Use an efficient feature matching algorithm (GMS feature point matching) to accurately match feature points and determine corresponding points between video frames. Based on these corresponding points, the overlapping area can be accurately located by calculating geometric transformation parameters such as the homography matrix, and the overlapping boundary can be further optimized by using multi-frame fusion or motion estimation.

[0010] Step 3: Comprehensively consider the seam line criteria of color consistency, geometric continuity and image content. Using a graph cut-based algorithm, the seam line search is transformed into an energy minimization problem to ensure the rationality and optimality of the seam line position.

[0011] Step 4: Use the dynamic programming algorithm to gradually build solutions in the overlapping area, retain the optimal solution of each step, and ensure that the global optimal suture line position is finally found.

[0012] Step 5: Smooth the seam lines through morphological operations (dilation, erosion), and perform fine pixel-level adjustments to eliminate artifacts and chromatic aberrations to further improve the quality of the seam lines.

[0013] Step 6. After the optimal overlapping area and stitching line position are determined, an advanced image fusion algorithm is used to ensure that the video frames transition naturally and seamlessly in the overlapping area to avoid stitching marks.

[0014] The advantages and beneficial effects of the present invention are as follows:

[0015] 1. The present invention proposes a method for improving the quality of video stitching based on an improved optimal stitching algorithm, which can more accurately find the overlapping area between two or more images through more refined feature detection and matching. This helps to reduce the stitching error caused by inaccurate registration and improve the clarity and fidelity of the final video.

[0016] 2. The present invention introduces an optimal stitching algorithm to further eliminate ghost images or artifacts caused by moving objects or exposure differences in the overlapping area, making the stitching result more natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of the algorithm proposed for the present invention.

[0018] Figure 2 This is the AXI read-write control simulation timing diagram of the algorithm proposed in this invention.

[0019] Figure 3 This is a functional logic simulation test diagram of the algorithm proposed in this invention. DETAILED DESCRIPTION

[0020] The specific use process of the present invention is achieved by the following steps:

[0021] Step 1: Use an improved feature detection algorithm (optimized version such as ORB) to reduce the computational complexity and mismatch rate while ensuring the stability of feature points, and quickly and accurately extract potential matching points in video frames.

[0022] After constructing the Gaussian scale space, the traditional ORB algorithm will use the FAST algorithm to extract corner points. The principle is to set a fixed threshold by taking the difference between the grayscale value of the pixel point and the grayscale value of the pixel point on a circle with a radius of 3 pixels in each layer. If the grayscale difference is greater than this threshold, this point is a corner point. In some images with unclear textures in some parts, the FAST algorithm cannot extract corner points in this part, but in positions with rich textures, a large number of clustered corner points will be extracted. In order to make the extracted corner points evenly distributed on the image to be spliced, the present invention uses an improved FAST algorithm to extract corner points. Compared with the traditional algorithm, this algorithm divides each layer of the Gaussian scale space into 30*30 grids, and changes the FAST threshold so that feature points can be extracted from each grid. The specific algorithm flow is as follows:

[0023] 1) Construct a Gaussian scale space (Gaussian pyramid).

[0024] 2) Divide each layer of the Gaussian pyramid into a grid of 30*30 pixels.

[0025] 3) Set the FAST threshold and extract corner points for each grid. If no corner points can be extracted, lower the threshold so that at least one corner point can be extracted for each grid.

[0026] 4) Use the non-maximum suppression method to select the corner point with the largest response value in each grid. The method for calculating whether point p is the maximum response is as shown in formula (1), where V is the maximum response value and t is the threshold.

[0027]

[0028] After the improved FAST algorithm extracts a large number of corner points, the Oriented FAST algorithm is used to calculate the rotation angle of each corner point, and then the corner point rotation angle solved in the previous step is used to guide BRIEF to describe the corner point. The BRIEF descriptor is a binary descriptor, and the graphics processing unit (GPU) can be used to accelerate the subsequent calculation to further improve the matching efficiency. The specific process is to first randomly select n point pairs in the neighborhood of the feature point to form an n-dimensional binary descriptor and define a binary comparison of a pixel point P.

[0029]

[0030] Where P(x) is the image at the pixel point p(y) is the grayscale of the image at position y; p(y) is the grayscale of the image at position y; by selecting n (x, y) pixel position pairs, an n-dimensional binary code string can be obtained.

[0031] f (n) (p) = Σ 1≤i≤n 2 -1τ(p:x,y) (3)

[0032] The BRIEF descriptor obtained at this time does not have rotation invariance, so a 2×n matrix S is defined.

[0033]

[0034]

[0035] Calculate the rotation matrix R by the pixel direction θ θ , at this time R θ The matrix is

[0036] S θ =R θ (6) Combining formula (4) we can obtain the BRIEF descriptor with rotation invariance

[0037] g n (p, θ) = f n (p)|(x i ,y i ) (7)

[0038] Step 2: Use an efficient feature matching algorithm (GMS feature point matching) to accurately match feature points and determine corresponding points between video frames. Based on these corresponding points, the overlapping area can be accurately located by calculating geometric transformation parameters such as the homography matrix, and the overlapping boundary can be further optimized by using multi-frame fusion or motion estimation.

[0039] The present invention screens the correct matching points by combining brute force matching with GMS, which has the characteristics of faster screening speed and more correct matching points retained, effectively improving the efficiency of image stitching. The GMS algorithm is a fast, ultra-robust feature matching algorithm based on the statistical characteristics of grid motion. The core idea of ​​the algorithm is that there must be several correct matching points next to a correct matching point as support, with motion consistency; but a wrong matching point has almost no similarly wrong matching points next to it as support (assuming that the wrong matching occurs randomly). The main principles of the GMS algorithm are as follows: Figure 2 There are regions a and b in the left and right pictures respectively. Assuming the correct probability of matching region a is t, the probability that a matching point in region a is wrong but the matching point still falls in region b is,

[0040]

[0041] The probability of matching the correct feature points in area a to area b is,

[0042]

[0043] in, The match for region a falls in region b; is a wrong match; M is the number of feature points in the entire image; β is an additional factor, mainly for repeated structures. From formula (9), we can get that pt is close to t and p f Close to 0, let S i is the feature point x i The support number of matching points at S is , because each feature point matching is independent, so S i It can be approximated as a binomial distribution

[0044]

[0045] Where n is the number of feature points in region a. From formula (10), we can get S i The probability density map of . In an image, the motion of a small area is smooth, and the motion of a larger area is also smooth. For a 3×3 area S i It becomes formula (12), S i The distribution of becomes formula (13), where K represents the number of grids in the selected area.

[0046]

[0047]

[0048] At this time, the standard deviation S t , S f They are

[0049]

[0050]

[0051] From S i From the distribution diagram, we can see that the supporting point distributions of correct matching points and incorrect matching points are different. In order to quickly filter out incorrect matching points, we use S i The mean and variance of the set an index P, defined as follows:

[0052]

[0053] From formula (15), we can get that P is proportional to In order to quickly traverse all feature matching points, the GMS algorithm generally divides the image into grids and calculates the support of each grid. Therefore, using formula (16) can determine whether the current grid matching is correct.

[0054]

[0055] That is S ijIf equation (16) is satisfied, the matching pair is correct, otherwise the matching pair is wrong. ij Represents the grid area {i k , j k} and the total number of matching points in the 8 adjacent grid areas around it. α is a constant factor, N i is the total number of matching points in the 3×3 grid area, usually N i =6.

[0056] Step 3: Comprehensively consider the seam line criteria of color consistency, geometric continuity and image content. Using a graph cut-based algorithm, the seam line search is transformed into an energy minimization problem to ensure the rationality and optimality of the seam line position.

[0057] The graph cut-based algorithm transforms the image stitching problem into the maximum flow minimum cut problem in graph theory. In the graph, each pixel is regarded as a node, and the relationship between pixels (such as color difference, position relationship, etc.) is regarded as an edge and assigned a corresponding weight. By minimizing an energy function, the optimal seam line position can be found so that the color difference and geometric discontinuity on both sides of the seam line are minimized.

[0058] An undirected graph G = (V, E) is used to represent the image to be stitched, where V is a set of vertices, each vertex corresponds to a pixel in the image; E is a set of edges, representing the connection relationship between vertices (pixels). In Graph Cuts, in addition to ordinary vertices, two special vertices "source point S" and "sink point T" are introduced, representing two different areas in the image (such as foreground and background).

[0059] The energy function E(L) usually consists of two parts: the region term R(L) and the boundary term B(L), namely:

[0060] E(L)=aR(L)+B(L) (17)

[0061] Where L is a set of labels, indicating the attribution of each pixel (such as foreground or background); a is a parameter that adjusts the relative importance of the region term and the boundary term. Region term R(L): For each pixel p, a penalty for assigning a label lp (such as 0 for background and 1 for foreground). This penalty can be obtained by comparing the grayscale or color of the pixel with the grayscale histogram of the given target and background, that is:

[0062] R(L)=p∈V∑R p (l p ) (18)

[0063] R p (1) = -ln P r (I p |obj) (19)

[0064] R p (0) = -ln P r (I p |bkg) (20)

[0065] Boundary term B(L): measures the discontinuity between adjacent pixels, encouraging similar pixels to have the same label and different pixels to have different labels. It is usually calculated using the color difference or other feature differences between pixels:

[0066] B(L)=∑ (p，q)∈N B p，q (l p , l q ) (twenty one)

[0067] B p，q (l p , l q )=δ(l p ≠l q )V (p，q) (twenty two)

[0068] Step 4: Use the dynamic programming algorithm to gradually build solutions in the overlapping area, retain the optimal solution of each step, and ensure that the global optimal suture line position is finally found.

[0069] The dynamic programming algorithm avoids repeated calculations by breaking down the problem into smaller, overlapping sub-problems and storing the solutions to these sub-problems. In image stitching, it uses the similarities and differences between adjacent pixels in the overlapping area to construct an energy matrix and finds the path with the minimum energy, i.e. the optimal stitching line, through step-by-step iteration.

[0070] Define the energy matrix: Let the height of the overlapping area be h and the width be w, then initialize an energy matrix E of size h×w, where E(i, j) represents the total energy cost of the optimal stitching line from the first row to the position (i, j).

[0071] Similarity measure: Define the similarity measure function e(i, j) between pixels in adjacent images, usually based on the color difference or brightness difference of the pixels. For example, the squared difference can be used to measure:

[0072] e(i,j)=[f(B(i,j))-g(B(i,j))] 2 (twenty three)

[0073] Among them, f(B(i, j)) and g(B(i, j)) represent the pixel values ​​of two adjacent images f and g at the position (i, j) in the overlapping area B respectively.

[0074] Step 5: Smooth the seam lines through morphological operations (dilation, erosion), and perform fine pixel-level adjustments to eliminate artifacts and chromatic aberrations to further improve the quality of the seam lines.

[0075] In the post-processing of image stitching, smoothing the seams through morphological operations (such as dilation and erosion) and implementing fine pixel-level adjustments are important steps to eliminate artifacts and chromatic aberrations and further improve the quality of the seams.

[0076] The dilation operation is often used to expand the foreground area to fill small gaps or holes near the seam line to make the seam line smoother. Define the structural element: First define a structural element (also called a kernel or template), which determines the scope and shape of the dilation operation. Common structural elements are rectangles, circles, etc. Apply the dilation operation: Slide the structural element over the image and replace the pixel value at the center of the structural element with the maximum value of the pixel in the area covered by the structural element (1 for binary images and the maximum value for grayscale images). Apply Gaussian blur or other smoothing filters to smooth the edges near the seam line to reduce jagged edges or sharp transitions.

[0077] Step 6. After the optimal overlapping area and stitching line position are determined, an advanced image fusion algorithm is used to ensure that the video frames transition naturally and seamlessly in the overlapping area to avoid stitching marks.

Claims

1. A method for improving video stitching quality based on an improved optimal stitching algorithm, characterized in that: The method is implemented by the following steps: Step 1: Use an improved feature detection algorithm (optimized version such as ORB) to reduce the computational complexity and mismatch rate while ensuring the stability of feature points, and quickly and accurately extract potential matching points in the video frame; Step 2: Use an efficient feature matching algorithm (GMS feature point matching) to accurately match feature points and determine corresponding points between video frames. Based on these corresponding points, calculate geometric transformation parameters such as homography matrix to accurately locate overlapping areas, and use multi-frame fusion or motion estimation to further optimize overlapping boundaries. Step 3: Considering the seam line criteria of color consistency, geometric continuity and image content, a graph cut-based algorithm is used to transform the seam line search into an energy minimization problem to ensure the rationality and optimality of the seam line position. Step 4: Use the dynamic programming algorithm to gradually build solutions in the overlapping area, retain the optimal solution of each step, and ensure that the global optimal suture line position is finally found; Step 5: Smooth the stitching line through morphological operations (dilation, erosion), and perform fine pixel-level adjustments to eliminate artifacts and chromatic aberrations, further improving the quality of the stitching line. Step 6. After the optimal overlapping area and stitching line position are determined, an advanced image fusion algorithm is used to ensure that the video frames transition naturally and seamlessly in the overlapping area to avoid stitching marks.

2. The method for improving video stitching quality based on an improved optimal stitching algorithm according to claim 1, characterized in that: In step one, the traditional ORB algorithm will use the FAST algorithm to extract corner points after constructing the Gaussian scale space. The principle is to set a fixed threshold by taking the difference between the grayscale value of the pixel point and the grayscale value of the pixel point on a circle with a radius of 3 pixels in each layer. If the grayscale difference is greater than this threshold, this point is a corner point. In some images with unclear textures in some parts, the FAST algorithm cannot extract corner points in this part, but in positions with rich textures, a large number of clustered corner points will be extracted. In order to make the extracted corner points evenly distributed on the image to be spliced, the present invention uses an improved FAST algorithm to extract corner points. Compared with the traditional algorithm, this algorithm divides each layer of the Gaussian scale space into 30*30 grids, and changes the FAST threshold so that feature points can be extracted from each grid. The specific algorithm flow is as follows: 1) Construct Gaussian scale space (Gaussian pyramid); 2) Divide each layer of the Gaussian pyramid into a grid of 30*30 pixels; 3) Set the FAST threshold to extract corner points for each grid. If no corner points can be extracted, lower the threshold so that at least one corner point can be extracted for each grid. 4) Use the non-maximum suppression method to select the corner point with the largest response value in each grid. The method for calculating whether point p is the maximum response is as shown in formula (1), where V is the maximum response value; t is the threshold; After a large number of corner points are extracted by the improved FAST algorithm, the OrientedFAST algorithm is used to calculate the rotation angle of each corner point, and then the corner point rotation angle solved in the previous step is used to guide BRIEF to describe the corner point. The BRIEF descriptor is a binary descriptor. In subsequent calculations, a graphics processing unit (GPU) can be used to accelerate and further improve the matching efficiency. The specific process is to first randomly select n point pairs in the neighborhood of the feature point to form an n-dimensional binary descriptor and define a binary comparison of a pixel point P; Where P(x) is the image at the pixel point p(y) is the grayscale of the image at position y; select n (x, y) pixel position pairs to obtain an n-dimensional binary code string; f (n) (p)=Σ 1≤i≤n 2 i-1 τ(p:x,y) (3) The BRIEF descriptor obtained at this time is not rotationally invariant, so a 2×n matrix S is defined; Calculate the rotation matrix R by the pixel direction θ θ , at this time R θ The matrix is S θ =R θ (6) Combining formula (4), we can get the BRIEF descriptor with rotation invariance: g n (p,θ)=f n (p)|(x i ,y i ) (7) 3. The method for improving video stitching quality based on an improved optimal stitching algorithm according to claim 1, characterized in that: In step 2, the present invention screens the correct matching points by combining brute force matching with GMS, which has the characteristics of faster screening speed and more correct matching points, effectively improving the efficiency of image stitching. The GMS algorithm is a fast and ultra-robust feature matching algorithm based on the statistical characteristics of grid motion. The core idea of ​​the algorithm is that there must be several correct matching points as support next to a correct matching point, which has motion consistency; but for an incorrect matching point, there are almost no similarly incorrect matching points as support next to it (assuming that the incorrect matching occurs randomly). The main principle of the GMS algorithm is as follows: In the left and right pictures of Figure 2, there are area a and area b respectively. Assuming that the correct probability of matching to area a is t, the probability that a matching point in area a is wrong but the matching point still falls in area b is, The probability that the correct feature point in region a matches region b is, in, The match for region a falls in region b; is a wrong match; M is the number of feature points in the entire image; β is an additional factor, mainly for repeated structures. From formula (9), we can get p t Close to t and p f Close to 0, let S i is the feature point x i The support number of matching points at S is , because each feature point matching is independent, so S i It can be approximated as a binomial distribution Where n is the number of feature points in region a. From formula (10), we can get S i Probability density map of . In an image, the motion of a small area is smooth, and the motion of a larger area is also smooth. For a 3×3 area S i It becomes formula (12), S i The distribution of becomes formula (13), where K represents the number of grids in the selected area, At this time, the standard deviation S t , S f They are From S i From the distribution diagram, we can see that the supporting point distributions of correct matching points and incorrect matching points are different. In order to quickly filter out incorrect matching points, we use S i The mean and variance of the set an index P, defined as follows: From formula (15), we can get that P is proportional to In order to quickly traverse all feature matching points, the GMS algorithm generally divides the image into grids and calculates the support of each grid. Therefore, formula (16) can be used to determine whether the current grid matching is correct. That is S ij If equation (16) is satisfied, the matching pair is correct, otherwise the matching pair is wrong, where S ij Represents the grid area {i k ,j k } and the total number of matching points in the 8 adjacent grid areas around it, α is a constant factor, N i is the total number of matching points in the 3×3 grid area, usually N i =6.

4. The method for improving video stitching quality based on an improved optimal stitching algorithm according to claim 1, characterized in that: In step 3, the algorithm based on graph cut transforms the image stitching problem into the maximum flow minimum cut problem in graph theory. In the graph, each pixel is regarded as a node, and the relationship between pixels (such as color difference, position relationship, etc.) is regarded as an edge and assigned a corresponding weight. By minimizing an energy function, the optimal seam line position can be found so that the color difference and geometric discontinuity on both sides of the seam line are minimized. An undirected graph G = (V, E) is used to represent the image to be stitched, where V is a set of vertices, each vertex corresponds to a pixel in the image; E is a set of edges, which represents the connection relationship between vertices (pixels). In GraphCuts, in addition to ordinary vertices, two special vertices "source point S" and "sink point T" are introduced, representing two different areas in the image (such as foreground and background). The energy function E(L) usually consists of two parts: the region term R(L) and the boundary term B(L), namely: E(L)=aR(L)+B(L) (17) Among them, L is a label set, indicating the attribution of each pixel (such as belonging to the foreground or background); a is a parameter that adjusts the relative importance of the region term and the boundary term, and the region term R(L): for each pixel p, the penalty for assigning a label lp (such as 0 for background and 1 for foreground). This penalty can be obtained by comparing the grayscale or color of the pixel with the grayscale histogram of the given target and background, that is: R(L)=p∈VΣR p (l p ) (18) R p (1)=-ln P r (I p |obj) (19) R p (0)=-ln P r (I p |bkg) (20) Boundary term B(L): measures the discontinuity between adjacent pixels, encourages similar pixels to have the same label, and different pixels to have different labels. It is usually calculated using the color difference or other feature differences between pixels: B(L)=Σ (p,q)∈N B p,q (l p ,l q ) (21) B p,q (l p ,l q )=δ(l p ≠l q )V (p,q) (22) The algorithm based on graph cut transforms the image stitching problem into the maximum flow minimum cut problem in graph theory. In the graph, each pixel is regarded as a node, and the relationship between pixels (such as color difference, position relationship, etc.) is regarded as an edge and assigned corresponding weights. By minimizing an energy function, the optimal stitching line position can be found so that the color difference and geometric discontinuity on both sides of the stitching line are minimized. An undirected graph G = (V, E) is used to represent the image to be stitched, where V is a set of vertices, each vertex corresponds to a pixel in the image; E is a set of edges, which represents the connection relationship between vertices (pixels). In GraphCuts, in addition to ordinary vertices, two special vertices "source point S" and "sink point T" are introduced, representing two different areas in the image (such as foreground and background). The energy function E(L) usually consists of two parts: the region term R(L) and the boundary term B(L), namely: E(L)=aR(L)+B(L) (17) Where L is a set of labels, indicating the attribution of each pixel (such as foreground or background); a is a parameter that adjusts the relative importance of the region term and the boundary term. Region term R(L): For each pixel p, a penalty is assigned to the label lp (such as 0 for background and 1 for foreground). This penalty can be obtained by comparing the grayscale or color of the pixel with the grayscale histogram of the given target and background, that is: R(L)=p∈VΣR p (l p ) (18) R p (1)=-ln P r (I p |obj) (19) R p (0)=-ln P r (I p |bkg) (20) Boundary term B(L): measures the discontinuity between adjacent pixels, encouraging similar pixels to have the same label and different pixels to have different labels. It is usually calculated using the color difference or other feature differences between pixels: B(L)=j (p,q)∈N B p,q (l p ,l q ) (21) B p,q (l p ,l q )=δ(l p ≠l q )V (p,q) (22) 5. The method for improving video stitching quality based on an improved optimal stitching algorithm according to claim 1, characterized in that: In step 4, the dynamic programming algorithm avoids repeated calculations by decomposing the problem into smaller, overlapping sub-problems and storing the solutions to these sub-problems. In image stitching, it uses the similarities and differences between adjacent pixels in the overlapping area to construct an energy matrix and finds the path with the minimum energy, i.e. the optimal stitching line, through step-by-step iteration. Define the energy matrix: Let the height of the overlapping area be h and the width be w, then initialize an energy matrix E of size h×w, where E(i,j) represents the total energy cost of the optimal suture line from the first row to the position (i,j); Similarity measure: Define the similarity measure function e(i,j) between pixels in adjacent images, usually based on the color difference or brightness difference of the pixels. For example, the squared difference can be used to measure: e(i,j)=[f(B(i,j))-g(B(i,j))] 2 (23) Among them, f(B(i,j)) and g(B(i,j)) represent the pixel values ​​of the two adjacent images f and g at the position (i,j) in the overlapping area B respectively.

6. The method for improving video stitching quality based on an improved optimal stitching algorithm according to claim 1, characterized in that: In step 5, in the post-processing of image stitching, smoothing the seam line through morphological operations (such as dilation and erosion) and implementing fine pixel-level adjustments are important steps to eliminate artifacts and chromatic aberrations and further improve the quality of the seam line; The dilation operation is often used to expand the foreground area to fill the tiny gaps or holes near the seam line to make the seam line smoother. Define the structural element: First define a structural element (also called a kernel or template), which determines the scope and shape of the dilation operation. Common structural elements are rectangles, circles, etc. Apply the dilation operation: Slide the structural element on the image, replace the pixel value at the center of the structural element with the maximum value of the pixel in the area covered by the structural element (1 for binary images and the maximum value for grayscale images), and apply Gaussian blur or other smoothing filters to smooth the edges near the seam line to reduce jagged edges or sharp transitions.

7. The method for improving video stitching quality based on an improved optimal stitching algorithm according to claim 1, characterized in that: In step 6, after the optimal overlap area and stitching line position are determined, an advanced image fusion algorithm is used to ensure that the video frames transition naturally and seamlessly in the overlap area to avoid stitching marks; Image fusion adopts a fade-in and fade-out weighted average fusion algorithm based on seam lines. The algorithm principle of the fade-in and fade-out weighted average fusion method is mainly to take into account the positional relationship of pixel points in the area when calculating pixel values, and add a weight coefficient; if the fade-in and fade-out fusion algorithm is performed in the entire overlapping area, although the stitching traces such as seam lines can be reduced or even eliminated, the artifact problem caused by parallax cannot be well solved, especially when there are foreground targets.

Images