A Video Continuous Vanishing Point Tracking Method Based on Optimized Bayesian Sampling Consistency

Abstract

This invention discloses a video continuous vanishing point tracking method based on optimized Bayesian sampling consistency. Vanishing points provide geometric information about a scene and are used to assist in camera calibration, scene understanding, and 3D reconstruction. Vanishing point detection has always been a fundamental problem in computer vision. The RANSAC method, as a traditional method for vanishing point detection, suffers from low efficiency and insufficient robustness. The BaySAC algorithm, with the assistance of prior information, can quickly find the correct parameters, mitigating these problems to some extent. This paper proposes an improved BaySAC vanishing point detection algorithm. It utilizes a line grouping strategy to increase the proportion of intrapoints in different groups, accelerating convergence. Simultaneously, it uses the detection result of the previous frame as prior information for subsequent frames in continuous vanishing point tracking, enabling rapid detection and tracking of vanishing points in the video. This method achieves high accuracy and efficiency and can run in real-time on real image datasets.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional laser point cloud data processing technology, specifically to a video continuous vanishing point tracking method utilizing optimized Bayesian sampling consistency. Background Technology

[0002] With the widespread application of computer vision technology in 3D reconstruction, autonomous driving, and robot navigation, vanishing point detection, as one of the core tasks of scene geometry analysis, has received continuous attention. Vanishing points reflect the projection patterns of parallel lines on the imaging plane in 3D space, and their accurate estimation can support key applications such as camera calibration, pose estimation, and structured scene understanding. Traditional methods are mostly based on image line feature extraction and parameter estimation frameworks, generating candidate models through random sampling strategies and iteratively selecting the optimal solution. However, they are susceptible to noise interference in dynamic scenes, and there are limitations in balancing efficiency and global optimality. Especially for continuous frame scenes, existing methods lack cross-frame information fusion mechanisms, initial estimation bias cannot be dynamically corrected, and the computational complexity increases significantly when detecting multiple vanishing points. In addition, algorithms that rely on strong geometric assumptions lack robustness in unstructured environments, limiting their generalization ability in practical applications. How to optimize vanishing point tracking efficiency through temporal continuity while balancing the accuracy and computational cost of multi-model fitting has become a technical bottleneck that urgently needs to be overcome.

[0003] Vanishing points, as a constraint on the main direction of a scene, have many applications in regular scenes such as Manhattan and Atlanta, including line classification, camera parameter estimation, camera orientation, and single-view 3D reconstruction. Currently, vanishing point detection mainly focuses on single-image estimation, with relatively few cases involving continuous frames. However, the continuous structured information and continuous motion of vanishing points in changing scenes can serve as effective prior information for estimation. Therefore, this paper builds upon RANSAC vanishing point detection and proposes an improved BaySAC-based vanishing point detection and tracking algorithm. The main contributions of this paper include:

[0004] (1) The BaySAC algorithm is used for vanishing point detection and tracking. The BaySAC algorithm is improved so that it can converge faster while ensuring accuracy. At the same time, the vanishing point continuity of consecutive frames in the video is used to determine the prior probability of the straight line in the next frame by using the vanishing point detection result of the previous frame as prior information. This greatly improves the efficiency of vanishing point tracking in the video without reducing accuracy.

[0005] (2) Combining the BaySAC algorithm with the line classification strategy, the problem of finding three optimal vanishing points globally is transformed into the problem of finding one and two vanishing points in two local areas respectively. The proportion of local points in each subset is greater, which further improves the efficiency of vanishing point detection. Summary of the Invention

[0006] The purpose of this invention is to provide a video continuous vanishing point tracking method based on optimized Bayesian sampling consistency, in order to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a video continuous vanishing point tracking method based on optimized Bayesian sampling consistency, comprising the following steps:

[0008] Step 1: Use the LSD algorithm to obtain the line parameters from the image;

[0009] Step 2: Multiple random selections of the smallest sample set to estimate the vanishing point parameter assumptions for the straight line;

[0010] Step 3: Construct the optimal vanishing point parameter hypothesis for spatial histogram statistics;

[0011] Step 4: Tracking status judgment. Check whether the current assumed parameters meet the tracking conditions. If yes, proceed to step 5; otherwise, return to step 3 and re-filter the parameters.

[0012] Step 5: Using this assumed vanishing point, calculate the distance of all lines to the vanishing point, and then calculate the prior probability of all lines. The formula for calculating the prior probability is:

[0013]

[0014] Among them, P i Let D be the prior probability of line i. i Let be the distance from line i to the assumed vanishing point, and m be a predefined threshold. An adaptive threshold m can be defined using the mean square error of the global points.

[0015] Step 6: Select the set of lines with the highest prior probability to construct a minimum sampling set. Calculate the position of the intersection point of the corresponding lines from this minimum sampling set, and use it as the vanishing point hypothesis for this cycle.

[0016] Step 7: Update the probability for all lines. The simplified Bayesian probability update formula (Kang, Zhang et al. 2014) is as follows:

[0017]

[0018] Where i is the set of local points, P t (i∈i) and P t-1 (i∈I) represent the probabilities of the lines in the t-th and t-1-th iterations, respectively; k is the number of lines whose distance from the vanishing point is less than the threshold in this iteration; and D is the total number of lines detected by the LSD algorithm in the current image.

[0019] Step 8: Use the probability change of each line during the iteration process as a condition for model convergence to check whether the probability distribution has converged or reached the maximum number of iterations. The convergence of the BaySAC algorithm is described by the average log-likelihood:

[0020]

[0021] Among them, H t It is the set of hypotheses for the straight line used in the t-th iteration of the hypothesis testing process, and x corresponds to the straight line at the vanishing point of the hypothesis in the loop; check whether the probability change result meets the judgment condition. If yes, output the vanishing point parameter of the current frame and generate a prior hypothesis for the next frame. If no, return to step 3 and continue iterative optimization.

[0022] Step 9: Repeat steps 1 to 8. The final processing result is the vanishing point parameter of the current frame, which is also used as the prior input for the next frame to form a continuous tracking loop.

[0023] Furthermore, in step 1, straight lines within a certain angle range in the vertical direction are selected based on the straight line parameters, and these straight lines are separated from all the straight lines extracted from the image as another set of parameters to be estimated.

[0024] Further, in step 2, during vanishing point estimation, image vanishing points corresponding to one vertical direction and two horizontal directions in three-dimensional space are extracted from the set of vertical lines and other set of lines, respectively. For vanishing point detection in the three orthogonal directions of the Manhattan world, the lines are first grouped. For each grouped line, the vanishing point hypotheses in the horizontal and vertical directions are solved separately. The vertical vanishing point hypotheses can be directly solved using the BaySAC algorithm. For solving the two vanishing points in the horizontal direction, the BaySAC algorithm is used to solve two different sets of prior probabilities for each line to correspond to the two vanishing points. Assume the distances of line l from the prior vanishing point hypotheses in the horizontal direction are respectively... The prior probability of line l in the horizontal grouping will then be represented by the probabilities corresponding to the two vanishing points, where the probability P corresponding to the closer vanishing point is... l It can be represented as

[0025]

[0026] Where m is a predefined threshold, and the probability of a vanishing point that is farther away will be represented as 0. The prior probabilities of the two vanishing points corresponding to the line will be updated after each loop.

[0027] Furthermore, in step 8, by calculating the change in the average log-likelihood before and after the iteration, the probability change of the straight line during the iteration process can be determined, and the convergence judgment process of the algorithm can be described as follows:

[0028]

[0029] Since each iteration requires selecting the minimum sample set (MSS) from the points with the highest probabilities, the convergence before and after iteration must also consider the consistency of the minimum sample set during iteration. Therefore, the convergence condition is defined as:

[0030] δ t =D MSS +Δlog(p(x|H t ))

[0031] in,

[0032] When the minimum sample set before and after the iteration is consistent, the convergence of the algorithm is determined by comparing the change in the mean log-likelihood. When the minimum sample set before and after the iteration is inconsistent, the algorithm is considered not to have converged, and iterations continue to obtain better vanishing point hypotheses. For the robustness of the algorithm, the number of samplings calculated based on the proportion of in-place points and the confidence level is used as an upper limit for the number of iterations to prevent multiple linear cycles with similar probabilities from appearing simultaneously in the minimum sample set, which could lead to D... MSS The case where the value is not 0 occurs.

[0033] Furthermore, in step 9, after exiting the loop, the obtained vanishing point hypothesis is used as a vanishing point corresponding to the image. In the problem of vanishing point tracking in consecutive frames, the calculated vanishing point result is used as the result of the current frame, and also as the prior information of the next frame image, which is used to calculate the prior probability hypothesis of the line in the next frame image. Attached Figure Description

[0034] Figure 1 This is a flowchart of the present invention.

[0035] Figure 2 The figure shows the experimental results of this invention. Detailed Implementation

[0036] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0037] Reference Figure 1 The present invention proposes a technical solution: a video continuous vanishing point tracking method based on optimized Bayesian sampling consistency, comprising the following steps:

[0038] Step 1: Use the LSD algorithm to obtain the line parameters from the image;

[0039] Step 2: Multiple random selections of the smallest sample set to estimate the vanishing point parameter assumptions for the straight line;

[0040] Step 3: Construct the optimal vanishing point parameter hypothesis for spatial histogram statistics;

[0041] Step 4: Tracking status judgment. Check whether the current assumed parameters meet the tracking conditions. If yes, proceed to step 5; otherwise, return to step 3 and re-filter the parameters.

[0042] Step 5: Using this assumed vanishing point, calculate the distance of all lines to the vanishing point, and then calculate the prior probability of all lines. The formula for calculating the prior probability is:

[0043]

[0044] Among them, P i Let D be the prior probability of line i. i Let be the distance from line i to the assumed vanishing point, and m be a predefined threshold. An adaptive threshold m can be defined using the mean square error of the global points.

[0045] Step 6: Select the set of lines with the highest prior probability to construct a minimum sampling set. Calculate the position of the intersection point of the corresponding lines from this minimum sampling set, and use it as the vanishing point hypothesis for this cycle.

[0046] Step 7: Update the probability for all lines. The simplified Bayesian probability update formula (Kang, Zhang et al. 2014) is as follows:

[0047]

[0048] Where I is the set of interior points, P t (i∈I) and P t-1 (i∈I) represent the probabilities of the lines in the t-th and t-1-th iterations, respectively; k is the number of lines whose distance from the vanishing point is less than the threshold in this iteration; and D is the total number of lines detected by the LSD algorithm in the current image.

[0049] Step 8: Use the probability change of each line during the iteration process as a condition for model convergence to check whether the probability distribution has converged or reached the maximum number of iterations. The convergence of the BaySAC algorithm is described by the average log-likelihood:

[0050]

[0051] Among them, H t This is the set of hypotheses for the straight line used in the t-th iteration of the hypothesis testing process, where x corresponds to the straight line at the vanishing point of the hypothesis in the loop; check whether the probability change result meets the judgment condition. If yes, output the vanishing point parameter of the current frame and generate a prior hypothesis for the next frame. If no, return to step 3 and continue iterative optimization;

[0052] Step 9: Repeat steps 1 to 8. The final processing result is the vanishing point parameter of the current frame, which is also used as the prior input for the next frame to form a continuous tracking loop.

[0053] This invention proposes a video continuous vanishing point tracking method based on optimized Bayesian sampling consistency. Compared with BaySAC, it adds new cyclic constraints, enabling the algorithm to quickly obtain results after convergence and thus avoiding the impact of redundant loops on algorithm efficiency. Furthermore, in the vanishing point tracking problem, it utilizes the continuity between frames, using the vanishing point detection results of the previous frame as prior information to provide constraints for subsequent vanishing point detections. This fully leverages the advantages of the BaySAC algorithm with the assistance of prior information, enabling the algorithm to converge quickly.

[0054] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A video continuous vanishing point tracking method based on optimizing Bayesian sampling consistency, characterized in that, The method comprises the following steps: Step 1, obtaining straight line parameters from the image using the LSD algorithm; Step 2, randomly selecting a minimum sample set multiple times to estimate vanishing point parameter hypotheses; Step 3, constructing a spatial histogram to count the best vanishing point parameter hypothesis; Step 4, tracking state judgment, checking whether the current hypothesis parameter satisfies the tracking condition, if yes, entering step 5, if no, returning to step 3 to reselect the parameter; Step 5, using the vanishing point hypothesis to calculate the distance of all straight lines to the vanishing point, and then calculating the prior probability of all straight lines, the calculation formula of the prior probability is: where P i is the a priori probability of line i, D i is the distance of line i to the hypothesized vanishing point, and m is a pre-defined threshold value, which is defined by the median error of the global points; Step 6, selecting a group of straight lines with the highest prior probability to construct a minimum sample set, and calculating the position of the intersection point corresponding to the group of straight lines from the minimum sample set as the vanishing point hypothesis of this cycle; Step 7, update the probabilities of all lines, the simplified Bayes probability update formula is: where I is the set of inliers, P t (i∈I) and P t-1 (i∈I) are the probabilities of the line correspondences in the tth and (t-1)th iterations, respectively, k is the number of lines in the current iteration that are within a threshold distance of the epipole, and D is the total number of lines detected in the current image using the LSD algorithm. Step 8, using the probability change of each straight line in the iteration process as the convergence condition of the model to check whether the probability distribution converges or reaches the maximum iteration number, and the average log likelihood is used to describe the convergence of the BaySAC algorithm: where H t is the hypothesis set of straight lines used in the tth iteration in the hypothesis testing process, x corresponds to the straight line of the corresponding hypothesis vanishing point in the loop; check whether the probability change result meets the judgment condition, if yes, output the current frame vanishing point parameter, and generate the prior hypothesis for the next frame, if not, return to step 3 and continue the iterative optimization; Step 9, repeating steps 1 to 8, and finally obtaining the processing result of the current frame of vanishing point parameters, which is used as the prior input of the next frame to form a continuous tracking loop; Further, in the step 2 of the method, when performing vanishing point estimation, the image vanishing points corresponding to one vertical direction and two horizontal directions in three-dimensional space are extracted from the vertical direction line set and the other line set respectively, for the vanishing point detection of the three orthogonal directions in Manhattan world, the lines are grouped first, and then the vanishing point hypotheses of the horizontal direction and the vertical direction are solved respectively, the vanishing point of the vertical direction can be directly solved by using the BaySAC algorithm, and for the solving of the two vanishing points of the horizontal direction, two different prior probabilities of each line are solved by using the BaySAC algorithm to correspond to the two vanishing points, assuming that the distance between the line l and the prior vanishing point hypothesis of the horizontal direction is respectively The prior probability of the line l in the horizontal group will be represented by the probabilities corresponding to the two vanishing points respectively, wherein the probability P l is represented as Wherein, m is a predefined threshold value, at the same time, the probability corresponding to the vanishing point farther away will be represented as 0, and the prior probability of the two vanishing points corresponding to the straight line will be updated after each cycle. 2.The video continuous vanishing point tracking method based on optimized Bayesian sampling consistency according to claim 1, wherein: In step 1, straight lines within a certain angle range in the vertical direction are selected according to the straight line parameters, and these straight lines are separated from all straight lines extracted from the image as another parameter set to be estimated. 3.The video continuous vanishing point tracking method based on optimized Bayesian sampling consistency according to claim 1, characterized in that: In step 8, the change value of the average log likelihood before and after iteration is calculated to judge the probability change of the straight line in the iteration process, and the convergence judgment process of the algorithm is described as: Since the minimum sample set needs to be selected from the several points with the highest probability each time, the consistency of the minimum sample set during iteration also needs to be considered when judging the convergence before and after iteration, and therefore the convergence condition is defined as: wherein That is, when the minimum sample sets before and after iteration are consistent, the convergence of the algorithm is determined by comparing the change of average log-likelihood, and when the minimum sample sets before and after iteration are inconsistent, it is considered that the algorithm has not converged, and then a better vanishing point hypothesis is obtained by continuing iteration. In order to ensure the robustness of the algorithm, the sampling times calculated based on the proportion of local points and confidence will be the upper limit of the number of loops, so as to prevent multiple straight lines with close corresponding probabilities from appearing in the minimum sample set and causing D MSS The case where the value is not 0 occurs.

4. The video continuous vanishing point tracking method based on optimized Bayesian sampling consistency according to claim 1, characterized in that: In step 9, after the loop is exited, the vanishing point hypothesis obtained is used as a vanishing point corresponding to the image, and in the vanishing point tracking problem of continuous frames, the calculated vanishing point result is used as the prior information of the next frame image to calculate the prior probability hypothesis of the straight line in the corresponding next frame image.

Images