Vehicle tracking method for constructing virtual trajectory based on ice and snow pavement rut traces

Abstract

The invention is applicable to the technical field of automatic driving, and provides a vehicle tracking method for constructing a virtual trajectory based on ice and snow pavement rut traces, which comprises the following steps of: acquiring ice and snow pavement rut images by fusing a vehicle-mounted camera with multiple sensors; distortion removal processing is carried out; color clustering is carried out on the pixel points, irrelevant element data are removed, and a preliminary ROI image is obtained; further pretreatment; converting the image into an aerial view through inverse perspective transformation; based on the distance range of left and right tires of the vehicle, pixel point coordinates of the rut edge are extracted through line-by-line scanning; fitting the cubic polynomial model by adopting a BANSAC algorithm based on nonlinear regression to generate a smooth curve; and performing perspective transformation on the fitted image into an original view angle, and taking a fitted curve as a virtual trajectory for vehicle tracking. According to the invention, by identifying the rut trace and fitting the virtual trajectory, the driving assistance system works normally, automatic tracking of the vehicle in snowy days is realized, guidance is provided for individual travel in snowy days, and travel intention and safety are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and particularly relates to a vehicle tracking method for constructing a virtual trajectory line based on rut marks on an ice and snow road surface. Background Art

[0002] In the technical field of autonomous driving, the function of an advanced driver assistance system to automatically identify lanes and achieve lane keeping is gradually being widely applied. This system collects images through sensors to identify lane lines and detects whether the vehicle deviates from the lane. It can not only prompt the driver in a timely manner but also assist the vehicle to drive smoothly within the lane. However, once in a snowy day scenario, many problems follow. The lane lines on the road surface are often covered by ice and snow and are difficult to identify; at the same time, the performance of visible light cameras is also greatly affected, and overexposure easily occurs, which causes the driving assistance system to be unable to accurately and effectively identify lane lines, thereby reducing the driving efficiency and safety.

[0003] Currently, foreign research on autonomous vehicles in ice and snow environments mostly focuses on intelligent performance evaluation methods, mainly analyzing the intelligent performance of autonomous vehicles under different ice and snow conditions; domestic research focuses more on vehicle trajectory prediction, safety assessment, and evaluation and simulation of driving performance in ice and snow environments. There is relatively little research on vehicle tracking driving methods in ice and snow environments. Although there are some studies on improving the accuracy of lane line detection in complex weather, these studies are neither systematic nor mature, especially in extreme cases where lane lines are completely covered by snow and cannot play a role at all. In short, in snowy environments, problems such as road markings being buried by snow, a significant reduction in color contrast, and limited sensor performance make autonomous driving technology face severe challenges.

[0004] Therefore, the existing technology urgently needs a new technical solution to achieve automatic tracking driving of vehicles in snowy day scenarios. Based on this, the present invention proposes a vehicle tracking method for constructing a virtual trajectory line based on rut marks on an ice and snow road surface. Summary of the Invention

[0005] The purpose of the present invention is to provide a vehicle tracking method for constructing a virtual trajectory line based on rut marks on an ice and snow road surface, aiming to solve the problems raised in the above background art.

[0006] The purpose of the present invention is achieved through the following technical solutions: A vehicle tracking method for constructing a virtual trajectory line based on rut marks on an ice and snow road surface includes the following steps: Step 1: Collect rut images of an ice and snow road surface through an in-vehicle camera by fusing multiple sensors; Step 2: Calibrate the in-vehicle camera to obtain the camera internal parameter matrix and distortion coefficients, and perform distortion correction on the collected images; Step 3: Perform color clustering on the pixel points, and preliminarily remove some irrelevant feature data before grayscale conversion to obtain a preliminary ROI image; Step 4: Further preprocess the image obtained in Step 3, including grayscale conversion, bilateral filtering, histogram equalization, adaptive threshold binarization, morphological erosion operation (erosion first and then dilation), and contour extraction; Step 5: Convert the image processed in Step 4 into a bird's-eye view through inverse perspective transformation; Step 6: Based on the vehicle's left and right tire spacing range, extract the pixel coordinates of the rut edges by scanning line by line; Step 7: Use the BANSAC algorithm based on non-linear regression to fit a cubic polynomial model to generate a smooth curve; Step 8: After perspective-transforming the fitted image back to the original perspective, use the fitted curve as a virtual trajectory line, draw it in the original image, and use it as the basis for vehicle path tracking.

[0007] Further, in Step 2, the camera internal parameter matrix , the distortion coefficient , and the undistortion formula is: ; ; ; where, is the focal length of the camera; is the principal point coordinate of the image; are the second-order, fourth-order, and sixth-order coefficients of radial distortion respectively; is the coefficient of tangential distortion; is the non-distorted coordinate position; is the distorted coordinate position; r is the radial distance from the point to the origin of the image coordinate system.

[0008] Further, the specific steps of Step 3 are as follows: Step 31: Read the processed image and convert it into a pixel point matrix, and use the K-Means clustering algorithm to perform color clustering on the pixel points in the image in the RGB space to obtain clusters with different color value ranges; Step 32: According to the snow pixel characteristics, that is, the color of snow is white and the three-channel values tend to 255; and the rut pixel characteristics, that is, the color of the rut is dark and the three-channel values tend to 0, select the number of clustering centers; Step 33: Analyze the clustering results, and by calculating the similarity between the center color of the cluster and the known snow and rut colors, extract the target clusters that are closest to the snow and rut colors; Step 34: Set the pixel points of non-target clusters to the background color, i.e., white, and retain the pixel points related to snow and ruts. Step 35: Generate a new image based on the filtered pixel points, retaining only the areas related to snow and ruts to form a preliminary ROI image.

[0009] Further, in step 4, the grayscale formula is as follows: ; where is the pixel value after grayscaling; R, G, and B respectively represent the pixel values of the red, green, and blue channels in the color image; The steps of histogram equalization are as follows: Calculate the frequency of each gray value occurrence, which is the histogram of the input image; Calculate the cumulative distribution function, i.e., CDF; According to the CDF, map each gray value of the input image to a new gray value, and the formula is as follows: ; where ; is the transformation function of histogram equalization; is the gray value of the input image; is the rounding function; is the cumulative distribution function of the input image; is the non-zero minimum cumulative distribution function value; N is the total number of pixels in the image; L is the total number of gray levels.

[0010] Further, in step 5, the inverse perspective transformation formula is: ; where is the point in the original image; is the transformed point; is the scaling factor; H is the inverse perspective transformation matrix.

[0011] Further, in step 6, the range of the vehicle's left and right tire spacing is obtained as follows: Based on the front wheel spacing and the rear wheel spacing range of different vehicle models, select the minimum wheelbase and the maximum wheelbase , and convert them into pixel values according to the resolution of the image and the scale after perspective transformation , ; The specific steps to extract the pixel point coordinates of the rut edge are as follows: Step 61: Initialize the row pointer \(i = m\) and the column pointer \(j = 0\), where \(m\) is the total number of rows of the image; scan the \(i\)-th row, that is, scan the binary image obtained row by row starting from the bottom of the image, and then determine whether the pixel values \(P\) of the current row are all 255. If so, there is no rut edge, let \(i = i - 1\) and scan the previous row; otherwise, go to Step 62; Step 62: Let \(j = j + 1\), start scanning pixel points from the first column from left to right, and find the pixel point with pixel value \(P\) of 0 in the \(j\)-th column of the current row; if \(P\) j is not 0, it is a non-target point, let \(j = j + 1\) and continue scanning the next column; if j is 0, record the column number of the current point ; In the current row, starting from the column, check the pixel values in the range from to ; if there are pixel points with pixel value 0 within the range, it is considered that the current point is part of the rut edge, and record the coordinates of the current point ; otherwise, regard the current point as an interference point, let to eliminate the interference; Step 63: Continue to scan row by row upward, repeat Step 62 until all rows are scanned; Step 64: After the scanning is completed, extract the coordinates of all pixel points with gray value 0, which are the coordinate values of the pixel points on the rut edge.

[0012] Furthermore, the specific steps of Step 7 are as follows: Step 71: Initialize the inlier probability of all data points to 0.5, and set the maximum number of iterations, error threshold, and confidence level; Step 72: Perform iterations. In each iteration, calculate the inlier probability of each data point according to the current model; update the inlier probability using a dynamic Bayesian network, considering the fitting error between the data point and the current model; Step 73: Perform weighted sampling according to the updated inlier probability, and select sample points for model estimation; Step 74: In the BANSAC framework, use non-linear regression to fit a cubic polynomial model, and optimize the objective function \(Z\) to minimize the fitting error; The mathematical expression of the cubic polynomial model is: ; where \(a, b, c, d\) are the coefficients of the polynomial; is the dependent variable; \(x\) is the independent variable; The optimized objective function \(Z\) is: ; ​Where N is the number of data points; are the coordinates of the data points; Step 75: Calculate the number of internal points and fitting error of the current model, and then determine whether the stopping criterion is met. If the stopping criterion is met, stop iteration; output the final cubic polynomial model parameters a, b, c, d, and generate a smooth curve.

[0013] Furthermore, in step 8, the specific method of transforming the fitted image perspective to the original perspective is: to obtain the inverse matrix of the inverse perspective transformation matrix H obtained in step 5 , use OpenCV's cv2.warpPerspective function to transform the fitted image to the original perspective.

[0014] A driving assistance system comprises a memory, a processor and a vehicle tracking program stored in the memory and executable on the processor, wherein the vehicle tracking program implements the steps of the vehicle tracking method described above when executed by the processor.

[0015] A vehicle comprises a vehicle body and the driving assistance system as described above, wherein the driving assistance system is arranged in the vehicle body.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention is designed for the scenario where the lane lines on the snowy road are covered with ice and snow. By identifying and extracting the snow ruts, a smooth virtual track line is fitted. The driving assistance system can operate normally, realize the automatic tracking driving of the vehicle in the snowy weather, provide track guidance for personal travel in the snowy weather, improve travel willingness and safety, and make up for the shortcomings of existing research in this scenario.

[0017] 2. The present invention adopts a fusion of multiple sensors. Considering that snow reflections can easily cause overexposure of images and loss of details, an infrared camera is used in combination to make up for the defects of visible light cameras. To address the problem of low contrast of images on snowy days, the image contrast is enhanced through histogram equalization.

[0018] 3. Before the image is grayed, the present invention uses the K-Means clustering algorithm to distinguish snow, ruts and other elements, remove irrelevant pixels, reduce noise and irrelevant information, and improve the efficiency and accuracy of subsequent processing. The new image only retains the areas related to snow and ruts, which reduces the complexity of subsequent processing and improves the accuracy of target detection or trajectory fitting.

[0019] 4. The present invention uses bilateral filtering to calculate weights by combining spatial distance and color difference, which can preserve edge information while smoothing the image, and is very important for subsequent binarization and morphological operations. For ice and snow road surface images containing complex textures and reflected light, bilateral filtering effectively removes noise, preserves rut contours, and enhances image quality, laying a good foundation for subsequent rut detection and analysis.

[0020] 5. The present invention uses line-by-line scanning and combines vehicle wheelbase information to extract the pixel values of rut edge points. This method can effectively filter out interference points that do not fall within the wheelbase range, can be adjusted according to the wheelbase ranges of different vehicle types, and is applicable to various vehicle types and ruts of different widths. Even if the rut edges are discontinuous or there is noise, it can effectively extract the rut edges. It converts the wheelbase range into pixel values, can adapt to images with different resolutions and perspective transformations, has a relatively low computational complexity, is suitable for application in real-time systems, can quickly process image data and extract rut edges; by setting points that do not fall within the wheelbase range to the background color, it can effectively eliminate interference points and improve the continuity and integrity of rut edges.

[0021] 6. The present invention uses the BANSAC algorithm based on non-linear regression to fit a cubic polynomial model to generate a smooth curve. BANSAC can better handle noise and outliers by dynamically updating the inlier probability; the new stopping criterion can reduce unnecessary iterations and improve the algorithm efficiency; BANSAC can work without prior information and can further improve performance when there is prior information. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is the flowchart of the method of the present invention.

[0023] Figure 2 is the framework diagram of the method of the present invention.

[0024] Figure 3 is the schematic diagram of the process of screening and extracting the coordinate of rut edge pixel points by line-by-line scanning of the present invention.

[0025] Figure 4 is the schematic diagram of the process of using the BANSAC algorithm based on non-linear regression of the present invention to fit a cubic polynomial model to generate a smooth curve. DETAILED DESCRIPTION OF THE INVENTION

[0026] In order to have a clearer understanding of the technical features, objectives, and beneficial effects of the present invention, the technical solutions of the present invention are described in detail below, but it should not be construed as a limitation on the applicable scope of the present invention.

[0027] Considering that the lane lines on the ice and snow road surface are covered in snowy weather, and there are often rut marks on them, which are similar to lane lines and have a certain prompting and assisting effect on driving tracking. The present invention replaces the recognition of lane lines with the recognition of rut marks, and proposes a vehicle tracking method for constructing a virtual trajectory line based on rut marks on the ice and snow road surface. By identifying and extracting rut marks on the ice and snow road surface and fitting to construct a virtual trajectory line, the driving assistance system can normally perform other functions based on the provided virtual trajectory line, realizing automatic tracking driving of the vehicle. The flow chart and framework diagram of this method are as shown in Figure 1 and Figure 2 shown below, and it includes the following steps: Step 1: Collect rut images on the ice and snow road surface through an in-vehicle camera that integrates multiple sensors (a visible light camera combined with an infrared camera and an ambient light detection sensor). Considering that the light reflected by the snow may cause overexposure of the image and loss of details, an infrared camera is used in combination to make up for the deficiencies of the visible light camera.

[0028] Step 2: Use OpenCV to calibrate the in-vehicle camera to obtain the camera internal parameter matrix and distortion coefficients. According to the obtained distortion coefficients, use the cv2.undistort function to perform undistortion processing on the collected images.

[0029] Among them, the camera internal parameter matrix is , the distortion coefficients are , and the undistortion formula is: ; ; ; Among them, is the focal length of the camera; is the principal point coordinate of the image; are the second-order, fourth-order, and sixth-order coefficients of radial distortion respectively; is the coefficient of tangential distortion; is the non-distorted coordinate position; is the distorted coordinate position; r is the radial distance from the point to the origin of the image coordinate system.

[0030] Step 3: Since the scene is snowy weather, the region of interest is the dark rut part, and the white snow is adjacent to it and can form a contrast with the dark part for contrast. Except for these two parts, the rest are irrelevant elements. By removing pixel points of other irrelevant colors before grayscale conversion according to its characteristics, the noise and irrelevant information in subsequent processing can be effectively reduced, and the efficiency and accuracy of subsequent processing can be improved. Therefore, color clustering is performed on the pixel points, and some irrelevant element data are initially removed before grayscale conversion to obtain a preliminary ROI image. The specific steps are as follows: Step 31: Read the processed image and convert it into a pixel point matrix. Apply the K-Means clustering algorithm to perform color clustering on the pixel points in the image using the RGB space to obtain clusters with different color value ranges. Step 32: According to the snow pixel characteristics (the color of snow is white, and the three-channel values tend to 255) and the rut pixel characteristics (the color of ruts is darker, and the three-channel values tend to 0), select an appropriate number of clustering centers. Step 33: Analyze the clustering results. By calculating the similarity between the center color of the cluster and the known snow and rut colors, extract the target clusters that are closest to the snow and rut colors. Step 34: Set the pixel points of non-target clusters to the background color (i.e., white, and replace the three-channel values with 255), and retain the pixel points related to snow and ruts. Step 35: Generate a new image based on the filtered pixel points, only retain the areas related to snow and ruts, and form a preliminary ROI image.

[0031] Step 4: Further preprocess the image obtained in Step 3. The specific steps are as follows: Step 41: Perform grayscale processing. Convert the color image into a grayscale image to reduce the amount of data while retaining the basic structural information of the image. The grayscale formula is as follows: ; where, is the pixel value after grayscale processing; R, G, and B respectively represent the pixel values of the red, green, and blue channels in the color image.

[0032] Step 42: Perform bilateral filtering to remove noise while retaining the rut edge information; Step 43: Considering that the overall contrast of snow day images is usually low, perform histogram equalization on the image to enhance the image contrast. The steps of histogram equalization are as follows: Calculate the frequency of each grayscale value occurrence, the histogram of the input image; Calculate the cumulative distribution function (CDF); According to the CDF, map each grayscale value of the input image to a new grayscale value. The formula is as follows: ; where, ; is the transformation function of histogram equalization; is the grayscale value of the input image; is the rounding function; is the cumulative distribution function of the input image; is the non-zero minimum cumulative distribution function value; N is the total number of pixels in the image; L is the total number of gray levels, which is 256 for 8-bit images.

[0033] Step 44: Use adaptive thresholding cv2.adaptiveThreshold to convert the grayscale image into a black-and-white image; Step 45: The morphological erosion operation erodes first and then dilates, which can remove small objects in the image, disconnect the connection between objects, and eliminate the hair edges, effectively removing the noise in the rut depressions; Step 46: Extract the contours through cv2.findContours.

[0034] Step 5: Convert the image obtained in Step 4 into a bird's-eye view through inverse perspective transformation. Specifically, select four points in the image as the source points, define the corresponding points in the target image, and use the getPerspectiveTransform function of OpenCV to calculate the inverse perspective transformation matrix to convert the image into a bird's-eye view.

[0035] The inverse perspective transformation formula is: ; where, is the point in the original image; is the transformed point; is the scaling factor to normalize the pixel coordinates; H is the inverse perspective transformation matrix. At least four pairs of corresponding points need to be found in the original image and the target image, and these points cannot have three or more collinear points. The H matrix can be calculated by the cv2.getPerspectiveTransform function.

[0036] Step 6: Select the minimum wheelbase and the maximum wheelbase according to the front-wheel spacing and rear-wheel spacing ranges of different vehicle models, and convert them into pixel values , (rounded) based on the resolution of the image and the scale after perspective transformation to obtain the range of the left and right tire spacings of the vehicle . Extract the coordinate values of the rut edge pixel points according to the following steps (see Figure 3 ): Step 61: Initialize the row pointer i = m and the column pointer j = 0 (m is the total number of rows in the image), and scan the i-th row, that is, scan the obtained binary image row by row starting from the bottom of the image. Then determine whether the pixel value P of this row is all 255. If so, it means that this row is all white and there is no rut edge, so let i = i - 1 and scan the previous row. Otherwise, proceed to Step 62.

[0037] Step 62: Starting from this row, let j = j + 1 and scan the pixel points starting from the first column from left to right to find the pixel point where P j (the pixel value of the j-th column in this row) is 0. If P jIf it is not 0, it is a non-target point, and continue to scan the next column (j = j + 1); if it is 0, record the column number of this point , and then make the next judgment.

[0038] In the current row, starting from the column, check the pixel values in the range from to . If there is a pixel point with a pixel value of 0 in this range, it is considered that the current point is part of the rut edge, and record the coordinates of this point ; otherwise, regard the current point as an interference point, and let to eliminate the interference.

[0039] Step 63: Continue to scan row by row upwards, repeat Step 62 until all rows are scanned (judge whether i is 1. If i = 1, it means that the highest row has been scanned and the scanning ends. Otherwise, let i = i - 1 and continue to scan the previous row).

[0040] Step 64: After the scanning is completed, extract the coordinates of all pixel points with a gray value of 0. These coordinates are the pixel point coordinate values of the rut edge.

[0041] Step 7: Use BANSAC to replace the traditional RANSAC, and adopt the BANSAC algorithm based on non-linear regression to fit a cubic polynomial model to generate a smooth curve.

[0042] BANSAC is an improved RANSAC algorithm that updates the inlier probability of data points through a dynamic Bayesian network, thereby realizing adaptive sampling. In each iteration, BANSAC updates the inlier probability of data points according to the current model instead of randomly selecting sample points; based on the updated inlier probability, BANSAC proposes a new stopping criterion (during the iteration process, when the change in the inlier probability of most data points is less than the confidence threshold, or the inlier probability changes little in several consecutive iterations, or the model error is lower than the error threshold, or when the maximum number of iterations is reached, the algorithm stops iterating), which can terminate the iteration earlier and improve the efficiency; BANSAC can work without prior data point scoring, and can also use prior information to further improve the performance.

[0043] The specific steps are as follows (see Figure 4 ): Step 71: Initialize the inlier probability of all data points = 0.5; set parameters such as the maximum number of iterations, error threshold, and confidence level.

[0044] Step 72: Perform iteration. In each iteration, calculate the inlier probability of each data point according to the current model; update the inlier probability using a dynamic Bayesian network, considering the fitting error between the data point and the current model.

[0045] Step 73: Perform weighted sampling according to the updated inlier probability, and select sample points for model estimation.

[0046] Step 74: Under the BANSAC framework, use a non-linear regression method to fit a cubic polynomial model, and optimize the objective function Z to minimize the fitting error.

[0047] The mathematical expression of the cubic polynomial model is: ; where a, b, c, d are the coefficients of the polynomial; is the dependent variable; x is the independent variable.

[0048] The optimized objective function Z is: ; where N is the number of data points; are the coordinates of the data points.

[0049] Step 75: Calculate the number of inliers and the fitting error of the current model. Then determine whether the stopping criterion is met. If it is met, stop the iteration, output the final cubic polynomial model parameters a, b, c, d, and generate a smooth curve. Otherwise, continue the iteration until the stopping criterion is met and the loop ends, generating the final smooth curve.

[0050] Step 8: Invert the inverse perspective transformation matrix H obtained in Step 5 , and use the cv2.warpPerspective function in OpenCV to perform perspective transformation on the fitted image to the original perspective, draw the fitted curve as a virtual trajectory line in the original image, and the vehicle tracking is based on this.

[0051] An embodiment of the present invention provides a driving assistance system. The driving assistance system includes a memory, a processor, and a vehicle tracking program stored on the memory and executable on the processor. When the vehicle tracking program is executed by the processor, it implements the steps of the vehicle tracking method as described above, so that functions such as automatic driving cruise, lane keeping, and departure warning are still effective when the snow-covered lane lines are covered.

[0052] Another embodiment of the present invention provides a vehicle, including a vehicle body and the driving assistance system as described above. The driving assistance system is arranged in the vehicle body, realizing that there is a predetermined trajectory to follow for private snow-day travel, improving the willingness and safety of personal snow-day travel.

[0053] The following is a detailed description of the specific implementation of the present invention in combination with specific embodiments.

[0054] Example 1: The dataset used in this example is the PhotoTourism dataset. Multiple experiments are conducted considering the curve fitting problem, and the inlier rate varies in the range of 15% - 50%. Each experiment contains 300 data points within the range of [-1, 1]. Among them, the inliers are disturbed by Gaussian noise with a mean of 0 and a variance of 0.02, and the outliers are simulated by a uniform distribution with a maximum absolute value of 1.0.

[0055] Compare BANSAC with RANSAC and BaySAC. As the estimated parameters, set an error threshold of 0.02, a maximum number of iterations of 3000, and an estimation confidence level of 0.99. In BANSAC, the initial probability of all data points is set to 0.5, and the stopping criterion threshold is set to 0.01. Measure the root mean square error (RMSE) of the geometric distance from each point in the estimated model to the ideal model and the number of iterations, and show the average values obtained after 1000 randomly generated trials. The results are shown in Table 1 and Table 2.

[0056] Table 1 Comparison of the number of iterations

[0057] Table 2 Comparison of the root mean square error

[0058] It can be seen from the data in Table 1 that as the inlier rate gradually increases, the number of iterations of the three methods shows a downward trend, and it is obvious that the number of iterations required by BANSAC at each inlier rate is significantly less than that of RANSAC and BaySAC. It can be seen from the data in Table 2 that overall, even when the inlier rate is low, the accuracy of BANSAC is comparable to or higher than that of RANSAC and BaySAC.

[0059] Generally speaking, through the analysis of the experimental data, it can be known that compared with RANSAC and BaySAC, BANSAC not only has higher accuracy, but also significantly reduces the number of iterations required while achieving similar or better accuracy, greatly improving the efficiency, fully demonstrating its advantages in dealing with the curve fitting problem.

[0060] The above is only the preferred implementation manner of the present invention. It should be noted that for those skilled in the art, without departing from the concept of the present invention, several deformations and improvements can still be made, which should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent.

Claims

1. A vehicle tracking method for constructing a virtual trajectory line based on rut traces on ice and snow roads, characterized in that, It includes the following steps: Step 1: Collect rut images on snow and ice roads by fusing multiple sensors through an in-vehicle camera; Step 2: Calibrate the in-vehicle camera to obtain the camera internal parameter matrix and distortion coefficients, and perform undistortion processing on the collected images; Step 3: Perform color clustering on pixel points, preliminarily remove some irrelevant element data before grayscale conversion, and obtain a preliminary ROI image; Step 4: Further preprocess the image obtained in Step 3, including grayscale conversion, bilateral filtering, histogram equalization, adaptive threshold binaryzation, morphological erosion operation (erosion first and then dilation), and contour extraction; Step 5: Convert the image processed in Step 4 into a bird's-eye view through inverse perspective transformation; Step 6: Based on the range of the vehicle's left and right tire spacings, extract the pixel coordinates of the rut edges by scanning line by line; Step 7: Use the BANSAC algorithm based on non-linear regression to fit a cubic polynomial model to generate a smooth curve; Step 8: After perspective-transforming the fitted image back to the original perspective, use the fitted curve as a virtual trajectory line, draw it in the original image, and use it as the basis for vehicle tracking.

2. The vehicle tracking method for constructing a virtual track line based on the rut marks on an ice and snow road surface according to claim 1, wherein In the said step 2, the camera intrinsic matrix , and the distortion coefficient . The undistortion formula is as follows: ； ； ； Among them, is the focal length of the camera; are the coordinates of the principal point of the image; are the second-order, fourth-order, and sixth-order coefficients of radial distortion, respectively; is the coefficient of tangential distortion; is the non-distorted coordinate position; is the distorted coordinate position; r is the radial distance of point to the origin of the image coordinate system.

3. The vehicle tracking method for constructing a virtual track line based on ruts on an ice and snow road surface according to claim 2, characterized in that, The specific steps of Step 3 are as follows: Step 31: Read the processed image and convert it into a pixel point matrix, and use the K-Means clustering algorithm to perform color clustering on the pixel points in the image using the RGB space to obtain clusters with different color value ranges; Step 32: According to the snow pixel characteristics, that is, the color of snow is white and the three-channel values tend to 255; and the rut pixel characteristics, that is, the color of the rut is darker and the three-channel values tend to 0, select the number of clustering centers; Step 33: Analyze the clustering results, and by calculating the similarity between the center color of the cluster and the known snow and rut colors, extract the target clusters that are closest to the snow and rut colors; Step 34: Set the pixel points of non-target clusters to the background color, that is, white, and retain the pixel points related to snow and ruts; Step 35: Generate a new image based on the filtered pixel points, only retain the areas related to snow and ruts, and form a preliminary ROI image.

4. The vehicle tracking method for constructing a virtual track line based on rut traces on an ice and snow road surface according to claim 1, characterized in that, In Step 4, the grayscale conversion formula is as follows: ； Among them, is the pixel value after grayscale conversion; R, G, and B respectively represent the pixel values of the red, green, and blue channels in the color image; The steps of histogram equalization are as follows: Calculate the frequency of each gray value occurrence, and input the histogram of the image; Calculate the cumulative distribution function, that is, CDF; According to the CDF, map each gray value of the input image to a new gray value, and the formula is as follows: ； Among them, ; is the transformation function of histogram equalization; is the gray value of the input image; is the rounding function; is the cumulative distribution function of the input image; is the minimum non-zero cumulative distribution function value; N is the total number of pixels in the image; L is the total number of gray levels.

5. The vehicle tracking method for constructing a virtual track line based on ruts on an ice and snow road surface according to claim 1, wherein In Step 5, the inverse perspective transformation formula is: ； Among them, is a point in the original image; is the transformed point; is the scaling factor; H is the inverse perspective transformation matrix.

6. The vehicle tracking method for constructing a virtual trajectory line based on ruts on an ice and snow road surface according to claim 1, characterized in that, In step 6, the range of the distance between the left and right tires of the vehicle is obtained as follows: Based on the front wheel distance and the rear wheel distance range of different vehicle models, select the minimum wheelbase and the maximum wheelbase , and convert them into pixel values according to the resolution of the image and the scale after perspective transformation , ; The specific steps of extracting the pixel coordinates of the rut edges are as follows: Step 61: Initialize the row pointer i = m, the column pointer j = 0, where m is the total number of rows of the image; Scan the i-th row, that is, scan the binary image obtained line by line from the bottom of the image, and then determine whether the pixel value P of the current row is all 255. If so, there is no rut edge, let i = i - 1 and scan the previous row; Otherwise, proceed to Step 62; Step 62: Let j = j + 1, scan the pixel points starting from the first column from left to right, and find the pixel point with pixel value P j equal to 0 in the j-th column of the current row; if P j is not equal to 0, it is a non-target point, let j = j + 1 and continue to scan the next column; if it is 0, record the column number of the current point ; In the current row, starting from column , check the pixel values in the range from to ; If there is a pixel point with a pixel value of 0 within the range, it is considered that the current point is part of the rut edge, and the coordinates of the current point are recorded ; Otherwise, consider the current point as an interference point and set to eliminate interference; Step 63: Continue to scan upward line by line, repeat Step 62 until all rows are scanned; Step 64: After the scanning is completed, extract the coordinates of all pixel points with a gray value of 0, which are the pixel coordinate values of the rut edges.

7. The vehicle tracking method for constructing a virtual track line based on the rut marks on an ice and snow road surface according to claim 1, wherein, The specific steps of Step 7 are as follows: Step 71: Initialize the inlier probabilities of all data points to 0.5, and set the maximum number of iterations, the error threshold, and the confidence level; Step 72: Perform iterations. In each iteration, calculate the inlier probability of each data point according to the current model; Update the inlier probability using a dynamic Bayesian network, considering the fitting error between the data points and the current model; Step 73: Perform weighted sampling based on the updated inlier probability and select sample points for model estimation; Step 74: Under the BANSAC framework, use nonlinear regression to fit a cubic polynomial model, and optimize the objective function Z to minimize the fitting error; The mathematical expression of the cubic polynomial model is: ； where a, b, c, and d are the coefficients of the polynomial; is the dependent variable; x is the independent variable; The optimized objective function Z is: ； where N is the number of data points; are the coordinates of the data points; Step 75: Calculate the number of inliers and the fitting error of the current model, and then determine whether the stopping criterion is satisfied. If the stopping criterion is satisfied, stop the iteration; output the parameters a, b, c, d of the final cubic polynomial model, and generate a smooth curve.

8. The vehicle tracking method for constructing a virtual trajectory line based on rut traces on an ice and snow road surface according to claim 5, characterized in that, In the above step 8, the specific method for perspective-transforming the fitted image back to the original perspective is as follows: calculate the inverse matrix of the inverse perspective transformation matrix H obtained in step 5 , and use the cv2.warpPerspective function in OpenCV to transform the fitted image back to the original perspective.

9. A driving assistance system, characterized in that, The driving assistance system includes a memory, a processor, and a vehicle tracking program stored on the memory and executable on the processor. When the vehicle tracking program is executed by the processor, it implements the steps of the vehicle tracking method according to any one of claims 1-8.

10. A vehicle, characterized in that, It includes a vehicle body and the driving assistance system according to claim 9, and the driving assistance system is arranged in the vehicle body.

Images