Vehicle tracking method based on constructing virtual trajectory lines based on rutting traces on icy and snowy roads

Abstract

The present invention is applicable to the field of autonomous driving technology and provides a vehicle tracking method for constructing a virtual track line based on rutting traces on icy and snowy roads. The method comprises the following steps: collecting rutting images on icy and snowy roads using a vehicle-mounted camera fusing multiple sensors; performing dedistortion processing; color clustering the pixels to remove irrelevant element data to obtain a preliminary ROI image; further preprocessing; converting the image into a bird's-eye view using an inverse perspective transformation; extracting the pixel coordinates of the rutting edge through line-by-line scanning based on the distance between the left and right tires of the vehicle; fitting a cubic polynomial model using the BANSAC algorithm based on nonlinear regression to generate a smooth curve; perspective transforming the fitted image back to the original perspective, and using the fitted curve as a virtual track line for vehicle tracking. By identifying rutting traces and fitting a virtual track line, the present invention enables the normal operation of the driving assistance system, achieving automatic vehicle tracking in snowy conditions, providing guidance for personal travel in snowy conditions, and enhancing travel willingness and safety.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving technology, and in particular relates to a vehicle tracking method for constructing a virtual trajectory line based on rutting traces on icy and snowy roads. Background Art

[0002] In the field of autonomous driving technology, advanced driver assistance systems (ADAS) that automatically identify and maintain lanes are gaining widespread adoption. These systems use sensors to capture images, identify lane markings, and detect if the vehicle is drifting out of its lane. This not only provides timely driver alerts but also helps the vehicle maintain stable lane control. However, snowy conditions present numerous challenges. Lane markings are often covered in ice and snow, making them difficult to discern. Furthermore, the performance of visible light cameras is significantly impacted, making them prone to overexposure. This prevents the ADA system from accurately and effectively identifying lane markings, reducing driving efficiency and safety.

[0003] Currently, international research on autonomous vehicles in snowy and icy environments has largely focused on intelligent performance evaluation methods, primarily analyzing the intelligent performance of autonomous vehicles under various snowy and icy conditions. Domestic research, however, has focused on vehicle trajectory prediction, safety assessment, and driving performance evaluation and simulation in snowy and icy environments. There is relatively little research on vehicle tracking methods in snowy and icy environments. While some research exists on improving lane detection accuracy in complex weather conditions, this research is neither systematic nor mature, and is particularly ineffective in extreme cases where lanes are completely covered by snow. In short, snowy environments present significant challenges for autonomous driving technology, as road markings are buried under snow, color contrast is significantly reduced, and sensor performance is limited.

[0004] Therefore, the existing technology urgently needs a new technical solution to achieve automatic tracking driving of vehicles in snowy scenes. Based on this, the present invention proposes a vehicle tracking method that constructs a virtual track line based on rutting traces on icy and snowy roads. Summary of the Invention

[0005] The purpose of the present invention is to provide a vehicle tracking method for constructing a virtual trajectory line based on rutting traces on icy and snowy roads, aiming to solve the problems raised in the above background technology.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] The vehicle tracking method for constructing a virtual track line based on rutting traces on an icy and snowy road surface includes the following steps:

[0008] Step 1: Collect rut images on icy and snowy roads using a vehicle-mounted camera and multiple sensors.

[0009] Step 2: Calibrate the vehicle-mounted camera, obtain the camera's intrinsic parameter matrix and distortion coefficient, and perform dedistortion processing on the captured image;

[0010] Step 3: Perform color clustering on the pixels and remove some irrelevant element data before grayscale conversion to obtain a preliminary ROI image;

[0011] Step 4: The image obtained in step 3 is further preprocessed, including grayscale conversion, bilateral filtering, histogram equalization, adaptive threshold binarization, morphological erosion operation (erosion followed by dilation), and contour extraction.

[0012] Step 5: Convert the image processed in step 4 into a bird's-eye view through inverse perspective transformation;

[0013] Step 6: Based on the distance between the left and right tires of the vehicle, extract the pixel coordinates of the rut edge by scanning line by line;

[0014] Step 7: Use the BANSAC algorithm based on nonlinear regression to fit the cubic polynomial model to generate a smooth curve;

[0015] Step 8: After transforming the fitted image into the original perspective, use the fitted curve as a virtual trajectory line, draw it in the original image and use it as the basis for vehicle tracking.

[0016] Furthermore, in step 2, the camera intrinsic parameter matrix , distortion coefficient , the distortion removal formula is:

[0017] ;

[0018] ;

[0019] ;

[0020] in, is the focal length of the camera; are the principal point coordinates of the image; are the second-order, fourth-order, and sixth-order coefficients of radial distortion, respectively; is the coefficient of tangential distortion; is the undistorted coordinate position; is the coordinate position after distortion; r is the point The radial distance to the origin of the image coordinate system.

[0021] Furthermore, the specific steps of step 3 are as follows:

[0022] Step 31: Read the processed image and convert it into a pixel matrix. Use the K-Means clustering algorithm to perform color clustering on the pixels in the image using the RGB space to obtain clusters with different color value ranges.

[0023] Step 32: Select the number of cluster centers based on the snow pixel characteristics, i.e., the snow color is white and the three channel values ​​are all close to 255; and the wheel track pixel characteristics, i.e., the wheel track color is dark and the three channel values ​​are all close to 0;

[0024] Step 33: Analyze the clustering results and extract the target cluster closest to the snow and rut colors by calculating the similarity between the cluster center color and the known snow and rut colors;

[0025] Step 34: Set the pixels of non-target clusters to the background color, i.e. white, and retain the pixels related to snow and ruts;

[0026] Step 35: Generate a new image based on the filtered pixels, retaining only the areas related to snow and ruts to form a preliminary ROI image.

[0027] Furthermore, in step 4, the grayscale formula is as follows:

[0028] ;

[0029] in, is the grayscale pixel value; R, G, and B represent the pixel values ​​of the red, green, and blue channels in the color image respectively;

[0030] The steps of histogram equalization are: calculate the frequency of occurrence of each grayscale value, the histogram of the input image; calculate the cumulative distribution function, i.e. CDF; according to the CDF, map each grayscale value of the input image to a new grayscale value. The formula is as follows:

[0031] ;

[0032] in, ; is the transformation function of histogram equalization; is the grayscale value of the input image; is the rounding function; is the cumulative distribution function of the input image; is the minimum non-zero cumulative distribution function value; N is the total number of pixels in the image; L is the total number of gray levels.

[0033] Furthermore, in step 5, the inverse perspective transformation formula is:

[0034] ;

[0035] in, is a point in the original image; is the transformed point; is the scaling factor; H is Inverse perspective transformation matrix.

[0036] Furthermore, in step 6, the distance between the left and right tires of the vehicle is The method of obtaining is: according to the front wheel spacing and rear wheel spacing range of different models, select the minimum wheel spacing And the maximum wheelbase , and converted into pixel values ​​according to the image resolution and the scale after perspective transformation , ;

[0037] The specific steps for extracting the pixel coordinates of the rut edge are as follows:

[0038] Step 61: Initialize row pointer i = m and column pointer j = 0, where m is the total number of rows in the image; scan row i, i.e., scan the binary image row by row starting from the bottom of the image, and then determine whether all pixel values ​​P in the current row are 255. If so, there is no rutting edge, and set i = i-1 to scan the previous row; otherwise, proceed to step 62;

[0039] Step 62: Let j = j + 1, scan the pixels starting from the first column from left to right, and find the pixel value P in the jth column of the current row j The pixel point is 0; if P j If it is not 0, it is not the target point, let j=j+1 and continue to scan the next column; if 0, then record the current point column number ;

[0040] In the current row, from Starting from the column, check arrive Pixel values ​​within the range; if there is a pixel point with a pixel value of 0 within the range, the current point is considered to be part of the rut edge and the coordinates of the current point are recorded Otherwise, the current point is regarded as an interference point, and , to eliminate interference;

[0041] Step 63: Continue scanning upwards row by row, repeating step 62 until all rows are scanned;

[0042] Step 64: After the scanning is completed, the coordinates of all pixels with a grayscale value of 0 are extracted, which are the pixel coordinate values ​​of the rut edge.

[0043] Furthermore, the specific steps of step 7 are as follows:

[0044] Step 71: Initialize the inlier probabilities of all data points is 0.5, setting the maximum number of iterations, error threshold and confidence level;

[0045] Step 72: Iterate, and in each iteration, calculate the inlier probability of each data point based on the current model; use the dynamic Bayesian network to update the inlier probability, taking into account the fitting error of the data point with the current model;

[0046] Step 73: Perform weighted sampling based on the updated inlier probability and select sample points for model estimation;

[0047] Step 74: Using nonlinear regression to fit the cubic polynomial model in the BANSAC framework, the objective function Z is optimized to minimize the fitting error.

[0048] The mathematical expression of the cubic polynomial model is:

[0049] ;

[0050] Among them, a, b, c, d are the coefficients of the polynomial; is the dependent variable; x is the independent variable;

[0051] The optimization objective function Z is:

[0052] ;

[0053] Where N is the number of data points; are the coordinates of the data points;

[0054] Step 75: Calculate the number of internal points and fitting error of the current model, and then determine whether the stopping criteria are met. If the stopping criteria are met, stop the iteration; output the final cubic polynomial model parameters a, b, c, d, and generate a smooth curve.

[0055] Furthermore, in step 8, the specific method of transforming the fitted image perspective to the original perspective is: to obtain the inverse matrix of the inverse perspective transformation matrix H obtained in step 5 , use OpenCV's cv2.warpPerspective function to transform the fitted image to the original perspective.

[0056] A driving assistance system includes a memory, a processor, and a vehicle tracking program stored in the memory and executable on the processor. When the vehicle tracking program is executed by the processor, the steps of the vehicle tracking method described above are implemented.

[0057] A vehicle comprises a vehicle body and the driving assistance system as described above, wherein the driving assistance system is arranged in the vehicle body.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. This invention is specifically designed for scenarios where lane lines on snowy roads are covered in ice and snow. By identifying and extracting snow rutting, it creates a smooth virtual trajectory. This allows the driver assistance system to function properly, enabling the vehicle to automatically follow the track in snowy conditions. This provides trajectory guidance for individuals traveling in snowy conditions, enhancing travel willingness and safety, and addressing the shortcomings of existing research in this context.

[0060] 2. This invention uses a fusion of multiple sensors. Considering that snow reflections can easily lead to overexposure and loss of detail in images, an infrared camera is used to compensate for the shortcomings of visible light cameras. To address the problem of low contrast in snowy images, histogram equalization is used to enhance image contrast.

[0061] 3. Before grayscaling the image, this method uses a K-Means clustering algorithm to distinguish snow, ruts, and other elements, removing irrelevant pixels, reducing noise and irrelevant information, and improving the efficiency and accuracy of subsequent processing. The new image retains only areas related to snow and ruts, reducing the complexity of subsequent processing and improving the accuracy of target detection or trajectory fitting.

[0062] 4. This invention uses bilateral filtering to calculate weights by combining spatial distance and color differences. This method smooths the image while preserving edge information, which is crucial for subsequent binarization and morphological operations. For images of icy and snowy roads containing complex textures and reflected light, bilateral filtering effectively removes noise, preserves rutting outlines, and enhances image quality, laying a solid foundation for subsequent rutting detection and analysis.

[0063] 5. The present invention utilizes line-by-line scanning, combined with vehicle wheelbase information, to extract pixel values ​​of rut edge points. This method can effectively filter out interfering points that do not conform to the wheelbase range, can be adjusted based on the wheelbase range of different vehicle models, and is applicable to a variety of vehicle types and ruts of varying widths. Rut edges can be effectively extracted even if they are discontinuous or noisy. The method converts the wheelbase range into pixel values, adapting to images of varying resolutions and perspective transformations. Its computational complexity is relatively low, making it suitable for use in real-time systems. It can rapidly process image data and extract rut edges. By setting points that do not conform to the wheelbase range to the background color, interfering points can be effectively eliminated, improving the continuity and integrity of the rut edges.

[0064] 6. This paper uses the BANSAC algorithm, based on nonlinear regression, to fit a cubic polynomial model and generate a smooth curve. BANSAC dynamically updates inlier probabilities, enabling better handling of noise and outliers. A new stopping criterion reduces unnecessary iterations, improving algorithm efficiency. BANSAC operates without prior information and can further improve performance when prior information is available. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 Flow chart of the method of the present invention.

[0066] Figure 2 This is a framework diagram of the method of the present invention.

[0067] Figure 3 This is a schematic diagram of the process of scanning, screening and extracting the coordinates of pixel points at the edge of ruts line by line in the present invention.

[0068] Figure 4 The figure is a flow chart of the process of fitting a cubic polynomial model to generate a smooth curve using the BANSAC algorithm based on nonlinear regression in the present invention. DETAILED DESCRIPTION

[0069] In order to have a clearer understanding of the technical features, objectives and beneficial effects of the present invention, the technical solution of the present invention is now described in detail below, but it should not be understood as limiting the scope of implementation of the present invention.

[0070] Considering that the lane lines on icy and snowy roads in snowy scenes are covered, there are often ruts left on them. Ruts, like lane lines, have a certain prompt and auxiliary effect on driving tracking. The present invention replaces the identification of lane lines with the identification of ruts, and proposes a vehicle tracking method based on the construction of virtual track lines based on ruts on icy and snowy roads. By identifying and extracting ruts on icy and snowy roads, a virtual track line is constructed by fitting, so that the driving assistance system can perform other functions normally based on the provided virtual track line, realizing automatic tracking driving of the vehicle. The flow chart and framework diagram of this method are shown in the figure below. Figure 1 and Figure 2 As shown, the following steps are included:

[0071] Step 1: Use the vehicle's camera to fuse multiple sensors (visible light camera, infrared camera, and ambient light detection sensor) to capture images of ruts on icy and snowy roads. Considering that light reflected from the snow can cause overexposure and loss of detail, the infrared camera is used to compensate for the limitations of the visible light camera.

[0072] Step 2: Use OpenCV to calibrate the on-board camera, obtain the camera intrinsic parameter matrix and distortion coefficient, and use the cv2.undistort function to dedistort the captured image based on the distortion coefficient obtained from the calibration.

[0073] Among them, the camera intrinsic parameter matrix , distortion coefficient , the distortion removal formula is:

[0074] ;

[0075] ;

[0076] ;

[0077] in, is the focal length of the camera; are the principal point coordinates of the image; are the second-order, fourth-order, and sixth-order coefficients of radial distortion, respectively; is the coefficient of tangential distortion; is the undistorted coordinate position; is the coordinate position after distortion; r is the point The radial distance to the origin of the image coordinate system.

[0078] Step 3: Since the scene is snowy, the region of interest is the dark ruts. The white snow is adjacent to them and contrasts with the darker parts. Apart from these two parts, the rest are irrelevant elements. By taking advantage of these characteristics, removing pixels of other irrelevant colors before grayscaling can effectively reduce noise and irrelevant information in subsequent processing, improving the efficiency and accuracy of subsequent processing. Therefore, color clustering is performed on the pixels, and some irrelevant element data is initially removed before grayscaling to obtain a preliminary ROI image. The specific steps are as follows:

[0079] Step 31: Read the processed image and convert it into a pixel matrix. Use the K-Means clustering algorithm to perform color clustering on the pixels in the image using the RGB space to obtain clusters with different color value ranges.

[0080] Step 32: Select the appropriate number of cluster centers based on the characteristics of snow pixels (snow is white, and the values ​​of the three channels are all close to 255) and rut pixels (ruts are dark, and the values ​​of the three channels are all close to 0);

[0081] Step 33: Analyze the clustering results and extract the target cluster closest to the snow and rut colors by calculating the similarity between the cluster center color and the known snow and rut colors;

[0082] Step 34: Set the pixels of non-target clusters to the background color (i.e., white, and replace the three-channel values ​​with 255), and retain the pixels related to snow and ruts;

[0083] Step 35: Generate a new image based on the filtered pixels, retaining only the areas related to snow and ruts to form a preliminary ROI image.

[0084] Step 4: Further preprocess the image obtained in step 3. The specific steps are as follows:

[0085] Step 41: Perform grayscale processing. Convert the color image to a grayscale image to reduce the amount of data while retaining the basic structural information of the image. The grayscale formula is as follows:

[0086] ;

[0087] in, It is the grayscale pixel value; R, G, and B represent the pixel values ​​of the red, green, and blue channels in the color image respectively.

[0088] Step 42: bilateral filtering is performed to remove noise while preserving rut edge information;

[0089] Step 43: Considering that snowy images generally have low overall contrast, histogram equalization is performed on the image to enhance the contrast. The steps of histogram equalization are as follows: calculating the frequency of occurrence of each grayscale value, the histogram of the input image; calculating the cumulative distribution function (CDF); and mapping each grayscale value of the input image to a new grayscale value based on the CDF. The formula is as follows:

[0090] ;

[0091] in, ; is the transformation function of histogram equalization; is the grayscale value of the input image; is the rounding function; is the cumulative distribution function of the input image; is the minimum non-zero CDF value; N is the total number of pixels in the image; L is the total number of gray levels, which is 256 for 8-bit images.

[0092] Step 44: Use adaptive threshold binarization cv2.adaptiveThreshold to convert the grayscale image into a black and white image;

[0093] Step 45: The morphological erosion operation first erodes and then dilates, which can remove small objects in the image, disconnect objects, and eliminate burrs, effectively removing noise in ruts.

[0094] Step 46: Extract the contours through cv2.findContours.

[0095] Step 5: Convert the image processed in step 4 to a bird's-eye view using an inverse perspective transformation. Specifically, select four points in the image as source points, define corresponding points in the target image, and use OpenCV's getPerspectiveTransform function to calculate the inverse perspective transformation matrix to convert the image to a bird's-eye view.

[0096] The inverse perspective transformation formula is:

[0097] ;

[0098] in, is a point in the original image; is the transformed point; is the scaling factor that normalizes the pixel coordinates; H is The inverse perspective transformation matrix requires finding at least four pairs of corresponding points in the original image and the target image. Three or more of these points cannot be collinear. The H matrix can be calculated by the cv2.getPerspectiveTransform function.

[0099] Step 6: Select the minimum wheelbase based on the front and rear wheel spacing ranges of different models And the maximum wheelbase , and converted into pixel values ​​according to the image resolution and the scale after perspective transformation , (rounded up) to get the range of distance between the left and right tires of the vehicle Follow the steps below to extract the pixel coordinates of the rut edge (see Figure 3 ):

[0100] Step 61: Initialize row pointer i = m and column pointer j = 0 (m is the total number of image rows). Scan row i, starting from the bottom of the image and scanning the resulting binary image row by row. Then determine whether all pixel values ​​P in that row are 255. If so, the row is entirely white and has no rutting edges. Therefore, set i = i-1 and scan the previous row. Otherwise, proceed to step 62.

[0101] Step 62: Starting from the row, let j = j + 1, scan the pixels from the first column from left to right, and find P j (pixel value of the row and column j) is 0. If P j If it is not 0, it is not the target point, and continue to scan the next column (j=j+1); if 0, then record the point column number , and then proceed to the next step of judgment.

[0102] In the current row, from Starting from the column, check arrive If there is a pixel point with a pixel value of 0 in this range, the current point is considered to be part of the rut edge and the coordinates of the point are recorded. Otherwise, the current point is regarded as an interference point, and , to eliminate interference.

[0103] Step 63: Continue scanning upward row by row, repeating step 62 until all rows are scanned (determine whether i is 1. If i=1, it indicates that the scan has reached the highest row and the scan is completed. Otherwise, set i=i-1 to continue scanning the previous row).

[0104] Step 64: After the scanning is completed, the coordinates of all pixels with a grayscale value of 0 are extracted. These coordinates are the pixel coordinate values ​​of the rut edge.

[0105] Step 7: Use BANSAC instead of traditional RANSAC, and use the BANSAC algorithm based on nonlinear regression to fit the cubic polynomial model to generate a smooth curve.

[0106] BANSAC is an improved RANSAC algorithm that updates the inlier probabilities of data points through a dynamic Bayesian network, thereby achieving adaptive sampling. In each iteration, BANSAC updates the inlier probabilities of data points based on the current model, rather than randomly selecting sample points. Based on the updated inlier probabilities, BANSAC proposes a new stopping criterion (the algorithm stops when the change in the inlier probabilities of most data points is less than the confidence threshold, or when the inlier probabilities do not change much over several consecutive iterations, or when the model error is below the error threshold, or when the maximum number of iterations is reached). This allows for earlier termination of iterations and improves efficiency. BANSAC can operate without prior data point scores and can also leverage prior information to further improve performance.

[0107] The specific steps are as follows (see Figure 4 ):

[0108] Step 71: Initialize the inlier probabilities of all data points =0.5; set parameters such as maximum number of iterations, error threshold and confidence level.

[0109] Step 72: Iterate. In each iteration, the inlier probability of each data point is calculated based on the current model; the inlier probability is updated using the dynamic Bayesian network, taking into account the fitting error of the data point with the current model.

[0110] Step 73: Perform weighted sampling based on the updated inlier probability and select sample points for model estimation.

[0111] Step 74: Under the BANSAC framework, use the nonlinear regression method to fit the cubic polynomial model, and optimize the objective function Z to minimize the fitting error.

[0112] The mathematical expression of the cubic polynomial model is:

[0113] ;

[0114] Among them, a, b, c, d are the coefficients of the polynomial; is the dependent variable; x is the independent variable.

[0115] The optimization objective function Z is:

[0116] ;

[0117] Where N is the number of data points; are the coordinates of the data points.

[0118] Step 75: Calculate the number of inliers and the fitting error of the current model. Then determine whether the stopping criterion is met. If so, the iteration stops, outputs the final cubic polynomial model parameters a, b, c, d, and generates a smooth curve. Otherwise, the iteration continues until the stopping criterion is met, and the final smooth curve is generated.

[0119] Step 8: Calculate the inverse matrix of the inverse perspective transformation matrix H obtained in step 5 ,Use OpenCV’s cv2.warpPerspective function to transform the fitted image into the original perspective, and use the fitted curve as a virtual trajectory line, which is drawn in the original image, and the vehicle tracking is based on this.

[0120] One embodiment of the present invention provides a driving assistance system, which includes a memory, a processor, and a vehicle tracking program stored in the memory and runnable on the processor. When the vehicle tracking program is executed by the processor, the steps of the vehicle tracking method described above are implemented, so that functions such as automatic driving cruise, lane keeping, and departure warning remain effective when the lane lines are covered in snow.

[0121] Another embodiment of the present invention provides a vehicle, including a vehicle body and the driving assistance system as described above, wherein the driving assistance system is arranged in the vehicle body, so that private travel on snowy days has a predetermined trajectory to follow, thereby improving personal willingness and safety of traveling on snowy days.

[0122] The specific implementation of the present invention is described in detail below with reference to specific embodiments.

[0123] Example 1: This example uses the PhotoTourism dataset. Considering the curve fitting problem, multiple experiments were conducted with inlier rates ranging from 15% to 50%. Each experiment consisted of 300 data points in the range [-1, 1]. Inliers were simulated using Gaussian noise with a mean of 0 and a variance of 0.02, while outliers were simulated using a uniform distribution with a maximum absolute value of 1.0.

[0124] BANSAC is compared with RANSAC and BaySAC. As the estimation parameters, the error threshold of 0.02, the maximum number of iterations of 3000 and the estimation confidence of 0.99 are set. In BANSAC, the initial probability of all data points is Set to 0.5, the stopping criterion threshold The root mean square error (RMSE) of the geometric distance between each point in the estimated model and the ideal model and the number of iterations were measured, and the average value obtained after 1000 randomly generated trials is shown. The results are shown in Tables 1 and 2.

[0125] Table 1 Comparison of iteration times

[0126]

[0127] Table 2 Root mean square error comparison

[0128]

[0129] As can be seen from the data in Table 1, the number of iterations for all three methods decreases as the inlier rate increases, and it is clear that BANSAC requires significantly fewer iterations than RANSAC and BaySAC at all inlier rates. As can be seen from the data in Table 2, overall, even at low inlier rates, BANSAC's accuracy is comparable to or higher than that of RANSAC and BaySAC.

[0130] In summary, through the analysis of experimental data, it can be seen that compared with RANSAC and BaySAC, BANSAC is not only more accurate, but also significantly reduces the number of iterations required while achieving similar or better accuracy, and its efficiency is greatly improved, fully demonstrating its advantages in processing curve fitting problems.

[0131] The above are only preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, several variations and improvements can be made without departing from the concept of the present invention. These should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A vehicle tracking method for constructing a virtual track line based on rutting traces on an icy and snowy road, characterized in that: The following steps are involved: Step 1: Collect rut images on icy and snowy roads using a vehicle-mounted camera and multiple sensors. Step 2: Calibrate the vehicle-mounted camera, obtain the camera's intrinsic parameter matrix and distortion coefficient, and perform dedistortion processing on the captured image; Step 3: Perform color clustering on the pixels and remove some irrelevant element data before grayscale conversion to obtain a preliminary ROI image; Step 4: The image obtained in step 3 is further preprocessed, including grayscale conversion, bilateral filtering, histogram equalization, adaptive threshold binarization, morphological erosion operation (erosion followed by dilation), and contour extraction. Step 5: Convert the image processed in step 4 into a bird's-eye view through inverse perspective transformation; Step 6: Based on the distance between the left and right tires of the vehicle, extract the pixel coordinates of the rut edge by scanning line by line; Step 7: Use the BANSAC algorithm based on nonlinear regression to fit the cubic polynomial model to generate a smooth curve; Step 8: After transforming the fitted image to the original perspective, use the fitted curve as a virtual trajectory line, draw it in the original image and use it as the basis for vehicle tracking; In step 2, the camera intrinsic parameter matrix , distortion coefficient , the distortion removal formula is: ； ； ； in, is the focal length of the camera; are the principal point coordinates of the image; are the second-order, fourth-order, and sixth-order coefficients of radial distortion, respectively; is the coefficient of tangential distortion; is the undistorted coordinate position; is the coordinate position after distortion; r is the point The radial distance to the origin of the image coordinate system; The specific steps of step 3 are as follows: Step 31: Read the processed image and convert it into a pixel matrix. Use the K-Means clustering algorithm to perform color clustering on the pixels in the image using the RGB space to obtain clusters with different color value ranges. Step 32: Select the number of cluster centers based on the snow pixel characteristics, i.e., the snow color is white and the three channel values ​​are all close to 255; and the wheel track pixel characteristics, i.e., the wheel track color is dark and the three channel values ​​are all close to 0; Step 33: Analyze the clustering results and extract the target cluster closest to the snow and rut colors by calculating the similarity between the cluster center color and the known snow and rut colors; Step 34: Set the pixels of non-target clusters to the background color, i.e. white, and retain the pixels related to snow and ruts; Step 35: Generate a new image based on the filtered pixels, retaining only the areas related to snow and ruts to form a preliminary ROI image; In step 4, the grayscale formula is as follows: ； in, is the grayscale pixel value; R, G, and B represent the pixel values ​​of the red, green, and blue channels in the color image respectively; The steps of histogram equalization are: calculate the frequency of occurrence of each grayscale value, the histogram of the input image; calculate the cumulative distribution function, i.e. CDF; according to the CDF, map each grayscale value of the input image to a new grayscale value. The formula is as follows: ； in, ; is the transformation function of histogram equalization; is the grayscale value of the input image; is the rounding function; is the cumulative distribution function of the input image; is the minimum non-zero cumulative distribution function value; N is the total number of pixels in the image; L is the total number of gray levels; In step 6, the distance between the left and right tires of the vehicle is The method of obtaining is: according to the front wheel spacing and rear wheel spacing range of different models, select the minimum wheel spacing And the maximum wheelbase , and converted into pixel values ​​according to the image resolution and the scale after perspective transformation , ; The specific steps for extracting the pixel coordinates of the rut edge are as follows: Step 61: Initialize row pointer i = m and column pointer j = 0, where m is the total number of rows in the image; scan row i, i.e., scan the binary image row by row starting from the bottom of the image, and then determine whether all pixel values ​​P in the current row are 255. If so, there is no rutting edge, and set i = i-1 to scan the previous row; otherwise, proceed to step 62; Step 62: Let j = j + 1, scan the pixels starting from the first column from left to right, and find the pixel value P in the jth column of the current row j The pixel point is 0; if P j If it is not 0, it is not the target point, let j=j+1 and continue to scan the next column; if 0, then record the current point column number ; In the current row, from Starting from the column, check arrive Pixel values ​​within the range; if there is a pixel point with a pixel value of 0 within the range, the current point is considered to be part of the rut edge and the coordinates of the current point are recorded Otherwise, the current point is regarded as an interference point, and , to eliminate interference; Step 63: Continue scanning upwards row by row, repeating step 62 until all rows are scanned; Step 64: After the scanning is completed, the coordinates of all pixels with a grayscale value of 0 are extracted, which are the pixel coordinate values ​​of the rut edge; The specific steps of step 7 are as follows: Step 71: Initialize the inlier probabilities of all data points is 0.5, setting the maximum number of iterations, error threshold and confidence level; Step 72: Iterate, and in each iteration, calculate the inlier probability of each data point based on the current model; use the dynamic Bayesian network to update the inlier probability, taking into account the fitting error of the data point with the current model; Step 73: Perform weighted sampling based on the updated inlier probability and select sample points for model estimation; Step 74: Using nonlinear regression to fit the cubic polynomial model in the BANSAC framework, the objective function Z is optimized to minimize the fitting error. The mathematical expression of the cubic polynomial model is: ； Among them, a, b, c, d are the coefficients of the polynomial; is the dependent variable; x is the independent variable; The optimization objective function Z is: ； Where N is the number of data points; are the coordinates of the data points; Step 75: Calculate the number of internal points and fitting error of the current model, and then determine whether the stopping criteria are met. If the stopping criteria are met, stop the iteration; output the final cubic polynomial model parameters a, b, c, d, and generate a smooth curve.

2. The vehicle tracking method for constructing a virtual track line based on rutting traces on an icy and snowy road according to claim 1, characterized in that: In step 5, the inverse perspective transformation formula is: ； in, is a point in the original image; is the transformed point; is the scaling factor; H is Inverse perspective transformation matrix.

3. The vehicle tracking method for constructing a virtual track line based on rutting traces on an icy and snowy road according to claim 1, characterized in that: In step 8, the specific method of transforming the fitted image perspective to the original perspective is: to obtain the inverse matrix of the inverse perspective transformation matrix H obtained in step 5 , use OpenCV's cv2.warpPerspective function to transform the fitted image to the original perspective.

4. A driving assistance system, characterized in that: The driving assistance system includes a memory, a processor, and a vehicle tracking program stored in the memory and executable on the processor. When the vehicle tracking program is executed by the processor, the steps of the vehicle tracking method as described in any one of claims 1 to 3 are implemented.

5. A vehicle, characterized in that: The invention comprises a vehicle body and the driving assistance system as claimed in claim 4, wherein the driving assistance system is arranged in the vehicle body.

Images