A virtual orbit tracking method and apparatus

Abstract

The application discloses a virtual track tracking method and device, steps comprising: S1. Obtain the road image containing the virtual track to be tracked, and extract the pixel point set belonging to the virtual track to be tracked in each frame image; S2. Extract the left and right boundary point sets from the pixel point set of each frame image and perform twice or more than three times of curve fitting to obtain the left and right track line fitting results, and perform twice or more times of curve fitting on the pixel point set of each frame image to obtain the center track line fitting result; S3. Perform Kalman filtering on each frame image using the corresponding left and right track line fitting results and the center track line fitting result to obtain the final virtual track curve fitting result, so as to realize virtual track tracking. The application has the advantages of simple implementation method, high tracking efficiency and precision, anti-interference and reliable stability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent / automatic vehicle driving control, and particularly relates to a virtual track tracking method and device. BACKGROUND

[0002] Vehicle driving is developing towards intelligence and automation. A virtual track is provided for a smart track vehicle to run with the ground virtual track as a guide, so as to realize intelligent / automatic driving of the vehicle. In order to realize intelligent / automatic driving of the vehicle, or to provide an auxiliary driving function for the driver, such as a lane keeping function, or to realize an auxiliary automatic driving function of tracking the virtual track on the road surface, it is necessary to identify the virtual track in the road image, that is, to track the virtual track, so as to provide spatial position information of the vehicle on the road for intelligent / automatic driving or auxiliary driving, so as to control the vehicle by the control system.

[0003] In the prior art, a lane line automatic tracking method usually adopts a lane line image real-time identification manner, that is, a lane line image is extracted in real time during vehicle running to perform image recognition and determine the type of the lane line. Such a lane line automatic tracking method has the following problems:

[0004] 1. Since the type of the lane line needs to be identified in real time, the implementation is complex, the tracking efficiency is low, the tracking precision is not high, and the anti-interference performance is poor, especially in a road environment with a small turning radius, it is usually difficult to accurately identify the change of the lane line, and thus the method is not suitable for a road scene with a small turning radius. The virtual track is usually applied to urban roads, and the turning radius is generally small. The above-mentioned traditional lane line automatic tracking method is not suitable for tracking of the virtual track, and has problems of low tracking efficiency, low tracking precision, poor anti-interference performance, and the like.

[0005] 2. During virtual track detection, the virtual track may not be detected or may not be detected completely. When the above-mentioned traditional lane line automatic tracking method occurs in the case that the virtual track is not detected or is not detected completely, the track line cannot be identified for tracking, resulting in interruption of tracking, that is, real-time and stable lane line tracking cannot be realized.

[0006] There are practitioners to combine the way of Kalman filter to realize lane line tracking, so that the track can be tracked according to the front and back frame relationship, but in this kind of way, a straight line fitting model is usually used, that is, since the curved road can also be approximated as a straight line in general, all types of lane lines including curved roads and straight lines are fitted with a straight line fitting model, and then Kalman filter is combined to filter to realize track line tracking. However, for the curved road with small turning radius, it cannot be approximated as a straight line, and the above-mentioned method is still only applicable to straight road sections or road scenes with large turning radius such as highways, and is not applicable to road scenes with small turning radius, that is, it is not applicable to virtual track tracking.

[0007] In summary, there is an urgent need for a virtual track tracking method that can be applied to virtual track tracking in a small turning radius road scene, while ensuring continuous and stable tracking of the virtual track. SUMMARY

[0008] The technical problem to be solved by the present application is that in view of the technical problems existing in the prior art, the present application provides a virtual track tracking method and device with simple implementation method, high tracking efficiency and precision, anti-interference and reliability and stability, which can continuously and stably realize accurate virtual track tracking.

[0009] To solve the above technical problems, the technical solution provided by the present application is:

[0010] A virtual track tracking method, comprising the following steps:

[0011] S1. Obtain a road image containing a virtual track to be tracked, and extract a set of pixel points belonging to the virtual track to be tracked in each image frame;

[0012] S2. Extract left and right boundary point sets from the set of pixel points of each image frame and perform two or more times of curve fitting to obtain left and right track line fitting results, and perform two or more times of curve fitting on the set of pixel points of each image frame to obtain a center track line fitting result;

[0013] S3. Perform Kalman filtering on each image frame using the corresponding left and right track line fitting results and the center track line fitting result to obtain a final virtual track curve fitting result, thereby realizing virtual track tracking.

[0014] Further, when the image is obtained in step S1, a front-view road image is collected by a vehicle-mounted image collection device and image distortion correction is performed, the corrected image is changed to a bird's eye view angle, and a road image with a bird's eye view angle is obtained.

[0015] Further, in the step S1, when the pixel point set is extracted, the coordinate data of each pixel point in the image coordinate system is converted into coordinate data in a pre-constructed rectangular coordinate system, and the rectangular coordinate system takes a specified position in the image as the origin and the vehicle forward direction as the X-axis direction.

[0016] Further, after the step S1 and before the step S2, the edges of the pixel point set are smoothed to obtain a smoothed pixel point set.

[0017] Further, in the step S2, a curve fitting method of the second order or more than the third order is used to obtain a curve equation of the second order or more than the third order.

[0018] Further, in the step S3, any one or more than two coefficients in the curve equation is used for Kalman filtering.

[0019] Further, in the step S3, the Kalman filtering is performed by using the center track line fitting result to construct a state vector and taking the state vector as the input of a state transition equation, using the left and right track line fitting results to construct an observation vector and taking the observation vector as the input of an observation equation, and solving a Kalman filtering equation to obtain a final virtual track curve fitting result.

[0020] Further, the state vector X of the kth image is constructed by using the curve equation parameters in the center track line fitting result of the kth image. k wherein the center track line fitting result is obtained by using an N-order curve fitting method, and N≥2, and the state transition equation of the kth image is constructed according to the following formula:

[0021]

[0022]

[0023] wherein, is the state vector estimation value of the kth image, X k-1 is the state vector of the (k-1)th image, and w is a system error vector, and each element in the system error vector w corresponds to a system error of each parameter in the state vector X.

[0024] Further, the observation vector Z of the kth image is constructed by using the curve equation parameters in the left and right track line fitting results of the kth image. k wherein the left and right track line fitting results are obtained by using an N-order curve fitting method, and N≥2, and the observation equation of the kth image is constructed according to the following formula:

[0025] Z k = H * Xk-1 +v

[0026]

[0027] wherein, X k-1 is the state vector of the k-1th frame image, v is the observation error vector, each element in v corresponds to the observation error of each parameter in the observation vector Z, H is the observation matrix.

[0028] Further, when solving the Kalman filtering equation, the following Kalman filtering equation is solved:

[0029]

[0030] and the covariance matrix is updated:

[0031]

[0032] wherein, K k is calculated by the following formula:

[0033]

[0034]

[0035] In the above formula, Q is a diagonal matrix composed of statistical parameters of system errors of each curve equation parameter in the center track line fitting result, R is a diagonal matrix composed of statistical parameters of system errors of curve equation parameters in the left and right track line fitting results, P0 is a diagonal matrix composed of different numbers belonging to the interval (0, 1) when k = 1.

[0036] Further, in the step S3, the Kalman filtering is performed, and the state vector and / or the observation vector further includes a virtual track overall width parameter.

[0037] A virtual track tracking device, comprising:

[0038] An acquisition module is configured to acquire a road image containing a virtual track to be tracked, and extract a pixel point set belonging to the virtual track to be tracked in each frame of image;

[0039] A fitting module is configured to extract left and right boundary point sets from the pixel point set of each frame of image and perform curve fitting for two or more times to obtain left and right track line fitting results, and perform curve fitting for two or more times on the pixel point set of each frame of image to obtain a center track line fitting result.

[0040] The filtering module is used for Kalman filtering of the left and right track line fitting results and the center track line fitting result of each frame of image to obtain a final virtual track curve fitting result, so as to realize virtual track tracking.

[0041] Further, when Kalman filtering is performed in the filtering module, the center track line fitting result is used to construct a state vector and as an input of a state transition equation, and the left and right track line fitting results are used to construct an observation vector and as an input of an observation equation, and a Kalman filtering equation is solved to obtain the final virtual track curve fitting result.

[0042] A virtual track tracking device, comprising a processor and a memory, the memory is used for storing a computer program, and the processor is used for executing the computer program, characterized in that the processor is used for executing the computer program to execute the above method.

[0043] An intelligent driving vehicle, comprising a control system for controlling the operation of the vehicle, characterized in that it further comprises the above virtual track tracking device, the input end of the virtual track tracking device is connected with an image acquisition device, and the output end is connected with the control system, and the control module controls the vehicle to operate according to the virtual track curve fitting result obtained by the virtual track tracking device, so as to realize virtual track tracking.

[0044] Compared with the prior art, the present application has the following advantages:

[0045] 1. In the virtual track tracking process, the present application combines the Kalman filtering method, and performs two or more times of curve fitting on the left, center and right track lines of each frame of image, then performs Kalman filtering based on the curve fitting results to obtain the final virtual track curve fitting result of each frame of image.

[0046] 2. The present application combines the curve fitting results of the track lines of the virtual track to perform Kalman filtering, so that even if the virtual track cannot be detected or is not completely detected for a short time, the virtual track can still be tracked by using the relationship between the front and rear frames, ensuring the continuity and stability of virtual track detection, so as to continuously provide accurate and stable virtual track position information for vehicle lateral control and assist vehicle automatic driving.

[0047] 3、The present application further uses the center orbit line fitting result as the input of the state transition equation, and uses the left and right orbit line fitting results as the input of the observation equation, so that the curve fitting results of the three orbit lines can be fully utilized to achieve optimal estimation of the virtual orbit, and the virtual orbit can be quickly and accurately tracked. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a flowchart of the implementation of the virtual orbit tracking method of the present embodiment.

[0049] Figure 2 is a schematic diagram of the top view and bird's eye view of the road image obtained in the specific application embodiment.

[0050] Figure 3 is a detailed flowchart of the implementation of the virtual orbit tracking by the method of the present application in the specific application embodiment. DETAILED DESCRIPTION

[0051] The present application will be further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present application is not limited thereby.

[0052] As shown in Figure 1 , the steps of the virtual orbit tracking method of the present embodiment include:

[0053] S1. Obtain the road image containing the virtual orbit to be tracked, and extract the pixel point set belonging to the virtual orbit to be tracked in each frame of image;

[0054] S2. Extract the left and right boundary point sets from the pixel point set of each frame of image, and perform two or more times of curve fitting to obtain the left and right orbit line fitting results, and perform two or more times of curve fitting on the pixel point set of each frame of image to obtain the center orbit line fitting result;

[0055] S3. Perform Kalman filtering on each frame of image using the corresponding left and right orbit line fitting results and the center orbit line fitting result to obtain the final virtual orbit curve fitting result, so as to achieve virtual orbit tracking.

[0056] In the virtual track tracking process, the Kalman filtering method is combined, two or more times of curve fitting is performed on the left, middle and right track lines of each frame of image respectively, then the Kalman filtering is performed based on the curve fitting result, the final virtual track curve fitting result of each frame of image is formed, the two or more times of curve fitting can obtain a fitting result closer to the real state of the virtual track, so that the virtual track can be tracked quickly and accurately, and the Kalman filtering is combined with the fitting result, the Kalman filtering result can be used for prediction, even if the virtual track cannot be detected or is not completely detected for a short time, the virtual track can still be tracked by using the relationship between the front and rear frames, the continuity and stability of the virtual track detection are ensured, and the method is especially suitable for a road section with many small-radius curves, can continuously provide accurate and stable virtual track position information for vehicle lateral control, and thus assists the vehicle automatic driving.

[0057] In the step S1 of the embodiment, when the image is acquired, the front-view road image is first acquired by the vehicle-mounted image acquisition device and image distortion calibration is performed, the calibrated image is changed in the bird's-eye view angle, and the road image in the bird's-eye view angle is obtained. The vehicle-mounted image acquisition device is specifically a front-view camera at the front end of the vehicle, and the original front-view road image acquired by the front-view camera is usually a top view. Due to the limitation of the view angle, the distances between the virtual track lines obtained in the top view are usually small, and there is a difference from the actual virtual track lines. After the change in the bird's-eye view angle, a road image closer to the real state can be obtained, so that the pixel point set of the virtual track line can be more easily and accurately extracted from the image.

[0058] In a specific application embodiment, a front-view road image is photographed by a vehicle-mounted front-view camera during vehicle operation, image distortion calibration is performed by using the internal and external parameters of the camera, then the calibrated image is changed in the bird's-eye view angle, and the road image in the bird's-eye view angle is obtained. The road image in the bird's-eye view angle obtained in the specific application embodiment is as shown in FIG. 1. Figure 2 As shown in FIG. 1, the left side of the figure is the obtained original top view, and the right side of the figure is the road image in the bird's-eye view angle obtained after the conversion.

[0059] In the step S1 of the embodiment, when the pixel point set is extracted, the coordinate data of each pixel point in the pixel point set in the image coordinate system is converted into the coordinate data in the pre-constructed rectangular coordinate system, and the rectangular coordinate system is a rectangular coordinate system with a specified position in the image as the origin and the vehicle forward direction as the X-axis direction. The rectangular coordinate system can be an XOY coordinate system with the midpoint of the bottom edge of the image as the origin, the vehicle forward direction as the positive direction of the X-axis and the left side of the vehicle as the positive direction of the Y-axis. Of course, the specific selection of the origin and the positive directions of the X-axis and the Y-axis can also be set according to actual needs, for example, the vertical direction of the X-axis can also be set as the direction of the Y-axis.

[0060] In step S1 of the embodiment, a virtual track image recognition algorithm is used to detect the pixel point set belonging to the virtual track in the road image. The recognition algorithm can be selected according to actual requirements.

[0061] In a specific application embodiment, the detailed steps of step S1 are as follows: during the operation of the vehicle, a front-view road image is captured by a front-view camera mounted on the vehicle, and image distortion correction is performed using the internal and external parameters of the camera, and then the corrected image is changed to a bird's-eye view angle to obtain a road image in the bird's-eye view angle; in the road image in the bird's-eye view angle, an XOY coordinate system is established with the midpoint of the bottom edge of the image as the origin, the forward direction of the vehicle as the positive direction of the X axis, and the left side of the vehicle as the positive direction of the Y axis; then a virtual track image recognition algorithm is used to detect the pixel point set belonging to the virtual track in the road image, and the coordinate data of the pixel points belonging to the virtual track in the image coordinate system are converted into coordinate data in the XOY coordinate system to obtain the final pixel point set of the virtual track to be tracked.

[0062] The embodiment further includes smoothing the edges of the pixel point set before step S2 to smooth the edges of the virtual track pixel point set and obtain the pixel point set after smoothing. The smoothing can be achieved by using image closing operation or other methods.

[0063] Since the left and right lane lines are located at the two side edges in the three lane lines, the left and right lane lines can be easily distinguished by using an edge extraction method after the pixel point set of the virtual track is extracted. In step S2 of the embodiment, an edge extraction algorithm is used to extract the left and right edges of the smoothed virtual track pixel point set, that is, the left edge and the right edge in the image, and the left edge and the right edge correspond to the left lane line and the right lane line, respectively. The edge extraction algorithm can also be selected according to actual requirements. In the embodiment, curve fitting is further performed on all pixel point sets of each frame of image, and the curve fitting result is used as the fitting result of the center track line.

[0064] In step S2 of the embodiment, a cubic curve fitting method is used to obtain a cubic curve equation, and then in step S3, Kalman filtering is performed using the coefficients of the cubic term, the quadratic term, the linear term and the constant term in the cubic curve equation.

[0065] For example, in the first frame of image, after the left and right edges of the smoothed virtual track pixel point set are extracted using the edge extraction algorithm, the obtained point sets are denoted as P 左集 and P 右集 , respectively. The fitting algorithm is used to fit P 左集 and P 右集Three times curve fitting is performed to obtain the cubic curve equation of the left and right lane lines under XOY coordinates, as shown in the following formulas (1) and (2), wherein the formula (1) corresponds to the curve fitting equation of the left lane line, and the formula (2) corresponds to the curve fitting equation of the right lane line.

[0066] x = a 左1 y 3 +b 左1 y 2 +c 左1 y 1 +d 左1 (1)

[0067] x = a 右1 y 3 +b 右1 y 2 +c 右1 y 1 +d 右1 (2)

[0068] wherein a 左1 , a 右1 are cubic coefficients, b 左1 , b 右1 are quadratic coefficients, c 左1 , c 右1 are linear coefficients, and d 左1 , d 右1 are constant coefficients.

[0069] In the first frame image, all pixel point sets of the obtained virtual tracks are subjected to three times curve fitting using a fitting algorithm to obtain a cubic curve equation of the middle track line, as shown in the formula (3).

[0070] x = a 中0 y 3 +b 中0 y 2 +c 中0 y 1 +d 中0 (3)

[0071] wherein a 中0 is a cubic coefficient, b 中0 is a quadratic coefficient, c 中0 is a linear coefficient, and d 中0 is a constant coefficient.

[0072] The above coefficients are taken as input data of the Kalman filter and input into the Kalman filter equation to realize virtual track tracking by using the front and rear frame relationship.

[0073] It can be understood that the curve fitting method can also use other multiple curve fitting methods, such as 4th curve fitting, or even 2nd curve fitting method to reduce the complexity of implementation, which can be configured according to actual needs.

[0074] In the step S3 of the embodiment, the Kalman filtering is performed by using the center orbit line fitting result to construct a state vector and as an input of a state transition equation, and using the left and right orbit line fitting results to construct an observation vector and as an input of an observation equation, solving the Kalman filtering equation to obtain the final virtual orbit curve fitting result. Based on the Kalman filtering, the optimal estimation of the system can be obtained. By using the center orbit line fitting result as the input of the state transition equation and using the left and right orbit line fitting results as the input of the observation equation, the curve fitting results of the three orbit lines can be fully utilized to achieve the optimal estimation of the virtual orbit, so that the virtual orbit can be quickly and accurately tracked.

[0075] In the embodiment, the curve equation parameters in the center orbit line fitting result of the kth image are used to construct the state vector X k of the kth image, and the state transition equation of the kth image is constructed according to the following formula (4) and (5):

[0076]

[0077] wherein, is the state vector estimation value of the kth image, X k-1 is the state vector of the (k-1)th image, and w is a system error vector, each element of w corresponding to the system error of each parameter in the state vector X.

[0078] In the embodiment, the center orbit line fitting result is obtained by using 3rd curve fitting, and the matrix A in the formula (4) is:

[0079]

[0080] If the center orbit line fitting result is obtained by using other types of Nth curve fitting, i.e. N=2 or N≥4, the matrix A in the formula (4) needs to be matched and constructed in a similar manner as when the 3rd curve fitting is used, i.e.:

[0081]

[0082] In the embodiment, the state vector further includes a virtual orbit overall width parameter, such as the state vector X k of the kth image, which is specifically X k =[a 中k ,b 中k ,c 中k ,d 中k,width] T , a 中k , b 中k , c 中k , d 中k respectively represent four parameters of the cubic curve equation of the central track line (i.e. the cubic term coefficient, the quadratic term coefficient, the linear term coefficient and the constant term coefficient), and width represents the overall width of the virtual track. w is specifically: w = [w 中a , w 中b , w 中c , w 中d , w width ], w 中a , w 中b , w 中c , w 中d respectively represent systematic errors of the four parameters of the cubic curve equation of the central track line and the overall width of the virtual track width, and w 中a obeys a normal distribution (0, б 中a ), w 中b obeys a normal distribution (0, б 中b ), w 中c obeys a normal distribution (0, б 中c ), w 中d obeys a normal distribution (0, б 中d ), w width obeys a normal distribution (0, б width ).

[0083] Since according to the geometric relationship of the virtual track, the cubic curve equation of the left boundary line of the virtual track (the left track line) and the cubic curve equation of the middle track line will have the following relationship:

[0084]

[0085]

[0086] According to formulas (6) and (7), the observation matrix H shown in formula (8) can be obtained, and the curve equation parameters in the left and right track line fitting results of the kth image are used to construct the observation vector Z k of the kth image, and the observation equation of the kth image is constructed according to the following formula and shown in formula (9).

[0087]

[0088] Z k = H * X k-1 + v (9)

[0089] Wherein, X k-1 is the state vector of the k-1th image, v is the observation error vector, and each element in v corresponds to the observation vector Zk Observation errors of each parameter.

[0090] If other types of N-order curve fitting are used to obtain the fitting results of the left and right track lines, i.e. N=2 or N≥4, the H matrix in formula (9) needs to be matched and constructed accordingly in a similar form as using 3-order curve fitting, i.e.:

[0091]

[0092] In this embodiment, the observation vector also includes a virtual overall width parameter of the track, and the observation vector is specifically Z=[a 左 , b 左 , c 左 , d 左 , a 右 , b 右 , d 右 , d 右 ] T , where a 左 , b 左 , c 左 , d 左 , a 右 , b 右 , d 右 , d 右 respectively represent 8 parameters (coefficients of each order term) of the 3-order curve equation of the left and right track lines, and the observation error vector is specifically v=[v 左a , v 左a , v 左c , v 左d , v 右a , v 右b , v 右c , v 右d ], v 左a , v 左a , v 左c , v 左d , v 右a , v 右b , v 右c , v 右d respectively represent observation errors of the 8 parameters of the 3-order curve equation of the left and right track lines, and wherein v 左a obeys normal distribution (0,б 左a ), v 左b obeys normal distribution (0,б 左b ), v 左c obeys normal distribution (0,б 左c ), v 左d obeys normal distribution (0,б 左d ), v 右a obeys normal distribution (0,б 右a ), v 左bobeys normal distribution (0,б 右b obeys normal distribution (0,б 左c obeys normal distribution (0,б 右c obeys normal distribution (0,б 右d obeys normal distribution (0,б 右d .

[0093] The coefficients of the 3rd order curve equation of the virtual track line and the overall width of the virtual track are used to construct the state vector and the observation vector, and if other curve equations such as 2nd order or 4th order are used, the corresponding state vector and observation vector can be constructed by using the coefficients of the corresponding order curve equation and the overall width of the virtual track, and of course, other types of virtual track parameters can be used or introduced to construct the state vector and the observation vector to further improve the tracking accuracy according to actual needs.

[0094] In this embodiment, the Kalman filter equation is solved, and the following Kalman filter equation is solved:

[0095]

[0096] and the covariance matrix is updated:

[0097]

[0098] wherein K k is calculated as follows:

[0099]

[0100]

[0101] In the above formula, Q is a diagonal matrix composed of statistical parameters of system errors of each curve equation parameter in the center track line fitting result, R is a diagonal matrix composed of statistical parameters of system errors of curve equation parameters in the left and right track line fitting results, and P0 is a diagonal matrix composed of different numbers belonging to the interval (0, 1) when k = 1.

[0102] In the specific application embodiment, the above Q, R and P0 are specifically:

[0103]

[0104]

[0105]

[0106] wherein k p1 ,k p1 ,k p1 ,k p1 ,k p1Take different numbers belonging to the interval (0,1). When k=1, specifically X0=[a 中0 ,b 中0 ,c 中0 ,d 中0 [,width],Z1=[a 左1 b 左1 c 左1 d 左1 a 右1 b 右1 d 右1 d 右1 ].

[0107] In a specific application embodiment, the detailed steps of step S3 above are as follows:

[0108] For the k-th frame image, first convert the state vector X obtained from the (k-1)-th frame... k-1 =[a k-1 ,b k-1 ,c k-1 ,d k-1 ,width] T Substituting into equations (4) and (5), we obtain the state vector of the k-frame images. The prior estimate is then used to calculate K according to equations (12) and (13). k ;

[0109] The observation vector Z is obtained by fitting two cubic curve equations using the left and right orbit lines in the k-th frame image. k =[a 左k b 左k c 左k d 左k a 右k b 右k d 右k d 右k ];as well as Substituting into the Kalman filter formula (10), we obtain the state vector X of the k-th frame. k That is, the parameters of the cubic curve equation of the predicted virtual track middle line and the virtual track width parameters are obtained, and used as the final virtual track curve fitting result of the k-th frame image, and the covariance matrix P is updated according to equation (11).

[0110] By repeating the above steps, the cubic curve equation of the middle track line of the virtual track corresponding to each frame image can be obtained, thus completing the virtual track tracking.

[0111] like Figure 3As shown, in a specific application embodiment, the detailed steps of the virtual track tracking realized by the present application are as follows: first, a front perspective image of a vehicle is acquired, and after the front perspective image is converted into a bird's eye view, an XOY coordinate system is established in the bird's eye view; a virtual track recognition algorithm is used to detect a virtual track region, then left and right edges in the virtual track region are extracted, and the left and right edges are respectively subjected to three times of curve fitting to obtain a left lane line cubic curve equation and a right lane line cubic curve equation; the entire virtual track region is subjected to three times of curve fitting to obtain a middle lane line cubic curve equation; the coefficients in the middle lane line cubic curve equation and the overall width of the virtual track are used as state equation parameters of Kalman filtering to construct a state vector, and the coefficients in the left and right lane line cubic curve equations and the 1 / 2 width of the overall width of the virtual track are used as observation equation parameters of Kalman filtering to construct an observation vector; after the Kalman filtering equation is established, each frame of image is used to track the virtual track cubic curve direction parameters (coefficients), wherein the prior estimate value of the state vector X of the kth frame of image and the observation vector Z k of the kth frame of image are substituted into the Kalman filtering formula (10) to obtain the state vector X k of the kth frame of image.

[0112] The virtual track tracking device of the embodiment comprises:

[0113] An acquisition module is configured to acquire a road image containing a virtual track to be tracked, and extract a pixel point set belonging to the virtual track to be tracked in each frame of image;

[0114] A fitting module is configured to extract left and right boundary point sets from the pixel point set of each frame of image and perform two or more times of curve fitting to obtain left and right track line fitting results, and perform two or more times of curve fitting on the pixel point set of each frame of image to obtain a center track line fitting result;

[0115] A filtering module is configured to perform Kalman filtering on the left and right track line fitting results and the center track line fitting result corresponding to each frame of image to obtain a final virtual track curve fitting result, so as to realize virtual track tracking.

[0116] In the filtering module, when performing Kalman filtering, the center track line fitting result is used to construct a state vector and as an input of a state transition equation, the left and right track line fitting results are used to construct an observation vector and as an input of an observation equation, a Kalman filtering equation is solved, and a final virtual track curve fitting result is obtained.

[0117] The virtual track tracking device of the embodiment is one-to-one corresponding to the virtual track tracking method, and will not be described one by one here.

[0118] In another embodiment, the virtual track tracking device of the present application can further comprise a processor and a memory, the memory is used to store a computer program, the processor is used to execute the computer program, and the processor is used to execute the computer program to perform the virtual track tracking method described above.

[0119] The embodiment also provides an intelligent driving vehicle, which comprises a control system for controlling the operation of the vehicle, and further comprises the virtual track tracking device as described above, the input end of the virtual track tracking device is connected with the image acquisition device, and the output end is connected with the control system, and the control module controls the vehicle to operate according to the virtual track curve fitting result obtained by the virtual track tracking device, so as to realize virtual track tracking.

[0120] The image acquisition device is specifically a front-view camera on the vehicle, and the front-view camera is used to acquire front-view road images when the vehicle is running, and the images are output to the virtual track tracking device for virtual track tracking.

[0121] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed as above with the preferred embodiment, it is not intended to limit the present application. Therefore, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present application, without departing from the technical solution of the present application, shall fall within the protection scope of the technical solution of the present application.

Claims

1. A virtual orbit tracking method characterized by the steps of The method comprises the following steps: Step S1. Obtain a road image containing a virtual track to be tracked, and extract a pixel point set belonging to the virtual track to be tracked in each frame of image; Step S2. Extract left and right boundary point sets from the pixel point set of each frame of image, and perform two or more times of curve fitting to obtain left and right track line fitting results, and perform two or more times of curve fitting on the pixel point set of each frame of image to obtain a center track line fitting result; Step S3. Perform Kalman filtering on each frame of image using the corresponding left and right track line fitting results and the center track line fitting result to obtain a final virtual track curve fitting result, thereby realizing virtual track tracking. In the step S3, the center track line fitting result is used to construct a state vector and is taken as an input of a state transition equation, the left and right track line fitting results are used to construct an observation vector and are taken as an input of an observation equation, a Kalman filtering equation is solved, and a final virtual track curve fitting result is obtained.

2. The virtual orbit tracking method of claim 1, wherein: In the step S1, a front-view road image is collected by a vehicle-mounted image collection device, and image distortion correction is performed, a bird's-eye view angle change is performed on the corrected image, and a road image in a bird's-eye view angle is obtained.

3. The virtual orbit tracking method of claim 1, wherein: In the step S1, when the pixel point set is extracted, the coordinate data of each pixel point in the pixel point set in an image coordinate system is converted into coordinate data in a previously constructed rectangular coordinate system, and the rectangular coordinate system takes a specified position in the image as an origin and a vehicle advancing direction as an X-axis direction.

4. The virtual orbit tracking method of claim 1, wherein: After the step S1 and before the step S2, the edges of the pixel point set are smoothed to obtain a smoothed pixel point set.

5. The virtual orbit tracking method according to any one of claims 1 to 4, characterized in that: In the step S2, a curve fitting method with two or more than three times is used to obtain a curve equation with two or more than three times.

6. The virtual orbit tracking method of claim 5, wherein: In the step S3, any one or more than two coefficients in the curve equation is used for Kalman filtering.

7. The virtual orbit tracking method of claim 1, wherein: The state vector X of the kth frame image is constructed using the curve equation parameters in the center orbit line fitting result of the kth frame image k wherein the center orbit line fitting result adopts N is fitted by a secondary curve fitting method, and N ≥2, and the state transition equation of the kth frame image is constructed according to the following formula: wherein, Xk is the state vector estimate for the kth image, k-1 Xk-1 is the state vector for the (k-1)th image, and w is a system error vector, the elements of which correspond to the system errors of the parameters in the state vector X.

8. The virtual orbit tracking method of claim 7, wherein: The curve equation parameters in the left and right track line fitting results of the kth frame image are used to construct an observation vector Z of the kth frame image k wherein the left and right track line fitting results adopt N The curve fitting method is used to fit, and N ≥2, and an observation equation of the kth frame image is constructed according to the following formula: Z k = H*X k-1 + v wherein X k-1 is the state vector of the k-1th frame image, v is the observation error vector, each element of v corresponds to the observation error of each parameter in the observation vector Z, and H is the observation matrix.

9. The virtual orbit tracking method of claim 8, wherein: In the step of solving the Kalman filtering equation, the following Kalman filtering equation is solved: and a covariance matrix is updated: wherein was calculated as follows: In the above formula, Q is a diagonal matrix consisting of statistical parameters of systematic errors of the curve equation parameters in the center track line fitting result, R is a diagonal matrix consisting of statistical parameters of systematic errors of the curve equation parameters in the left and right track line fitting results, when k = 1, P 0 is a diagonal matrix consisting of different numbers belonging to the interval (0, 1).

10. The virtual orbit tracking method according to any one of claims 1 to 4, characterized in that: In the step S3, the state vector and / or the observation vector further comprises a virtual track overall width parameter.

11. A virtual track following device, characterized by The method comprises the following steps: An acquisition module is configured to obtain a road image containing a virtual track to be tracked, and extract a pixel point set belonging to the virtual track to be tracked in each frame of image; A fitting module is configured to extract left and right boundary point sets from the pixel point set of each frame of image, and perform two or more times of curve fitting to obtain left and right track line fitting results, and perform two or more times of curve fitting on the pixel point set of each frame of image to obtain a center track line fitting result; A filtering module is configured to perform Kalman filtering on each frame of image using the corresponding left and right track line fitting results and the center track line fitting result to obtain a final virtual track curve fitting result, thereby realizing virtual track tracking. When Kalman filtering is performed in the filtering module, the central track line fitting result is used to construct a state vector and as an input of a state transition equation, the left and right track line fitting results are used to construct an observation vector and as an input of an observation equation, a Kalman filtering equation is solved, and a final virtual track curve fitting result is obtained.

12. A virtual track following device comprising a processor and a memory, the memory for storing a computer program, the processor for executing the computer program, characterized in that, The processor is configured to execute the computer program to perform the method of any one of claims 1-10.

13. An intelligent driver vehicle comprising a control system for controlling operation of the vehicle, characterized in that The virtual track tracking device of any one of claims 11-12 is further included, an input end of the virtual track tracking device is connected with the image acquisition device, an output end is connected with the control system, and the control module controls the vehicle to run according to the virtual track curve fitting result obtained by the virtual track tracking device, so as to realize virtual track tracking.

Images