Lane line tracking method, device and equipment based on spline curve

Abstract

The invention provides a lane line tracking method, device and equipment based on a spline curve. The method comprises the following steps: initializing a filter according to acquired vehicle motion information and lane line information; according to the vehicle motion information and the lane line information, the control point at the previous moment is transferred to the vehicle system at the current moment, and meanwhile, the covariance is transferred to the vehicle system at the current moment; according to the transfer result and the obtained lane line information at the current moment, establishing an association relationship between the sensed lane line and the lane line in the filter; according to the incidence relation, constructing an observation equation for each filter, and updating the filters; and checking the relationship between the control points in the filter and the vehicle and the currently sensed lane line, if the sensed lane line exceeds the coverage range of the control points in the updated filter, adding a new control point, and if the control points exceed a certain range of the tail part of the vehicle, deleting the corresponding control points so as to realize the tracking of the lane line.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a lane line tracking method, device and equipment based on a spline curve. Background Art

[0002] Lane lines play a vital role in assisted driving systems. They are an important prerequisite for ensuring that the vehicle is centered and complies with driving regulations. They are also the basis for vehicle positioning, path planning and control. High-quality lane line information can effectively guide vehicle driving and improve driving safety and comfort. With the continuous development of autonomous driving technology, the accuracy and real-time requirements for lane line tracking are becoming higher and higher.

[0003] At present, the commonly used methods for generating lane lines generally include two steps: detection and tracking. The detection step is usually implemented through a neural network, which can identify the lane lines on the road. However, due to the unexplainability of neural networks, there are problems such as false detection, missed detection, and jumping in many cases. Therefore, directly using the detection results may have serious consequences, so lane lines are usually tracked or smoothed based on the detection results. Lane line tracking can be implemented by graph optimization or filtering. To ensure real-time performance, filtering is often used on the vehicle side. This method takes the detection results as input and ultimately outputs smooth, non-jumping lane lines for planning and control.

[0004] Although the existing lane tracking methods can achieve lane smoothing and tracking to a certain extent, there are still some significant defects. When modeling, traditional lane tracking methods generally represent lanes as quadratic equations or polynomials under the vehicle system, and the state of the filter is the parameter of the equation or polynomial. The nonlinearity of lanes is handled by Kalman filtering or mixed state Kalman, and prior information such as lane width and spacing is used. However, this fixed-order modeling method is difficult to handle special-shaped lanes, resulting in unsatisfactory lane tracking effects in complex road environments. In addition, the state quantity is the coefficient of the equation, and its covariance does not have actual physical meaning. Therefore, the setting of the covariance of the filter becomes very tricky, and it is usually necessary to rely on debugging experience for design, which not only increases the complexity of the system, but also reduces the accuracy and stability of tracking. Summary of the invention

[0005] The present invention provides a lane line tracking method, device and equipment based on spline curves, which solves the problem that traditional tracking methods cannot handle special-shaped lane lines.

[0006] The present invention provides a lane line tracking method based on a spline curve, comprising the following steps: Initialize the filter according to the acquired vehicle motion information and lane line information; the filter state quantity is the coordinates of the control points in the vehicle system obtained by sampling according to the distance on the perceived lane line, and initialize the covariance of the control points at the same time; According to the vehicle motion information and lane line information, the control point at the previous moment is transferred to the vehicle system at the current moment, and the covariance is transferred to the vehicle system at the current moment; According to the transfer result and the lane line information obtained at the current moment, an association relationship between the perceived lane line and the lane line in the filter is established; According to the association relationship, construct an observation equation for each filter and update the filter; According to the association relationship, check whether the currently sensed lane line exceeds the coverage of the control points in the corresponding updated filter. If the sensed lane line exceeds the coverage of the control points in the updated filter, add a new control point; check the relationship between the control point and the vehicle position. If it exceeds a certain range from the rear of the vehicle, delete the corresponding control point to achieve lane line tracking.

[0007] According to a spline curve-based lane line tracking method provided by the present invention, before initializing the filter according to the acquired vehicle motion information and lane line information, the method also includes: acquiring vehicle motion information and lane line information; the lane line information includes the timestamp of the current frame camera sampling moment, the ID of each lane line, the 3D point coordinates and confidence of the perceived lane line under the vehicle system; the vehicle motion information includes the vehicle position and vehicle posture corresponding to the timestamp of the current frame camera sampling moment.

[0008] According to a spline curve-based lane tracking method provided by the present invention, the control point at the previous moment is transferred to the vehicle system at the current moment according to the vehicle motion information and the lane line information, and the covariance is transferred to the vehicle system at the current moment, which specifically includes: obtaining the increment between the vehicle information corresponding to the current perceived lane line timestamp and the vehicle information in the filter; and performing state transfer and error propagation according to the increment.

[0009] According to a spline curve-based lane line tracking method provided by the present invention, the association relationship between the perceived lane line and the lane line in the filter is established based on the transfer result and the lane line information obtained at the current moment, specifically including: constructing a K-dimensional tree from the control points under the current vehicle system obtained through state transfer and error propagation; performing a nearest neighbor search for the 3D point coordinates of the perceived lane line corresponding to each ID in the constructed K-dimensional tree; voting for the lane line ID in the filter based on the search results within a preset distance threshold range; and establishing an association relationship between the ID of each lane line and the filter based on the voting result.

[0010] According to a lane line tracking method based on spline curves provided by the present invention, the observation equation is constructed for each filter according to the association relationship, and the filter is updated, which specifically includes: step 1: for the coordinates of a lane line 3D point under the current vehicle system, the two closest control points in the control points are searched to obtain the interpolation parameters of the cubic spline curve; step 2: according to the two control points, the two adjacent control points are respectively obtained to obtain four consecutive control points; step 3: the interpolation parameters and the obtained four control points are brought into the interpolation function, and the observation equation is established with the perception observation point; step 4: according to steps 2 and 3, the state update is completed based on the extended Kalman filter formula; the process of steps 1 to 4 is repeated until all the lane line points perceived and observed are traversed to complete the update of one filter; and the update of all associated filters is completed through steps 1 to 4.

[0011] According to a spline curve-based lane line tracking method provided by the present invention, if the perceived lane line exceeds the coverage range of the control points in the updated filter, a new control point is added; the relationship between the control point and the vehicle position is checked, and if it exceeds a certain range from the rear of the vehicle, the corresponding control point is deleted to achieve lane line tracking, specifically including: if it exceeds the coverage range of the control point in the corresponding filter, the 3D point coordinates of the perceived lane line that exceeds the coverage range by a certain distance and its covariance are used to expand the state quantity of the filter; all association relationships are traversed to complete the state expansion; if the control point in the filter exceeds a certain range from the rear of the vehicle, the exceeded control point and the corresponding covariance are deleted.

[0012] The present invention also provides a lane line tracking device based on a spline curve, comprising the following modules: A filter initialization module is used to initialize the filter according to the acquired vehicle motion information and lane line information; the filter state quantity is the coordinates of the control points in the vehicle system obtained by sampling according to the distance on the perceived lane line, and the covariance of the control points is initialized at the same time; A state quantity transfer module, used to transfer the control point at the previous moment to the vehicle system at the current moment according to the vehicle motion information and lane line information, and transfer the covariance to the vehicle system at the current moment; An association relationship establishment module, used to establish an association relationship between the lane line sensed and the lane line in the filter according to the transfer result and the lane line information obtained at the current moment; A filter updating module, used to construct an observation equation for each filter according to the association relationship and update the filter; The control point coverage range check and update module is used to check whether the currently sensed lane line exceeds the control point coverage range in the corresponding updated filter according to the association relationship. If the sensed lane line exceeds the control point coverage range in the updated filter, a new control point is added; the relationship between the control point and the vehicle position is checked. If it exceeds a certain range at the rear of the vehicle, the corresponding control point is deleted to achieve lane line tracking.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a lane line tracking method based on a spline curve as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the lane line tracking method based on a spline curve as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the lane line tracking method based on a spline curve as described above is implemented.

[0016] The present invention provides a lane line tracking method, device and equipment based on spline curves, including the following beneficial effects: by using control points on cubic spline curves to represent lane lines, the shape of lane lines can be more accurately fitted and tracked, especially for those lane lines with large curvature changes, which helps to improve the accuracy and safety of automatic driving or assisted driving systems in lane keeping, lane changing and other operations. By initializing and updating the filter state quantity and transferring the covariance, the noise and uncertainty in lane line detection can be effectively processed, and even when the lane line is partially missing, blurred or blocked, good tracking performance can be maintained, thereby enhancing the robustness of the system. By transferring the control point at the previous moment to the vehicle system at the current moment, it can quickly adapt to the change of the lane line and realize real-time lane line tracking, which is particularly important for high-speed vehicles, and can ensure that the system responds to the change of the lane line in the shortest time. When the perceived lane line distance exceeds the coverage range of the existing control points in the filter, new control points can be automatically added, thereby ensuring continuous and seamless tracking of the lane line. This adaptive expansion mechanism enables the present invention to be applicable to lane line scenes of different lengths and complexities. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the lane line tracking method based on spline curves provided by the present invention.

[0019] Figure 2 It is a structural schematic diagram of a lane line tracking device based on a spline curve provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Lane markings play an irreplaceable role in assisted driving. They are an important prerequisite for ensuring that the vehicle is centered and complies with driving regulations. They are the basis for vehicle positioning, planning and control. Low-quality lane markings can easily cause problems such as vehicles crossing the line, deviating to the left or right, etc.

[0023] The commonly used method of generating lane lines generally includes two steps: detection and tracking. Detection is achieved through neural networks. Due to the unexplainability of neural networks, there are problems such as false detection, missed detection and jumping in many cases. Directly using the detection results may bring serious consequences. Therefore, lane line tracking or smoothing is performed based on the detection results.

[0024] Lane tracking can be achieved by graph optimization or filtering. To ensure real-time performance, filtering is often used on the vehicle side. The detection results are used as input, and the final output is a smooth, non-jumping lane line for planning and control.

[0025] Traditional lane tracking methods generally represent lanes as quadratic equations or polynomials in the vehicle system when modeling, and the state of the filter is the parameters of the equation or polynomial. The nonlinearity of lanes is handled by Kalman filtering or mixed state Kalman filtering, and prior information such as lane width and spacing is used.

[0026] This fixed-order modeling method is difficult to handle irregular lane lines. In addition, the state quantity is the coefficient of the equation, and its covariance has no actual physical meaning. Therefore, setting the covariance of the filter becomes very difficult, causing it to become a numbers game designed based on debugging experience.

[0027] The present invention provides a 3D lane tracking method based on Catmull-Rom spline and Kalman filter, the main innovations of which are: 1. The filter state is the control point of the Catmull-Rom spline curve. Since the Catmull-Rom spline curve passes through the control point, the covariance of the state has actual physical meaning, that is, the position uncertainty of the point on the lane line.

[0028] 2. Taking the third-order spline curve as an example, the curve between every two adjacent control points can be expressed as a cubic polynomial, and n control points represent n-1 cubic polynomials. Therefore, the lane line is divided into multiple cubic polynomials, which can handle the problem of irregular lane lines well.

[0029] 3. In the assisted driving mode, the positioning module usually outputs a continuous vehicle trajectory. The present invention decouples lane line tracking from positioning, while taking into account the uncertainty of positioning to ensure the real-time tracking.

[0030] Combine the following Figure 1-Figure 3 The embodiments of the present invention are described in detail. The implementation of the present invention is introduced according to the filtering process: the whole process includes: data subscription, initialization, state transfer and error propagation, observation association, error correction, and state expansion.

[0031] Figure 1 is one of the flow charts of the lane line tracking method based on spline curves provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: S110, initializing a filter according to the acquired vehicle motion information and lane line information; the filter state quantity is the coordinates of the control points in the vehicle system obtained by sampling according to the distance on the perceived lane line, and the covariance of the control points is initialized at the same time.

[0032] According to a spline curve-based lane line tracking method provided by the present invention, before initializing the filter based on the acquired vehicle motion information and lane line information, vehicle motion information and lane line information are acquired; the lane line information includes the timestamp of the current frame camera sampling moment, the ID of each lane line, the 3D point coordinates and confidence of the perceived lane line under the vehicle system; the vehicle motion information includes the vehicle position and vehicle posture corresponding to the timestamp of the current frame camera sampling moment.

[0033] Specifically, data subscription.

[0034] Subscribe to the vehicle motion information output by the positioning module, including the first timestamp, vehicle position, and posture. Subscribe to the lane line information output by the perception module, including the second timestamp, the ID of each lane line, the 3D point coordinates of the lane line under the vehicle system, and the confidence.

[0035] initialization.

[0036] The filter is initialized using the lane line information and positioning information obtained during the subscription process. The state of the filter is: the control points on the Catmull-Rom spline curve under the vehicle system obtained by sampling at distance on the lane line, and the covariance of the control points.

[0037] Specifically, according to the perceived lane line information and vehicle motion information obtained through subscription, the vehicle information corresponding to the perceived lane line timestamp is obtained, including timestamp t_i, vehicle position p_i, and vehicle posture as rotation matrix R_i.

[0038] The filter is initialized using the ID of the perceived lane line and the 3D coordinate points under the vehicle system. Each ID corresponds to a filter, and the state of the filter is the 3D coordinate point corresponding to the ID. The control point obtained after downsampling is recorded as X(t_i), and the covariance of all control points is recorded as P(t_i).

[0039] S120 . According to the vehicle motion information and the lane line information, the control point at the previous moment is transferred to the vehicle system at the current moment, and the covariance is transferred to the vehicle system at the current moment.

[0040] According to a spline curve-based lane tracking method provided by the present invention, the control point at the previous moment is transferred to the vehicle system at the current moment according to the vehicle motion information and the lane line information, and the covariance is transferred to the vehicle system at the current moment. Specifically, the method includes: obtaining the increment between the vehicle information corresponding to the current perceived lane line timestamp and the vehicle information in the filter; and performing state transfer and error propagation according to the increment.

[0041] Specifically, state transfer and error propagation.

[0042] According to the obtained vehicle motion information, the control point at the previous moment is transferred to the vehicle system at the current moment, and the covariance is transferred to the vehicle system at the current moment, and the input noise is added.

[0043] Specifically, according to the current perceived lane line information and vehicle motion information, the vehicle information corresponding to the perceived lane line timestamp is obtained, including timestamp t_j, vehicle position p_j, and vehicle posture R_j.

[0044] Calculate the delta_R, delta_p increments between the vehicle information and the vehicle information in the filter. The rotation increment delta_R = R_j.inverse*R_i, and the position increment delta_p = R_j.inverse*(t_i-t_j). R_i and R_j: represent the rotation matrix, which is used to describe the direction of the vehicle at time points t_i and t_j. In three-dimensional space, the rotation matrix is ​​a 3x3 matrix used to transform points in one coordinate system to another coordinate system. t_i and t_j: represent the translation vector at time points t_i and t_j, which is a three-dimensional vector that describes the position of the vehicle.

[0045] According to the increments calculated above, the state and error propagation are completed. The state transfer process can be expressed as: X(t_j)=delta_R*X(t_i)+delta_p. The error propagation can be expressed as: P(t_j)=delta_R*P(t_i)*delta_R.transpose+Q. Where Q is the process noise.

[0046] Calculate the process noise Q. To simplify the problem, take delta_R and delta_p as input quantities, and assume that delta_R only has the error dyaw on the yaw angle, and conforms to the Gaussian distribution with a mean of 0 and a standard deviation of sigma_yaw, recorded as dyaw~N(0,sigma_yaw). Assume that delta_p is an error of only dx and dy, and x~N(0,sigma_dx), y~N(0,sigma_dy), then the covariance matrix of the input noise is the matrix M with diagonal values ​​of (sigma_yaw^2,sigma_dx^2,sigma_dy^2), and the partial derivative of the state transfer equation with respect to the input quantity is G, then Q=G*M*G.transpose.

[0047] S130. According to the transfer result and the lane line information obtained at the current moment, an association relationship between the sensed lane line and the lane line in the filter is established.

[0048] According to a spline curve-based lane line tracking method provided by the present invention, an association relationship between the perceived lane line and the lane line in the filter is established according to the transfer result and the lane line information obtained at the current moment, specifically including: constructing a K-dimensional tree from the control points under the current vehicle system obtained after state transfer and error propagation; performing a nearest neighbor search for the 3D point coordinates of the perceived lane line corresponding to each ID in the constructed K-dimensional tree; voting for the lane line ID in the filter according to the search results within a preset distance threshold range; and establishing an association relationship between the ID of each lane line and the filter according to the voting result.

[0049] Specifically, based on the result after the transfer and the perceived lane line at the current moment, an association relationship between the perceived lane line and the lane line in the filter is established.

[0050] Specifically, a kd-tree is constructed based on the control points of the current vehicle system. The 3D points of the perceived lane lines are used to perform a nearest neighbor search for each ID, and the association relationship between each ID of the perceived lane line and the filter is established based on the matching relationship.

[0051] S140: construct an observation equation for each filter according to the association relationship and update the filter.

[0052] According to a lane line tracking method based on spline curves provided by the present invention, an observation equation is constructed for each filter according to the association relationship, and the filter is updated, which specifically includes: step 1: for the coordinates of a lane line 3D point under the current vehicle system, the two closest control points in the control points are searched to obtain the interpolation parameters of the cubic spline curve; step 2: according to the two control points, the two adjacent control points are respectively obtained to obtain four consecutive control points; step 3: the interpolation parameters and the obtained four control points are brought into the interpolation function, and the observation equation is established with the perception observation point; step 4: according to steps 2 and 3, the state update is completed based on the extended Kalman filter formula; the process of steps 1 to 4 is repeated until all the lane line points perceived and observed are traversed to complete the update of one filter; and steps 1 to 4 are continued to complete the update of all associated filters.

[0053] Specifically, a. for a perceived 3D point coordinate under the current vehicle system, search for the two closest control points among the control points, and calculate the interpolation parameter u of the Catmull-Rom spline curve.

[0054] b. Based on the two control points obtained, obtain the two adjacent control points to obtain four consecutive control points, recorded as p_i, p_i+1, p_i+2, p_i+3. Use the calculated interpolation parameter u and the third-order Catmull-Rom spline curve interpolation function f to obtain the observation equation: p_measure=f(p_i+1, p_i+2, p_i+3, p_i+4, u). p_measure is the 3D point coordinate of the lane line obtained by perception.

[0055] c. Based on the observation method in b, the state update is completed using the standard extended Kalman filter EKF formula.

[0056] d. Traverse all perception points and loop from a to c to complete the update of a filter.

[0057] e. Traverse all associations and use the ad step to complete the update of all filters.

[0058] S150. Check whether the currently sensed lane line exceeds the coverage of the control points in the corresponding updated filter based on the association relationship. If the sensed lane line exceeds the coverage of the control points in the updated filter, add a new control point. Check the relationship between the control point and the vehicle position. If it exceeds a certain range from the rear of the vehicle, delete the corresponding control point to achieve lane line tracking.

[0059] According to a spline curve-based lane line tracking method provided by the present invention, if the perceived lane line exceeds the coverage range of the control points in the updated filter, a new control point is added; the relationship between the control point and the vehicle position is checked, and if it exceeds a certain range from the rear of the vehicle, the corresponding control point is deleted to achieve lane line tracking, specifically including: if it exceeds the coverage range of the control point in the corresponding filter, the 3D point coordinates of the perceived lane line that exceeds a certain distance from the coverage range and its covariance are used to expand the state quantity of the filter; all association relationships are traversed to complete the state expansion; if the control point in the filter exceeds a certain range from the rear of the vehicle, the exceeded control point and the corresponding covariance are deleted.

[0060] Specifically, according to the obtained association relationship between the perceived lane line and the lane line in the filter, it is checked whether the currently perceived lane line exceeds the coverage range of the control point in the corresponding filter.

[0061] For situations that exceed the coverage of the control points, the 3D point coordinates of the perceived lane line that exceed the coverage by a certain distance and their covariance are used to expand the state of the filter. All associations are traversed, and the situation that exceeds the coverage of the control points is checked in a loop to complete the state expansion.

[0062] The present invention realizes lane line tracking by using Catmull-Rom spline curve and EKF filter, and solves the problem that the traditional tracking method cannot handle special-shaped lane lines by using the characteristics of Catmull-Rom spline curve passing through control points. At the same time, the control points are actual physical coordinates, which solves the problem that the coefficients of the parameter equation of the traditional filter tracking curve are difficult to reflect the actual physical meaning of the coefficient covariance. In addition, the EKF filter of the tracking parameter equation is difficult to handle the high nonlinearity of the lane line. Using the Catmull-Rom spline curve, the lane line can be divided into multiple third-order polynomials to better handle the nonlinearity of the lane line.

[0063] The lane line tracking device based on a spline curve provided by the present invention is described below. The lane line tracking device based on a spline curve described below and the lane line tracking method based on a spline curve described above can be referred to each other.

[0064] like Figure 2 The present invention shows a lane tracking device based on a spline curve, comprising: A filter initialization module 210 is used to initialize the filter according to the acquired vehicle motion information and lane line information; the filter state quantity is the coordinates of the control points in the vehicle system obtained by sampling according to the distance on the perceived lane line, and the covariance of the control points is initialized at the same time; A state quantity transfer module 220, for transferring the control point at the previous moment to the vehicle system at the current moment according to the vehicle motion information and lane line information, and transferring the covariance to the vehicle system at the current moment; An association relationship establishing module 230, for establishing an association relationship between the sensed lane line and the lane line in the filter according to the transfer result and the lane line information obtained at the current moment; A filter updating module 240, configured to construct an observation equation for each filter according to the association relationship and update the filter; The control point coverage range check and update module 250 is used to check whether the currently sensed lane line exceeds the control point coverage range in the corresponding updated filter based on the association relationship. If the sensed lane line exceeds the control point coverage range in the updated filter, a new control point is added; the relationship between the control point and the vehicle position is checked. If it exceeds a certain range from the rear of the vehicle, the corresponding control point is deleted to achieve lane line tracking.

[0065] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3As shown, the electronic device may include: a processor 310 , a communications interface 320 , a memory 330 and a communication bus 340 , wherein the processor 310 , the communications interface 320 , and the memory 330 communicate with each other via the communication bus 340 . The processor 310 can call the logic instructions in the memory 330 to execute the lane line tracking method based on the spline curve, which includes: initializing the filter according to the acquired vehicle motion information and lane line information; the filter state quantity is the coordinates of the control points under the vehicle system obtained by sampling according to the distance on the perceived lane line, and the covariance of the control points is initialized at the same time; according to the vehicle motion information and the lane line information, the control point at the previous moment is transferred to the vehicle system at the current moment, and the covariance is transferred to the vehicle system at the current moment; according to the transfer result and the lane line information obtained at the current moment, an association relationship between the perceived lane line and the lane line in the filter is established; according to the association relationship, an observation equation is constructed for each filter, and the filter is updated; according to the association relationship, whether the currently perceived lane line exceeds the coverage range of the control points in the corresponding updated filter is checked, if the perceived lane line exceeds the coverage range of the control points in the updated filter, a new control point is added; the relationship between the control point and the vehicle position is checked, if it exceeds a certain range of the rear of the vehicle, the corresponding control point is deleted to achieve lane line tracking.

[0066] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0067] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the lane line tracking method based on the spline curve provided by the above methods, and the method includes: initializing the filter according to the acquired vehicle motion information and lane line information; the filter state quantity is the coordinates of the control points under the vehicle system obtained by sampling according to the distance on the perceived lane line, and the covariance of the control points is initialized at the same time; according to the vehicle motion information and the lane line information, the control point at the previous moment is transferred to the vehicle system at the current moment, and the covariance is transferred to the vehicle system at the current moment; according to the transfer result and the lane line information obtained at the current moment, an association relationship between the perceived lane line and the lane line in the filter is established; according to the association relationship, an observation equation is constructed for each filter, and the filter is updated; according to the association relationship, whether the currently perceived lane line exceeds the coverage range of the control points in the corresponding updated filter is checked, if the perceived lane line exceeds the coverage range of the control points in the updated filter, a new control point is added; the relationship between the control point and the vehicle position is checked, if it exceeds a certain range of the rear of the vehicle, the corresponding control point is deleted to achieve lane line tracking.

[0068] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the lane line tracking method based on the spline curve provided by the above methods is implemented, and the method includes: initializing the filter according to the acquired vehicle motion information and lane line information; the filter state quantity is the coordinates of the control points under the vehicle system obtained by sampling according to the distance on the perceived lane line, and the covariance of the control points is initialized at the same time; according to the vehicle motion information and the lane line information, the control point at the previous moment is transferred to the vehicle system at the current moment, and the covariance is transferred to the vehicle system at the current moment; according to the transfer result and the lane line information obtained at the current moment, an association relationship between the perceived lane line and the lane line in the filter is established; according to the association relationship, an observation equation is constructed for each filter, and the filter is updated; according to the association relationship, whether the currently perceived lane line exceeds the coverage range of the control points in the corresponding updated filter, if the perceived lane line exceeds the coverage range of the control points in the updated filter, a new control point is added; the relationship between the control point and the vehicle position is checked, if it exceeds a certain range of the rear of the vehicle, the corresponding control point is deleted to achieve lane line tracking.

[0069] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0070] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lane tracking method based on spline curves, characterized in that: include: Initialize the filter according to the acquired vehicle motion information and lane line information; the filter state quantity is the coordinates of the control points in the vehicle system obtained by sampling according to the distance on the perceived lane line, and initialize the covariance of the control points at the same time; According to the vehicle motion information and lane line information, the control point at the previous moment is transferred to the vehicle system at the current moment, and the covariance is transferred to the vehicle system at the current moment; According to the transfer result and the lane line information obtained at the current moment, an association relationship between the perceived lane line and the lane line in the filter is established; According to the association relationship, construct an observation equation for each filter and update the filter; According to the association relationship, check whether the currently sensed lane line exceeds the coverage of the control points in the corresponding updated filter. If the sensed lane line exceeds the coverage of the control points in the updated filter, add a new control point; check the relationship between the control point and the vehicle position. If it exceeds a certain range from the rear of the vehicle, delete the corresponding control point to achieve lane line tracking.

2. The lane tracking method based on spline curve according to claim 1, characterized in that: Before initializing the filter according to the acquired vehicle motion information and lane line information, the method further includes: Obtain vehicle motion information and lane line information; The lane line information includes the timestamp of the camera sampling time of the current frame, the ID of each lane line, the 3D point coordinates and confidence of the perceived lane line under the vehicle system; The vehicle motion information includes the vehicle position and vehicle posture corresponding to the timestamp of the camera sampling moment of the current frame.

3. The lane tracking method based on spline curve according to claim 2, characterized in that: The method of transferring the control point at the previous moment to the vehicle system at the current moment according to the vehicle motion information and the lane line information, and transferring the covariance to the vehicle system at the current moment, specifically includes: Get the increment between the vehicle information corresponding to the timestamp of the currently sensed lane line and the vehicle information in the filter; State transfer and error propagation are performed according to the increment.

4. The lane tracking method based on spline curve according to claim 2, characterized in that: The step of establishing an association relationship between the lane line sensed and the lane line in the filter based on the transfer result and the lane line information obtained at the current moment specifically includes: The control points of the current vehicle system obtained through state transfer and error propagation are constructed into a K-dimensional tree; Perform nearest neighbor search on the constructed K-dimensional tree for the 3D point coordinates of the perceived lane line corresponding to each ID; vote for the lane line ID in the filter based on the search results within the preset distance threshold range; According to the voting results, an association between the ID of each lane line and the filter is established.

5. The lane tracking method based on spline curve according to claim 1, characterized in that: According to the association relationship, constructing an observation equation for each filter and updating the filter specifically includes: Step 1: For the 3D point coordinates of a lane line under the current vehicle system, search for the two closest control points among the control points to obtain the interpolation parameters of the cubic spline curve; Step 2: According to the two control points, two adjacent control points are obtained respectively to obtain four consecutive control points; Step 3: Substitute the interpolation parameters and the obtained four control points into the interpolation function, and establish an observation equation with the perception observation points; Step 4: According to steps 2 and 3, complete the state update based on the extended Kalman filter formula; The above steps 1 to 4 are repeated until all lane line points perceived and observed are traversed and a filter update is completed; Continue to update all associated filters through steps 1 to 4.

6. The lane tracking method based on spline curve according to claim 1, characterized in that: If the sensed lane line exceeds the coverage of the control points in the updated filter, a new control point is added; Check the relationship between the control point and the vehicle position. If it exceeds a certain range from the rear of the vehicle, delete the corresponding control point to achieve lane tracking, including: If the coverage of the control point in the corresponding filter is exceeded, the 3D coordinates of the lane line points and their covariance that are perceived at a certain distance beyond the coverage range are used to expand the state of the filter; Traverse all associations and complete state expansion; If the control points in the filter exceed a certain range from the rear of the vehicle, the exceeding control points and the corresponding covariance will be deleted.

7. A lane tracking device based on a spline curve, characterized in that: include: A filter initialization module is used to initialize the filter according to the acquired vehicle motion information and lane line information; the filter state quantity is the coordinates of the control points in the vehicle system obtained by sampling according to the distance on the perceived lane line, and the covariance of the control points is initialized at the same time; A state quantity transfer module, used to transfer the control point at the previous moment to the vehicle system at the current moment according to the vehicle motion information and lane line information, and transfer the covariance to the vehicle system at the current moment; An association relationship establishment module, used to establish an association relationship between the lane line sensed and the lane line in the filter according to the transfer result and the lane line information obtained at the current moment; A filter updating module, used to construct an observation equation for each filter according to the association relationship and update the filter; The control point coverage range check and update module is used to check whether the currently sensed lane line exceeds the control point coverage range in the corresponding updated filter according to the association relationship. If the sensed lane line exceeds the control point coverage range in the updated filter, a new control point is added; the relationship between the control point and the vehicle position is checked. If it exceeds a certain range at the rear of the vehicle, the corresponding control point is deleted to achieve lane line tracking.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the lane line tracking method based on the spline curve as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the lane line tracking method based on a spline curve as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the lane line tracking method based on a spline curve as claimed in any one of claims 1 to 6 is implemented.

Images