Lane line detection method and device, medium and vehicle

Abstract

The invention provides a lane line detection method and device, a medium and a vehicle, and the method comprises the steps: predicting a state vector of a Kalman filter at a current moment through employing a state vector at a previous moment if visual lane line data is determined to be in an invalid state at the current moment, and obtaining a prediction state vector corresponding to the current moment; correcting the high-precision map lane line data at the current moment by using the prediction state vector, and determining a target lane line by using the corrected high-precision map lane line data; therefore, when the visual lane line is valid, the target lane line can be directly determined according to the visual lane line data; once determining that the visual lane line data is invalid, predicting a state vector of a Kalman filter to obtain a predicted state vector, and correcting the high-precision map lane line data in real time by using the predicted state vector to improve the positioning precision of the lane line; moreover, only two data sources, namely the visual lane line data and the high-precision map, need to be fused, so that the hardware implementation cost and complexity are reduced.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a lane line detection method, device, medium and vehicle. Background Art

[0002] In autonomous driving systems and advanced driving assistance systems (ADAS), it is very important to achieve lane centering control of the vehicle on the road.

[0003] In order to achieve accurate lane centering, the existing technology generally relies on visual perception or map data. However, when lane line detection relies solely on visual data or map data, relying solely on visual perception is easily disturbed by external conditions such as ambient light, bad weather and road obstructions, while relying solely on map data has the problems of limited real-time and accuracy.

[0004] Therefore, how to improve the accuracy of lane line detection is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the problems existing in the prior art, the embodiments of the present invention provide a lane line detection method, device, medium and vehicle to solve or partially solve the technical problem in the prior art that the detection accuracy cannot be ensured when detecting lane lines.

[0006] A first aspect of the present invention provides a lane line detection method, the method comprising:

[0007] If it is determined that the visual lane line data is in an invalid state at the current moment, the state vector of the Kalman filter at the current moment is predicted using the state vector at the previous moment to obtain a predicted state vector corresponding to the current moment;

[0008] The predicted state vector is used to correct the high-precision map lane line data at the current moment, and the target lane line is determined using the corrected high-precision map lane line data.

[0009] In the above scheme, the state vector of the Kalman filter at the current moment is predicted by using the state vector at the previous moment to obtain the predicted state vector corresponding to the current moment, including:

[0010] Acquire a first transfer relationship matrix and a second transfer relationship matrix; the first relationship transfer matrix and the second relationship transfer matrix are determined according to a lane line data error sequence, and the lane line data error sequence is determined according to multiple frames of historical visual lane line data and multiple frames of historical high-precision map lane line data;

[0011] Determine a state transfer matrix of the Kalman filter according to the first transfer relationship matrix and the second transfer relationship matrix;

[0012] The predicted state vector corresponding to the current moment is determined using the state vector at the previous moment and the state transfer matrix; the predicted state vector is the product of the state vector at the previous moment and the state transfer matrix.

[0013] In the above scheme, after determining the predicted state vector corresponding to the current moment by using the state vector at the previous moment and the state transfer matrix, the method further includes:

[0014] Determine the process noise covariance matrix of the Kalman filter according to the lane line data error sequence, the state transfer matrix and a preset unit matrix;

[0015] The state covariance matrix of the Kalman filter at the current moment is predicted according to the state covariance matrix at the previous moment, the state transfer matrix and the process noise covariance matrix to obtain the predicted covariance matrix corresponding to the current moment.

[0016] In the above scheme, the method further comprises:

[0017] If it is determined that the visual lane line data is in a valid state at the current moment, the Kalman gain matrix at the current moment is determined using the initialized observation matrix, the measurement noise covariance matrix, and the state covariance matrix at the previous moment;

[0018] Determine the observation vector of the Kalman filter at the current moment, and determine the updated state vector at the current moment based on the Kalman gain matrix at the current moment, the observation vector, the state vector of the Kalman filter at the moment before the current moment, and the initialized observation matrix.

[0019] In the above scheme, the Kalman gain matrix at the current moment is determined by using the initialized observation matrix, the measurement noise covariance matrix and the state covariance matrix at the previous moment, including:

[0020] According to the formula K = P × H T ×(H×P×H T +R) -1 Determining the Kalman gain matrix K;

[0021] Wherein, P is the state covariance matrix of the previous moment, H is the initialization observation matrix, and H T is the transposed matrix of the H, and R is the measurement noise covariance matrix.

[0022] In the above scheme, the updating state vector at the current moment is determined according to the Kalman gain matrix at the current moment, the observation vector, the state vector of the Kalman filter at the previous moment before the current moment, and the initialized observation matrix, including:

[0023] The updated state vector State′ is determined according to the formula State′=State+K×(LH×State); wherein,

[0024] The State is the state vector of the Kalman filter at the previous moment before the current moment, the K is the Kalman gain matrix at the current moment, the L is the observation vector, and the H is the initialized observation matrix.

[0025] In the above scheme, after determining the updated state vector at the current moment according to the Kalman gain matrix at the current moment, the observation vector, the state vector of the Kalman filter at the previous moment of the current moment, and the initialized observation matrix, the method further includes:

[0026] The updated state covariance matrix P' at the current moment is determined according to the formula P'=(EK×H)×P; wherein,

[0027] The E is the preset fourth-order unit matrix, the K is the Kalman gain matrix at the current moment, the H is the initialized observation matrix, and the P is the state covariance matrix at the previous moment.

[0028] A second aspect of the present invention provides a lane line detection device, the device comprising:

[0029] A prediction unit, configured to predict a state vector of the Kalman filter at the current moment using a state vector at a previous moment if it is determined that the visual lane line data is in an invalid state at the current moment, to obtain a predicted state vector corresponding to the current moment;

[0030] A correction unit is used to correct the high-precision map lane line data at the current moment using the predicted state vector, and determine the target lane line using the corrected high-precision map lane line data.

[0031] In the above solution, the prediction unit is specifically used for:

[0032] Acquire a first transfer relationship matrix and a second transfer relationship matrix; the first relationship transfer matrix and the second relationship transfer matrix are determined according to a lane line data error sequence, and the lane line data error sequence is determined according to multiple frames of historical visual lane line data and multiple frames of historical high-precision map lane line data;

[0033] Determine a state transfer matrix of the Kalman filter according to the first transfer relationship matrix and the second transfer relationship matrix;

[0034] The predicted state vector corresponding to the current moment is determined using the state vector at the previous moment and the state transfer matrix; the predicted state vector is the product of the state vector at the previous moment and the state transfer matrix.

[0035] In the above solution, the prediction unit is specifically used for:

[0036] Determine the process noise covariance matrix of the Kalman filter according to the lane line data error sequence, the state transfer matrix and a preset unit matrix;

[0037] The state covariance matrix of the Kalman filter at the current moment is predicted according to the state covariance matrix at the previous moment, the state transfer matrix and the process noise covariance matrix to obtain the predicted covariance matrix corresponding to the current moment.

[0038] In the above solution, the device further includes: an updating unit, configured to:

[0039] If it is determined that the visual lane line data is in a valid state at the current moment, the Kalman gain matrix at the current moment is determined using the initialized observation matrix, the measurement noise covariance matrix, and the state covariance matrix at the previous moment;

[0040] Determine the observation vector of the Kalman filter at the current moment, and determine the updated state vector at the current moment according to the Kalman gain matrix at the current moment, the observation vector, the state vector of the Kalman filter at the previous moment of the current moment, and the initialized observation matrix;

[0041] The target lane line is determined according to the updated state vector and the visual lane line at the current moment.

[0042] In the above scheme, the update unit is used to:

[0043] According to the formula K = P × H T ×(H×P×H T +R) -1 Determining the Kalman gain matrix K;

[0044] Wherein, P is the state covariance matrix of the previous moment, H is the initialization observation matrix, and H T is the transposed matrix of the H, and R is the measurement noise covariance matrix.

[0045] In the above scheme, the update unit is used to:

[0046] The updated state vector State′ is determined according to the formula State′=State+K×(LH×State); wherein,

[0047] The State is the state vector of the Kalman filter at the previous moment before the current moment, the K is the Kalman gain matrix at the current moment, the L is the observation vector, and the H is the initialized observation matrix.

[0048] In the above scheme, the update unit is used to:

[0049] The updated state covariance matrix P' at the current moment is determined according to the formula P'=(EK×H)×P; wherein,

[0050] The E is the preset fourth-order unit matrix, the K is the Kalman gain matrix at the current moment, the H is the initialized observation matrix, and the P is the state covariance matrix at the previous moment.

[0051] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which performs the steps of any one of the methods described in the first aspect when the program is executed by a processor.

[0052] According to a fourth aspect of the present invention, a vehicle is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the methods described in the first aspect are implemented.

[0053] The present invention provides a lane line detection method, device and vehicle, the method comprising: if it is determined that the visual lane line data is invalid at the current moment, using the state vector of the previous moment to predict the state vector of the Kalman filter at the current moment, and obtaining the predicted state vector corresponding to the current moment; using the predicted state vector to correct the high-precision map lane line data at the current moment, and using the corrected high-precision map lane line data to determine the target lane line; in this way, when the visual lane line is valid, the target lane line can be directly determined according to the visual lane line data; once it is determined that the visual lane line data is invalid, the state vector of the Kalman filter is predicted to obtain the predicted state vector, and the predicted state vector is used to correct the high-precision map lane line data in real time to improve the positioning accuracy of the lane line; and the present invention only needs to fuse two data sources, the visual lane line data and the high-precision map, and can also reduce the hardware implementation cost and complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0055] Figure 1 A schematic diagram of a lane line detection method flow chart according to an embodiment of the present invention is shown;

[0056] Figure 2 A schematic structural diagram of a lane line detection device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0057] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0058] The present invention provides a lane line detection method, such as Figure 1 As shown, the method comprises the following steps:

[0059] S110: If it is determined that the visual lane line data is invalid at the current moment, the state vector of the Kalman filter at the current moment is predicted using the state vector at the previous moment to obtain a predicted state vector corresponding to the current moment.

[0060] Visual lane line data is collected using on-board cameras. In complex environments, such as heavy fog, rainy and snowy weather, or insufficient light at night, the visual lane line detection results may be unreliable (the visual lane line data is invalid in such scenarios). The lane line cannot be determined based on the visual lane line data, causing the vehicle to deviate from the center of the lane.

[0061] However, the lane line data of the high-precision map (LDMap, Local Dynamic Map) does not have the ability to make real-time dynamic adjustments and cannot reflect dynamic changes on the road in real time, resulting in insufficient positioning accuracy of the lane lines.

[0062] Based on this, the present invention provides a lane line detection method to ensure that the lane line position can be accurately output under various road conditions. For example, when it is determined that the visual lane line is invalid at the current moment, the state vector of the Kalman filter at the current moment is predicted using the state vector at the previous moment to obtain the predicted state vector corresponding to the current moment.

[0063] When it is determined that the visual lane line data is in a valid state at the current moment, the target lane line is directly determined based on the visual lane line data.

[0064] In one implementation, if it is determined that the visual lane line data is valid at the current moment, the method further includes:

[0065] Determine the Kalman gain matrix at the current moment using the initialized observation matrix, the measurement noise covariance matrix and the state covariance matrix at the previous moment;

[0066] Determine the observation vector of the Kalman filter at the current moment, and determine the updated state vector at the current moment based on the Kalman gain matrix at the current moment, the observation vector, the state vector of the Kalman filter at the previous moment before the current moment, and the initialized observation matrix.

[0067] Specifically, before the lane detection strategy is executed each time the vehicle is started, the measurement noise covariance matrix, the initial state covariance matrix, the state transfer matrix, and the process noise covariance matrix need to be determined based on the historical lane data. When the lane detection strategy is subsequently executed, the state covariance matrix is ​​continuously updated, while the measurement noise covariance matrix, the state transfer matrix, and the process noise covariance matrix can be fixed.

[0068] Therefore, if the current moment is the initial moment, the state covariance matrix used in determining the Kalman gain matrix at the current moment is the state covariance matrix at the initial moment; if the current moment is not the initial moment, the state covariance matrix used in determining the Kalman gain matrix at the current moment is the state covariance matrix at the previous moment.

[0069] In the present invention, the implementation process of determining the measurement noise covariance matrix, the state covariance matrix at the initial moment, the state transfer matrix and the process noise covariance matrix according to the historical lane line data is as follows:

[0070] The historical lane line data includes multiple frames of historical visual lane line data and multiple frames of historical high-precision map lane line data. The lane line data error sequence can be determined based on the multiple frames of historical visual lane line data and the multiple frames of historical high-precision map lane line data, and the mean of the lane line data error sequence can be determined;

[0071] The state covariance matrix and the measurement noise covariance matrix of the Kalman filter at the initial moment are determined according to the lane line data error sequence and the mean value of the lane line data error sequence.

[0072] Specifically, the visual lane lines and high-precision map lane lines are both fitted using cubic polynomials. For example, the cubic polynomial of the visual lane line corresponds to the equation: y = c3x 3 +c2x2 +c1x+c0, the equation corresponding to the cubic polynomial of the lane line of the high-precision map is: y′=c3′x 3 +c′2x 2 +c1′x+c′0. Among them, c0, c1, c2, c3 are the coefficients of the cubic polynomial of the visual lane line, and c′3, c′2, c1′, c′0 are the coefficients of the cubic polynomial of the high-precision map lane line.

[0073] Then each frame of historical visual lane line data is a four-dimensional vector [c3, c2, c1, c0], and each frame of historical high-precision map lane line data is a four-dimensional vector [c3′, c′2, c1′, c′0]. The present invention can obtain multiple frames of historical visual lane line data and multiple frames of historical high-precision map lane line data, and then determine the lane line data error sequence.

[0074] For example, if you need to obtain 20 frames of lane line data, you can obtain any continuous 20 frames of lane line data in the historical period. The 20 frames of historical visual lane line data obtained can be:

[0075] [[-1.522491,0.001355,-0.000193,0.000002,1],

[0076] [-1.519787,0.001071,-0.000190,0.000002,1],

[0077] [-1.519324,0.000953,-0.000190,0.000002,1],

[0078] [-1.516590,0.000593,-0.000178,0.000001,1],

[0079] [-1.507254,-0.000575,-0.000163,0.000002,1],

[0080] [-1.507332,-0.000436,-0.000138,0.000001,1],

[0081] [-1.506478,-0.001451,-0.000107,0.000001,1],

[0082] [-1.501855,-0.002134,-0.000081,0.000000,1],

[0083] [-1.495637,-0.003238,-0.000051,0.000000,1],

[0084] [-1.493939,-0.004710,-0.000056,0.000000,1],

[0085] [-1.484918,-0.006370,-0.000055,0.000000,1],

[0086] [-1.497364,-0.006658,-0.000030,-0.000000,1],

[0087] [-1.533787,-0.014937,-0.000011,-0.000000,1],

[0088] [-1.531881,-0.015492,-0.000017,-0.000000,1],

[0089] [-1.541503,-0.015629,-0.000020,-0.000000,1],

[0090] [-1.547832,-0.015443,-0.000018,-0.000000,1],

[0091] [-1.562501,-0.015493,-0.000005,-0.000001,1],

[0092] [-1.559824,-0.016138,0.000010,-0.000001,1],

[0093] [-1.567866,-0.017526,0.000014,-0.000001,1],

[0094] [-1.570176,-0.017893,0.000021,-0.000001,1].

[0095] In the above historical visual lane line data, the meanings of the five parameters respectively represent c0, c1, c2, c3 and valid flag 1.

[0096] The 20 frames of historical high-precision map lane line data are:

[0097] [[-1.271362,-0.001243,0.000148,0.000001,1],

[0098] [-1.268023,-0.001393,0.000148,0.000001,1],

[0099] [-1.264550,-0.001528,0.000149,0.000001,1],

[0100] [-1.272047,-0.001600,0.000149,0.000001,1],

[0101] [-1.268416,-0.001671,0.000150,0.000001,1],

[0102] [-1.267695,-0.001738,0.000151,0.000001,1],

[0103] [-1.274840,-0.001832,0.000152,0.000001,1],

[0104] [-1.267922,-0.001858,0.000153,0.000001,1],

[0105] [-1.271947,-0.001971,0.000154,0.000001,1],

[0106] [-1.275945,-0.002094,0.000154,0.000001,1],

[0107] [-1.276683,-0.002365,0.000155,0.000001,1],

[0108] [-1.272162,-0.003535,0.000157,0.000001,1],

[0109] [-1.294430,-0.009031,0.000161,0.000001,1],

[0110] [-1.301673,-0.009813,0.000162,0.000001,1],

[0111] [-1.298685,-0.011651,0.000163,0.000001,1],

[0112] [-1.300286,-0.013229,0.000165,0.000001,1],

[0113] [-1.306452,-0.013740,0.000166,0.000001,1],

[0114] [-1.302452,-0.013551,0.000168,0.000001,1],

[0115] [-1.297247,-0.013258,0.000169,0.000001,1],

[0116] [-1.307487,-0.012869,0.000171,0.000001,1]].

[0117] In the above historical high-precision map lane line data, the meanings of the five parameters represent c′0, c1′, c′2, c′3 and valid flag 1 respectively.

[0118] Then, the lane line data error sequence is determined based on the above 20 frames of historical visual lane line data and 20 frames of historical high-precision map lane line data. For example, the difference between the first four parameters of the historical visual lane line data and the historical high-precision map lane line data at the same acquisition time is defined as the lane line data error. Assume that the four-dimensional vector corresponding to the lane line data error is e i , i is any lane line data error, then the lane line data error sequence is {e1,e2...e i ...e N}, N = 20, N is the total number of lane line data errors.

[0119] Then, the mean value e of the lane line data error sequence is calculated according to formula (1):

[0120]

[0121] Due to e i is a four-dimensional vector, then the final mean is is also a four-dimensional vector.

[0122] Then, the state covariance matrix and the measurement noise covariance matrix of the Kalman filter at the initial moment can be determined according to the lane line data error sequence and the mean of the lane line data error sequence, which can be implemented as follows:

[0123] The state covariance matrix P1 of the Kalman filter at the initial moment is determined according to formula (2), and the measurement noise covariance matrix R of the Kalman filter is determined according to formula (3);

[0124]

[0125] In formula (2) and formula (3), T is the transposed matrix identifier, and I4 is the fourth-order identity matrix

[0126] It should be noted that when determining the state covariance matrix at the initial moment, only M frames of data are needed, and M can be half of N. When calculating the measurement noise covariance matrix, N frames of data are needed. To ensure the positive definiteness of the state covariance matrix and the measurement noise covariance matrix at the initial moment, an offset (for example, 0.1) needs to be added to the diagonal of the state covariance matrix at the initial moment, and an offset (for example, 0.001) needs to be added to the diagonal of the measurement noise covariance matrix.

[0127] Continuing with the above example, the state covariance matrix P1 at the initial moment determined based on the above actual data is:

[0128]

[0129] Similarly, the measurement noise covariance matrix R can be obtained as:

[0130]

[0131] Then, the first transfer relationship matrix and the second transfer relationship matrix are determined according to the lane line data error sequence. The first transfer relationship matrix can be determined according to formula (4), and the second transfer relationship matrix can be determined according to formula (5):

[0132]

[0133] The state transfer matrix of the Kalman filter is determined according to the first transfer relationship matrix and the second transfer relationship matrix, as shown in formula (6):

[0134]

[0135] In formula (6), det(A) represents the determinant value of the first transfer relationship matrix A, and I4 is a fourth-order unit matrix.

[0136] Following the above example, the determined state transfer matrix F is:

[0137]

[0138] The implementation method of determining the process noise covariance matrix Q of the Kalman filter based on the lane line data error sequence, the state transfer matrix and the preset unit matrix is ​​shown in formula (7):

[0139]

[0140] In formula (11), S = N-1, F is the state transfer matrix, e iis the data error of the i-th lane line, e i+1 is the i+1th lane line data error.

[0141] Continuing with the above example, the process noise covariance matrix Q of the Kalman filter is determined as:

[0142]

[0143] After the state covariance matrix and the measurement noise covariance matrix at the initial moment are determined, if the current moment is the initial moment and the visual lane line data at the current moment is in a valid state, the Kalman gain matrix at the initial moment can be determined according to the initialized observation matrix, the measurement noise covariance matrix and the state covariance matrix at the initial moment;

[0144] Determine the observation vector of the Kalman filter at the initial moment, and determine the updated state vector at the initial moment based on the Kalman gain matrix at the initial moment, the observation vector at the initial moment, the preset initial state vector (preset value) of the Kalman filter and the initialized observation matrix.

[0145] Similarly, if the current moment is not the initial moment and the visual lane line data at the current moment is in a valid state, the Kalman gain matrix at the current moment can also be determined according to the initialized observation matrix, the measurement noise covariance matrix and the state covariance matrix at the previous moment;

[0146] Determine the observation vector of the Kalman filter at the current moment, and determine the updated state vector at the current moment based on the Kalman gain matrix at the current moment, the observation vector, the state vector of the Kalman filter at the previous moment before the current moment, and the initialized observation matrix.

[0147] It should be noted that the formulas used to determine the Kalman gain matrix at the initial moment, the state vector at the initial moment, and the updated covariance matrix at the initial moment are the same as those used to determine the Kalman gain matrix at the non-initial moment, the state vector at the non-initial moment, and the updated covariance matrix at the non-initial moment. The only difference is that the covariance matrix used to determine the Kalman gain matrix at the initial moment, the state vector at the initial moment, and the updated covariance matrix at the initial moment is the covariance matrix at the initial moment determined based on the historical lane line data; while the covariance matrix used to determine the Kalman gain matrix at the non-initial moment, the state vector at the initial moment, and the updated covariance matrix at the initial moment is the covariance matrix at the previous moment.

[0148] The following takes the non-initial moment as an example to further illustrate the specific implementation process.

[0149] In one implementation, if the current moment is not the initial moment, the Kalman gain matrix at the current moment is determined using the initialized observation matrix, the measurement noise covariance matrix, and the state covariance matrix at the previous moment, including:

[0150] The Kalman gain matrix K is determined according to formula (8):

[0151] K=P×H T ×(H×P×H T +R) -1 (8)

[0152] In formula (8), P is the state covariance matrix of the previous moment, H is the initialized observation matrix (which is a 4th-order unit matrix), and H T is the transposed matrix of H, and R is the measurement noise covariance matrix.

[0153] Then determine the observation vector of the Kalman filter at the current moment, and then determine the updated state vector based on the Kalman gain matrix, the observation vector, the state vector of the Kalman filter at the previous moment of the current moment, and the initialized observation matrix, including:

[0154] The updated state vector State′ is determined according to formula (9):

[0155] State′=State+K×(LH×State) (9)

[0156] In formula (9), State is the state vector of the Kalman filter at the previous moment, K is the Kalman gain matrix, L is the observation vector at the current moment, and H is the initialized observation matrix. The observation vector refers to the actual difference between the cubic polynomial coefficients corresponding to the visual lane line data at the current moment and the lane line data on the high-precision map.

[0157] In addition, it can be seen from formula (9) that the Kalman gain matrix is ​​needed when the state vector is updated, and it can be seen from formula (8) that the update of the Kalman gain matrix is ​​related to the covariance matrix. Therefore, in order to continuously update the state vector, after determining the updated state vector at the current moment according to the Kalman gain matrix, the observation vector, the state vector of the Kalman filter at the previous moment of the current moment, and the initialized observation matrix, the method also includes:

[0158] According to formula (10), the updated state covariance matrix P′ at the current moment is determined:

[0159] P′=(EK×H)×P (10)

[0160] In formula (10), E is the preset fourth-order unit matrix, K is the Kalman gain matrix, H is the initialized observation matrix, and P is the state covariance matrix at the previous moment.

[0161] In this way, if the visual lane line data at each moment is in a valid state, the state vector of the Kalman filter can be continuously updated according to formulas (8) to (10) to obtain the updated state vector at each moment.

[0162] It should be noted that if the visual lane line at the current moment is in a valid state and the visual lane line at the next moment is in an invalid state, then the state vector at the current moment used to determine the predicted state vector at the next moment using the following formulas (11) and (12) is the updated state vector corresponding to the current moment determined according to formula (8), and the covariance matrix at the current moment used to determine the predicted covariance matrix at the next moment is the updated covariance matrix at the current moment determined according to formula (10).

[0163] That is to say, if the visual lane line at the current moment is in a valid state, the determined updated state vector and updated state covariance matrix are mainly for prediction, and the target lane line at the current moment can be directly determined based on the visual lane line data at the current moment.

[0164] In another embodiment, if it is determined that the visual lane line is invalid at the current moment, the target lane line position cannot be determined based on the visual lane line data, and the target lane line needs to be determined using high-precision map lane line data. However, since the high-precision map lane line data cannot reflect the dynamic changes on the road in real time, it is necessary to use the state vector of the previous moment to predict the state vector of the Kalman filter at the current moment to obtain the predicted state vector corresponding to the current moment. The predicted state vector is then used to correct the high-precision map lane line data at the current moment, and the target lane line is determined using the corrected high-precision map lane line data.

[0165] In one implementation, the state vector of the Kalman filter at the current moment is predicted using the state vector at the previous moment to obtain the predicted state vector corresponding to the current moment, including:

[0166] Obtaining a first transfer relationship matrix and a second transfer relationship matrix; the first relationship transfer matrix and the second relationship transfer matrix are determined according to a lane line data error sequence, and the lane line data error sequence is determined according to multiple frames of historical visual lane line data and multiple frames of historical high-precision map lane line data;

[0167] Determine a state transfer matrix of the Kalman filter according to the first transfer relationship matrix and the second transfer relationship matrix;

[0168] The predicted state vector corresponding to the current moment is determined using the state vector and state transfer matrix of the previous moment; the predicted state vector is the product of the state vector of the previous moment and the state transfer matrix.

[0169] The method for determining the state transfer matrix has been described in detail above and will not be repeated here.

[0170] Then, the predicted state vector State″ corresponding to the current moment is determined according to the state vector at the previous moment and the state transfer matrix, as shown in formula (11):

[0171] State″=F×State (11)

[0172] In formula (10), State is the state vector at the previous moment.

[0173] In addition, after predicting the state vector of the Kalman filter at the current moment using the state vector at the previous moment to obtain the predicted state vector corresponding to the current moment, the method further includes:

[0174] Determine the process noise covariance matrix of the Kalman filter according to the lane line data error sequence, the state transfer matrix and the preset unit matrix;

[0175] The state covariance matrix of the Kalman filter at the current moment is predicted according to the state covariance matrix, transfer matrix and process noise covariance matrix at the previous moment to obtain the predicted covariance matrix corresponding to the current moment.

[0176] The state covariance matrix of the Kalman filter at the current moment is predicted based on the state covariance matrix, transfer matrix and process noise covariance matrix at the previous moment, and the implementation method of obtaining the prediction covariance matrix P″ corresponding to the current moment is shown in formula (12):

[0177] P″=F×P×F T +Q (12)

[0178] In formula (12), F is the state transfer matrix, P is the state covariance matrix at the previous moment, and F T is the transposed matrix of the state transfer matrix.

[0179] In this way, if the visual lane line data at each moment is invalid, the predicted state vector and predicted covariance matrix of the Kalman filter can be continuously predicted according to formulas (11) to (12), and then the target lane line can be determined according to the predicted state vector.

[0180] In summary, if the current time is not the initial time, this application may include four scenarios, namely:

[0181] The first type: the visual lane line at the current moment is valid, and the visual lane line at the previous moment is valid;

[0182] The second type: the visual lane line at the current moment is valid, and the visual lane line at the previous moment is invalid;

[0183] The third type: the visual lane line at the current moment is invalid, and the visual lane line at the previous moment is valid;

[0184] The fourth type: the visual lane line at the current moment is invalid, and the visual lane line at the previous moment is invalid.

[0185] For the above four scenarios, if the visual lane line data at the current moment is in a valid state, the updated state vector and updated state covariance matrix at the current moment are determined according to formulas (8) to (10); if the visual lane line at the previous moment is in a valid state, the state vector and state covariance matrix at the previous moment used in determining the updated state vector and updated state covariance matrix at the current moment are also determined according to formulas (8) to (10). If the visual lane line at the previous moment is in an invalid state, the state vector and state covariance matrix at the previous moment used in determining the updated state vector and updated state covariance matrix at the current moment are determined according to formulas (11) and (12).

[0186] If the visual lane line data at the current moment is invalid, the predicted state vector and predicted state covariance matrix at the current moment can be determined according to (11) to (12); if the visual lane line at the previous moment is valid, the state vector and state covariance matrix at the previous moment used in determining the predicted state vector and predicted state covariance matrix at the current moment are also determined according to formulas (8) to (10). If the visual lane line at the previous moment is invalid, the state vector and state covariance matrix at the previous moment used in determining the predicted state vector and predicted state covariance matrix at the current moment are determined according to formulas (11) and (12).

[0187] S111, using the predicted state vector to correct the high-precision map lane line data at the current moment, and using the corrected high-precision map lane line data to determine the target lane line.

[0188] After the predicted state vector is determined, the predicted state vector is also a four-dimensional vector, and the predicted state vector can be used to correct the high-precision map lane line data at the current moment, and the target lane line can be determined using the corrected high-precision map lane line data.

[0189] Specifically, the element values ​​in the predicted state vector can be added to the coefficients of the cubic polynomial corresponding to the high-precision map lane line data to obtain the corrected cubic polynomial corresponding to the high-precision map lane line data, and the target lane line can be output according to the updated cubic polynomial.

[0190] For example, if the predicted state vector is [g1, g2, g3, g4], and the coefficients of the cubic polynomial corresponding to the high-precision map lane line data are [c3′, c′2, c1′, c′0], then the cubic polynomial corresponding to the updated high-precision map lane line data is: y′=(c3′+g1)x 3 +(c′2+g2)x 2 +(c1′+g3)x+c′0+g4.

[0191] In this way, when the visual lane line data is valid, the target lane line can be determined directly using the visual lane line data; even when the visual lane line data is invalid, the high-precision map lane line data can be dynamically corrected to accurately determine the target lane line, ensuring the output accuracy and stability of the target lane line under various conditions.

[0192] Based on the same inventive concept as in the above-mentioned embodiment, this embodiment also provides a lane line detection device, such as Figure 2 As shown, the device comprises:

[0193] A prediction unit 21 is used to predict the state vector of the Kalman filter at the current moment by using the state vector at the previous moment if it is determined that the visual lane line data is in an invalid state at the current moment, so as to obtain a predicted state vector corresponding to the current moment;

[0194] The correction unit 22 is used to correct the high-precision map lane line data at the current moment using the predicted state vector, and determine the target lane line using the corrected high-precision map lane line data.

[0195] Since the device introduced in the embodiment of the present invention is a device used to implement the lane line detection method of the embodiment of the present invention, based on the method introduced in the embodiment of the present invention, the person skilled in the art can understand the specific structure and deformation of the device, so it is not repeated here. All devices used in the method of the embodiment of the present invention belong to the scope of protection of the present invention.

[0196] Based on the same inventive concept, the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any of the methods described above are implemented.

[0197] Based on the same inventive concept, this embodiment provides a vehicle, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, any step of the method described above is implemented.

[0198] Through one or more embodiments of the present invention, the present invention has the following beneficial effects or advantages:

[0199] The present invention provides a lane line detection method, device, medium and vehicle, the method comprising:

[0200] If it is determined that the visual lane line data is invalid at the current moment, the state vector of the Kalman filter at the current moment is predicted using the state vector of the previous moment to obtain the predicted state vector corresponding to the current moment; the high-precision map lane line data at the current moment is corrected using the predicted state vector, and the target lane line is determined using the corrected high-precision map lane line data; in this way, when the visual lane line is valid, the target lane line can be directly determined based on the visual lane line data; once it is determined that the visual lane line data is invalid, the state vector of the Kalman filter is predicted to obtain the predicted state vector, and the high-precision map lane line data is corrected in real time using the predicted state vector to improve the positioning accuracy of the lane line; and the present invention only needs to fuse two data sources, the visual lane line data and the high-precision map, which can also reduce the cost and complexity of hardware implementation.

[0201] Compared with the prior art, the lane line detection method, device, medium and vehicle provided by the present invention can bring the following beneficial effects:

[0202] (1) Improving the robustness of lane detection: The multi-information fusion of visual lane lines, high-precision map lane lines, and Kalman filters effectively improves the robustness of lane detection, especially in low light or camera obstruction. The stability of lane centering control can still be guaranteed.

[0203] (2) Reduce hardware complexity and cost: Compared with existing solutions that rely on the fusion of multiple sensors, the present invention mainly uses two data sources: camera data and high-precision maps, which reduces the hardware complexity and cost of the system, making the solution more suitable for practical applications and convenient for large-scale mass production.

[0204] (3) Improve real-time performance and accuracy: Through the state prediction and error correction of the Kalman filter, the lane lines of the high-precision map are corrected in real time, which improves the accuracy of lane positioning. In particular, when the lane line data changes dynamically, the lane line detection can still maintain a high accuracy.

[0205] (4) Ensure the continuity of the lane centering function: When the visual lane lines are not available (invalid state), it can automatically switch to using the high-precision map lane lines for detection and correct the high-precision map lane line data to ensure the continuity and stability of the lane centering assist function and improve the driving safety of the vehicle.

[0206] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0207] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A lane line detection method, characterized in that: The method comprises: If it is determined that the visual lane line data is in an invalid state at the current moment, the state vector of the Kalman filter at the current moment is predicted using the state vector at the previous moment to obtain a predicted state vector corresponding to the current moment; The predicted state vector is used to correct the high-precision map lane line data at the current moment, and the target lane line is determined using the corrected high-precision map lane line data.

2. The method according to claim 1, characterized in that The method of using the state vector at the previous moment to predict the state vector of the Kalman filter at the current moment to obtain the predicted state vector corresponding to the current moment includes: Acquire a first transfer relationship matrix and a second transfer relationship matrix; the first relationship transfer matrix and the second relationship transfer matrix are determined according to a lane line data error sequence, and the lane line data error sequence is determined according to multiple frames of historical visual lane line data and multiple frames of historical high-precision map lane line data; Determine a state transfer matrix of the Kalman filter according to the first transfer relationship matrix and the second transfer relationship matrix; The predicted state vector corresponding to the current moment is determined using the state vector at the previous moment and the state transfer matrix; the predicted state vector is the product of the state vector at the previous moment and the state transfer matrix.

3. The method according to claim 2, characterized in that After determining the predicted state vector corresponding to the current moment by using the state vector at the previous moment and the state transfer matrix, the method further includes: Determine the process noise covariance matrix of the Kalman filter according to the lane line data error sequence, the state transfer matrix and a preset unit matrix; The state covariance matrix of the Kalman filter at the current moment is predicted according to the state covariance matrix at the previous moment, the state transfer matrix and the process noise covariance matrix to obtain the predicted covariance matrix corresponding to the current moment.

4. The method according to claim 1, characterized in that The method further comprises: If it is determined that the visual lane line data is in a valid state at the current moment, the Kalman gain matrix at the current moment is determined using the initialized observation matrix, the measurement noise covariance matrix, and the state covariance matrix at the previous moment; Determine the observation vector of the Kalman filter at the current moment, and determine the updated state vector at the current moment based on the Kalman gain matrix at the current moment, the observation vector, the state vector of the Kalman filter at the moment before the current moment, and the initialized observation matrix.

5. The method according to claim 4, characterized in that The method of determining the Kalman gain matrix at the current moment by using the initialized observation matrix, the measurement noise covariance matrix, and the state covariance matrix at the previous moment includes: According to the formula K = P × H T ×(H×P×H T +R) -1 Determining the Kalman gain matrix K; Wherein, P is the state covariance matrix of the previous moment, H is the initialization observation matrix, and H T is the transposed matrix of the H, and R is the measurement noise covariance matrix.

6. The method according to claim 4, characterized in that The step of determining the updated state vector at the current moment according to the Kalman gain matrix at the current moment, the observation vector, the state vector of the Kalman filter at the previous moment before the current moment, and the initialized observation matrix comprises: The updated state vector State′ is determined according to the formula State′=State+K×(LH×State); wherein, The State is the state vector of the Kalman filter at the previous moment before the current moment, the K is the Kalman gain matrix at the current moment, the L is the observation vector, and the H is the initialized observation matrix.

7. The method according to claim 4, characterized in that After determining the updated state vector at the current moment according to the Kalman gain matrix at the current moment, the observation vector, the state vector of the Kalman filter at the previous moment of the current moment, and the initialized observation matrix, the method further includes: The updated state covariance matrix P' at the current moment is determined according to the formula P'=(EK×H)×P; wherein, The E is the preset fourth-order unit matrix, the K is the Kalman gain matrix at the current moment, the H is the initialized observation matrix, and the P is the state covariance matrix at the previous moment.

8. A lane line detection device, characterized in that: The device comprises: A prediction unit, configured to predict a state vector of the Kalman filter at the current moment using a state vector at a previous moment if it is determined that the visual lane line data is in an invalid state at the current moment, to obtain a predicted state vector corresponding to the current moment; The correction unit is used to correct the high-precision map lane line data at the current moment using the predicted state vector, and determine the target lane line using the corrected high-precision map lane line data.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A vehicle comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images