Lane line determination method and device, vehicle and storage medium
By fusing the output information and historical recognition results of multiple lane detection models, and utilizing noise distribution characteristics and the Kalman filter algorithm, the problem of low recognition accuracy and stability of lane detection models is solved, achieving higher detection accuracy and robustness.
Patent Information
- Application Number
- CN202410592974.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-13
- Publication Date
- 2025-11-18
AI Technical Summary
In existing technologies, lane detection models have low recognition accuracy and stability, and cannot effectively combine the advantages of multiple lane detection models.
By acquiring the output information of multiple lane detection models and historical lane recognition results, and performing fusion processing, the lane parameters at the target time are determined, including noise distribution characteristics and confidence analysis. The Kalman filter algorithm is used for data correction, integrating the advantages of multiple models.
It improves the stability and accuracy of lane line detection, enhances robustness, integrates the advantages of multiple models, and improves the overall performance of lane line detection.
Smart Images
Figure CN120976880A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of autonomous driving technology, and in particular to a method, apparatus, vehicle, and storage medium for determining lane lines. Background Technology
[0002] Lane detection is a core function of autonomous driving systems. It uses computer recognition algorithms to identify lane markings on the road and determine the vehicle's position within the lane. Based on this information, it controls the vehicle's steering to keep it within the lane.
[0003] There are various lane detection models available on the market, all of which can be used for lane detection, and each model has its own advantages. However, currently, lane detection on the same vehicle is based on the output of a single lane detection model, failing to combine the advantages of multiple models, resulting in relatively low accuracy and stability. Summary of the Invention
[0004] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, vehicle, and storage medium for determining lane lines.
[0005] According to a first aspect of the present disclosure, a method for determining lane lines is provided, applied to a vehicle, the method comprising:
[0006] Obtain multiple lane line information corresponding to the target time from the output of multiple lane line detection models; where different lane line information corresponds to different lane line detection models;
[0007] Obtain the historical lane line recognition results corresponding to the previous time step of the target time;
[0008] Based on the historical lane line recognition results, the multiple lane line information are fused to obtain the target lane line parameters corresponding to the target time.
[0009] The target lane line is determined based on the target lane line parameters.
[0010] Optionally, the step of fusing the multiple lane line information based on the historical lane line recognition results to obtain the target lane line parameters corresponding to the target time includes:
[0011] Based on the multiple lane line information, determine the lane line detection point corresponding to the target time.
[0012] Based on the lane line detection points, the noise distribution characteristics corresponding to the target time are determined, and the noise distribution characteristics are used to reflect the distribution characteristics of noise data in the lane line detection points;
[0013] The target lane line parameters are determined based on the lane line detection points, the historical lane line identification results, and the noise distribution characteristics.
[0014] Optionally, each lane line information includes multiple lane line points, first location information corresponding to each lane line point, and a confidence level corresponding to each lane line point. The confidence level is used to characterize the degree of confidence that the lane line point is a point on the target lane line. Determining the lane line detection point corresponding to the target time based on the multiple lane line information includes:
[0015] Based on the confidence level and first location information corresponding to each lane line point, the lane line detection point is determined from the plurality of lane line points.
[0016] Optionally, determining the lane line detection point from the plurality of lane line points based on the confidence level and first location information corresponding to each lane line point includes:
[0017] Determine the first lane line point with the highest confidence level from among the multiple lane line points;
[0018] Based on the first location information corresponding to each lane line point, the lane line point located outside the target range among the multiple lane line points is designated as the second lane line point, and the target range is the area determined with the first lane line point as the center.
[0019] The second lane line point is used as a new plurality of lane line points, and the steps of determining the first lane line point with the highest confidence from the plurality of lane line points to using the second lane line point as a new plurality of lane line points are repeated until there are no lane line points outside the target range, and the obtained first lane line point is used as the lane line detection point.
[0020] Optionally, determining the noise distribution characteristics corresponding to the target time based on the lane detection points includes:
[0021] Based on the second location information corresponding to the lane line detection point, the lane line detection point is divided into multiple lane line point sets;
[0022] For each set of lane line points, the noise distribution function corresponding to each lane line detection model in the set of lane line points is determined sequentially. The distribution function is used to characterize the noise distribution features corresponding to the target time.
[0023] Optionally, determining the noise distribution function corresponding to each lane line detection model in each lane line point set includes:
[0024] For each lane line detection model, the lane line detection points corresponding to the lane line detection model in the lane line point set are taken as the third lane line points;
[0025] Based on the third lane line point, determine the noise distribution function corresponding to the lane line detection model.
[0026] Optionally, determining the noise distribution function corresponding to the lane line detection model based on the third lane line point includes:
[0027] Determine the target weight corresponding to the third lane line point in the lane line point set;
[0028] Based on the third position information corresponding to the third lane line point and the target weight, the noise distribution function corresponding to the lane line detection model is determined.
[0029] Optionally, determining the target weight corresponding to the third lane line point in the set of lane line points includes:
[0030] Determine the first number corresponding to the third lane line point, and the second number of lane line detection points included in the lane line point set;
[0031] The target weight is determined based on the ratio of the first quantity to the second quantity.
[0032] Optionally, determining the target lane line parameters based on the lane line detection points, the historical lane line recognition results, and the noise distribution characteristics includes:
[0033] Based on the historical lane line identification results and the noise distribution characteristics, the estimated values of the lane line parameters corresponding to the target time are determined;
[0034] The target lane line parameters are determined based on the lane line parameter estimates and the lane line detection points.
[0035] Optionally, determining the target lane line parameters based on the lane line parameter estimates and the lane line detection points includes:
[0036] The lane line detection points are corrected based on the estimated lane line parameters.
[0037] The target lane line parameters are determined based on the corrected lane line detection points.
[0038] Optionally, the lane line information is obtained in the following way:
[0039] Obtain multiple detection data points, including lane lines, corresponding to the target time.
[0040] For each of the detected data, the detected data is input into the corresponding lane line detection model to obtain the lane line information output by the lane line detection model.
[0041] According to a second aspect of the present disclosure, a lane line determining device is provided, the device comprising:
[0042] The first acquisition module is configured to acquire multiple lane line information output by multiple lane line detection models at the target time; wherein, different lane line information corresponds to different lane line detection models;
[0043] The second acquisition module is configured to acquire the historical lane line recognition results corresponding to the previous time of the target time;
[0044] The fusion module is configured to fuse the multiple lane line information based on the historical lane line recognition results to obtain the target lane line parameters corresponding to the target time.
[0045] The determination module is configured to determine the target lane line based on the target lane line parameters.
[0046] Optionally, the fusion module is configured to: determine the lane detection point corresponding to the target time based on the multiple lane line information; determine the noise distribution characteristics corresponding to the target time based on the lane line detection point, wherein the noise distribution characteristics are used to reflect the distribution characteristics of noise data in the lane line detection point; and determine the target lane line parameters based on the lane line detection point, the historical lane line recognition results, and the noise distribution characteristics.
[0047] Optionally, each lane line information includes multiple lane line points, first location information corresponding to each lane line point, and confidence level corresponding to each lane line point. The confidence level is used to characterize the degree of confidence that the lane line point is a point on the target lane line. The fusion module is configured to determine the lane line detection point from the multiple lane line points based on the confidence level and first location information corresponding to each lane line point.
[0048] Optionally, the fusion module is configured to: determine a first lane line point with the highest confidence from a plurality of lane line points; based on the first location information corresponding to each lane line point, designate a lane line point located outside a target range from the plurality of lane line points as a second lane line point, wherein the target range is an area defined centered on the first lane line point; designate the second lane line point as a new plurality of lane line points, and repeat the steps from determining the first lane line point with the highest confidence from the plurality of lane line points to designating the second lane line point as a new plurality of lane line points, until no lane line points exist outside the target range, and designate the obtained first lane line point as the lane line detection point.
[0049] Optionally, the fusion module is configured to divide the lane line detection points into multiple lane line point sets according to the second location information corresponding to the lane line detection points; and for each lane line point set, sequentially determine the noise distribution function corresponding to each lane line detection model in the lane line point set, wherein the distribution function is used to characterize the noise distribution features corresponding to the target time.
[0050] Optionally, the fusion module is configured to, for each lane line detection model, take the lane line detection points corresponding to the lane line detection model in the lane line point set as third lane line points; and determine the noise distribution function corresponding to the lane line detection model based on the third lane line points.
[0051] Optionally, the fusion module is configured to determine the target weight corresponding to the third lane line point in the lane line point set; and to determine the noise distribution function corresponding to the lane line detection model based on the third position information corresponding to the third lane line point and the target weight.
[0052] Optionally, the fusion module is configured to determine a first number corresponding to the third lane line point and a second number of lane line detection points included in the lane line point set; and to determine the target weight based on the ratio of the first number to the second number.
[0053] Optionally, the fusion module is configured to determine the lane line parameter estimate corresponding to the target time based on the historical lane line recognition results and the noise distribution characteristics; and to determine the target lane line parameters based on the lane line parameter estimate and the lane line detection points.
[0054] Optionally, the fusion module is configured to correct the lane line detection points based on the lane line parameter estimates; and to determine the target lane line parameters based on the corrected lane line detection points.
[0055] Optionally, the lane line information is obtained in the following way:
[0056] Obtain multiple detection data points, including lane lines, corresponding to the target time.
[0057] For each of the detected data, the detected data is input into the corresponding lane line detection model to obtain the lane line information output by the lane line detection model.
[0058] According to a third aspect of the present disclosure, a vehicle is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the steps of the lane line determination method provided in the first aspect of the present disclosure when the executable instructions stored in the memory are invoked.
[0059] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the lane line determination method provided in the first aspect of the present disclosure.
[0060] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: acquiring multiple lane line information output by multiple lane line detection models corresponding to a target time; wherein different lane line information corresponds to different lane line detection models; acquiring historical lane line recognition results corresponding to the previous time of the target time; fusing the multiple lane line information according to the historical lane line recognition results to obtain target lane line parameters corresponding to the target time; and determining the target lane line according to the target lane line parameters. This disclosure can integrate multiple lane line detection models and acquire lane line information output by each lane line detection model at the target time, as well as historical lane line recognition results. Based on the historical lane line recognition results, the multiple lane line information is fused to obtain target lane line parameters corresponding to the target time. And based on the target lane line parameters, the target lane line is obtained. In this way, by fusing multiple lane line information, the advantages of multiple lane line models can be combined and maximized, improving the stability and accuracy of lane line detection. Furthermore, the multi-model fusion method has stronger robustness compared to a single lane line detection model scheme.
[0061] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0062] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0063] Figure 1 This is a flowchart illustrating a method for determining lane lines according to an exemplary embodiment.
[0064] Figure 2 This is a flowchart illustrating another method for determining lane lines according to an exemplary embodiment.
[0065] Figure 3 This is a flowchart illustrating another method for determining lane lines according to an exemplary embodiment.
[0066] Figure 4 This is a flowchart illustrating another method for determining lane lines according to an exemplary embodiment.
[0067] Figure 5 This is a block diagram illustrating a lane line determination device according to an exemplary embodiment.
[0068] Figure 6 This is a functional block diagram of a vehicle according to an exemplary embodiment. Detailed Implementation
[0069] Some embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a particular order. Furthermore, for clarity and brevity, descriptions of features known in the art may be omitted.
[0070] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0071] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily construed as indicating a specific order or sequence. Furthermore, in the description with reference to the accompanying drawings, the same reference numerals in different drawings denote the same elements. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features.
[0072] In the description of this disclosure, unless otherwise stated, "multiple" means two or more, and other quantifiers are similar; "at least one," "one or more," or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one 'a' can represent any number of 'a's; as another example, one or more of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.
[0073] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0074] Figure 1 This is a flowchart illustrating a method for determining lane lines according to an exemplary embodiment, such as... Figure 1 As shown, this method can be applied to vehicles, and the method may include the following steps:
[0075] In step S101, the lane line information corresponding to the target time is obtained from the output of multiple lane line detection models.
[0076] Different lane line information corresponds to different lane line detection models.
[0077] In some embodiments, the lane line information can be obtained by: first, acquiring multiple detection data points containing lane lines corresponding to the target time; then, for each detection data point, inputting the detection data into the corresponding lane line detection model to obtain the lane line information output by the lane line detection model.
[0078] Multiple detection data points can be acquired through different sensors, and the lane detection model can be any detection model capable of sensing lane lines. For example, image sensors installed on the vehicle can acquire image data containing lane lines along the vehicle's direction of travel; that is, the detection data includes image data. Alternatively, LiDAR sensors installed on the vehicle can acquire LiDAR data containing lane lines along the vehicle's direction of travel; that is, the detection data includes LiDAR data. Accordingly, the lane detection model can include visual segmentation models, 2D detection models, visual 3D detection models, LiDAR 3D detection models, etc. In practical applications, different lane detection models can be selected based on the type of detection data, and the detection data can be input into the corresponding lane detection model to obtain the lane line information output by the lane detection model.
[0079] It should be noted that the above examples are for illustrative purposes only, and this disclosure is not limited to the above methods.
[0080] In step S102, the historical lane line recognition result corresponding to the previous time of the target time is obtained.
[0081] The historical lane line recognition result is used to reflect the lane line recognition result corresponding to the target lane line identified at the previous time.
[0082] In step S103, based on the historical lane line recognition result, the multiple lane line information is fused to obtain the target lane line parameters corresponding to the target time.
[0083] In some embodiments, firstly, the noise distribution characteristics corresponding to noise data (i.e., lane line data that does not belong to the target lane line) in the multiple lane line information can be determined based on the multiple lane line information. Then, Kalman filtering can be performed on the multiple lane line information based on the noise distribution characteristics and historical lane line identification results to obtain the target lane line parameters corresponding to the target time.
[0084] In step S104, the target lane line is determined based on the target lane line parameters.
[0085] After obtaining the target lane line parameters, the target lane line parameters can be substituted into the corresponding lane line equation to obtain the lane line expression. This lane line expression can represent the location and shape of the target lane line, thus determining the target lane line.
[0086] For example, the lane line equation can be expressed as follows:
[0087]
[0088] Among them, k0, k1, k2 and k3 are the lane line parameters.
[0089] In this way, by fusing lane line information from multiple models, the advantages of each lane line detection model can be maximized. For example, the visual segmentation model has a better effect on straight line perception within 100 meters, while the laser 3D detection model can make more accurate judgments about space and has a certain guessing ability. By fusing the lane line information output by various lane line detection models, the advantages of each model can be combined to improve the stability and accuracy of lane line detection.
[0090] The above method integrates multiple lane detection models and acquires the lane information output by each model at the target time, as well as historical lane recognition results. Based on the historical lane recognition results, the multiple lane information sets are fused to obtain the target lane parameters corresponding to the target time. Then, the target lane line is obtained based on these parameters. In this way, by fusing multiple lane information sets, the advantages of multiple lane detection models can be combined and maximized, improving the stability and accuracy of lane detection. Furthermore, the multi-model fusion approach exhibits stronger robustness compared to a single lane detection model scheme.
[0091] The following is a detailed explanation of step S103 above, such as... Figure 2 As shown, step S103, based on the historical lane line recognition result, fuses the multiple lane line information to obtain the target lane line parameters corresponding to the target time, which may include the following steps:
[0092] In step S1031, the lane detection point corresponding to the target time is determined based on the multiple lane line information.
[0093] Each lane line information includes multiple lane line points, the first location information corresponding to each lane line point, and the confidence score corresponding to each lane line point. The confidence score is used to characterize the degree to which the lane line point is a point on the target lane line. For a lane line, different lane line detection models output a set of lane line points (i.e., lane line information) P. i Where 0 < i ≤ K, there are K lane line detection models, and each lane line point P ij Represents the coordinates (x, y) in the vehicle coordinate system. j y j And the confidence level c of each lane line point. j The vehicle coordinate system is a special moving coordinate system used to describe the vehicle's motion; its origin coincides with the center of mass. When the vehicle is stationary on a horizontal road, the X-axis points to the driver's left, the Y-axis is parallel to the ground and points in front of the vehicle, and the Z-axis passes through the vehicle's center of mass and points upward.
[0094] In some embodiments, a lane line detection point can be determined from a plurality of lane line points based on the confidence level and first location information corresponding to each lane line point. Here, the plurality of lane line points are all P... ij That is, the lane line points output by all lane line detection models.
[0095] Specifically, such as Figure 3 As shown, determining the lane line detection point from multiple lane line points based on the confidence level and first location information corresponding to each lane line point may include the following steps:
[0096] Step A: Determine the first lane line point with the highest confidence level from multiple lane line points.
[0097] In other words, all lane line points can be sorted from high to low confidence level, and the lane line point with the highest confidence level can be used as the first lane line point.
[0098] Step B: Based on the first position information corresponding to each lane line point, the lane line point located outside the target range among the multiple lane line points is designated as the second lane line point.
[0099] The target range is the area defined centered on the first lane line point. This area can be of any shape, such as a circle, rectangle, or square. For example, the target range can be defined as the area falling within the first lane line point with dimensions r. In this step, designating lane line points outside the target range as second lane line points is essentially equivalent to deleting lane line points within the target range. For instance, points falling within the area centered on the first lane line point with dimensions r can be deleted, and points outside this area can be designated as second lane line points.
[0100] Step C: Use the second lane line point as a new plurality of lane line points, and repeat the step of determining the first lane line point with the highest confidence from the plurality of lane line points until the second lane line point is used as a new plurality of lane line points, until there are no lane line points outside the target range, and use the obtained first lane line point as the lane line detection point.
[0101] In step B, lane line points within the target range are deleted. Next, the lane line point with the highest confidence level is selected from the remaining second lane line points, and steps A to C are repeated until there are no more lane line points to delete, i.e., no lane line points outside the target range. All the resulting first lane line points can be used as lane line detection points. In this way, the lane line points are filtered using the Non-Maximum Suppression (NMS) method, thereby removing redundant lane line points and retaining the lane line points most likely to be on the target lane, reducing the amount of subsequent data processing, and thus obtaining the lane line detection points.
[0102] In step S1032, the noise distribution characteristics corresponding to the target time are determined based on the lane detection point.
[0103] The noise distribution characteristic is used to reflect the distribution characteristics of noise data at the lane detection point. This step mainly aims to determine the distribution characteristics of the noise data at the lane detection point to facilitate subsequent filtering.
[0104] In some embodiments, as Figure 4 shown, determining the noise distribution characteristics corresponding to the target moment according to the lane line detection points in step S1032 may include the following steps:
[0105] In step S1032a, according to the second position information corresponding to the lane line detection points, the lane line detection points are divided into multiple lane line point sets.
[0106] Specifically, the lane line may be segmented along the y-axis (the forward direction of the vehicle may be used as the positive direction of the y-axis), and the lane line detection points are divided into N [y j , y j+1 , where 0 < j < N, obtaining N lane line point sets, and the following steps are all performed separately for each interval range.
[0107] In step S1032b, for each of the lane line point sets, the noise distribution function corresponding to each lane line detection model in the lane line point set is determined in sequence.
[0108] Among them, the distribution function is used to characterize the noise distribution characteristics corresponding to the target moment.
[0109] In some embodiments, it can be set that within each interval range (that is, within each lane line point set), the lane line detection points output by each lane line detection model follow a Gaussian distribution, that is, the x coordinates of the lane line detection points output by each lane line detection model follow a Gaussian distribution with a mean of μ and a variance of Σ. Among them, the mean of μ and the variance of Σ can be determined according to the x coordinates of the lane line detection points output by each lane line detection model:
[0110]
[0111] The above Gaussian distribution is set for a single model. For the lane line detection points of multiple models, in some embodiments, it can be set that the lane line detection points corresponding to multiple lane line detection models follow a Gaussian mixture model distribution, that is, the x coordinates of the lane line detection points corresponding to multiple lane line detection models follow the following distribution:
[0112]
[0113] Among them, a i is the target weight corresponding to each lane line detection model, which can be obtained from the proportion of the lane line detection points of the lane line detection model in the segmented interval.
[0114] Furthermore, determining the noise distribution function corresponding to each lane line detection model in the lane line point set for each of the lane line point sets may include:
[0115] (1) For each lane line detection model, the lane line detection points corresponding to the lane line detection model in the lane line point set are taken as the third lane line points.
[0116] (2) Based on the third lane line point, determine the noise distribution function corresponding to the lane line detection model.
[0117] For example, firstly, the target weight corresponding to the third lane line point in the set of lane line points can be determined. For instance, a first number corresponding to the third lane line point and a second number of lane line detection points included in the set of lane line points can be determined, and the target weight (i.e., a) can be determined based on the ratio of the first number to the second number. i Then, based on the third position information corresponding to the third lane line point and the target weight, the noise distribution function corresponding to the lane line detection model is determined.
[0118] Specifically, in some embodiments, the noise data at lane detection points can be set to follow a variance Σ. n Given a Gaussian distribution, the variance Σ of the lane detection points obtained in the above steps is used. i This is used to estimate the noise data at lane detection points, thus enabling subsequent filtering by the user. For example, the noise data at lane detection points may follow the following noise distribution function:
[0119]
[0120] In step S1033, the target lane line parameters are determined based on the lane line detection point, the historical lane line identification result, and the noise distribution characteristics.
[0121] Specifically, it may include the following steps:
[0122] (1) Based on the historical lane line identification results and the noise distribution characteristics, determine the estimated values of the lane line parameters corresponding to the target time.
[0123] In some embodiments, the estimated value of the lane line parameters corresponding to the target time can be determined by using a Kalman filter algorithm based on the historical lane line identification results and the noise distribution characteristics.
[0124] For example, the historical lane line identification result may include lane line parameters k0, k1, k2, and k3 corresponding to the previous time step of the target time. Determining the estimated lane line parameters using the Kalman filter algorithm may include the following steps:
[0125] Step 1: Calculate the prior lane line parameters k0, k1, k2, and k3 using the lane line state transition matrix F and the lane line parameters k0, k1, k2, and k3 from the previous time step at the target time.
[0126] The lane line state transition matrix F (which can be determined based on the second position information corresponding to the lane line detection point) may include:
[0127]
[0128] Step 2: Calculate the prior estimated covariance matrix P from the process excitation noise Q (which can be preset) at the previous time step of the target time. - ;
[0129] Step 3: Estimate the covariance matrix P from prior knowledge. - The Kalman gain matrix K is calculated using the transformation matrix H and the observation noise R.
[0130] The transformation matrix H (which can be determined based on the second position information corresponding to the lane line detection point) may include:
[0131]
[0132] The observed noise R (which can be determined based on the noise distribution function) may include:
[0133] R = [∑ n ]
[0134] Step 4: Calculate the posterior state estimate from the Kalman gain matrix K, which is the target lane line parameters k0, k1, k2 and k3 at the target time;
[0135] Step 5: Update the posterior estimate of covariance P; repeat steps 1 through 5.
[0136] The lane line parameters can be updated and estimated through the Kalman filtering process described above. The specific implementation of the Kalman filtering process can be found in the relevant technologies, and will not be elaborated here.
[0137] (2) Determine the target lane line parameters based on the estimated value of the lane line parameters and the lane line detection point.
[0138] For example, the equation for the lane line to be determined can be established based on the estimated lane line parameters. Then, the lane line detection point is corrected based on the equation. This corrects and filters out noise data in the lane line detection point. Finally, the target lane line parameters are determined based on the corrected lane line detection point. Specifically, the location information corresponding to the corrected lane line detection point can be substituted into the lane line equation described above to solve for the target lane line parameters.
[0139] The above method integrates multiple lane detection models and acquires the lane information output by each model at the target time, as well as historical lane recognition results. Based on the historical lane recognition results, the multiple lane information sets are fused to obtain the target lane parameters corresponding to the target time. Then, the target lane line is obtained based on these parameters. In this way, by fusing multiple lane information sets, the advantages of multiple lane detection models can be combined and maximized, improving the stability and accuracy of lane detection. Furthermore, the multi-model fusion approach exhibits stronger robustness compared to a single lane detection model scheme.
[0140] Figure 5 This is a block diagram illustrating a lane marking determination device according to an exemplary embodiment. (Refer to...) Figure 5 The device 200 includes:
[0141] The first acquisition module 201 is configured to acquire multiple lane line information output by multiple lane line detection models at a target time; wherein, different lane line information corresponds to different lane line detection models;
[0142] The second acquisition module 202 is configured to acquire the historical lane line recognition result corresponding to the previous time of the target time;
[0143] The fusion module 203 is configured to fuse the multiple lane line information based on the historical lane line recognition result to obtain the target lane line parameters corresponding to the target time.
[0144] The determination module 204 is configured to determine the target lane line based on the target lane line parameters.
[0145] Optionally, the fusion module 203 is configured to determine the lane detection point corresponding to the target time based on the multiple lane line information; determine the noise distribution characteristics corresponding to the target time based on the lane detection point, the noise distribution characteristics being used to reflect the distribution characteristics of noise data in the lane detection point; and determine the target lane line parameters based on the lane detection point, the historical lane line identification results, and the noise distribution characteristics.
[0146] Optionally, each lane line information includes multiple lane line points, first location information corresponding to each lane line point, and confidence level corresponding to each lane line point. The confidence level is used to characterize the degree of confidence that the lane line point is a point on the target lane line. The fusion module 203 is configured to determine the lane line detection point from multiple lane line points based on the confidence level and first location information corresponding to each lane line point.
[0147] Optionally, the fusion module 203 is configured to determine the first lane line point with the highest confidence from the plurality of lane line points; based on the first location information corresponding to each lane line point, take the lane line point located outside the target range from the plurality of lane line points as the second lane line point, the target range being the area range determined with the first lane line point as the center; take the second lane line point as the new plurality of lane line points, and repeat the steps from determining the first lane line point with the highest confidence from the plurality of lane line points to taking the second lane line point as the new plurality of lane line points, until there are no lane line points outside the target range, and take the obtained first lane line point as the lane line detection point.
[0148] Optionally, the fusion module 203 is configured to divide the lane line detection point into multiple lane line point sets according to the second position information corresponding to the lane line detection point; for each lane line point set, the noise distribution function corresponding to each lane line detection model in the lane line point set is determined in sequence, and the distribution function is used to characterize the noise distribution characteristics corresponding to the target time.
[0149] Optionally, the fusion module 203 is configured to, for each lane line detection model, take the lane line detection points corresponding to the lane line detection model in the lane line point set as the third lane line points; and determine the noise distribution function corresponding to the lane line detection model based on the third lane line points.
[0150] Optionally, the fusion module 203 is configured to determine the target weight corresponding to the third lane line point in the lane line point set; and to determine the noise distribution function corresponding to the lane line detection model based on the third position information corresponding to the third lane line point and the target weight.
[0151] Optionally, the fusion module 203 is configured to determine a first number corresponding to the third lane line point and a second number of lane line detection points included in the lane line point set; and to determine the target weight based on the ratio of the first number to the second number.
[0152] Optionally, the fusion module 203 is configured to determine the lane line parameter estimate corresponding to the target time based on the historical lane line recognition result and the noise distribution characteristics; and to determine the target lane line parameters based on the lane line parameter estimate and the lane line detection point.
[0153] Optionally, the fusion module 203 is configured to correct the lane line detection point based on the lane line parameter estimate; and to determine the target lane line parameters based on the corrected lane line detection point.
[0154] Optionally, the lane line information is obtained in the following way:
[0155] Acquire multiple detection data points, including lane lines, corresponding to the target time.
[0156] For each detection data point, the detection data is input into the corresponding lane line detection model to obtain the lane line information output by the lane line detection model.
[0157] Using the aforementioned device, multiple lane detection models can be integrated, and the lane information output by each lane detection model at the target time, as well as historical lane recognition results, can be acquired. Based on the historical lane recognition results, the multiple lane information sets are fused to obtain the target lane parameters corresponding to the target time. The target lane line is then obtained based on these parameters. In this way, by fusing multiple lane information sets, the advantages of multiple lane detection models can be combined and maximized, improving the stability and accuracy of lane detection. Furthermore, the multi-model fusion approach exhibits stronger robustness compared to a single lane detection model scheme.
[0158] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0159] Figure 6 This is a block diagram illustrating a vehicle 300 according to an exemplary embodiment. For example, vehicle 300 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicle. Vehicle 300 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.
[0160] Reference Figure 6 The vehicle 300 may include various subsystems, such as an infotainment system 310, a perception system 320, a decision control system 330, a drive system 340, and a computing platform 350. The vehicle 300 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of the vehicle 300 can be interconnected via wired or wireless means.
[0161] In some embodiments, the infotainment system 310 may include a communication system, an entertainment system, and a navigation system, etc.
[0162] The perception system 320 may include several sensors for sensing information about the environment surrounding the vehicle 300. For example, the perception system 320 may include a global positioning system (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.
[0163] The decision control system 330 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.
[0164] The drive system 340 may include components that provide powered motion to the vehicle 300. In one embodiment, the drive system 340 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.
[0165] Some or all of the functions of the vehicle 300 are controlled by a computing platform 350. The computing platform 350 may include at least one processor 351 and a memory 352, the processor 351 being able to execute instructions 353 stored in the memory 352.
[0166] Processor 351 can be any conventional processor, such as a commercially available CPU. The processor may also include, for example, a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), a System on Chip (SOC), an Application Specific Integrated Circuit (ASIC), or a combination thereof.
[0167] The memory 352 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0168] In addition to instruction 353, memory 352 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 352 can be used by computing platform 350.
[0169] In this embodiment of the disclosure, processor 351 may execute instruction 353 to complete all or part of the steps of the lane line determination method described above.
[0170] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the lane line determination method provided in this disclosure.
[0171] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the lane line determination method described above when executed by the programmable device.
[0172] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”
[0173] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding the specification and drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”
[0174] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.
[0175] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A method for determining lane lines, characterized in that, Applied to vehicles, the method includes: Obtain multiple lane line information corresponding to the target time from the output of multiple lane line detection models; where different lane line information corresponds to different lane line detection models; Obtain the historical lane line recognition results corresponding to the previous time step of the target time; Based on the historical lane line recognition results, the multiple lane line information are fused to obtain the target lane line parameters corresponding to the target time. The target lane line is determined based on the target lane line parameters.
2. The method according to claim 1, characterized in that, The step of fusing the multiple lane line information based on the historical lane line recognition results to obtain the target lane line parameters corresponding to the target time includes: Based on the multiple lane line information, determine the lane line detection point corresponding to the target time. Based on the lane line detection points, the noise distribution characteristics corresponding to the target time are determined, and the noise distribution characteristics are used to reflect the distribution characteristics of noise data in the lane line detection points; The target lane line parameters are determined based on the lane line detection points, the historical lane line identification results, and the noise distribution characteristics.
3. The method according to claim 2, characterized in that, Each lane line information includes multiple lane line points, first location information corresponding to each lane line point, and a confidence level corresponding to each lane line point. The confidence level is used to characterize the degree of confidence that the lane line point is a point on the target lane line. Determining the lane line detection point corresponding to the target time based on the multiple lane line information includes: Based on the confidence level and first location information corresponding to each lane line point, the lane line detection point is determined from the plurality of lane line points.
4. The method according to claim 3, characterized in that, The step of determining the lane line detection point from a plurality of lane line points based on the confidence level and first position information corresponding to each lane line point includes: Determine the first lane line point with the highest confidence level from among the multiple lane line points; Based on the first location information corresponding to each lane line point, the lane line point located outside the target range among the multiple lane line points is designated as the second lane line point, and the target range is the area determined with the first lane line point as the center. The second lane line point is used as a new plurality of lane line points, and the steps of determining the first lane line point with the highest confidence from the plurality of lane line points to using the second lane line point as a new plurality of lane line points are repeated until there are no lane line points outside the target range, and the obtained first lane line point is used as the lane line detection point.
5. The method according to claim 2, characterized in that, The step of determining the noise distribution characteristics corresponding to the target time based on the lane detection points includes: Based on the second location information corresponding to the lane line detection point, the lane line detection point is divided into multiple lane line point sets; For each set of lane line points, the noise distribution function corresponding to each lane line detection model in the set of lane line points is determined sequentially. The distribution function is used to characterize the noise distribution features corresponding to the target time.
6. The method according to claim 5, characterized in that, The step of determining the noise distribution function corresponding to each lane line detection model in each lane line point set includes: For each lane line detection model, the lane line detection points corresponding to the lane line detection model in the lane line point set are taken as the third lane line points; Based on the third lane line point, determine the noise distribution function corresponding to the lane line detection model.
7. The method according to claim 6, characterized in that, The step of determining the noise distribution function corresponding to the lane line detection model based on the third lane line point includes: Determine the target weight corresponding to the third lane line point in the lane line point set; Based on the third position information corresponding to the third lane line point and the target weight, the noise distribution function corresponding to the lane line detection model is determined.
8. The method according to claim 7, characterized in that, Determining the target weight corresponding to the third lane line point in the set of lane line points includes: Determine the first number corresponding to the third lane line point, and the second number of lane line detection points included in the lane line point set; The target weight is determined based on the ratio of the first quantity to the second quantity.
9. The method according to claim 2, characterized in that, The step of determining the target lane line parameters based on the lane line detection points, the historical lane line identification results, and the noise distribution characteristics includes: Based on the historical lane line identification results and the noise distribution characteristics, the estimated values of the lane line parameters corresponding to the target time are determined; The target lane line parameters are determined based on the lane line parameter estimates and the lane line detection points.
10. The method according to claim 9, characterized in that, Determining the target lane line parameters based on the lane line parameter estimates and the lane line detection points includes: The lane line detection points are corrected based on the estimated lane line parameters. The target lane line parameters are determined based on the corrected lane line detection points.
11. The method according to any one of claims 1 to 10, characterized in that, The lane line information is obtained in the following way: Obtain multiple detection data points, including lane lines, corresponding to the target time. For each of the detected data, the detected data is input into the corresponding lane line detection model to obtain the lane line information output by the lane line detection model.
12. A lane marking device, characterized in that, The device includes: The first acquisition module is configured to acquire multiple lane line information output by multiple lane line detection models at the target time; wherein, different lane line information corresponds to different lane line detection models; The second acquisition module is configured to acquire the historical lane line recognition results corresponding to the previous time of the target time; The fusion module is configured to fuse the multiple lane line information based on the historical lane line recognition results to obtain the target lane line parameters corresponding to the target time. The determination module is configured to determine the target lane line based on the target lane line parameters.
13. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the steps of the method according to any one of claims 1 to 11 when executable instructions stored in the memory are invoked.
14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 11.