A lane line fitting method and device, electronic equipment and storage medium

Abstract

This invention relates to the field of autonomous driving technology, and discloses a lane line fitting method, apparatus, electronic device, and storage medium. The method involves acquiring a lane line dataset at the current moment; this dataset includes the position of each sampling point among multiple sampling points located on the lane line; if the slope value of any two adjacent sampling points in the lane line dataset is greater than or equal to a first preset threshold, segmentation points are determined from the lane line dataset, and then a target lane line dataset is determined. Based on the target lane line dataset at the current moment, the vehicle motion state at the current moment, the vehicle running state at historical moments, and the predicted lane line state quantity at the current moment, an extended Kalman filter is performed to obtain the target lane line state quantity at the current moment; the target lane line at the current moment is determined based on the target lane line state quantity at the current moment. This invention provides a lane line fitting method with high accuracy and reliability in the fitting results.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a lane line fitting method, apparatus, electronic device, and storage medium. Background Technology

[0002] Lane fitting is a crucial parameter in autonomous driving technology, impacting vehicle safety and legality during operation. Existing solutions typically use cubic curves to fit lane lines; however, the characteristics of cubic curves make it difficult to represent curves with large curvatures when the longitudinal distance is large, resulting in low fitting accuracy. Summary of the Invention

[0003] To address the problems of existing technologies, this application provides a lane line fitting method, apparatus, electronic device, and storage medium. The technical solution is as follows:

[0004] On the one hand, a lane line fitting method is provided, including:

[0005] Obtain the lane line dataset at the current time; this lane line dataset includes the position of each of the multiple sampling points located on the lane lines;

[0006] If the slope value of any two adjacent sampling points in the lane line dataset is greater than or equal to the first preset threshold, a target sampling point is determined from the lane line dataset; the distance between the target sampling point and the vehicle is within the preset threshold range.

[0007] Based on the target sampling point, a target lane line dataset is determined from the lane line dataset; the distance between each sampling point in the target lane line dataset and the vehicle is less than the distance between the target sampling point and the vehicle.

[0008] Based on the target lane line dataset at the current moment, the vehicle motion state at the current moment, the vehicle operation state at historical moments, and the predicted lane line state at the current moment, an extended Kalman filter is performed to obtain the target lane line state at the current moment.

[0009] The target lane line at the current moment is determined based on the target lane line state variable at the current moment.

[0010] In one exemplary implementation, determining the target sampling point from the lane line dataset includes:

[0011] The lane line dataset is determined from the lane line dataset; the distance between each sampling point in the lane line dataset and the vehicle is greater than or equal to a first threshold and less than or equal to a second threshold; the second threshold is greater than the first threshold;

[0012] For each sampling point in the lane line dataset, the slope of that sampling point is determined based on its location and the locations of its neighboring sampling points.

[0013] If the lane line dataset meets the first preset condition, a target sampling point set is determined from the lane line dataset based on the slope of each sampling point in the lane line dataset; the slope of the sampling points in the target sampling point set is greater than zero or less than zero.

[0014] The target sampling point is determined based on the target sampling point set.

[0015] In one exemplary implementation, the first preset condition includes:

[0016] The sampling points in this lane line dataset are arranged from closest to furthest from the vehicle and belong to the first dataset, the second dataset, and the third dataset in sequence. The slope of each sampling point in the first dataset and the slope of the third dataset are both greater than zero, and the slope of each sampling point in the second dataset is less than zero; or, the slope of each sampling point in the first dataset and the slope of the third dataset are both less than zero, and the slope of each sampling point in the second dataset is greater than zero.

[0017] In one exemplary implementation, a target sampling point set is determined from the lane line dataset based on the slope of each sampling point in the lane line dataset; the slope of the sampling points in the target sampling point set is greater than zero or less than zero; determining the target sampling point based on the target sampling point set includes:

[0018] This third dataset is designated as the target sampling point set;

[0019] Based on the positions of each sampling point in the target sampling point set and the positions of each sampling point in the second dataset, the target sampling point is determined from the target sampling point set; one adjacent sampling point of the target sampling point belongs to the second dataset; another adjacent sampling point of the target sampling point belongs to the second dataset.

[0020] In an exemplary implementation, when the target lane line dataset includes at least two sampling points, the extended Kalman filter is performed based on the target lane line dataset at the current time, the vehicle motion state at the current time, the vehicle operation state at historical times, and the predicted lane line state quantity at the current time to obtain the target lane line state quantity at the current time, including:

[0021] Determine quasi-target sampling points from the target lane line dataset;

[0022] Based on the location of the quasi-target sampling point, the vehicle motion state at the current moment, the vehicle operation state at historical moments, and the predicted lane line state at the current moment, an extended Kalman filter is performed to obtain the initial target lane line state at the current moment.

[0023] Update the initial target lane state at the current moment to the updated predicted lane state at the current moment;

[0024] The target sampling point is determined from the remaining lane line dataset; the remaining lane line dataset is the dataset excluding the target sampling point.

[0025] Based on the location of the target sampling point, the vehicle motion state at the current moment, the vehicle operation state at the historical moment, and the updated predicted lane line state quantity at the current moment, an extended Kalman filter is performed to obtain the quasi-target lane line state quantity at the current moment.

[0026] The quasi-target lane state quantity at the current moment is used as the updated predicted lane state quantity at the current moment. Then, the target sampling point is determined from the remaining lane data set. Based on the position of the target sampling point, the vehicle motion state at the current moment, the vehicle running state at the historical moment, and the updated predicted lane state quantity at the current moment, the extended Kalman filter is performed until there are no sampling points in the remaining lane data set, and the target lane state quantity at the current moment is obtained.

[0027] In one exemplary implementation, the vehicle operating state includes the vehicle's position and speed;

[0028] The predicted lane line state variables include the offset between the vehicle's centerline and the centerline of the lane it is in, the angle between the vehicle's centerline and the centerline of the lane it is in, the curvature of the lane line, and the rate of change of the curvature of the lane line.

[0029] In one exemplary implementation, obtaining the lane line dataset at the current moment includes:

[0030] The current road scene image is acquired using a data acquisition device; this road scene image includes the lane lines of the lanes in which the vehicles are located.

[0031] Image recognition processing is performed on the road scene image at the current moment to obtain the lane line dataset at the current moment.

[0032] On the other hand, a lane line fitting device is provided, the device comprising:

[0033] The acquisition module is used to acquire the lane line dataset at the current time; the lane line dataset includes the position of each sampling point among multiple sampling points located on the lane lines;

[0034] The first determining module is used to determine a target sampling point from the lane line dataset if the slope value of any two adjacent sampling points in the lane line dataset is greater than or equal to a first preset threshold; the distance between the target sampling point and the vehicle is within the preset threshold range.

[0035] The second determining module is used to determine a target lane line dataset from the lane line dataset based on the target sampling point; the distance between each sampling point in the target lane line dataset and the vehicle is less than the distance between the target sampling point and the vehicle.

[0036] The extended Kalman filter module is used to perform extended Kalman filtering based on the target lane line dataset at the current time, the vehicle motion state at the current time, the vehicle running state at historical time, and the predicted lane line state quantity at the current time to obtain the target lane line state quantity at the current time.

[0037] The third determination module is used to determine the target lane line at the current moment based on the target lane line state quantity at the current moment.

[0038] On the other hand, an electronic device is provided, including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the lane line fitting method of any of the above aspects.

[0039] On the other hand, a computer-readable storage medium is provided that stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the lane line fitting method as described above.

[0040] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the lane fitting method of any of the above aspects.

[0041] This application embodiment obtains a lane line dataset at the current moment; the lane line dataset includes the position of each sampling point among multiple sampling points located on the lane line; if the slope value of any two adjacent sampling points in the lane line dataset is greater than or equal to a first preset threshold, a target sampling point is determined from the lane line dataset; the distance between the target sampling point and the vehicle is within the preset threshold range; based on the target sampling point, a target lane line dataset is determined from the lane line dataset; the distance between each sampling point in the target lane line dataset and the vehicle is less than the distance between the target sampling point and the vehicle; based on the target lane line dataset at the current moment, the vehicle motion state at the current moment, the vehicle running state at historical moments, and the predicted lane line state quantity at the current moment, an extended Kalman filter is performed to obtain the target lane line state quantity at the current moment; the target lane line at the current moment is determined based on the target lane line state quantity at the current moment; in this case, by segmenting the lane line dataset, performing curve fitting based on the segmented dataset, and combining extended Kalman filtering, the accuracy and reliability of the lane line fitting results are achieved. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0044] Figure 2 This is a schematic flowchart of a lane line fitting method provided in an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of the distribution of sampling points provided in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of a process for determining target sampling points provided in an embodiment of this application;

[0047] Figure 5 This is a flowchart illustrating a method for determining the state quantity of a target lane line, as provided in an embodiment of this application.

[0048] Figure 6 This is a schematic diagram of a lane model provided in an embodiment of this application;

[0049] Figure 7 This is a schematic diagram of a process for determining the initial target lane line state quantity provided in an embodiment of this application;

[0050] Figure 8 This is a schematic diagram of another process for determining the initial target lane line state quantity provided in an embodiment of this application;

[0051] Figure 9 This is a schematic diagram of an image coordinate system provided in this application;

[0052] Figure 10 This is a schematic diagram of a vehicle coordinate system provided in an embodiment of this application;

[0053] Figure 11 This is a schematic diagram of the structure of a lane line fitting device provided in an embodiment of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0055] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0056] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0057] Please see Figure 1The diagram illustrates an implementation environment provided in this application embodiment. This environment includes a vehicle 10 and a lane line fitting system 20 located on the vehicle 10. The lane line fitting system 20 includes a data acquisition device 201 and a processing unit 202. The data acquisition device 201 acquires road scene data at the current moment and sends it to the processing unit 202. The processing unit 202 processes the road scene data at the current moment to obtain a lane line dataset. This lane line dataset includes the position of each sampling point among multiple sampling points located on the lane lines. If the slope value of any two adjacent sampling points in the lane line dataset is greater than or equal to a first pre-defined slope, the system can determine the lane line fitting system. A threshold is set, and a target sampling point is determined from the lane line dataset. The distance between the target sampling point and the vehicle falls within the preset threshold range. Based on the target sampling point, a target lane line dataset is determined from the lane line dataset. The distance between each sampling point in the target lane line dataset and the vehicle is less than the distance between the target sampling point and the vehicle. An extended Kalman filter is performed based on the target lane line dataset at the current moment, the vehicle motion state at the current moment, the vehicle running state at historical moments, and the predicted lane line state quantity at the current moment to obtain the target lane line state quantity at the current moment. The target lane line at the current moment is determined based on the target lane line state quantity at the current moment.

[0058] The processing unit 202 can also be located on a server or a terminal.

[0059] Terminals include, but are not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, and aircraft.

[0060] It should be noted that the server involved in the embodiments of this application can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0061] The embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0062] Please see Figure 2 The diagram shown is a flowchart of a lane line fitting method provided in an embodiment of this application. This method can be applied to... Figure 1The system describes a lane line fitting method. It should be noted that this specification provides the operational steps of the method as shown in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual system or product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown... Figure 2 As shown, the method may include:

[0063] S201: Obtain the lane line dataset at the current time; the lane line dataset includes the position of each of the multiple sampling points located on the lane lines.

[0064] In an exemplary embodiment, step S201 may include: acquiring a road scene image at the current moment using an acquisition device; the road scene image includes lane lines of the lane where the vehicle is located; performing image recognition processing on the road scene image at the current moment to obtain a lane line dataset at the current moment.

[0065] In some embodiments, the acquisition device can be a camera or radar; the following description will use a camera as the acquisition device. Optionally, a target detection network model can be used to perform feature extraction, classification, and other processing on the road scene image, thereby outputting a dataset of pixels belonging to the lane lines; one pixel can be used as a sampling point; optionally, the pixels can be further subjected to initial screening processing, for example, selecting multiple pixels spaced at a preset distance to obtain multiple sampling points spaced at the preset distance; to further simulate the perspective effect of data points on the lane lines actually acquired, the preset interval of sampling points that are gradually moving away from the vehicle can also be gradually increased, thus presenting a distribution where sampling points closer to the vehicle are denser and sampling points farther away from the vehicle are sparser. Optionally, the target detection network model is a deep learning network model.

[0066] S203: If the slope value of any two adjacent sampling points in the lane line dataset is greater than or equal to the first preset threshold, a target sampling point is determined from the lane line dataset; the distance between the target sampling point and the vehicle is within the preset threshold range.

[0067] In this embodiment, the slope value of any two adjacent sampling points in the lane line dataset can be determined first. This slope value can refer to the absolute value of the slope, and the slope can refer to a positive or negative value. When there is a slope value greater than or equal to a first preset threshold, it indicates that the current lane line state of the vehicle will change, such as moving from a straight road to a curve, or from a curve to a straight road, or moving into a series of curves, etc. At this time, it is necessary to segment and refit to determine the lane line; otherwise, in order to improve data processing efficiency, segmentation can be omitted and the lane line can be directly refitted; of course, segmentation and refitting can also be performed in real time.

[0068] In this embodiment, the distance between the target sampling point and the center point of the vehicle may be a preset distance.

[0069] In one exemplary implementation, see [reference] Figure 3-4 , Figure 3 This is a schematic diagram of the distribution of sampling points provided in an embodiment of this application. Figure 4 This is a schematic flowchart illustrating the process of determining a target sampling point according to an embodiment of this application. Step S203, determining the target sampling point from the lane line dataset, may include:

[0070] S2031: For each sampling point in the lane line dataset, determine the slope of the sampling point based on its location and the locations of its neighboring sampling points.

[0071] See Figure 3 For sampling point a10, its adjacent sampling points are a9 and a11. In order to avoid repeatedly calculating the slope of the two sampling points and improve the calculation efficiency, it is optional to specify the adjacent sampling point of sampling point a10 as a11, that is, the adjacent sampling point that is far away from the vehicle.

[0072] S2033: If the lane line dataset meets the first preset condition, a target sampling point set is determined from the lane line dataset based on the slope of each sampling point in the lane line dataset; the slope of the sampling points in the target sampling point set is greater than zero or less than zero.

[0073] In an exemplary embodiment, the first preset condition includes that the lane line dataset comprises a first dataset, a second dataset, and a third dataset; the slope of each sampling point in the first dataset and the slope of the third dataset are both greater than zero, and the slope of each sampling point in the second dataset is less than zero; or, the slope of each sampling point in the first dataset and the slope of the third dataset are both less than zero, and the slope of each sampling point in the second dataset is greater than zero; wherein, the sampling points in the lane line dataset are arranged from closest to farthest from the vehicle, in the order of the first dataset, the second dataset, and the third dataset. See also... Figure 3The first dataset can be {a1, a2}, the second dataset can be {a3, a4, ..., a9}, and the third dataset can be {a10, a11, a12, a13, a14}.

[0074] S2035: Determine the target sampling point based on the target sampling point set.

[0075] Steps S2033-S2035 may include: determining the third dataset as the target sampling point set; determining target sampling points from the target sampling point set based on the positions of each sampling point in the target sampling point set and the positions of each sampling point in the second dataset; one adjacent sampling point of the target sampling point belongs to the second dataset; another adjacent sampling point of the target sampling point belongs to the third dataset. See also Figure 3 The sampling point a10 is the target sampling point. a10 includes two adjacent sampling points, a9 and a11. Among them, a9 belongs to the second dataset and a11 belongs to the third dataset.

[0076] To provide flexibility in the application of this application, in another feasible embodiment, in step S2033, if the lane line dataset does not meet the first preset condition, a target sampling point set is determined from the lane line dataset based on the slope of each sampling point in the lane line dataset; the slope value of the sampling points in the target sampling point set is greater than or equal to a second preset threshold; the second preset threshold is greater than the first preset threshold; optionally, the method for determining the target sampling point set includes: determining the maximum slope value from the slope values ​​of each sampling point in the lane line dataset; determining the target sampling point set from the lane line dataset based on the maximum slope value; the target sampling point set includes two sampling points corresponding to the maximum slope value. Optionally, if the lane line dataset does not meet the first preset condition and also does not meet the second preset condition, then segmentation is not required, i.e., target sampling points do not need to be determined; the second preset condition includes: the slope value of each sampling point in the lane line dataset is less than the second preset threshold.

[0077] In this embodiment, refer to Figure 3 When the slope values ​​of sampling points a10 and a11 are at their maximum, sampling point a10 can be used as the target sampling point, or a11 can be selected as the target sampling point.

[0078] In one optional embodiment, the target sampling point can be determined based on the maximum slope as described above; in another optional embodiment, when the target sampling point set includes at least three sampling points, any one of the sampling points in the target sampling point set can be used as the target sampling point, and a new sampling point can be obtained by averaging the target sampling point set.

[0079] It should be noted that, in order to ensure that the determined sampling points are located in appropriate areas, such as Figure 3 The area where sampling point a10 is located can be further filtered by selecting sampling points in the target sampling point dataset. Sampling points located within a preset threshold area can be selected as unprocessed sampling points, and multiple consecutive sampling points can be identified from these. The positions of these selected sampling points can be averaged to determine a new sampling point, which is then used as the target sampling point. Similarly, even in the method of determining the target sampling point based on the maximum slope described above, it is necessary to ensure that the target sampling point is located within a suitable area.

[0080] To further enhance the application flexibility of this application, in another exemplary embodiment, the process of determining the target sampling point set based on slope can be replaced by: fitting a line to the lane line dataset to obtain a fitted line; for each sampling point in the multiple sampling points of the lane line dataset, determining the residual of the sampling point based on the fitted line and the sampling point position; determining the target sampling point set from the lane line dataset based on the residuals of each sampling point in the lane line dataset; the residuals of the sampling points in the target sampling point set are greater than or equal to a fourth threshold; and determining the target sampling point based on the target sampling point set. Optionally, the fitted line can be a straight line or a curve. To improve data processing efficiency, the sampling point with the largest residual can be directly used as the target sampling point. If there are multiple sampling points with the largest residuals, and the distance between any two adjacent sampling points is greater than a preset distance or is not continuous, then a sampling point at a suitable position is obtained by filtering them through a preset position range; otherwise, any sampling point in the target sampling point set can be used as the target sampling point, or a new sampling point can be determined by averaging the positions of each sampling point in the target sampling point set, and this new sampling point can be used as the target sampling point.

[0081] S205: Based on the target sampling point, determine the target lane line dataset from the lane line dataset; the distance between each sampling point in the target lane line dataset and the vehicle is less than the distance between the target sampling point and the vehicle.

[0082] In this embodiment, the target sampling point is used as the segmentation point, dividing the data in the lane line dataset into two parts: the sampling points located before segmentation point a10 (e.g., ...). Figure 3 The sample points a1, ..., a9, and a11, ..., located after the segment point a10, can be used as the target lane line dataset, which includes the segment point a10 and the sample points before the segment point a10.

[0083] In this embodiment, the distance between the sampling point and the vehicle can be the distance between the sampling point and the center point of the vehicle; optionally, the center point can also be other reference points, such as the camera, the center point of the left front wheel, etc.

[0084] S207: Based on the target lane line dataset at the current moment, the vehicle motion state at the current moment, the vehicle running state at historical moments, and the predicted lane line state at the current moment, extended Kalman filtering is performed to obtain the target lane line state at the current moment.

[0085] In one exemplary implementation, see [reference] Figure 5 and Figure 6 , Figure 5 This is a flowchart illustrating a method for determining the state quantity of a target lane line, as provided in this application. Figure 6 This is a schematic diagram of a lane model provided in an embodiment of this application. When the target lane line dataset includes at least two sampling points, step S207 may include:

[0086] S2071: Determine the quasi-target sampling points from the target lane line dataset.

[0087] In this embodiment, the quasi-target sampling point can be the sampling point closest to the vehicle. Continuing with the above example, sampling point a1 can be used as the quasi-target sampling point.

[0088] S2073: Based on the location of the quasi-target sampling point, the vehicle motion state at the current moment, the vehicle operation state at historical moments, and the predicted lane line state at the current moment, extended Kalman filtering is performed to obtain the initial target lane line state at the current moment.

[0089] In one exemplary implementation, the vehicle operating state includes the vehicle's position and speed; the predicted lane line state quantity includes the offset e between the vehicle's centerline and the centerline of the lane it occupies. k The angle θ between the vehicle's centerline and the centerline of its lane. k The curvature C of the lane lines 0,k and the rate of change of curvature C of the lane lines 1,k Optionally, to improve prediction accuracy, the number of predicted lane line state variables can be increased, such as including the rate of change of the offset e between the vehicle's centerline and the centerline of its lane. k θ is the rate of change of the angle between the vehicle's centerline and the centerline of its lane. k '; The angle φ between the camera's optical axis and the road plane k Road width W k The embodiments of this application will primarily focus on predicting lane line state quantities, including e. k θ k C 0,k C1,k Let's take an example to illustrate this in detail.

[0090] In one exemplary implementation, see [reference] Figure 7 , Figure 7 This is a flowchart illustrating the determination of an initial target lane line state quantity according to an embodiment of this application. Step S2073 may include:

[0091] S701: Based on the position of the quasi-target sampling point, the vehicle motion state at the current moment, and the vehicle operation state at the historical moment, determine the first Jacobian matrix and the control matrix at the current moment. In this embodiment, the first Jacobian matrix at the current moment can be set as F. k The control matrix at the current moment is set to u. k ;F k Based on the vehicle's motion state and vehicle dynamics at the current and historical moments, we can obtain:

[0092]

[0093] Among them, considering the lateral motion Δx and the rotation Δyaw, the values ​​of the lateral motion Δx and the rotation Δyaw at time k can be obtained from the vehicle's sensors.

[0094] Then the control matrix u k =[-Δx,-Δyaw,0,0]

[0095] Among them, the above u k The relationships between Δx and Δyaw and the four state variables are as follows:

[0096] Therefore, we can obtain the following formula (1):

[0097]

[0098] Δl is the change in lane length from time k-1 to time k, which can also be expressed as

[0099] Due to each lane orientation change According to Taylor expansion Known Then Δz≈Δl;

[0100]

[0101] C 0,k =C 0,k-1 +C 1,k-1 Δl=C 0,k-1 +C 1,k-1 Δz...Formula (3)

[0102] Based on the above formulas (1)-(3), the control matrix u can be determined. k The relationship with the four state variables.

[0103] S703: Determine the predicted lane line state variables at the current moment based on the first Jacobian matrix at the current moment, the control matrix at the current moment, and the target lane line state variables at historical moments.

[0104] In this embodiment, the predicted lane volume state at the current moment can be determined based on the following formula:

[0105]

[0106] in, It can also be called the prior state estimate at the current time k. The target lane line state quantity at a historical moment, where k-1 in this embodiment can be represented as the previous moment.

[0107] S705: Determine the prediction error covariance at the current moment based on the first Jacobian matrix at the current moment, the target error covariance at the historical moment, and the process noise covariance at the current moment.

[0108] In an exemplary embodiment, the prediction error covariance at the current moment can be calculated using the following formula.

[0109]

[0110] Among them, P k-1 WQW represents the target error covariance at historical time points, Q represents the process noise covariance matrix, and WQW represents the target error covariance at historical time points. T Based on empirical values, it is set to an identity matrix here.

[0111] S707: Determine the initial target lane state quantity at the current moment based on the prediction error covariance at the current moment, the predicted lane state quantity at the current moment, the observation noise at the current moment, the location of the quasi-target sampling point at the current moment, the vehicle motion state at the current moment, and the vehicle running state at the historical moment.

[0112] In one exemplary implementation, see [reference] Figure 8 , Figure 8 This is a schematic diagram of another process for determining the initial target lane line state quantity provided in an embodiment of this application. Step S707 may include:

[0113] S7071: Determine the observation Jacobian matrix for the current moment based on the location of the quasi-target sampling point at the current moment, the vehicle motion state at the current moment, and the vehicle operation state at the historical moment.

[0114] In one feasible embodiment, see Figure 9 and 10 , Figure 9 This is a schematic diagram of an image coordinate system provided in this application. Figure 10 This is a schematic diagram of a vehicle coordinate system provided in an embodiment of this application; the observation Jacobian matrix H at the current moment can be calculated based on the following formula:

[0115]

[0116] Where, x [0] x [1] x [3] x [4] These correspond to the four state variables e mentioned above. k θ k C 0,k C 1,k One of them, x img This represents the image coordinates of the quasi-target sampling point in the image coordinate system; however, since the measured quantity is the x-coordinate of the vehicle coordinate system... road This involves the transformation between the vehicle coordinate system and the image coordinate system. Here, we assume the vehicle travels along... Figure 6 The image shows driving on a road in plane xz, x road ,z road For vehicle coordinates.

[0117] Given vehicle coordinates (x) road ,z road ) and image coordinates (x) img ,z img Conversion parameters

[0118] The conversion formula between the vehicle coordinate system and the image coordinate system is as follows:

[0119]

[0120] Where Zc is the z-axis coordinate in the camera coordinate system.

[0121] Z c x img =p[0]x road +p[1]y road +p[2]

[0122] Z c =p[6] road +p[7] road +[8]

[0123] Let φ(x) road ) = Z c x img ;t(xroad ) = Z c ; but

[0124] When the state variables are not limited to the above four, H also needs to be updated accordingly, that is, x[i] also includes x[5], x[6], etc.

[0125] S7073: Determine the Kalman gain at the current time based on the prediction error covariance, the observation noise at the current time, and the observation Jacobian matrix at the current time.

[0126] In one feasible embodiment, the Kalman gain K at the current moment k It can be represented as:

[0127] Among them, R k This represents the observation noise at the current moment;

[0128] S7075: Determine the initial target lane state at the current moment based on the Kalman gain, the predicted lane state at the current moment, and the observed initial state at the current moment.

[0129] The initial target lane line state at the current time, i.e., the initial posterior estimate at the current time. Where k k ∈[0,H -1 ]; when k k =0, completely trust the prediction, when k k =H -1 Completely trust the measured values.

[0130] Initial target error covariance at the current moment

[0131] S2075: Update the initial target lane state at the current time to the updated predicted lane state at the current time.

[0132] In this embodiment, the initial target lane line state quantity determined in step S703 at the current moment can be used as...

[0133] S2077: Determine the target sampling point from the remaining lane line dataset; the remaining lane line dataset is the dataset excluding the target sampling point.

[0134] Continuing with the example above, it could be that... Figure 3 a2 is used as the target sampling point.

[0135] S2079: Based on the location of the target sampling point, the vehicle motion state at the current moment, the vehicle operation state at the historical moment, and the updated predicted lane line state quantity at the current moment, extended Kalman filtering is performed to obtain the quasi-target lane line state quantity at the current moment.

[0136] In this embodiment, based on the known current time in step S703... The initial target lane line state quantity is determined in the same way as in steps S7071-S7075 above, and the quasi-target lane line state quantity at the current moment is updated.

[0137] S20711: Use the quasi-target lane state quantity at the current moment as the updated predicted lane state quantity at the current moment, and perform the steps of determining the target sampling point from the remaining lane data set, and performing extended Kalman filtering based on the position of the target sampling point, the vehicle motion state at the current moment, the vehicle running state at the historical moment, and the updated predicted lane state quantity at the current moment, until there are no sampling points in the remaining lane data set, and obtain the target lane state quantity at the current moment.

[0138] In this embodiment, for the target lane line set {a1, a2, ..., a10}, for a1, based on the previous time step... The predicted lane line state at the current moment can be determined. Then, following steps S7071-S7075 above, the initial target lane line state quantity at the current moment is determined. (Initial), and calculate the initial target error covariance P at the current time. k (Initial); For a2, calculate the value of a1. (Initial) as the current moment P k (Initial) as The quasi-target lane line state quantity at the current moment is determined according to the steps S7071-S7075 described above. (Accurate) and P k (Accurate), calculate a2 (Accurate) and P k (Preliminary) as the next sampling point and Repeat the above steps to calculate a3. and P k The calculation is iterated sequentially until all sampling points in the target lane line set are completed. and P k And the last sampling point, for example, sampling point a10 and P k The image at the current moment and Pk .

[0139] S209: Determine the target lane line at the current moment based on the target lane line state quantity at the current moment.

[0140] In a feasible embodiment, the target lane line at current time k satisfies the following formula:

[0141]

[0142] Where l is the specified lane line, for example, l = -1 represents the left lane line, and l = 1 represents the right lane line.

[0143] Corresponding to the lane line fitting methods provided in the above embodiments, this application also provides a lane line fitting device. Since the lane line fitting device provided in this application corresponds to the lane line fitting methods provided in the above embodiments, the implementation methods of the aforementioned lane line fitting methods are also applicable to the lane line fitting device provided in this embodiment, and will not be described in detail in this embodiment.

[0144] Please see Figure 11 The diagram shows a schematic representation of a lane line fitting device provided in an embodiment of this application. This device has the function of implementing the lane line fitting method described in the above-described method embodiments. This function can be implemented in hardware or by hardware executing corresponding software. Figure 11 As shown, the device may include:

[0145] The acquisition module 1101 is used to acquire the lane line dataset at the current time; the lane line dataset includes the position of each sampling point among multiple sampling points located on the lane lines;

[0146] The first determining module 1103 is used to determine a target sampling point from the lane line dataset if the slope value of any two adjacent sampling points in the lane line dataset is greater than or equal to a first preset threshold; the distance between the target sampling point and the vehicle is within the preset threshold range.

[0147] The second determining module 1105 is used to determine a target lane line dataset from the lane line dataset based on the target sampling point; the distance between each sampling point in the target lane line dataset and the vehicle is less than the distance between the target sampling point and the vehicle.

[0148] The extended Kalman filter module 1107 is used to perform extended Kalman filtering based on the target lane line dataset at the current time, the vehicle motion state at the current time, the vehicle running state at historical time and the predicted lane line state quantity at the current time, to obtain the target lane line state quantity at the current time.

[0149] The third determining module 1109 is used to determine the target lane line at the current time based on the target lane line state quantity at the current time.

[0150] In one exemplary implementation, the first determining module is configured to determine the slope of each sampling point in the lane line dataset based on the location of the sampling point and the locations of adjacent sampling points.

[0151] If the lane line dataset meets the first preset condition, a target sampling point set is determined from the lane line dataset based on the slope of each sampling point in the lane line dataset; the slope of the sampling points in the target sampling point set is greater than zero or less than zero.

[0152] The target sampling point is determined based on the target sampling point set.

[0153] In one exemplary implementation, the first preset condition includes:

[0154] The lane line dataset includes a first dataset, a second dataset, and a third dataset; the slope of each sampling point in the first dataset and the slope of the third dataset are both greater than zero, and the slope of each sampling point in the second dataset is less than zero; or, the slope of each sampling point in the first dataset and the slope of the third dataset are both less than zero, and the slope of each sampling point in the second dataset is greater than zero.

[0155] In this lane line dataset, the sampling points are arranged from closest to furthest from the vehicle, and are respectively the first dataset, the second dataset, and the third dataset.

[0156] In one exemplary implementation, a first determining module is configured to determine the third dataset as the target sampling point set;

[0157] Based on the positions of each sampling point in the target sampling point set and the positions of each sampling point in the second dataset, the target sampling point is determined from the target sampling point set; one adjacent sampling point of the target sampling point belongs to the second dataset; another adjacent sampling point of the target sampling point belongs to the second dataset.

[0158] In one exemplary implementation, when the target lane line dataset includes at least two sampling points, an extended Kalman filter module is used to determine quasi-target sampling points from the target lane line dataset.

[0159] Based on the location of the quasi-target sampling point, the vehicle motion state at the current moment, the vehicle operation state at historical moments, and the predicted lane line state at the current moment, an extended Kalman filter is performed to obtain the initial target lane line state at the current moment.

[0160] Update the initial target lane state at the current moment to the updated predicted lane state at the current moment;

[0161] The target sampling point is determined from the remaining lane line dataset; the remaining lane line dataset is the dataset excluding the target sampling point.

[0162] Based on the location of the target sampling point, the vehicle motion state at the current moment, the vehicle operation state at the historical moment, and the updated predicted lane line state quantity at the current moment, an extended Kalman filter is performed to obtain the quasi-target lane line state quantity at the current moment.

[0163] The quasi-target lane state quantity at the current moment is used as the updated predicted lane state quantity at the current moment. Then, the target sampling point is determined from the remaining lane data set. Based on the position of the target sampling point, the vehicle motion state at the current moment, the vehicle running state at the historical moment, and the updated predicted lane state quantity at the current moment, the extended Kalman filter is performed until there are no sampling points in the remaining lane data set, and the target lane state quantity at the current moment is obtained.

[0164] In one exemplary implementation, the vehicle operating state includes the vehicle's position and speed;

[0165] The predicted lane line state variables include the offset between the vehicle's centerline and the centerline of the lane it is in, the angle between the vehicle's centerline and the centerline of the lane it is in, the curvature of the lane line, and the rate of change of the curvature of the lane line.

[0166] In one exemplary embodiment, the acquisition module is used to acquire a road scene image at the current moment using an acquisition device; the road scene image includes lane lines of the lane in which the vehicle is located;

[0167] Image recognition processing is performed on the road scene image at the current moment to obtain the lane line dataset at the current moment.

[0168] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0169] This application provides an electronic device including a processor and a memory. The memory stores at least one instruction or at least one program segment, which is loaded and executed by the processor to implement any of the lane line fitting methods provided in the above method embodiments.

[0170] Memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created based on the use of the device, etc. Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory can also include a memory controller to provide the processor with access to the memory.

[0171] Embodiments of this application also provide a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a lane line fitting method. The at least one instruction or the at least one program is loaded and executed by the processor to implement any of the lane line fitting methods provided in the above-described method embodiments.

[0172] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform any of the lane line fitting methods provided in the above-described method embodiments.

[0173] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0174] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0175] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0176] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0177] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A lane line fitting method, characterized in that, include: Obtain the lane line dataset for the current time. The lane line dataset includes the position of each of multiple sampling points located on the lane lines; If the slope value of any two adjacent sampling points in the lane line dataset is greater than or equal to the first preset threshold, the target sampling point is determined from the lane line dataset. The distance between the target sampling point and the vehicle falls within a preset threshold range; Based on the target sampling points, a target lane line dataset is determined from the lane line dataset; The distance between each sampling point in the target lane line dataset and the vehicle is less than the distance between the target sampling point and the vehicle; Extended Kalman filtering is performed based on the target lane line dataset at the current moment, the vehicle motion state at the current moment, the vehicle operation state at historical moments, and the predicted lane line state at the current moment to obtain the target lane line state at the current moment. The target lane line at the current moment is determined based on the target lane line state quantity at the current moment; When the target lane line dataset includes at least two sampling points, the step of performing extended Kalman filtering based on the target lane line dataset at the current time, the vehicle motion state at the current time, the vehicle running states at historical times, and the predicted lane line state at the current time to obtain the target lane line state at the current time includes: Determine quasi-target sampling points from the target lane line dataset; Extended Kalman filtering is performed based on the location of the quasi-target sampling point, the vehicle motion state at the current moment, the vehicle operation state at historical moments, and the predicted lane line state at the current moment to obtain the initial target lane line state at the current moment. Update the initial target lane state at the current moment to the updated predicted lane state at the current moment; The target sampling point is determined from the remaining lane line dataset; the remaining lane line dataset is the dataset excluding the quasi-target sampling point. Based on the location of the target sampling point, the vehicle motion state at the current moment, the vehicle operation state at the historical moment, and the updated predicted lane line state quantity at the current moment, an extended Kalman filter is performed to obtain the quasi-target lane line state quantity at the current moment. The quasi-target lane state quantity at the current moment is used as the updated predicted lane state quantity at the current moment. Then, the following steps are performed: determining the target sampling point from the remaining lane data set; performing extended Kalman filtering based on the position of the target sampling point, the vehicle motion state at the current moment, the vehicle running state at the historical moment, and the updated predicted lane state quantity at the current moment; until no sampling point exists in the remaining lane data set, the target lane state quantity at the current moment is obtained.

2. The lane line fitting method according to claim 1, characterized in that, Determining the target sampling point from the lane line dataset includes: For each sampling point in the lane line dataset, the slope of the sampling point is determined based on the location of the sampling point and the locations of adjacent sampling points; If the lane line dataset satisfies the first preset condition, a target sampling point set is determined from the lane line dataset based on the slope of each sampling point in the lane line dataset; the slope of the sampling points in the target sampling point set is greater than zero or less than zero. The target sampling points are determined based on the target sampling point set.

3. The lane line fitting method according to claim 2, characterized in that, The first preset conditions include: The sampling points in the lane line dataset are arranged from closest to farthest from the vehicle and belong to the first dataset, the second dataset, and the third dataset in sequence. The slope of each sampling point in the first dataset and the slope of the third dataset are both greater than zero, and the slope of each sampling point in the second dataset is less than zero; or, the slope of each sampling point in the first dataset and the slope of the third dataset are both less than zero, and the slope of each sampling point in the second dataset is greater than zero.

4. The lane line fitting method according to claim 3, characterized in that, The target sampling point set is determined from the lane line dataset based on the slope of each sampling point in the lane line dataset; The slope of the sampling points in the target sampling point set is greater than zero or less than zero; Determining the target sampling points based on the target sampling point set includes: The third dataset is determined as the target sampling point set; Based on the positions of each sampling point in the target sampling point set and the positions of each sampling point in the second dataset, a target sampling point is determined from the target sampling point set; one adjacent sampling point of the target sampling point belongs to the second dataset; and another adjacent sampling point of the target sampling point belongs to the third dataset.

5. The lane line fitting method according to claim 1, characterized in that, The vehicle's operating status includes its position and speed; The predicted lane line state quantities include the offset between the vehicle centerline and the centerline of the lane, the angle between the vehicle centerline and the centerline of the lane, the curvature of the lane line, and the rate of change of the curvature of the lane line.

6. The lane line fitting method according to any one of claims 1-5, characterized in that, The process of obtaining the lane line dataset at the current moment includes: The current moment's road scene image is acquired using a data acquisition device; the road scene image includes the lane lines of the lane in which the vehicle is located; The road scene image at the current moment is processed by image recognition to obtain the lane line dataset at the current moment.

7. A lane line fitting device, characterized in that, The device includes: The acquisition module is used to acquire the lane line dataset at the current time; the lane line dataset includes the position of each sampling point among multiple sampling points located on the lane lines; The first determining module is used to determine a target sampling point from the lane line dataset if the slope value of any two adjacent sampling points in the lane line dataset is greater than or equal to a first preset threshold; the distance between the target sampling point and the vehicle is within the preset threshold range. The second determining module is used to determine a target lane line dataset from the lane line dataset based on the target sampling points; the distance between each sampling point in the target lane line dataset and the vehicle is less than the distance between the target sampling point and the vehicle. The extended Kalman filter module is used to perform extended Kalman filtering based on the target lane line dataset at the current time, the vehicle motion state at the current time, the vehicle running state at historical time, and the predicted lane line state quantity at the current time to obtain the target lane line state quantity at the current time. The third determining module is used to determine the target lane line at the current time based on the target lane line state quantity at the current time. When the target lane line dataset includes at least two sampling points, the step of performing extended Kalman filtering based on the target lane line dataset at the current time, the vehicle motion state at the current time, the vehicle running states at historical times, and the predicted lane line state at the current time to obtain the target lane line state at the current time includes: Determine quasi-target sampling points from the target lane line dataset; Extended Kalman filtering is performed based on the location of the quasi-target sampling point, the vehicle motion state at the current moment, the vehicle operation state at historical moments, and the predicted lane line state at the current moment to obtain the initial target lane line state at the current moment. Update the initial target lane state at the current moment to the updated predicted lane state at the current moment; The target sampling point is determined from the remaining lane line dataset; the remaining lane line dataset is the dataset excluding the quasi-target sampling point. Based on the location of the target sampling point, the vehicle motion state at the current moment, the vehicle operation state at the historical moment, and the updated predicted lane line state quantity at the current moment, an extended Kalman filter is performed to obtain the quasi-target lane line state quantity at the current moment. The quasi-target lane state quantity at the current moment is used as the updated predicted lane state quantity at the current moment. Then, the following steps are performed: determining the target sampling point from the remaining lane data set; performing extended Kalman filtering based on the position of the target sampling point, the vehicle motion state at the current moment, the vehicle running state at the historical moment, and the updated predicted lane state quantity at the current moment; until no sampling point exists in the remaining lane data set, the target lane state quantity at the current moment is obtained.

8. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the lane line fitting method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the lane line fitting method as described in any one of claims 1 to 6.

Images