Lane tracking method, system, electronic device, and storage medium

Abstract

The present invention provides a lane line tracking method, system, electronic device, and storage medium. The method includes: obtaining a first point set, which is used to fit and form a finalized lane line from a previous cycle; performing a compensatory transformation on the first point set based on the vehicle's own motion in the current cycle to form a second point set in the vehicle coordinate system for the current cycle; obtaining a detected lane line in the current cycle; if it is determined that the detected lane line matches the finalized lane line from the previous cycle, sampling a third point set on the detected lane line based on the longitudinal coordinates of the second point set in the vehicle coordinate system; fitting the second and third point sets to form a fourth point set; and fitting the fourth point set to form the finalized lane line tracked in the current cycle. This point-based lane line tracking method improves the accuracy of lane line tracking.

Description

Technical Field

[0001] The present invention relates to the field of lane line tracking technology, and in particular to a lane line tracking method, system, electronic device, and storage medium. Background Art

[0002] Lane detection and tracking are essential components of autonomous driving technology. Lane detection and tracking can be used to control autonomous vehicles along lane lines in areas without high-precision maps or where positioning is poor. It can also be used for lateral positioning of the positioning module. However, due to the variability of road conditions, simple lane detection cannot guarantee lane accuracy and stability, making lane tracking crucial.

[0003] Lane lines are typically represented using polynomial lane equations. Current lane tracking solutions typically track lane lines by directly tracking the lane equation parameters. Tracking polynomial parameters is sensitive to input detection and relies on upstream detection for stable detection. However, due to the inevitable shape errors and positional deviations in front-end lane detectors, lane parameters can easily diverge during tracking, resulting in abnormal lanes and poor tracking performance. Summary of the Invention

[0004] In response to the problems existing in the prior art, the present invention provides a lane line tracking method, system, electronic device, and storage medium.

[0005] The present invention provides a lane line tracking method, the method comprising:

[0006] Obtaining a first point set, where the first point set is used to fit and form a finalized lane line of the previous cycle;

[0007] Based on the vehicle's own motion in the current cycle, a compensating transformation is performed on the first point set to form a second point set in the vehicle coordinate system in the current cycle;

[0008] Get the detected lane lines of the current cycle;

[0009] If it is determined that the detected lane line matches the finalized lane line of the previous cycle, sampling the detected lane line to obtain a third point set based on the longitudinal axis coordinate of the second point set in the vehicle coordinate system;

[0010] Fitting the second point set and the third point set to form a fourth point set;

[0011] Based on the fourth point set, a finalized lane line tracked in the current cycle is formed by fitting.

[0012] According to a lane line tracking method provided by the present invention, the method further includes:

[0013] If it is determined that the detected lane line does not match the finalized lane line of the previous cycle, the detected lane line is used as the finalized lane line detected in the current cycle.

[0014] According to a lane line tracking method provided by the present invention, the method further includes:

[0015] If it is determined that the detected lane line does not match the finalized lane line of the previous cycle, sampling at equal intervals on the longitudinal axis of the vehicle coordinate system in the current cycle based on the detected lane line to obtain a fifth point set;

[0016] The fifth point set is used as the first point set of the next cycle.

[0017] According to a lane tracking method provided by the present invention, fitting the second point set and the third point set to form a fourth point set includes:

[0018] Obtaining projections of the second point set and the third point set on the longitudinal axis of the vehicle coordinate system;

[0019] Fitting the second point set and the third point set of the overlapping portion of the projection to form a first sub-point set of the overlapping portion;

[0020] Obtaining a translation amount between a first connecting line formed by the second point set and / or the third point set of the non-overlapping projection portion and a second connecting line formed by the first sub-point set;

[0021] translating the second point set and / or the third point set of the non-overlapping projections in a direction close to the second connecting line according to the translation amount to form a second sub-point set;

[0022] The first sub-point set and the second sub-point set are combined to form the fourth point set.

[0023] According to a lane tracking method provided by the present invention, fitting the second point set and the third point set to form a fourth point set includes:

[0024] Inputting the second point set and the third point set into a Kalman filter model;

[0025] The Kalman filter model outputs the fourth point set.

[0026] According to a lane line tracking method provided by the present invention, the method further includes:

[0027] Obtaining two finalized lane lines within a historical period, and measuring a first distance between the two finalized lane lines within the historical period;

[0028] Obtaining two finalized lane lines tracked in a current cycle, measuring a second distance between the two finalized lane lines tracked in the current cycle, and successfully matching the two finalized lane lines tracked in the current cycle with the two finalized lane lines in the historical cycle;

[0029] Calculating a difference between the first spacing and the second spacing;

[0030] Based on the difference, an offset adjustment is performed on the positions of the two finalized lane lines tracked in the current cycle so that the difference is zero.

[0031] According to a lane line tracking method provided by the present invention, the method further includes:

[0032] Get the finalized lane lines within multiple cycles;

[0033] Based on the finalized lane lines in the multiple cycles, lane lines across cycles are formed by fitting.

[0034] The present invention also provides a lane tracking system, the system comprising:

[0035] An acquisition module, configured to acquire a first point set, wherein the first point set is used to fit and form a finalized lane line of a previous cycle;

[0036] a compensation module, configured to perform a compensation transformation on the first point set based on the vehicle's own motion in the current cycle to form a second point set in the vehicle coordinate system in the current cycle;

[0037] Detection module, used to obtain the detection lane lines of the current cycle;

[0038] a sampling module, wherein if the sampling module determines that the detected lane line matches the finalized lane line of the previous cycle, sampling the detected lane line based on the longitudinal coordinate of the second point set in the vehicle coordinate system to obtain a third point set;

[0039] a point fitting module, configured to fit the second point set and the third point set to form a fourth point set;

[0040] The line fitting module is used to fit the final lane line tracked in the current cycle based on the fourth point set.

[0041] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the lane line tracking method as described in any one of the above items when executing the program.

[0042] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the lane line tracking method as described in any one of the above items.

[0043] The lane line tracking method, system, electronic device, and storage medium provided by the present invention improve the accuracy of lane line tracking through a point-based lane line tracking method. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A schematic flow chart of a lane line tracking method provided by the present invention;

[0046] Figure 2 A schematic diagram of point set fitting provided by the present invention;

[0047] Figure 3 A schematic diagram of the structure of a lane tracking system provided by the present invention;

[0048] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0049] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0050] The lane line tracking method provided in the embodiment of the present application is described in detail below with reference to specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0051] Figure 1 A flow chart of a lane line tracking method provided by the present invention is shown as follows: Figure 1 As shown, the present invention provides a lane line tracking method, which includes the following steps.

[0052] Preferably, the lane tracking method runs in a vehicle processor at the edge and / or a server in the cloud.

[0053] It should be noted that, in the following vehicle coordinate system, the vertical axis is the X-axis parallel to the ground and pointing to the front of the vehicle, and the horizontal axis is the Y-axis pointing to the left side of the driver.

[0054] S100: Obtain a first point set, where the first point set is used to fit and form the finalized lane line of the previous cycle.

[0055] Preferably, the first point set is an initialization point set obtained based on detector sampling, or an iterative point set obtained based on the lane line tracking method of the present invention.

[0056] Preferably, the period refers to the duration of each lane line detection, which may also be referred to as a frame.

[0057] S200 : Based on the vehicle's own motion in the current cycle, a compensating transformation is performed on the first point set to form a second point set in the vehicle coordinate system in the current cycle.

[0058] It's important to note that since lane lines detected in each cycle exist in the vehicle coordinate system, when lane lines detected in a new cycle are received, the lane lines from the previous cycle must be compensated to the target vehicle coordinate system of the current cycle based on the target vehicle's motion. Since lane lines are stationary, their own motion doesn't need to be considered; only the motion of the target vehicle is.

[0059] S300: Obtain the detected lane lines of the current cycle.

[0060] Optionally, the method further includes:

[0061] If it is determined that the detected lane line does not match the finalized lane line of the previous cycle, the detected lane line is used as the finalized lane line detected in the current cycle.

[0062] Preferably, based on the detector information, a detection point set about the lane line is obtained, and based on the detection point set, a lane line equation is formed by mathematical fitting as the initialization lane line.

[0063] Preferably, the distance between the initialized lane line of the current cycle and the finalized lane line of the previous cycle is calculated, and matching is performed using the Hungarian algorithm.

[0064] Optionally, the method further includes:

[0065] If it is determined that the detected lane line does not match the finalized lane line of the previous cycle, sampling at equal intervals on the longitudinal axis of the vehicle coordinate system in the current cycle based on the detected lane line to obtain a fifth point set;

[0066] The fifth point set is used as the first point set of the next cycle.

[0067] S400: If it is determined that the detected lane line matches the finalized lane line of the previous cycle, a third point set is sampled on the detected lane line based on the longitudinal axis coordinate of the second point set in the vehicle coordinate system.

[0068] S500: Fit the second point set and the third point set to form a fourth point set.

[0069] Optionally, Figure 2 This is a schematic diagram of the point set fitting provided by the present invention, such as Figure 2 As shown, the second point set 1 and the third point set 2 are fitted to form a fourth point set 3, including:

[0070] Obtaining projections of the second point set and the third point set on the longitudinal axis of the vehicle coordinate system;

[0071] Fitting the second point set and the third point set of the overlapping part of the projection to form a first sub-point set of the overlapping part;

[0072] Obtaining a translation amount between a first line formed by the second point set and / or the third point set of the non-overlapping portion of the projection and a second line formed by the first sub-point set;

[0073] The second point set and / or the third point set of the non-overlapping projections are translated in a direction close to the second connecting line according to the translation amount to form a second sub-point set;

[0074] The first sub-point set and the second sub-point set are combined to form a fourth point set.

[0075] It should be noted that for non-overlapping projections, the entire non-overlapping portion is translated to the overlapping portion, using the updated position of the overlapping portion as a reference. This prevents point position discontinuities caused by the update at the overlapping cutoff point. Through translation adjustment, the line connecting the fourth point set maintains lane continuity.

[0076] Optionally, fitting the second point set and the third point set to form a fourth point set includes:

[0077] Inputting the second point set and the third point set into the Kalman filter model;

[0078] The Kalman filter model outputs a fourth point set.

[0079] Preferably, referring to the Kalman filter formula, the update weights are calculated based on the horizontal coordinate variance of the third point set obtained based on detection, the horizontal coordinate variance of the second point set obtained based on tracking and maintenance, and the measurement matrix, and the second tracked and maintained point set is updated using the fourth point set formed by fitting. In the present invention, the measurement matrix is ​​the identity matrix, so the actual factors that influence the update weights are the horizontal coordinate variance of the third point set (obtained through offline evaluation or set based on experience) and the horizontal coordinate variance of the second point set (obtained through online iteration).

[0080] It should be noted that, since the vertical axis coordinates of the third point set are consistent with those of the second point set, when obtaining the fourth point set, its vertical axis coordinates are consistent with those of the third point set. In fact, only the horizontal axis coordinates of the fourth point set need to be obtained.

[0081] The calculation formula for updating the horizontal axis coordinate weight of the fourth point set is as follows:

[0082]

[0083] The update formula for the horizontal axis coordinate and variance of the fourth point set is:

[0084] X k|k =X k|k-1 +K k (z k -X k|k-1 )

[0085] P k|k =(IK k )P k|k-1

[0086] Where I represents the identity matrix, K k Represents the horizontal coordinate update weight of the fourth point set, P k|k-1 Represents the horizontal coordinate variance of the second point set, P k|k Represents the horizontal coordinate variance of the fourth point set, R k Represents the horizontal axis coordinate variance of the third point set, z k Represents the horizontal coordinate of the third point set, X k|k-1 Represents the horizontal coordinate of the second point set, X k|k Represents the horizontal axis coordinate of the fourth point set.

[0087] S600: Based on the fourth point set, fitting is performed to form a finalized lane line tracked in the current cycle.

[0088] Preferably, based on the fourth point set, a lane line equation with cubic polynomial coefficients is fitted to serve as the finalized lane line.

[0089] An embodiment of the present invention provides a point-based lane line tracking solution, which maintains the shape of the lane line by tracking the positions of points on the lane line. During the tracking process, a fourth point set is obtained by fitting the third point set obtained by detection and the second point set obtained by tracking and maintenance, and the final lane line is obtained by fitting based on the fourth point set.

[0090] In another embodiment, based on the above embodiment, the lane line tracking method further includes:

[0091] Obtain two finalized lane lines within a historical period, and measure a first distance between the two finalized lane lines within the historical period;

[0092] Obtain the two finalized lane lines tracked in the current cycle, measure the second distance between the two finalized lane lines tracked in the current cycle, and successfully match the two finalized lane lines tracked in the current cycle with the two finalized lane lines in the historical cycle.

[0093] Calculate the difference between the first spacing and the second spacing;

[0094] Based on the difference, the positions of the two finalized lane lines tracked in the current cycle are offset and adjusted so that the difference is zero.

[0095] It should be noted that on real roads, lane widths generally do not change abruptly. Based on this rule, the present invention applies a sliding window filter to the finalized lane line position. This sliding window filter compares the historical average lane width with the current lane width. If there is a discrepancy, the current lane line position is adjusted based on the target vehicle. This adjustment is done by applying an offset to all points on the lane line to ensure that it matches the historical lane width. This shift in the overall lane line position maintains lane width continuity without changing the lane line shape.

[0096] In another embodiment, based on the above embodiment, the lane line tracking method further includes:

[0097] Get the finalized lane lines within multiple cycles;

[0098] Based on the finalized lane lines in multiple cycles, cross-cycle lane lines are fitted.

[0099] By further fitting the finalized lane lines of multiple cycles obtained through continuous iteration, we can obtain lane lines across cycles, further improving the accuracy and reliability of the lane lines.

[0100] The lane line tracking system provided by the present invention is described below. The lane line tracking system described below and the lane line tracking method described above can be referenced to each other.

[0101] Figure 3 A schematic diagram of the lane tracking system provided by the present invention is shown in FIG. Figure 3 As shown, the present invention also provides a lane tracking system, the system comprising:

[0102] The acquisition module is used to obtain the first point set, which is used to fit the final lane line of the previous cycle;

[0103] The compensation module is used to perform a compensation transformation on the first point set based on the vehicle's own motion in the current cycle to form a second point set in the vehicle coordinate system in the current cycle;

[0104] Detection module, used to obtain the detection lane lines of the current cycle;

[0105] a sampling module, wherein if the sampling module determines that the detected lane line matches the finalized lane line of the previous cycle, sampling the detected lane line based on the longitudinal coordinate of the second point set in the vehicle coordinate system to obtain a third point set;

[0106] A point fitting module is used to fit the second point set and the third point set to form a fourth point set;

[0107] The line fitting module is used to fit the final lane line tracked in the current cycle based on the fourth point set.

[0108] This embodiment improves the accuracy of lane line tracking through a point-based lane line tracking method.

[0109] Figure 4 A schematic diagram of the physical structure of an electronic device provided by the present invention, such as Figure 4 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the lane tracking method, which includes:

[0110] Obtaining a first point set, where the first point set is used to fit and form a finalized lane line of the previous cycle;

[0111] Based on the vehicle's own motion in the current cycle, a compensating transformation is performed on the first point set to form a second point set in the vehicle coordinate system in the current cycle;

[0112] Get the detected lane lines of the current cycle;

[0113] If it is determined that the detected lane line matches the finalized lane line of the previous cycle, sampling the detected lane line to obtain a third point set based on the longitudinal axis coordinate of the second point set in the vehicle coordinate system;

[0114] Fitting the second point set and the third point set to form a fourth point set;

[0115] Based on the fourth point set, a finalized lane line tracked in the current cycle is formed by fitting.

[0116] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0117] In another aspect, the present invention further provides a computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program includes program instructions. When the program instructions are executed by a computer, the computer can perform the lane tracking method provided by each of the above methods, wherein the method includes:

[0118] Obtaining a first point set, where the first point set is used to fit and form a finalized lane line of the previous cycle;

[0119] Based on the vehicle's own motion in the current cycle, a compensating transformation is performed on the first point set to form a second point set in the vehicle coordinate system in the current cycle;

[0120] Get the detected lane lines of the current cycle;

[0121] If it is determined that the detected lane line matches the finalized lane line of the previous cycle, sampling the detected lane line to obtain a third point set based on the longitudinal axis coordinate of the second point set in the vehicle coordinate system;

[0122] Fitting the second point set and the third point set to form a fourth point set;

[0123] Based on the fourth point set, a finalized lane line tracked in the current cycle is formed by fitting.

[0124] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program is implemented to perform the lane tracking method provided above, the method comprising:

[0125] Obtaining a first point set, where the first point set is used to fit and form a finalized lane line of the previous cycle;

[0126] Based on the vehicle's own motion in the current cycle, a compensating transformation is performed on the first point set to form a second point set in the vehicle coordinate system in the current cycle;

[0127] Get the detected lane lines of the current cycle;

[0128] If it is determined that the detected lane line matches the finalized lane line of the previous cycle, sampling the detected lane line to obtain a third point set based on the longitudinal axis coordinate of the second point set in the vehicle coordinate system;

[0129] Fitting the second point set and the third point set to form a fourth point set;

[0130] Based on the fourth point set, a finalized lane line tracked in the current cycle is formed by fitting.

[0131] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0132] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A lane line tracking method, characterized in that: The method comprises: Obtaining a first point set, where the first point set is used to fit and form a finalized lane line of the previous cycle; Based on the vehicle's own motion in the current cycle, a compensating transformation is performed on the first point set to form a second point set in the vehicle coordinate system in the current cycle; Get the detection lane line of the current cycle Get the detection lane line of the current cycle; If it is determined that the detected lane line matches the finalized lane line of the previous cycle, sampling the detected lane line to obtain a third point set based on the longitudinal axis coordinate of the second point set in the vehicle coordinate system; Fitting the second point set and the third point set to form a fourth point set; Based on the fourth point set, a finalized lane line tracked in the current cycle is formed by fitting.

2. The lane line tracking method according to claim 1, characterized in that: The method further comprises: If it is determined that the detected lane line does not match the finalized lane line of the previous cycle, the detected lane line is used as the finalized lane line detected in the current cycle.

3. The lane line tracking method according to claim 2, characterized in that: The method further includes: if it is determined that the detected lane line does not match the finalized lane line of the previous cycle, sampling at equal intervals on the longitudinal axis of the vehicle coordinate system in the current cycle based on the detected lane line to obtain a fifth point set; The fifth point set is used as the first point set of the next cycle.

4. The lane line tracking method according to claim 1, characterized in that: Fitting the second point set and the third point set to form a fourth point set includes: Obtaining projections of the second point set and the third point set on the longitudinal axis of the vehicle coordinate system; Fitting the second point set and the third point set of the overlapping portion of the projection to form a first sub-point set of the overlapping portion; Obtaining a translation amount between a first connecting line formed by the second point set and / or the third point set of the non-overlapping projection portion and a second connecting line formed by the first sub-point set; translating the second point set and / or the third point set of the non-overlapping projections in a direction close to the second connecting line according to the translation amount to form a second sub-point set; The first sub-point set and the second sub-point set are combined to form the fourth point set.

5. The lane line tracking method according to claim 1, characterized in that: Fitting the second point set and the third point set to form a fourth point set includes: Inputting the second point set and the third point set into a Kalman filter model; The Kalman filter model outputs the fourth point set.

6. The lane line tracking method according to claim 1, characterized in that: The method further comprises: Obtaining two finalized lane lines within a historical period, and measuring a first distance between the two finalized lane lines within the historical period; Obtaining two finalized lane lines tracked in a current cycle, measuring a second distance between the two finalized lane lines tracked in the current cycle, and successfully matching the two finalized lane lines tracked in the current cycle with the two finalized lane lines in the historical cycle; Calculating a difference between the first spacing and the second spacing; Based on the difference, an offset adjustment is performed on the positions of the two finalized lane lines tracked in the current cycle so that the difference is zero.

7. The lane line tracking method according to claim 1, characterized in that: The method further comprises: Get the finalized lane lines within multiple cycles; Based on the finalized lane lines in the multiple cycles, lane lines across cycles are formed by fitting.

8. A lane tracking system, characterized in that: The system comprises: An acquisition module, configured to acquire a first point set, wherein the first point set is used to fit and form a finalized lane line of a previous cycle; a compensation module, configured to perform a compensation transformation on the first point set based on the vehicle's own motion in the current cycle to form a second point set in the vehicle coordinate system in the current cycle; Detection module, used to obtain the detection lane lines of the current cycle; a sampling module, wherein if the sampling module determines that the detected lane line matches the finalized lane line of the previous cycle, sampling the detected lane line based on the longitudinal coordinate of the second point set in the vehicle coordinate system to obtain a third point set; a point fitting module, configured to fit the second point set and the third point set to form a fourth point set; The line fitting module is used to fit the final lane line tracked in the current cycle based on the fourth point set.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the lane line tracking method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the lane line tracking method according to any one of claims 1 to 7 are implemented.

Images